Hazelcast 简明教程
Hazelcast - Introduction
Distributed In-memory Data Grid
数据网格是分布式缓存的超集。分布式缓存通常仅用于存储和检索跨缓存服务器分布的键值对。然而,数据网格除了支持键值对存储之外,还支持其他功能,例如,
A data grid is a superset to distributed cache. Distributed cache is typically used only for storing and retrieving key-value pairs which are spread across caching servers. However, a data grid, apart from supporting storage of key-value pairs, also supports other features, for example,
-
It supports other data structures like locks, semaphores, sets, list, and queues.
-
It provides a way to query the stored data by rich querying languages, for example, SQL.
-
It provides a distributed execution engine which helps to operate on the data in parallel.
Benefits of Hazelcast
-
Support multiple data structures − Hazelcast supports the usage of multiple data structures along with Map. Some of the examples are Lock, Semaphore, Queue, List, etc.
-
Fast R/W access − Given that all the data is in-memory, Hazelcast offers very high-speed data read/write access.
-
High availability − Hazelcast supports the distribution of data across machines along with additional support for backup. This means that the data is not stored on a single machine. So, even if a machine goes down, which occurs frequently in a distributed environment, the data is not lost.
-
High Performance − Hazelcast provides constructs which can be used to distribute the workload/computation/query among multiple worker machines. This means a computation/query uses resources from multiple machines which reduces the execution time drastically.
-
Easy to use − Hazelcast implements and extends a lot of java.util.concurrent constructs which make it very easy to use and integrate with the code. Configuration to start using Hazelcast on a machine just involves adding the Hazelcast jar to our classpath.
Hazelcast vs Other Caches & Key-Value stores
将 Hazelcast 与其他缓存(如 Ehcache、Guava 和 Caffeine)进行比较可能不是很有用。这是因为与其他缓存不同,Hazelcast 是一个分布式缓存,也就是说,它将数据跨机器/JVM 分发。尽管 Hazelcast 也可以在单个 JVM 上很好地工作,但它在分布式环境中更有用。
Comparing Hazelcast with other caches like Ehcache, Guava, and Caffeine may not be very useful. It is because, unlike other caches, Hazelcast is a distributed cache, that is, it spreads the data across machines/JVM. Although Hazelcast can work very well on single JVM as well, however, it is more useful is a distributed environment.
同样,将其与 MongoDB 之类的数据库进行比较也没有多大用处。这是因为 Hazelcast 主要将数据存储在内存中(尽管它也支持写入磁盘)。因此,它提供了较高的 R/W 速度,但限制在于数据需要存储在内存中。
Similarly comparing it with Databases like MongoDB is also of not much use. This is because, Hazelcast mostly stores data in memory (although it also supports writing to disk). So, it offers high R/W speed with the limitation that data needs to be stored in memory.
与其他数据存储不同,Hazelcast 还支持缓存/存储复杂数据类型并提供了查询它们的接口。
Hazelcast also supports caching/storing complex data types and provides an interface to query them, unlike other data stores.
然而,可以与 Redis 进行比较,它也提供了类似的功能。
A comparison, however, can be made with Redis which also offers similar features.
Hazelcast vs Redis
在功能方面,Redis 和 Hazelcast 非常相似。然而,以下几点是 Hazelcast 优于 Redis 的地方−
In terms of features, both Redis and Hazelcast are very similar. However, following are the points where Hazelcast scores over Redis −
-
Built for Distributed Environment from ground-up − Unlike Redis, which started as single machine cache, Hazelcast, from the very beginning, has been built for distributed environment.
-
Simple cluster scale in/out − Maintaining a cluster where nodes are added or removed is very simple in case of Hazelcast, for example, adding a node is a matter of launching the node with the required configuration. Removing a node requires simple shutting down of the node. Hazelcast automatically handles partitioning of data, etc. Having the same setup for Redis and performing the above operation requires more precaution and manual efforts.
-
Less resources needs to support failover − Redis follows master-slave approach. For failover, Redis requires additional resources to setup Redis Sentinel. These Sentinel nodes are responsible to elevate a slave to master if the original master node goes down. In Hazelcast, all nodes are treated equal, failure of a node is detected by other nodes. So, the case of a node going down is handled pretty transparently and that too without any additional set of monitoring servers.
-
Simple Distributed Compute − Hazelcast, with its EntryProcessor, provides a simple interface to send the code to the data for parallel processing. This reduces data transfer over the wire. Redis also supports this, however, achieving this requires one to be aware of Lua scripting which adds additional learning curve.
Hazelcast - Setup
Hazelcast 要求 Java 1.6 或更高版本。Hazelcast 还可以与 .NET、C++ 或其他基于 JVM 的语言(如 Scala 和 Clojure)一起使用。不过,对于本教程,我们将使用 Java 8。
Hazelcast requires Java 1.6 or above. Hazelcast can also be used with .NET, C++, or other JVM based languages like Scala and Clojure. However, for this tutorial, we are going to use Java 8.
在我们继续之前,以下是我们将用于本教程的项目设置。
Before we move on, following is the project setup that we will use for this tutorial.
hazelcast/
├── com.example.demo/
│ ├── SingleInstanceHazelcastExample.java
│ ├── MultiInstanceHazelcastExample.java
│ ├── Server.java
│ └── ....
├── pom.xml
├── target/
├── hazelcast.xml
├── hazelcast-multicast.xml
├── ...
现在,我们只需在 hazelcast 目录中创建包,即 com.example.demo。然后,直接 cd 到该目录。我们将在即将到来的部分中查看其他文件。
For now, we can just create the package, i.e., com.example.demo inside the hazelcast directory. Then, just cd to that directory. We will look at other files in the upcoming sections.
Installing Hazelcast
安装 Hazelcast 只需将 JAR 文件添加到您的构建文件。基于您使用 Maven 或 Gradle,POM 文件或 build.gradle。
Installing Hazelcast simply involves adding a JAR file to your build file. POM file or build.gradle based on whether you are using Maven or Gradle respectively.
如果您使用 Gradle,将以下内容添加到 build.gradle 文件中就足够了−
If you are using Gradle, adding the following to build.gradle file would be enough −
dependencies {
compile "com.hazelcast:hazelcast:3.12.12”
}
POM for the tutorial
我们将在教程中使用以下 POM −
We will use the following POM for our tutorial −
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>1.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo</name>
<description>Demo project for Hazelcast</description>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast</artifactId>
<version>3.12.12</version>
</dependency>
</dependencies>
<!-- Below build plugin is not needed for Hazelcast, it is being used only to created a shaded JAR so that -->
<!-- using the output i.e. the JAR becomes simple for testing snippets in the tutorial-->
<build>
<plugins>
<plugin>
<!-- Create a shaded JAR and specify the entry point class-->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Hazelcast - First Application
Hazelcast 可以单独运行(单节点),也可以运行多个节点以形成一个集群。让我们先尝试启动一个实例。
Hazelcast can be run in isolation (single node) or multiple nodes can be run to form a cluster. Let us first try starting a single instance.
Single Instance
Example
现在,让我们尝试创建和使用 Hazelcast 集群的一个实例。为此,我们将创建 SingleInstanceHazelcastExample.java 文件。
Now, let us try creating and using a single instance of Hazelcast cluster. For that, we will create SingleInstanceHazelcastExample.java file.
package com.example.demo;
import java.util.Map;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
public class SingleInstanceHazelcastExample {
public static void main(String... args){
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
System.out.println(“Hello world”);
// perform a graceful shutdown
hazelcast.shutdown();
}
}
现在,让我们编译代码并执行它−
Now let’s compile the code and execute it −
mvn clean install
java -cp target/demo-0.0.1-SNAPSHOT.jar
com.example.demo.SingleInstanceHazelcastExample
Output
如果执行上述代码,输出将是−
If you execute above code, the output would be −
Hello World
然而,更重要的是,你还会注意到 Hazelcast 的日志行,它表示 Hazelcast 已启动。由于我们只运行此代码一次,即单个 JVM,我们的集群中只会有一个成员。
However, more importantly, you will also notice log lines from Hazelcast which signifies that Hazelcast has started. Since we are running this code only once, i.e., a single JVM, we would only have one member in our cluster.
Jan 30, 2021 10:26:51 AM com.hazelcast.config.XmlConfigLocator
INFO: Loading 'hazelcast-default.xml' from classpath.
Jan 30, 2021 10:26:51 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [dev] [3.12.12] Prefer IPv4 stack is true.
Jan 30, 2021 10:26:52 AM com.hazelcast.instance.AddressPicker
INFO: [LOCAL] [dev] [3.12.12] Picked [localhost]:5701, using socket
ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
Jan 30, 2021 10:26:52 AM com.hazelcast.system
...
Members {size:1, ver:1} [
Member [localhost]:5701 - 9b764311-9f74-40e5-8a0a-85193bce227b this
]
Jan 30, 2021 10:26:56 AM com.hazelcast.core.LifecycleService
INFO: [localhost]:5701 [dev] [3.12.12] [localhost]:5701 is STARTED
...
You will also notice log lines from Hazelcast at the end which signifies
Hazelcast was shutdown:
INFO: [localhost]:5701 [dev] [3.12.12] Hazelcast Shutdown is completed in 784 ms.
Jan 30, 2021 10:26:57 AM com.hazelcast.core.LifecycleService
INFO: [localhost]:5701 [dev] [3.12.12] [localhost]:5701 is SHUTDOWN
Cluster: Multi Instance
现在,让我们创建 MultiInstanceHazelcastExample.java 文件(如下所示),它将用于多实例集群。
Now, let’s create MultiInstanceHazelcastExample.java file (as below) which would be used for multi-instance cluster.
package com.example.demo;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
public class MultiInstanceHazelcastExample {
public static void main(String... args) throws InterruptedException{
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
//print the socket address of this member and also the size of the cluster
System.out.println(String.format("[%s]: No. of hazelcast members: %s",
hazelcast.getCluster().getLocalMember().getSocketAddress(),
hazelcast.getCluster().getMembers().size()));
// wait for the member to join
Thread.sleep(30000);
//perform a graceful shutdown
hazelcast.shutdown();
}
}
让我们在 two different shells 上执行以下命令−
Let’s execute the following command on two different shells −
java -cp .\target\demo-0.0.1-SNAPSHOT.jar
com.example.demo.MultiInstanceHazelcastExample
你将在 1st shell 上注意到已启动 Hazelcast 实例,并且已分配了一个成员。请注意输出的最后一行,其中显示 single member using port 5701 。
You would notice on the 1st shell that a Hazelcast instance has been started and a member has been assigned. Note the last line of output which says that there is a single member using port 5701.
Jan 30, 2021 12:20:21 PM com.hazelcast.internal.cluster.ClusterService
INFO: [localhost]:5701 [dev] [3.12.12]
Members {size:1, ver:1} [
Member [localhost]:5701 - b0d5607b-47ab-47a2-b0eb-6c17c031fc2f this
]
Jan 30, 2021 12:20:21 PM com.hazelcast.core.LifecycleService
INFO: [localhost]:5701 [dev] [3.12.12] [localhost]:5701 is STARTED
[/localhost:5701]: No. of hazelcast members: 1
你将在 2nd shell 上注意到 Hazelcast 实例已加入第一个实例。请注意输出的最后一行,其中显示现在有 two members using port 5702 。
You would notice on the 2nd shell that a Hazelcast instance has joined the 1st instance. Note the last line of the output which says that there are now two members using port 5702.
INFO: [localhost]:5702 [dev] [3.12.12]
Members {size:2, ver:2} [
Member [localhost]:5701 - b0d5607b-47ab-47a2-b0eb-6c17c031fc2f
Member [localhost]:5702 - 037b5fd9-1a1e-46f2-ae59-14c7b9724ec6 this
]
Jan 30, 2021 12:20:46 PM com.hazelcast.core.LifecycleService
INFO: [localhost]:5702 [dev] [3.12.12] [localhost]:5702 is STARTED
[/localhost:5702]: No. of hazelcast members: 2
Hazelcast - Configuration
Hazelcast 支持基于编程的配置以及基于 XML 的配置。然而,由于易于使用,XML 配置在生产中被大量使用。但 XML 配置在内部使用基于编程的配置。
Hazelcast supports programmatic as well as XML-based configuration. However, it is the XML configuration which is heavily used in production, given its ease of use. But XML configuration internally uses the Programmatic configuration.
XML Configuration
hazelcast.xml 是需要放置这些配置的位置。以以下位置(按相同顺序)搜索该文件,并从第一个可用位置选择它−
The hazelcast.xml is where these configurations need to be placed. The file is searched for in the following location (in same order) and is chosen from the first available location −
-
Passing the location of the XML to the JVM via the system property - Dhazelcast.config=/path/to/hazelcast.xml
-
hazelcast.xml in the current working directory
-
hazelcast.xml in the classpath
-
default hazelcast.xml provided by Hazelcast
一旦找到 XML,Hazelcast 将从 XML 文件加载所需的配置。
Once the XML is found, Hazelcast would load the required configuration from the XML file.
让我们通过一个例子来尝试一下。在当前目录中创建一个名为 hazelcast.xml 的 XML。
Let’s try that out with an example. Create an XML in your current directory with the name hazelcast.xml.
<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config
http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<!-- name of the instance -->
<instance-name>XML_Hazelcast_Instance</instance-name>
</hazelcast>
现在的 XML 只包含 Hazelcast XML 的模式位置,该模式位置用于验证。但更重要的是,它包含实例名称。
The XML as of now only contains the schema location of the Hazelcast XML which is used for validation. But more importantly, it contains the instance name.
Example
现在创建 XMLConfigLoadExample.java 文件,其中包含以下内容。
Now create an XMLConfigLoadExample.java file with the following content.
package com.example.demo;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
public class XMLConfigLoadExample {
public static void main(String... args) throws InterruptedException{
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
//specified the name written in the XML file
System.out.println(String.format("Name of the instance: %s",hazelcast.getName()));
//perform a graceful shutdown
hazelcast.shutdown();
}
}
使用以下命令执行上面的 Java 文件 -
Execute the above Java file with the following command −
java -Dhazelcast.config=hazelcast.xml -cp .\target\demo-0.0.1-SNAPSHOT.jar
com.example.demo.XMLConfigLoadExample
Output
上述命令的输出将是 -
The output for above command would be −
Jan 30, 2021 1:21:41 PM com.hazelcast.config.XmlConfigLocator
INFO: Loading configuration hazelcast.xml from System property
'hazelcast.config'
Jan 30, 2021 1:21:41 PM com.hazelcast.config.XmlConfigLocator
INFO: Using configuration file at C:\Users\demo\eclipseworkspace\
hazelcast\hazelcast.xml
...
Members {size:1, ver:1} [
Member [localhost]:5701 - 3d400aed-ddb9-4e59-9429-3ab7773e7e09 this
]
Name of cluster: XML_Hazelcast_Instance
正如您所看到的,Hazelcast 已经加载了配置并打印了配置中指定的名字(最后一行)。
As you see, Hazelcast loaded the configuration and printed the name which was specified in the configuration (last line).
XML 中可以指定许多配置选项。完整列表可以在以下位置找到 -
There are a whole lot of configuration options which can be specified in the XML. The complete list can be found at −
在我们继续学习本教程时,我们将看到其中一些配置。
We will see a few of these configurations as we move along the tutorial.
Programmatic Configuration
如前所述,XML 配置最终通过编程配置来完成。因此,让我们针对我们在 XML 配置中看到的同一个示例尝试编程配置。为此,让我们创建一个 ProgramaticConfigLoadExample.java 文件,其中包含以下内容。
As stated earlier, XML configuration is ultimately done via programmatic configuration. So, let’s try programmatic configuration for the same example which we saw in XML configuration. For that, let’s create the ProgramaticConfigLoadExample.java file with the following content.
Example
package com.example.demo;
import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
public class ProgramaticConfigLoadExample {
public static void main(String... args) throws InterruptedException {
Config config = new Config();
config.setInstanceName("Programtic_Hazelcast_Instance");
// initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance(config);
// specified the name written in the XML file
System.out.println(String.format("Name of the instance: %s", hazelcast.getName()));
// perform a graceful shutdown
hazelcast.shutdown();
}
}
让我们在不传递任何 hazelcast.xml 文件的情况下执行代码 -
Let’s execute the code without passing any hazelcast.xml file by −
java -cp .\target\demo-0.0.1-SNAPSHOT.jar
com.example.demo.ProgramaticConfigLoadExample
Logging
为了避免依赖项,Hazelcast 默认使用基于 JDK 的日志记录。但它也支持通过 slf4j, log4j 进行日志记录。例如,如果我们想要通过 logback 设置使用 sl4j 的日志记录,我们可以更新 POM 以包含以下依赖项 -
To avoid dependencies, Hazelcast by default uses JDK based logging. But it also supports logging via slf4j, log4j. For example, if we want to setup logging via for sl4j with logback, we can update the POM to contain the following dependencies −
<!-- contains both sl4j bindings and the logback core -->
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
</dependency>
Example
定义一个配置 logback.xml 文件并将它添加到您的类路径,例如 src/main/resources。
Define a configuration logback.xml file and add it to your classpath, for example, src/main/resources.
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="STDOUT" />
</root>
<logger name="com.hazelcast" level="error">
<appender-ref ref="STDOUT" />
</logger>
</configuration>
现在,当我们执行以下命令时,我们注意到关于 Hazelcast 成员创建等的所有元信息都没有被打印出来。这是因为我们已经把 Hazelcast 的日志级别设置为 error,并要求 Hazelcast 使用 sl4j 日志记录器。
Now, when we execute the following command, we notice that all the meta information about the Hazelcast member creation etc. is not printed. And this is because we have set the logging level for Hazelcast to error and asked Hazelcast to use sl4j logger.
java -Dhazelcast.logging.type=slf4j -cp .\target\demo-0.0.1-SNAPSHOT.jar com.example.demo.SingleInstanceHazelcastExample
Variables
写入 XML 配置文件的值可能因环境而异。例如,在生产环境中,与开发环境相比,您可能对连接 Hazelcast 集群使用不同的用户名/密码。除了维护独立的 XML 文件外,人们还可以在 XML 文件中编写变量,然后通过命令行或以编程方式将这些变量传递给 Hazelcast。下面是通过命令行选择实例名称的一个示例。
Value written to XML configuration files can vary based on the environment. For example, in production, you may use a different username/password for connecting to the Hazelcast cluster compared to the dev environment. Instead of maintaining separate XML files, one can also write variables in the XML files and then pass those variables via command line or programmatically to Hazelcast. Here is an example for choosing the name of the instance from the command line.
因此,以下是我们的 XML 文件,其中包含变量 ${varname}
So, here is our XML file with the variable ${varname}
<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config
http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<instance-name>${instance_name}</instance-name>
</hazelcast>
Example
以下是我们将用来打印变量值的示例 Java 代码 -
And here is the sample Java code we would use to print the variable value −
package com.example.demo;
import java.util.Map;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
public class XMLConfigLoadWithVariable {
public static void main(String... args) throws InterruptedException {
// initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
// specified the name written in the XML file
System.out.println(String.format("Name of the instance: %s", hazelcast.getName()));
// perform a graceful shutdown
hazelcast.shutdown();
}
}
以下命令 -
And, following is the command −
java -Dhazelcast.config=others\hazelcast.xml -Dinstance_name=dev_cluster -cp
.\target\demo-0.0.1-SNAPSHOT.jar com.example.demo.XMLConfigLoadWithVariable
Hazelcast - Setting up multi node instances
鉴于 Hazelcast 是一个分布式的 IMDG,并且通常在多台计算机上设置,因此它需要访问内部/外部网络。最重要的用例是在集群内发现 Hazelcast 节点。
Given that Hazelcast is a distributed IMDG and typically is set up on multiple machines, it requires access to the internal/external network. The most important use-case being discovery of Hazelcast nodes within a cluster.
Hazelcast 需要以下端口 -
Hazelcast requires the following ports −
-
1 inbound port to receive pings/data from other Hazelcast nodes/clients
-
n number of outbound ports which are required to send ping/data to other members of the cluster.
此节点发现以几种方式发生 -
This node discovery happens in few ways −
-
Multicast
-
TCP/IP
-
Amazon EC2 auto discovery
其中,我们将了解多播和 TCP/IP
Of this, we will look at Multicast and TCP/IP
Multicast
默认情况下启用多播连接机制。 https://en.wikipedia.org/wiki/Multicast 是一种通信形式,其中消息会发送到组中的所有节点。这是 Hazelcast 用于发现集群中其他成员的方式。早前我们已了解到,所有示例均使用多播来发现成员。
Multicast joining mechanism is enabled by default. https://en.wikipedia.org/wiki/Multicast is a way of communication form in which message is transmitted to all the nodes in a group. And this is what Hazelcast uses to discover other members of the cluster. All the examples that we have looked at earlier use multicast to discover members.
Example
现在,让我们明确启用它。在 hazelcast-multicast.xml 中添加以下内容:
Let’s now explicitly turn it on. Save the following in hazelcast-multicast.xml
<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config
http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<network>
<join>
<multicast enabled="true" />
</join>
</network>
</hazelcast>
然后,让我们执行以下内容 -
And then, let us execute the following −
java -Dhazelcast.config=hazelcast-multicast.xml -cp .\target\demo-0.0.1-
SNAPSHOT.jar com.example.demo.XMLConfigLoadExample
Output
在输出中,我们注意到以下来自 Hazelcast 的行,这实际上表示使用多播连接器来发现成员。
In the output, we notice the following lines from Hazelcast which effectively means that multicast joiner is used to discover the members.
Jan 30, 2021 5:26:15 PM com.hazelcast.instance.Node
INFO: [localhost]:5701 [dev] [3.12.12] Creating MulticastJoiner
默认情况下,多播接受来自多播组内所有计算机的通信。这可能是一个安全隐患,这就是为什么通常在内部部署中会对多播通信使用防火墙的原因。因此,多播虽然适合开发工作,但在生产中,最好使用基于 TCP/IP 的发现。
Multicast, by default, accepts communication from all the machines in the multicast group. This may be a security concern and that is why typically, on-premise, multicast communication is firewalled. So, while multicast is good for development work, in production, it is best to use TCP/IP based discovery.
TCP/IP
由于陈述了多播的缺陷,因此 TCP/IP 成为首选的通信方式。在 TCP/IP 的情况下,一个成员只能连接到已知/列出的成员。
Due to the drawbacks stated for Multicast, TCP/IP is the preferred way for communication. In case of TCP/IP, a member can connect to only known/listed members.
Example
我们让 TCP/IP 用于发现机制。在 hazelcast-tcp.xml 中添加以下内容:
Let’s use TCP/IP for discovery mechanisms. Save the following in hazelcast-tcp.xml
<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config
http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<network>
<join>
<multicast enabled="false" />
<tcp-ip enabled="true">
<members>localhost</members>
</tcp-ip>
</join>
</network>
</hazelcast>
然后,我们来执行以下命令 -
And then, let’s execute the following command −
java -Dhazelcast.config=hazelcast-tcp.xml -cp .\target\demo-0.0.1-SNAPSHOT.jar
com.example.demo.XMLConfigLoadExample
Output
output 如下 -
The output is following −
INFO: [localhost]:5701 [dev] [3.12.12] Creating TcpIpJoiner
Jan 30, 2021 8:09:29 PM
com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
上述输出显示 TCP/IP 连接器用于连接两个成员。
The above output shows that TCP/IP joiner was use to join two members.
如果您在两个不同的 Shell 上执行以下命令 -
And if you execute following command on two different shells −
java '-Dhazelcast.config=hazelcast-tcp.xml' -cp .\target\demo-0.0.1-SNAPSHOT.jar
com.example.demo.MultiInstanceHazelcastExample
我们看到以下输出 -
We see the following output −
Members {size:2, ver:2} [
Member [localhost]:5701 - 62eedeae-2701-4df0-843c-7c3655e16b0f
Member [localhost]:5702 - 859c1b46-06e6-495a-8565-7320f7738dd1 this
]
上述输出表示节点能够使用 TCP/IP 连接,且两者都使用本地主机作为 IP 地址。
The above output means that the nodes were able to join using TCP/IP and both are using localhost as the IP address.
请注意我们在 XML 配置文件中可以指定更多 IP 或机器名称(会通过 DNS 解析)。
Note that we can specify more IPs or the machine names (which would be resolved by DNS) in the XML configuration file.
<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config
http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<network>
<join>
<multicast enabled="false" />
<tcp-ip enabled="true">
<members>machine1, machine2....</members>
</tcp-ip>
</join>
</network>
</hazelcast>
Hazelcast - Data Structures
java.util.concurrent 包提供了诸如 AtomicLong、CountDownLatch、ConcurrentHashMap 这样的数据结构,当有多个线程对数据结构读/写数据时,这些数据结构非常有用。但是为了提供线程安全性,所有这些线程都应在单个 JVM/机器上。
java.util.concurrent package provides data structures such as AtomicLong, CountDownLatch, ConcurrentHashMap, etc. which are useful when you have more than one thread reading/writing data to the data structure. But to provide thread safety, all of these threads are expected to be on a single JVM/machine.
分布式数据结构有两个主要好处:
There are two major benefits of distributing data structure −
-
Better Performance − If more than one machine has access to the data, all of them can work in parallel and complete the work in a lesser timespan.
-
Data Backup − If a JVM/machine goes down, we have another JVMs/machines holding the data
Hazelcast 提供了一种跨 JVM/机器分配数据结构的方法。
Hazelcast provides a way to distribute your data structure across JVMs/machines.
Hazelcast - Client
Hazelcast 客户端是对 Hazelcast 成员的轻量级客户端。Hazelcast 成员负责存储数据和分区。它们在传统的客户端-服务器模型中充当服务器。
Hazelcast clients are the lightweight clients to Hazelcast members. Hazelcast members are responsible to store data and the partitions. They act like the server in the traditional client-server model.
Hazelcast 客户端仅用于访问存储在集群的 Hazelcast 成员中的数据。它们不负责存储数据,也不承担存储数据的任何所有权。
Hazelcast clients are created only for accessing data stored with Hazelcast members of the cluster. They are not responsible to store data and do not take any ownership to store data.
这些客户端有自己的生命周期并且不会影响 Hazelcast 成员实例。
The clients have their own life cycle and do not affect the Hazelcast member instances.
我们先创建并运行 Server.java。
Let’s first create Server.java and run it.
import java.util.Map;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
public class Server {
public static void main(String... args){
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
//create a simple map
Map<String, String> vehicleOwners = hazelcast.getMap("vehicleOwnerMap");
// add key-value to map
vehicleOwners.put("John", "Honda-9235");
// do not shutdown, let the server run
//hazelcast.shutdown();
}
}
现在,运行上述类。
Now, run the above class.
java -cp .\target\demo-0.0.1-SNAPSHOT.jar com.example.demo.Server
对于设置客户端,我们还需要添加客户端 jar。
For setting up a client, we also need to add client jar.
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-client</artifactId>
<version>3.12.12</version>
</dependency>
现在让我们创建 Client.java。请注意,类似于 Hazelcast 成员,客户端也可以通过编程方式或通过 XML 配置(即通过 -Dhazelcast.client.config 或 hazelcast-client.xml)进行配置。
Let’s now create Client.java. Note that similar to Hazelcast members, clients can also be configured programmatically or via XML configuration (i.e., via -Dhazelcast.client.config or hazelcast-client.xml).
Example
我们使用默认配置,这意味着我们的客户端将能够连接到本地实例。
Let’s use the default configuration which means our client would be able to connect to local instances.
import java.util.Map;
import com.hazelcast.client.HazelcastClient;
import com.hazelcast.core.HazelcastInstance;
public class Client {
public static void main(String... args){
//initialize hazelcast client
HazelcastInstance hzClient = HazelcastClient.newHazelcastClient();
//read from map
Map<String, String> vehicleOwners = hzClient.getMap("vehicleOwnerMap");
System.out.println(vehicleOwners.get("John"));
System.out.println("Member of cluster: " +
hzClient.getCluster().getMembers());
// perform shutdown
hzClient.getLifecycleService().shutdown();
}
}
现在,运行上述类。
Now, run the above class.
java -cp .\target\demo-0.0.1-SNAPSHOT.jar com.example.demo.Client
Output
它将生成如下输出:
It will produce the following output −
Honda-9235
Member of cluster: [Member [localhost]:5701 - a47ec375-3105-42cd-96c7-fc5eb382e1b0]
从输出中看到 -
As seen from the output −
-
The cluster only contains 1 member which is from Server.java.
-
The client is able to access the map which is stored inside the server.
Load Balancing
Hazelcast 客户端支持使用各种算法进行负载均衡。负载均衡确保负载在成员之间共享,集群中的任何单个成员都不会过载。默认的负载均衡机制设置为循环。使用 config 中的 loadBalancer 标记可以更改此设置。
Hazelcast Client supports load balancing using various algorithms. Load balancing ensures that the load is shared across members and no single member of the cluster is overloaded. The default load balancing mechanism is set to round-robin. The same can be changed by using the loadBalancer tag in the config.
我们可以使用配置中的 load-balancer 标记指定负载均衡器的类型。下面是一个随意选择节点的策略的示例。
We can specify the type of load balancer using the load-balancer tag in the configuration. Here is a sample for choosing a strategy that randomly picks up a node.
<hazelcast-client xmlns="http://www.hazelcast.com/schema/client-config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.hazelcast.com/schema/client-config
http://www.hazelcast.com/schema/client-config/hazelcastclient-config-4.2.xsd">
<load-balancer type="random"/>
</hazelcast-client>
Failover
在分布式环境中,成员可能任意失败。为了支持故障转移,建议提供多个成员的地址。如果客户端可以访问任何一个成员,那它就能访问到其他成员。addressList 参数可以在客户端配置中指定。
In a distributed environment, members can fail arbitrarily. For supporting failover, it is recommended that address to multiple members is provided. If the client gets access to any one member, that is sufficient for it to get addressed to other members. The parameters addressList can be specified in the client configuration.
例如,如果我们使用以下配置 -
For example, if we use the following configuration −
<hazelcast-client xmlns="http://www.hazelcast.com/schema/client-config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.hazelcast.com/schema/client-config
http://www.hazelcast.com/schema/client-config/hazelcastclient-config-4.2.xsd">
<address-list>machine1, machine2</address-list>
</hazelcast-client>
即使 machine1 宕机,客户端也可以使用 machine2 访问集群的其他成员。
Even if, say, machine1 goes down, clients can use machine2 to get access to other members of the cluster.
Hazelcast - Serialization
Hazelcast 最好用于数据/查询分布在多台机器的环境中。在这些环境中,数据需要从我们的 Java 对象序列化为能够在网络上传输的字节数组。
Hazelcast is ideally used in an environment where data/query are distributed across machines. This requires data to be serialized from our Java objects to a byte array which can be transferred over the network.
Hazelcast 支持多种序列化类型。但是,让我们关注一些常用类型,即 Java 序列化和 Java Externalizable。
Hazelcast supports various types of Serialization. However, let’s look at some commonly used ones, i.e., Java Serialization and Java Externalizable.
Java Serialization
Example
首先让我们了解一下 Java 序列化。假设我们定义了一个实现了 Serializable 接口的 Employee 类。
First let’s look at Java Serialization. Let’s say, we define an Employee class with Serializable interface implemented.
public class Employee implements Serializable{
private static final long serialVersionUID = 1L;
private String name;
private String department;
public Employee(String name, String department) {
super();
this.name = name;
this.department = department;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getDepartment() {
return department;
}
public void setDepartment(String department) {
this.department = department;
}
@Override
public String toString() {
return "Employee [name=" + name + ", department=" + department + "]";
}
}
现在让我们编写代码将 Employee 对象添加到 Hazelcast 映射中。
Let’s now write code to add Employee object to the Hazelcast map.
public class EmployeeExample {
public static void main(String... args){
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
//create a set to track employees
Map<Employee, String> employeeOwners=hazelcast.getMap("employeeVehicleMap");
Employee emp1 = new Employee("John Smith", "Computer Science");
// add employee to set
System.out.println("Serializing key-value and add to map");
employeeOwners.put(emp1, "Honda");
// check if emp1 is present in the set
System.out.println("Serializing key for searching and Deserializing
value got out of map");
System.out.println(employeeOwners.get(emp1));
// perform a graceful shutdown
hazelcast.shutdown();
}
}
Output
它将生成如下输出:
It will produce the following output −
Serializing key-value and add to map
Serializing key for searching and Deserializing value got out of map
Honda
在此,一个非常重要的方面是,只需通过实现 Serializable 接口,我们就可以让 Hazelcast 使用 Java 序列化。另外请注意,Hazelcast 会存储键和值序列化的数据,而不是将它们像 HashMap 一样存储在内存中。因此,Hazelcast 负责序列化和反序列化的繁重工作。
A very important aspect here is that simply by implementing a Serializable interface, we can make Hazelcast use Java Serialization. Also note that Hazelcast stores serialized data for key and value instead of storing it in-memory like HashMap. So, Hazelcast does the heavy-lifting of Serialization and Deserialization.
Example
但是,这里有一个陷阱。在上述情况下,如果员工的部门发生改变怎么办?这个人仍然是同一个人。
However, there is a pitfall here. In the above case, what if the department of the employee changes? The person is still the same.
public class EmployeeExampleFailing {
public static void main(String... args){
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
//create a set to track employees
Map<Employee, String> employeeOwners=hazelcast.getMap("employeeVehicleMap");
Employee emp1 = new Employee("John Smith", "Computer Science");
// add employee to map
System.out.println("Serializing key-value and add to map");
employeeOwners.put(emp1, "Honda");
Employee empDeptChange = new Employee("John Smith", "Electronics");
// check if emp1 is present in the set
System.out.println("Checking if employee with John Smith is present");
System.out.println(employeeOwners.containsKey(empDeptChange));
Employee empSameDept = new Employee("John Smith", "Computer Science");
System.out.println("Checking if employee with John Smith is present");
System.out.println(employeeOwners.containsKey(empSameDept));
// perform a graceful shutdown
hazelcast.shutdown();
}
}
Output
它将生成如下输出:
It will produce the following output −
Serializing key-value and add to map
Checking if employee with name John Smith is present
false
Checking if employee with name John Smith is present
true
这是因为 Hazelcast 在比较时并不会对键(即 Employee)进行反序列化。它直接比较序列化键的字节码。因此,所有属性的值都相同的对象会被视为相同对象。但如果这些属性的值发生了改变,例如上述场景中的部门,则这两个键会被视为唯一键。
It is because Hazelcast does not deserialize the key, i.e., Employee while comparison. It directly compares the bytecode of the serialized key. So, an object with the same value to all the attributes would be treated the same. But if the value to those attributes changes, for example, department in the above scenario, those two keys are treated as unique.
Java Externalizable
如果在上述示例中,我们在对键进行序列化/反序列化时不关心部门的值怎么办。Hazelcast 也支持 Java Externalizable,它让我们可以控制用于序列化和反序列化的标签。
What if, in the above example, we don’t care about the value of the department while performing serialization/deserialization of keys. Hazelcast also supports Java Externalizable which gives us control over what tags are used for serialization and deserialization.
Example
让我们修改我们的 Employee 类 −
Let’s modify our Employee class accordingly −
public class EmplyoeeExternalizable implements Externalizable {
private static final long serialVersionUID = 1L;
private String name;
private String department;
public EmplyoeeExternalizable(String name, String department) {
super();
this.name = name;
this.department = department;
}
@Override
public void readExternal(ObjectInput in) throws IOException,
ClassNotFoundException {
System.out.println("Deserializaing....");
this.name = in.readUTF();
}
@Override
public void writeExternal(ObjectOutput out) throws IOException {
System.out.println("Serializing....");
out.writeUTF(name);
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getDepartment() {
return department;
}
public void setDepartment(String department) {
this.department = department;
}
@Override
public String toString() {
return "Employee [name=" + name + ", department=" + department + "]";
}
}
因此,正如您从代码中看到的那样,我们添加了 readExternal/writeExternal 方法,它们负责序列化/反序列化。鉴于我们在序列化/反序列化时是对部门不感兴趣的,所以我们在 readExternal/writeExternal 方法中排除了部门。
So, as you can see from the code, we have added readExternal/writeExternal methods which are responsible for serialization/deserialization. Given that we are not interested in the department while serialization/deserialization, we exclude those in readExternal/writeExternal methods.
Example
现在,如果我们执行以下代码 −
Now, if we execute the following code −
public class EmployeeExamplePassing {
public static void main(String... args){
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
//create a set to track employees
Map<EmplyoeeExternalizable, String> employeeOwners=hazelcast.getMap("employeeVehicleMap");
EmplyoeeExternalizable emp1 = new EmplyoeeExternalizable("John Smith", "Computer Science");
// add employee to map
employeeOwners.put(emp1, "Honda");
EmplyoeeExternalizable empDeptChange = new EmplyoeeExternalizable("John Smith", "Electronics");
// check if emp1 is present in the set
System.out.println("Checking if employee with John Smith is present");
System.out.println(employeeOwners.containsKey(empDeptChange));
EmplyoeeExternalizable empSameDept = new EmplyoeeExternalizable("John Smith", "Computer Science");
System.out.println("Checking if employee with John Smith is present");
System.out.println(employeeOwners.containsKey(empSameDept));
// perform a graceful shutdown
hazelcast.shutdown();
}
}
Output
我们得到的输出是 −
The output we get is −
Serializing....
Checking if employee with John Smith is present
Serializing....
true
Checking if employee with John Smith is present
Serializing....
true
正如输出所示,使用 Externalizable 接口,我们可以仅提供员工姓名的序列化数据给 Hazelcast。
As the output shows, using Externalizable interface, we can provide Hazelcast with serialized data for only the name of the employee.
另外请注意,Hazelcast 序列化我们的键两次 −
Also note that Hazelcast serializes our key twice −
-
Once while storing the key,
-
And, second for searching the given key in the map. As stated earlier, this is because Hazelcast uses serialized byte arrays for key comparison.
总体而言,与 Serializable 相比,使用 Externalizable 的优势更多,因为如果我们想要更多地控制要序列化的属性以及我们希望如何处理它们,它提供了更多控制权。
Overall, using Externalizable has more benefits as compared to Serializable if we want to have more control over what attributes are to be serialized and how we want to handle them.
Hazelcast - Spring Integration
Hazelcast 支持一种简单的方法来与 Spring Boot 应用程序集成。让我们通过一个示例来理解这一点。
Hazelcast supports an easy way to integrate with Spring Boot application. Let’s try to understand that via an example.
我们将创建一个简单的 API 应用程序,它为获取公司员工信息提供了一个 API。为此,我们将使用 Spring Boot 驱动的 RESTController 以及 Hazelcast 来缓存数据。
We will create a simple API application which provides an API to get employee information for a company. For this purpose, we will use Spring Boot driven RESTController along with Hazelcast for caching data.
请注意,要在 Spring Boot 中集成 Hazelcast,我们需要两件事——
Note that to integrate Hazelcast in Spring Boot, we will need two things −
-
Add Hazelcast as a dependency to our project.
-
Define a configuration (static or programmatic) and make it available to Hazelcast
让我们首先定义 POM。请注意,我们必须指定 Hazelcast JAR 以便在 Spring Boot 项目中使用它。
Let’s first define the POM. Note that we have to specify Hazelcast JAR to use it in the Spring Boot project.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>hazelcast</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo</name>
<description>Demo project to explain Hazelcast integration with Spring Boot</description>
<properties>
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.source>1.8</maven.compiler.source>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.4.0</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-all</artifactId>
<version>4.0.2</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
另外,将 hazelcast.xml 添加到 src/main/resources 中——
Also add hazelcast.xml to src/main/resources −
<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config
http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<instance-name>XML_Hazelcast_Instance</instance-name>
</hazelcast>
定义 Spring Boot 要使用的入口点文件。确保我们指定了 @EnableCaching
Define an entry point file for Spring Boot to use. Ensure that we have @EnableCaching specified −
package com.example.demo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;
@EnableCaching
@SpringBootApplication
public class CompanyApplication {
public static void main(String[] args) {
SpringApplication.run(CompanyApplication.class, args);
}
}
让我们定义我们的员工 POJO ——
Let us define our employee POJO −
package com.example.demo;
import java.io.Serializable;
public class Employee implements Serializable{
private static final long serialVersionUID = 1L;
private int empId;
private String name;
private String department;
public Employee(Integer id, String name, String department) {
super();
this.empId = id;
this.name = name;
this.department = department;
}
public int getEmpId() {
return empId;
}
public void setEmpId(int empId) {
this.empId = empId;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getDepartment() {
return department;
}
public void setDepartment(String department) {
this.department = department;
}
@Override
public String toString() {
return "Employee [empId=" + empId + ", name=" + name + ", department=" + department + "]";
}
}
最后,让我们定义一个基本的 REST 控制器来访问员工 ——
And ultimately, let us define a basic REST controller to access employee −
package com.example.demo;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/v1/")
class CompanyApplicationController{
@Cacheable(value = "employee")
@GetMapping("employee/{id}")
public Employee getSubscriber(@PathVariable("id") int id) throws
InterruptedException {
System.out.println("Finding employee information with id " + id + " ...");
Thread.sleep(5000);
return new Employee(id, "John Smith", "CS");
}
}
现在让我们通过运行命令来执行上述应用程序——
Now let us execute the above application, by running the command −
mvn clean install
mvn spring-boot:run
你会注意到命令的输出将包含 Hazelcast 成员信息,这意味着 Hazelcast 实例会使用 hazelcast.xml 配置自动为我们配置。
You will notice that the output of the command would contain Hazelcast member information which mean Hazelcast Instance is automatically configured for us using hazelcast.xml configuration.
Members {size:1, ver:1} [
Member [localhost]:5701 - 91b3df1d-a226-428a-bb74-6eec0a6abb14 this
]
现在让我们通过 curl 执行或使用浏览器来访问 API ——
Now let us execute via curl or use browser to access API −
curl -X GET http://localhost:8080/v1/employee/5
API 的输出将是我们的员工样本。
The output of the API would be our sample employee.
{
"empId": 5,
"name": "John Smith",
"department": "CS"
}
在服务器日志中(即运行 Spring Boot 应用程序的位置),我们看到以下行——
In the server logs (i.e. where Spring Boot application running), we see the following line −
Finding employee information with id 5 ...
但是,请注意,访问信息需要大约 5 秒(因为我们添加了睡眠)。但如果我们再次调用 API,API 的输出将立即显示。这是因为我们指定了 @Cacheable 符号。我们第一次 API 调用的数据已使用 Hazelcast 作为后端缓存。
However, note that it takes almost 5 secs (because of sleep we added) to access the information. But If we call the API again, the output of the API is immediate. This is because we have specified @Cacheable notation. The data of our first API call has been cached using Hazelcast as a backend.
Hazelcast - Monitoring
Hazelcast 提供有多种方法来监视集群。我们将研究如何通过 REST API 和 JMX 来进行监视。让我们首先研究 REST API。
Hazelcast provides multiple ways to monitor the cluster. We will look into how to monitor via REST API and via JMX. Let’s first look into REST API.
Monitoring Hazelcast via REST API
要通过 REST API 监视集群或成员状态,必须为成员启用基于 REST API 的通信。这可以通过配置以及以编程方式来完成。
To monitor health of the cluster or member state via REST API, one has to enable REST API based communication to the members. This can be done by configuration and also programmatically.
让我们通过 hazelcast-monitoring.xml 中的 XML 配置启用基于 REST 的监视 −
Let us enable REST based monitoring via XML configuration in hazelcast-monitoring.xml −
<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config
http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<instance-name>XML_Hazelcast_Instance</instance-name>
<network>
<rest-api enabled="true">
<endpoint-group name="CLUSTER_READ" enabled="true" />
<endpoint-group name="HEALTH_CHECK" enabled="true" />
</rest-api>
</network>
</hazelcast>
让我们创建一个在 Server.java 文件中无限期运行的 Hazelcast 实例 −
Let us create a Hazelcast instance which runs indefinitely in Server.java file −
public class Server {
public static void main(String... args){
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
// do not shutdown, let the server run
//hazelcast.shutdown();
}
}
现在让我们启动集群 −
And now let us execute start the cluster −
java '-Dhazelcast.config=hazelcast-monitoring.xml' -cp .\target\demo-0.0.1-
SNAPSHOT.jar com.example.demo.Server
启动后,可以通过调用 API 来了解集群的健康状况,如下所示 −
Once started, the health of the cluster can be found out by calling the API like −
http://localhost:5701/hazelcast/health
上述 API 调用的 output −
The output of the above API call −
Hazelcast::NodeState=ACTIVE
Hazelcast::ClusterState=ACTIVE
Hazelcast::ClusterSafe=TRUE
Hazelcast::MigrationQueueSize=0
Hazelcast::ClusterSize=1
这表明集群中有一个成员,它处于活动状态。
This displays that there is 1 member in our cluster and it is Active.
有关节点的更详细信息,例如,IP、端口、名称,可以使用 − 找到
More detailed information about the nodes, for example, IP, port, name can be found using −
http://localhost:5701/hazelcast/rest/cluster
上述 API 的输出 −
The output of the above API −
Members {size:1, ver:1} [
Member [localhost]:5701 - e6afefcb-6b7c-48b3-9ccb-63b4f147d79d this
]
ConnectionCount: 1
AllConnectionCount: 2
JMX monitoring
Hazelcast 还支持对其中嵌入的数据结构进行 JMX 监控,例如,IMap、Iqueue 等。
Hazelcast also supports JMX monitoring of the data structures embedded inside it, for example, IMap, Iqueue, and so on.
要启用 JMX 监控,我们首先需要启用基于 JVM 的 JMX 代理。可以通过将“-Dcom.sun.management.jmxremote”传递给 JVM 来完成此操作。若要使用不同的端口或使用身份验证,我们分别可以使用 -Dcom.sun.management.jmxremote.port、- Dcom.sun.management.jmxremote.authenticate。
To enable JMX monitoring, we first need to enable JVM based JMX agents. This can be done by passing "-Dcom.sun.management.jmxremote" to the JVM. For using different ports or use authentication, we can use -Dcom.sun.management.jmxremote.port, - Dcom.sun.management.jmxremote.authenticate, respectively.
除此之外,我们必须为 Hazelcast MBeans 启用 JMX。让我们通过 hazelcast-monitoring.xml 中的 XML 配置启用基于 JMX 的监控 −
Apart from this, we have to enable JMX for Hazelcast MBeans. Let us enable JMX based monitoring via XML configuration in hazelcast-monitoring.xml −
<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config
http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<instance-name>XML_Hazelcast_Instance</instance-name>
<properties>
<property name="hazelcast.jmx">true</property>
</properties>
</hazelcast>
让我们创建一个 Hazelcast 实例,该实例在 Server.java 文件中无限期运行,并添加一个映射 −
Let us create a Hazelcast instance which runs indefinitely in Server.java file and add a map −
class Server {
public static void main(String... args){
//initialize hazelcast server/instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
//create a simple map
Map<String, String> vehicleOwners = hazelcast.getMap("vehicleOwnerMap");
// add key-value to map
vehicleOwners.put("John", "Honda-9235");
// do not shutdown, let the server run
//hazelcast.shutdown();
}
}
现在我们执行以下命令启用 JMX −
Now we can execute the following command to enable JMX −
java '-Dcom.sun.management.jmxremote' '-Dhazelcast.config=others\hazelcastmonitoring.
xml' -cp .\target\demo-0.0.1-SNAPSHOT.jar com.example.demo.Server
现在,JMX 端口可以通过 JMX 客户端(如 jConsole、VisualVM 等)进行连接。
The JMX ports can now be connected by JMX clients like jConsole, VisualVM, etc.
以下是连接使用 jConsole 时获得的结果示例,并查看 VehicleMap 的属性。正如我们可以看到,映射的名称为 vehicleOwnerMap,映射的大小为 1。
Here is a snapshot of what we will get if we connect using jConsole and see the attributes for VehicleMap. As we can see, the name of the map as vehicleOwnerMap and the size of map being 1.

Hazelcast - Map Reduce & Aggregations
MapReduce 是一种计算模型,在您有大量数据且需要多台机器(即分布式环境)计算数据时,该模型对数据处理非常有用。它涉及将数据映射为键值对,然后进行“缩减”,即对这些键进行分组并在值上执行操作。
MapReduce is a computation model which is useful for data processing when you have lots of data and you need multiple machines, i.e., a distributed environment to calculate data. It involves 'map’ing of data into key-value pairs and then 'reducing', i.e., grouping these keys and performing operation on the value.
鉴于 Hazelcast 是在考虑分布式环境的情况下设计的,因此它自然而然地实现了 Map-Reduce 框架。
Given the fact that Hazelcast is designed keeping a distributed environment in mind, implementing Map-Reduce Frameworks comes naturally to it.
让我们用一个例子看看该怎么做。
Let’s see how to do it with an example.
例如,让我们假设我们有关于汽车(品牌和车号)及其车主的数据。
For example, let’s suppose we have data about a car (brand & car number) and the owner of that car.
Honda-9235, John
Hyundai-235, Alice
Honda-935, Bob
Mercedes-235, Janice
Honda-925, Catnis
Hyundai-1925, Jane
现在,我们必须找出每个品牌的汽车数量,即现代、本田等。
And now, we have to figure out the number of cars for each brand, i.e., Hyundai, Honda, etc.
Example
让我们尝试使用 MapReduce 找出 −
Let’s try to find that out using MapReduce −
package com.example.demo;
import java.lang.reflect.Array;
import java.util.ArrayList;
import java.util.Map;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.atomic.AtomicInteger;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.ICompletableFuture;
import com.hazelcast.core.IMap;
import com.hazelcast.mapreduce.Context;
import com.hazelcast.mapreduce.Job;
import com.hazelcast.mapreduce.JobTracker;
import com.hazelcast.mapreduce.KeyValueSource;
import com.hazelcast.mapreduce.Mapper;
import com.hazelcast.mapreduce.Reducer;
import com.hazelcast.mapreduce.ReducerFactory;
public class MapReduce {
public static void main(String[] args) throws ExecutionException,
InterruptedException {
try {
// create two Hazelcast instances
HazelcastInstance hzMember = Hazelcast.newHazelcastInstance();
Hazelcast.newHazelcastInstance();
IMap<String, String> vehicleOwnerMap=hzMember.getMap("vehicleOwnerMap");
vehicleOwnerMap.put("Honda-9235", "John");
vehicleOwnerMap.putc"Hyundai-235", "Alice");
vehicleOwnerMap.put("Honda-935", "Bob");
vehicleOwnerMap.put("Mercedes-235", "Janice");
vehicleOwnerMap.put("Honda-925", "Catnis");
vehicleOwnerMap.put("Hyundai-1925", "Jane");
KeyValueSource<String, String> kvs=KeyValueSource.fromMap(vehicleOwnerMap);
JobTracker tracker = hzMember.getJobTracker("vehicleBrandJob");
Job<String, String> job = tracker.newJob(kvs);
ICompletableFuture<Map<String, Integer>> myMapReduceFuture =
job.mapper(new BrandMapper())
.reducer(new BrandReducerFactory()).submit();
Map<String, Integer&g; result = myMapReduceFuture.get();
System.out.println("Final output: " + result);
} finally {
Hazelcast.shutdownAll();
}
}
private static class BrandMapper implements Mapper<String, String, String, Integer> {
@Override
public void map(String key, String value, Context<String, Integer>
context) {
context.emit(key.split("-", 0)[0], 1);
}
}
private static class BrandReducerFactory implements ReducerFactory<String, Integer, Integer> {
@Override
public Reducer<Integer, Integer> newReducer(String key) {
return new BrandReducer();
}
}
private static class BrandReducer extends Reducer<Integer, Integer> {
private AtomicInteger count = new AtomicInteger(0);
@Override
public void reduce(Integer value) {
count.addAndGet(value);
}
@Override
public Integer finalizeReduce() {
return count.get();
}
}
}
让我们尝试理解此代码 −
Let’s try to understand this code −
-
We create Hazelcast members. In the example, we have a single member, but there can well be multiple members.
-
We create a map using dummy data and create a Key-Value store out of it.
-
We create a Map-Reduce job and ask it to use the Key-Value store as the data.
-
We then submit the job to cluster and wait for completion.
-
The mapper creates a key, i.e., extracts brand information from the original key and sets the value to 1 and then emits that information as K-V to the reducer.
-
The reducer simply sums the value, grouping the data, based on key, i.e., brand name.
Hazelcast - Collection Listener
Hazelcast 在给定集合(例如队列、集合、列表等)更新时支持添加监听器。典型事件包括添加条目和删除条目。
Hazelcast supports addition of listeners when a given collection, for example, queue, set, list, etc. is updated. Typical events include entry added and entry removed.
让我们通过一个示例了解如何实现集合监听器。因此,假设我们要实现一个监听器,以跟踪集合中元素的数量。
Let’s see how to implement a set listener via an example. So, let’s say we want to implement a listener which tracks the number of elements in a set.
Example
因此,让我们首先实现生产者 −
So, let’s first implement the Producer −
public class SetTimedProducer{
public static void main(String... args) throws IOException,
InterruptedException {
//initialize hazelcast instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
Thread.sleep(5000);
// create a set
ISet<String> hzFruits = hazelcast.getSet("fruits");
hzFruits.add("Mango");
Thread.sleep(2000);
hzFruits.add("Apple");
Thread.sleep(2000);
hzFruits.add("Banana");
System.exit(0);
}
}
现在,让我们实现监听器 −
Now let’s implement the listener −
package com.example.demo;
import java.io.IOException;
import com.hazelcast.core.ISet;
import com.hazelcast.core.ItemEvent;
import com.hazelcast.core.ItemListener;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
public class SetListener{
public static void main(String... args) throws IOException, InterruptedException {
//initialize hazelcast instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
// create a set
ISet<String> hzFruits = hazelcast.getSet("fruits");
ItemListener<String> listener = new FruitListener<String>();
hzFruits.addItemListener(listener, true);
System.exit(0);
}
private static class FruitListener<String> implements ItemListener<String> {
private int count = 0;
@Override
public void itemAdded(ItemEvent<String> item) {
System.out.println("item added" + item);
count ++;
System.out.println("Total elements" + count);
}
@Override
public void itemRemoved(ItemEvent<String> item) {
count --;
}
}
}
我们将首先运行生产者 −
We will first run the producer −
java -cp .\target\demo-0.0.1-SNAPSHOT.jar com.example.demo.SetTimedProducer
然后,我们运行监听器并使其无限期运行 −
And then, we run the listeners and let it run indefinitely −
java -cp .\target\demo-0.0.1-SNAPSHOT.jar com.example.demo.SetListener
Output
监听器的 output 如下所示 −
The output from the Listener is as follows −
item added: ItemEvent{
event=ADDED, item=Mango, member=Member [localhost]:5701-c28a60b7-3259-44bf-8793-54063d244394 this}
Total elements: 1
item added: ItemEvent{
event=ADDED, item=Apple, member=Member [localhost]:5701-c28a60b7-3259-44bf-8793-54063d244394 this}
Total elements: 2
item added: ItemEvent{
event=ADDED, item=Banana, member=Member [localhost]:5701-c28a60b7-3259-44bf-8793-54063d244394 this}
Total elements: 3
hzFruits.addItemListener(listener, true) 的调用告诉 Hazelcast 提供成员信息。如果设置为 false,则我们只会被通知条目已添加/删除。这有助于避免对条目进行序列化和反序列化以使其可用于监听器。
The call with hzFruits.addItemListener(listener, true) tells Hazelcast to provide member information. If set to false, we will just be notified that an entry was added/removed. This helps in avoiding the need to serialize and deserialize the entry to make it accessible to the listener.
Hazelcast - Common Pitfalls & Performance Tips
Hazelcast Queue on single machine
Hazelcast 队列存储在单个成员上(以及不同机器上的备份)。这实际上意味着队列可以容纳一台机器可以容纳的任意多项。因此,队列容量不会通过添加更多成员而扩展。向队列中加载机器可以处理的数据量之外的数据会导致机器崩溃。
Hazelcast queues are stored on a single member (along with a backup on different machines). This effectively means the queue can hold as many items which can be accommodated on a single machine. So, the queue capacity does not scale by adding more members. Loading more data than what a machine can handle in a queue can cause the machine to crash.
Using Map’s set method instead of put
如果我们使用 IMap 的 put(key, newValue),Hazelcast 会返回旧值。这意味着,反序列化会花费额外的计算和时间。其中还包括从网络发送的更多数据。相反,如果我们对旧值不感兴趣,我们可以使用返回 void 的 set(key, value)。
If we use IMap’s put(key, newValue), Hazelcast returns the oldValue. This means, extra computation and time is spent in deserialization. This also includes more data sent from the network. Instead, if we are not interested in the oldValue, we can use set(key, value) which returns void.
让我们了解如何存储和注入对 Hazelcast 结构的引用。以下代码创建一个名为“stock”的地图,并在一个位置添加芒果,在另一个位置添加苹果。
Let’s see how to store and inject references to Hazelcast structures. The following code creates a map of the name "stock" and adds Mango at one place and Apple at another.
//initialize hazelcast instance
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
// create a map
IMap<String, String> hzStockTemp = hazelcast.getMap("stock");
hzStock.put("Mango", "4");
IMap<String, String> hzStockTemp2 = hazelcast.getMap("stock");
hzStock.put("Apple", "3");
然而,这里的问题在于我们使用了 getMap(“stock”) 两次。虽然在单个节点环境中此调用看起来无害,但它在集群环境中会造成缓慢。函数调用 getMap() 涉及到与集群的其他成员之间的网络往返。
However, the problem here is that we are using getMap("stock") twice. Although this call seems harmless in a single node environment, it creates slowness in a clustered environment. The function call getMap() involves network round trips to other members of the cluster.
因此,建议我们将对映射的引用存储在本地,并在操作映射时使用引用。例如 −
So, it is recommended that we store the reference to the map locally and use the referencing while operating on the map. For example −
// create a map
IMap<String, String> hzStock = hazelcast.getMap("stock");
hzStock.put("Mango", "4");
hzStock.put("Apple", "3");
Hazelcast uses serialized data for object comparison
正如我们在早期的示例中所看到的,非常重要的一点是,Hazelcast 在比较键时不使用反序列化对象。因此,它无法访问我们在 equals/hashCode 方法中编写的代码。根据 Hazelcast,如果两个 Java 对象的所有属性值相同,那么键是相等的。
As we have seen in the earlier examples, it is very critical to note that Hazelcast does not use deserialize objects while comparing keys. So, it does not have access to the code written in our equals/hashCode method. According to Hazelcast, keys are equal if the value to all the attributes of two Java objects is the same.
Use monitoring
在大型分布式系统中,监视起着非常重要的作用。使用 REST API 和 JMX 进行监视对于采取主动措施而不是采取被动措施非常重要。
In a large-scale distributed system, monitoring plays a very important role. Using REST API and JMX for monitoring is very important for taking proactive measures instead of being reactive.
Homogeneous cluster
Hazelcast 假设所有机器是相等的,即所有机器具有相同的资源。但是,如果我们的集群包含一台功能较弱的机器,例如内存较少、CPU 功能较弱等,那么如果计算发生在那台机器上,可能会造成缓慢。最糟糕的是,较弱的机器可能会耗尽资源,从而导致级联故障。因此,Hazelcast 成员必须具有相等的资源能力。
Hazelcast assumes all the machines are equal, i.e., all the machines have same resources. But if our cluster contains a less powerful machine, for example, less memory, lesser CPU power, etc., then it can create slowness if the computation happens on that machine. Worst, the weaker machine can run out of resources causing cascading failures. So, it is necessary that Hazelcast members have equal resource power.