Connecting to an Elasticsearch cluster

Elasticsearch 是一个众所周知的全文搜索引擎和 NoSQL 数据存储。

Elasticsearch is a well known full text search engine and NoSQL datastore.

在本指南中,我们将了解如何让你的 REST 服务与 Elasticsearch 集群交互。

In this guide, we will see how you can get your REST services to interact with an Elasticsearch cluster.

Quarkus 提供了两种访问 Elasticsearch 的方法:

Quarkus provides two ways of accessing Elasticsearch:

  • The lower level REST Client

  • The Elasticsearch Java client

对于“高级 REST 客户端”,曾经存在过一个第三个 Quarkus 扩展,但它已被移除,因为 Elastic 已将此客户端弃用,并且它有一些许可问题。

A third Quarkus extension for the "high level REST Client" used to exist, but was removed as this client has been deprecated by Elastic and has some licensing issues.

Prerequisites

include::{includes}/prerequisites.adoc[]* 已安装 Elasticsearch 或已安装 Docker

Unresolved directive in elasticsearch.adoc - include::{includes}/prerequisites.adoc[] * Elasticsearch installed or Docker installed

Architecture

本指南中的应用程序非常简单:用户可以使用窗体在列表中添加元素,然后更新列表。

The application built in this guide is quite simple: the user can add elements in a list using a form and the list is updated.

浏览器和服务器之间所有信息都采用 JSON 格式。

All the information between the browser and the server is formatted as JSON.

元素存储在 Elasticsearch 中。

The elements are stored in Elasticsearch.

Creating the Maven project

首先,我们需要一个新项目。使用以下命令创建一个新项目:

First, we need a new project. Create a new project with the following command:

Unresolved directive in elasticsearch.adoc - include::{includes}/devtools/create-app.adoc[]

此命令生成一个 Maven 结构,该结构会导入 Quarkus REST(以前的 RESTEasy Reactive)、Jackson 以及 Elasticsearch 低级 REST 客户端扩展。

This command generates a Maven structure importing the Quarkus REST (formerly RESTEasy Reactive), Jackson, and Elasticsearch low level REST client extensions.

Elasticsearch 低级 REST 客户端随 `quarkus-elasticsearch-rest-client`一起提供,该扩展已添加到您的构建文件中。

The Elasticsearch low level REST client comes with the quarkus-elasticsearch-rest-client extension that has been added to your build file.

如果您想改用 Elasticsearch Java 客户端,请使用 `quarkus-elasticsearch-java-client`扩展替换 `quarkus-elasticsearch-rest-client`扩展。

If you want to use the Elasticsearch Java client instead, replace the quarkus-elasticsearch-rest-client extension by the quarkus-elasticsearch-java-client extension.

我们在此处使用 `rest-jackson`扩展,未使用 JSON-B 变体,因为我们将使用 Vert.x `JsonObject`助手,利用此助手,可以将我们的对象序列化/反序列化为 Elasticsearch 的对象或从 Elasticsearch 的对象序列化/反序列化我们的对象,并且该助手实际上使用的是 Jackson。

We use the rest-jackson extension here and not the JSON-B variant because we will use the Vert.x JsonObject helper to serialize/deserialize our objects to/from Elasticsearch and it uses Jackson under the hood.

如需将扩展添加到现有项目,请按照以下说明进行操作。

To add the extensions to an existing project, follow the instructions below.

对于 Elasticsearch 低级 REST 客户端,请将以下依赖关系添加到您的构建文件:

For the Elasticsearch low level REST client, add the following dependency to your build file:

pom.xml
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-elasticsearch-rest-client</artifactId>
</dependency>
build.gradle
implementation("io.quarkus:quarkus-elasticsearch-rest-client")

对于 Elasticsearch Java 客户端,请将以下依赖关系添加到您的构建文件:

For the Elasticsearch Java client, add the following dependency to your build file:

pom.xml
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-elasticsearch-java-client</artifactId>
</dependency>
build.gradle
implementation("io.quarkus:quarkus-elasticsearch-java-client")

Creating your first JSON REST service

在此示例中,我们将创建一个应用程序来管理水果列表。

In this example, we will create an application to manage a list of fruits.

首先,我们按如下方式创建 `Fruit`bean:

First, let’s create the Fruit bean as follows:

package org.acme.elasticsearch;

public class Fruit {
    public String id;
    public String name;
    public String color;
}

没什么新奇之处。需要特别注意的是,JSON 序列化层要求有缺省构造函数。

Nothing fancy. One important thing to note is that having a default constructor is required by the JSON serialization layer.

现在创建 org.acme.elasticsearch.FruitService,它将成为我们应用程序的业务层,并将把水果存储到 Elasticsearch 实例或从 Elasticsearch 实例加载水果。在此,我们使用低级 REST 客户端,如果您想改用 Java API 客户端,请改而按照 Using the Elasticsearch Java Client段中的说明进行操作。

Now create a org.acme.elasticsearch.FruitService that will be the business layer of our application and will store/load the fruits from the Elasticsearch instance. Here we use the low level REST client, if you want to use the Java API client instead, follow the instructions in the using-the-elasticsearch-java-client paragraph instead.

package org.acme.elasticsearch;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;

import org.apache.http.util.EntityUtils;
import org.elasticsearch.client.Request;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;

import io.vertx.core.json.JsonArray;
import io.vertx.core.json.JsonObject;

@ApplicationScoped
public class FruitService {
    @Inject
    RestClient restClient; (1)

    public void index(Fruit fruit) throws IOException {
        Request request = new Request(
                "PUT",
                "/fruits/_doc/" + fruit.id); (2)
        request.setJsonEntity(JsonObject.mapFrom(fruit).toString()); (3)
        restClient.performRequest(request); (4)
    }

    public Fruit get(String id) throws IOException {
        Request request = new Request(
                "GET",
                "/fruits/_doc/" + id);
        Response response = restClient.performRequest(request);
        String responseBody = EntityUtils.toString(response.getEntity());
        JsonObject json = new JsonObject(responseBody); (5)
        return json.getJsonObject("_source").mapTo(Fruit.class);
    }

    public List<Fruit> searchByColor(String color) throws IOException {
        return search("color", color);
    }

    public List<Fruit> searchByName(String name) throws IOException {
        return search("name", name);
    }

    private List<Fruit> search(String term, String match) throws IOException {
        Request request = new Request(
                "GET",
                "/fruits/_search");
        //construct a JSON query like {"query": {"match": {"<term>": "<match"}}
        JsonObject termJson = new JsonObject().put(term, match);
        JsonObject matchJson = new JsonObject().put("match", termJson);
        JsonObject queryJson = new JsonObject().put("query", matchJson);
        request.setJsonEntity(queryJson.encode());
        Response response = restClient.performRequest(request);
        String responseBody = EntityUtils.toString(response.getEntity());

        JsonObject json = new JsonObject(responseBody);
        JsonArray hits = json.getJsonObject("hits").getJsonArray("hits");
        List<Fruit> results = new ArrayList<>(hits.size());
        for (int i = 0; i < hits.size(); i++) {
            JsonObject hit = hits.getJsonObject(i);
            Fruit fruit = hit.getJsonObject("_source").mapTo(Fruit.class);
            results.add(fruit);
        }
        return results;
    }
}
1 We inject an Elasticsearch low level RestClient into our service.
2 We create an Elasticsearch request.
3 We use Vert.x JsonObject to serialize the object before sending it to Elasticsearch, you can use whatever you want to serialize your objects to JSON.
4 We send the request (indexing request here) to Elasticsearch.
5 In order to deserialize the object from Elasticsearch, we again use Vert.x JsonObject.

现在,按以下方式创建类 org.acme.elasticsearch.FruitResource

Now, create the org.acme.elasticsearch.FruitResource class as follows:

package org.acme.elasticsearch;

import jakarta.inject.Inject;
import jakarta.ws.rs.BadRequestException;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.core.Response;
import java.io.IOException;
import java.net.URI;
import java.util.List;
import java.util.UUID;

import org.jboss.resteasy.reactive.RestQuery;

@Path("/fruits")
public class FruitResource {

    @Inject
    FruitService fruitService;

    @POST
    public Response index(Fruit fruit) throws IOException {
        if (fruit.id == null) {
            fruit.id = UUID.randomUUID().toString();
        }
        fruitService.index(fruit);
        return Response.created(URI.create("/fruits/" + fruit.id)).build();
    }

    @GET
    @Path("/{id}")
    public Fruit get(String id) throws IOException {
        return fruitService.get(id);
    }

    @GET
    @Path("/search")
    public List<Fruit> search(@RestQuery String name, @RestQuery String color) throws IOException {
        if (name != null) {
            return fruitService.searchByName(name);
        } else if (color != null) {
            return fruitService.searchByColor(color);
        } else {
            throw new BadRequestException("Should provide name or color query parameter");
        }
    }
}

实现方式非常简单,你只需使用 Jakarta REST 注释定义端点,并使用 FruitService 来列出/添加新水果。

The implementation is pretty straightforward and you just need to define your endpoints using the Jakarta REST annotations and use the FruitService to list/add new fruits.

Configuring Elasticsearch

要配置的主要属性是用于与 Elasticsearch 集群连接的 URL。

The main property to configure is the URL to connect to the Elasticsearch cluster.

对于一个典型的集群 Elasticsearch 服务,示例配置应如下所示:

For a typical clustered Elasticsearch service, a sample configuration would look like the following:

# configure the Elasticsearch client for a cluster of two nodes
quarkus.elasticsearch.hosts = elasticsearch1:9200,elasticsearch2:9200

在我们的例子中,我们使用在 localhost: 上运行的一个实例:

In our case, we are using a single instance running on localhost:

# configure the Elasticsearch client for a single instance on localhost
quarkus.elasticsearch.hosts = localhost:9200

如果你需要更高级的配置,你可以在此指南的末尾找到支持的配置属性的完整列表。

If you need a more advanced configuration, you can find the comprehensive list of supported configuration properties at the end of this guide.

Dev Services

Quarkus 支持一项称为 Dev Services 的功能,它允许你在没有配置的情况下启动各种容器。在 Elasticsearch 的情况下,这种支持扩展到了默认的 Elasticsearch 连接。这意味着如果你没有配置 quarkus.elasticsearch.hosts,Quarkus 将在运行测试或开发模式时自动启动一个 Elasticsearch 容器,并自动配置连接。

Quarkus supports a feature called Dev Services that allows you to start various containers without any config. In the case of Elasticsearch, this support extends to the default Elasticsearch connection. What that means practically is that, if you have not configured quarkus.elasticsearch.hosts, Quarkus will automatically start an Elasticsearch container when running tests or dev mode, and automatically configure the connection.

在运行应用程序的生产版本时,需要像往常一样配置 Elasticsearch 连接,因此如果你想在 application.properties 中包含生产数据库配置,并继续使用 Dev Services,我们建议你使用 %prod. 配置文件来定义 Elasticsearch 设置。

When running the production version of the application, the Elasticsearch connection needs to be configured as usual, so if you want to include a production database config in your application.properties and continue to use Dev Services we recommend that you use the %prod. profile to define your Elasticsearch settings.

更多信息,你可以阅读 Dev Services for Elasticsearch guide

For more information you can read the Dev Services for Elasticsearch guide.

Programmatically Configuring Elasticsearch

在参数化配置之上,你还可以通过实现 RestClientBuilder.HttpClientConfigCallback 并使用 ElasticsearchClientConfig 对其进行注释,以编程方式将其他配置应用于客户端。你可以提供多个实现,并且每个实现提供的配置将以随机排序级联方式应用。

On top of the parametric configuration, you can also programmatically apply additional configuration to the client by implementing a RestClientBuilder.HttpClientConfigCallback and annotating it with ElasticsearchClientConfig. You may provide multiple implementations and configuration provided by each implementation will be applied in a randomly ordered cascading manner.

例如,在访问在 HTTP 层上针对 TLS 设置的 Elasticsearch 集群时,客户端需要信任 Elasticsearch 使用的证书。以下是设置客户端信任由 Elasticsearch 使用的证书颁发机构 (CA) 的示例,当时该 CA 证书在 PKCS#12 密钥库中。

For example, when accessing an Elasticsearch cluster that is set up for TLS on the HTTP layer, the client needs to trust the certificate that Elasticsearch is using. The following is an example of setting up the client to trust the CA that has signed the certificate that Elasticsearch is using, when that CA certificate is available in a PKCS#12 keystore.

import io.quarkus.elasticsearch.restclient.lowlevel.ElasticsearchClientConfig;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.ssl.SSLContexts;
import org.elasticsearch.client.RestClientBuilder;

import jakarta.enterprise.context.Dependent;
import javax.net.ssl.SSLContext;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.KeyStore;

@ElasticsearchClientConfig
public class SSLContextConfigurator implements RestClientBuilder.HttpClientConfigCallback {
    @Override
    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
        try {
            String keyStorePass = "password-for-keystore";
            Path trustStorePath = Paths.get("/path/to/truststore.p12");
            KeyStore truststore = KeyStore.getInstance("pkcs12");
            try (InputStream is = Files.newInputStream(trustStorePath)) {
                truststore.load(is, keyStorePass.toCharArray());
            }
            SSLContextBuilder sslBuilder = SSLContexts.custom()
                    .loadTrustMaterial(truststore, null);
            SSLContext sslContext = sslBuilder.build();
            httpClientBuilder.setSSLContext(sslContext);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }

        return httpClientBuilder;
    }
}

有关此特定示例的更多详细信息,请参见 Elasticsearch documentation

See Elasticsearch documentation for more details on this particular example.

默认情况下,标有 @ElasticsearchClientConfig 的类将成为应用程序作用范围的 CDI bean。如果你更喜欢不同的作用域,可以在类级别覆盖该作用域。

Classes marked with @ElasticsearchClientConfig are made application scoped CDI beans by default. You can override the scope at the class level if you prefer a different scope.

Running an Elasticsearch cluster

由于默认情况下,Elasticsearch 客户端配置为访问端口 9200(默认 Elasticsearch 端口)上的本地 Elasticsearch 集群,如果你在这个端口上有一个本地运行实例,那么在测试之前不需要执行任何其他操作!

As by default, the Elasticsearch client is configured to access a local Elasticsearch cluster on port 9200 (the default Elasticsearch port), if you have a local running instance on this port, there is nothing more to do before being able to test it!

如果你想使用 Docker 运行 Elasticsearch 实例,可以使用以下命令启动一个:

If you want to use Docker to run an Elasticsearch instance, you can use the following command to launch one:

docker run --name elasticsearch  -e "discovery.type=single-node" -e "ES_JAVA_OPTS=-Xms512m -Xmx512m"\
       -e "cluster.routing.allocation.disk.threshold_enabled=false" -e "xpack.security.enabled=false"\
       --rm -p 9200:9200 {elasticsearch-image}

Running the application

我们在开发模式下启动应用程序:

Let’s start our application in dev mode:

Unresolved directive in elasticsearch.adoc - include::{includes}/devtools/dev.adoc[]

你可以通过以下 curl 命令向列表中添加新水果:

You can add new fruits to the list via the following curl command:

curl localhost:8080/fruits -d '{"name": "bananas", "color": "yellow"}' -H "Content-Type: application/json"

并通过以下 curl 命令按名称或颜色搜索水果:

And search for fruits by name or color via the following curl command:

curl localhost:8080/fruits/search?color=yellow

Using the Elasticsearch Java Client

这里有一个 FruitService 版本,其中使用 Elasticsearch Java 客户端,而不是底层版本:

Here is a version of the FruitService using the Elasticsearch Java Client instead of the low level one:

import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch._types.FieldValue;
import co.elastic.clients.elasticsearch._types.query_dsl.QueryBuilders;
import co.elastic.clients.elasticsearch.core.*;
import co.elastic.clients.elasticsearch.core.search.HitsMetadata;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import org.acme.elasticsearch.Fruit;

import java.io.IOException;
import java.util.List;
import java.util.stream.Collectors;

@ApplicationScoped
public class FruitService {
    @Inject
    ElasticsearchClient client; (1)

    public void index(Fruit fruit) throws IOException {
        IndexRequest<Fruit> request = IndexRequest.of(  (2)
            b -> b.index("fruits")
                .id(fruit.id)
                .document(fruit)); (3)
        client.index(request);  (4)
    }

    public Fruit get(String id) throws IOException {
        GetRequest getRequest = GetRequest.of(
            b -> b.index("fruits")
                .id(id));
        GetResponse<Fruit> getResponse = client.get(getRequest, Fruit.class);
        if (getResponse.found()) {
            return getResponse.source();
        }
        return null;
    }

    public List<Fruit> searchByColor(String color) throws IOException {
        return search("color", color);
    }

    public List<Fruit> searchByName(String name) throws IOException {
        return search("name", name);
    }

    private List<Fruit> search(String term, String match) throws IOException {
        SearchRequest searchRequest = SearchRequest.of(
            b -> b.index("fruits")
                .query(QueryBuilders.match().field(term).query(FieldValue.of(match)).build()._toQuery()));

        SearchResponse<Fruit> searchResponse = client.search(searchRequest, Fruit.class);
        HitsMetadata<Fruit> hits = searchResponse.hits();
        return hits.hits().stream().map(hit -> hit.source()).collect(Collectors.toList());
    }
}
1 We inject an ElasticsearchClient inside the service.
2 We create an Elasticsearch index request using a builder.
3 We directly pass the object to the request as the Java API client has a serialization layer.
4 We send the request to Elasticsearch.

Hibernate Search Elasticsearch

Quarkus 通过 `quarkus-hibernate-search-orm-elasticsearch`扩展支持使用 Elasticsearch 的 Hibernate 搜索。

Quarkus supports Hibernate Search with Elasticsearch via the quarkus-hibernate-search-orm-elasticsearch extension.

Hibernate Search Elasticsearch 允许将您的 Jakarta Persistence 实体同步到 Elasticsearch 集群,并提供通过 Hibernate Search API 查询 Elasticsearch 集群的方法。

Hibernate Search Elasticsearch allows to synchronize your Jakarta Persistence entities to an Elasticsearch cluster and offers a way to query your Elasticsearch cluster using the Hibernate Search API.

如果您对此感兴趣,请查阅 Hibernate Search with Elasticsearch guide

If you are interested in it, please consult the Hibernate Search with Elasticsearch guide.

Cluster Health Check

如果您使用 `quarkus-smallrye-health`扩展,这两个扩展将自动添加就绪性健康检查以验证集群的运行状况。

If you are using the quarkus-smallrye-health extension, both extensions will automatically add a readiness health check to validate the health of the cluster.

因此,当您访问 `/q/health/ready`应用程序的端点时,您将获得有关集群状态的信息。它使用集群运行状况端点,如果集群的状态是 red,或者集群不可用,则检查将关闭。

So when you access the /q/health/ready endpoint of your application, you will have information about the cluster status. It uses the cluster health endpoint, the check will be down if the status of the cluster is red, or the cluster is not available.

可以通过在 application.properties 中将 `quarkus.elasticsearch.health.enabled`属性设置为 `false`来禁用此行为。

This behavior can be disabled by setting the quarkus.elasticsearch.health.enabled property to false in your application.properties.

Building a native executable

您可以在本机可执行文件中使用这两个客户端。

You can use both clients in a native executable.

您可以使用以下常用命令构建一个本机可执行文件:

You can build a native executable with the usual command:

Unresolved directive in elasticsearch.adoc - include::{includes}/devtools/build-native.adoc[]

只需要执行 ./target/elasticsearch-low-level-client-quickstart-1.0.0-SNAPSHOT-runner 即可运行它。

Running it is as simple as executing ./target/elasticsearch-low-level-client-quickstart-1.0.0-SNAPSHOT-runner.

然后您可以将浏览器指向 http://localhost:8080/fruits.html,并使用您的应用程序。

You can then point your browser to http://localhost:8080/fruits.html and use your application.

Conclusion

Quarkus 使得从低级 REST 客户端或 Elasticsearch Java 客户端访问 Elasticsearch 集群变得轻而易举,因为它提供了简单的配置、CDI 集成以及对它的原生支持。

Accessing an Elasticsearch cluster from the low level REST client or the Elasticsearch Java client is easy with Quarkus as it provides easy configuration, CDI integration and native support for it.

Configuration Reference

Unresolved directive in elasticsearch.adoc - include::{generated-dir}/config/quarkus-elasticsearch-rest-client.adoc[]