Scripted and runtime fields

  • 脚本字段基于结果文档和现有的文档字段进行计算,并将其添加到返回的文档中。

  • 运行时字段基于存储的文档进行计算,可用于查询或返回在搜索结果中。

使用脚本字段可以动态计算字段,例如基于出生日期计算年龄。运行时字段可用于在查询中使用计算值,例如根据性别和最大年龄过滤人员。

Spring Data Elasticsearch 支持脚本字段和运行时字段。有关它们的详细信息,请参考 Elasticsearch 文档中的脚本 ([role="bare"][role="bare"]https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html) 和运行时字段 ([role="bare"][role="bare"]https://www.elastic.co/guide/en/elasticsearch/reference/8.9/runtime.html)。在 Spring Data Elasticsearch 的上下文中,可以使用

Spring Data Elasticsearch supports scripted fields and runtime fields. Please refer to the Elasticsearch documentation about scripting ([role="bare"]https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html) and runtime fields ([role="bare"]https://www.elastic.co/guide/en/elasticsearch/reference/8.9/runtime.html) for detailed information about this. In the context of Spring Data Elasticsearch you can use

  • scripted fields that are used to return fields that are calculated on the result documents and added to the returned document.

  • runtime fields that are calculated on the stored documents and can be used in a query and/or be returned in the search result.

以下代码片段将展示你可以执行的操作(这些代码显示命令式代码,但 reactive 实现类似)。

The following code snippets will show what you can do (these show imperative code, but the reactive implementation works similar).

The person entity

这些示例中使用的实体是 Person 实体。此实体具有 birthDateage 属性。出生日期是固定的,而年龄取决于发出查询的时间,需要动态计算。

The enity that is used in these examples is a Person entity. This entity has a birthDate and an age property. Whereas the birthdate is fix, the age depends on the time when a query is issued and needs to be calculated dynamically.

import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.DateFormat;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.ScriptedField;
import org.springframework.lang.Nullable;

import java.time.LocalDate;
import java.time.format.DateTimeFormatter;

import static org.springframework.data.elasticsearch.annotations.FieldType.*;

import java.lang.Integer;

@Document(indexName = "persons")
public record Person(
        @Id
        @Nullable
        String id,
        @Field(type = Text)
        String lastName,
        @Field(type = Text)
        String firstName,
        @Field(type = Keyword)
        String gender,
        @Field(type = Date, format = DateFormat.basic_date)
        LocalDate birthDate,
        @Nullable
        @ScriptedField Integer age                   1
) {
    public Person(String id,String lastName, String firstName, String gender, String birthDate) {
        this(id,                                     2
            lastName,
            firstName,
            LocalDate.parse(birthDate, DateTimeFormatter.ISO_LOCAL_DATE),
            gender,
            null);
    }
}
1 the age property will be calculated and filled in search results.
2 a convenience constructor to set up the test data.

请注意,age 属性加上了 @ScriptedField 注释。这会禁止在索引映射中写入对应的条目,并将该属性标记为放置搜索响应中计算字段的目标。

Note that the age property is annotated with @ScriptedField. This inhibits the writing of a corresponding entry in the index mapping and marks the property as a target to put a calculated field from a search response.

The repository interface

本例中使用的存储库:

The repository used in this example:

public interface PersonRepository extends ElasticsearchRepository<Person, String> {

    SearchHits<Person> findAllBy(ScriptedField scriptedField);

    SearchHits<Person> findByGenderAndAgeLessThanEqual(String gender, Integer age, RuntimeField runtimeField);
}

The service class

服务类拥有一个已注入存储库和 ElasticsearchOperations 实例,用于展示填充和使用 age 属性的几种方式。我们将代码分割成不同的段落,以便说明

The service class has a repository injected and an ElasticsearchOperations instance to show several ways of populating and using the age property. We show the code split up in different pieces to put the explanations in

import org.springframework.data.elasticsearch.core.ElasticsearchOperations;
import org.springframework.data.elasticsearch.core.SearchHits;
import org.springframework.data.elasticsearch.core.query.Criteria;
import org.springframework.data.elasticsearch.core.query.CriteriaQuery;
import org.springframework.data.elasticsearch.core.query.FetchSourceFilter;
import org.springframework.data.elasticsearch.core.query.RuntimeField;
import org.springframework.data.elasticsearch.core.query.ScriptData;
import org.springframework.data.elasticsearch.core.query.ScriptType;
import org.springframework.data.elasticsearch.core.query.ScriptedField;
import org.springframework.data.elasticsearch.core.query.StringQuery;
import org.springframework.stereotype.Service;

import java.util.List;

@Service
public class PersonService {
    private final ElasticsearchOperations operations;
    private final PersonRepository repository;

    public PersonService(ElasticsearchOperations operations, SaRPersonRepository repository) {
        this.operations = operations;
        this.repository = repository;
    }

    public void save() { 1
        List<Person> persons = List.of(
                new Person("1", "Smith", "Mary", "f", "1987-05-03"),
                new Person("2", "Smith", "Joshua", "m", "1982-11-17"),
                new Person("3", "Smith", "Joanna", "f", "2018-03-27"),
                new Person("4", "Smith", "Alex", "m", "2020-08-01"),
                new Person("5", "McNeill", "Fiona", "f", "1989-04-07"),
                new Person("6", "McNeill", "Michael", "m", "1984-10-20"),
                new Person("7", "McNeill", "Geraldine", "f", "2020-03-02"),
                new Person("8", "McNeill", "Patrick", "m", "2022-07-04"));

        repository.saveAll(persons);
    }
1 a utility method to store some data in Elasticsearch.

Scripted fields

下一部分展示了如何使用脚本字段计算并返回人员的年龄。脚本字段只能向返回的数据添加内容,年龄无法在查询中使用(为此请参见运行时字段)。

The next piece shows how to use a scripted field to calculate and return the age of the persons. Scripted fields can only add something to the returned data, the age cannot be used in the query (see runtime fields for that).

    public SearchHits<Person> findAllWithAge() {

        var scriptedField = ScriptedField.of("age",                               1
                ScriptData.of(b -> b
                        .withType(ScriptType.INLINE)
                        .withScript("""
                                Instant currentDate = Instant.ofEpochMilli(new Date().getTime());
                                Instant startDate = doc['birth-date'].value.toInstant();
                                return (ChronoUnit.DAYS.between(startDate, currentDate) / 365);
                                """)));

        // version 1: use a direct query
        var query = new StringQuery("""
                { "match_all": {} }
                """);
        query.addScriptedField(scriptedField);                                    2
        query.addSourceFilter(FetchSourceFilter.of(b -> b.withIncludes("*")));    3

        var result1 = operations.search(query, Person.class);                     4

        // version 2: use the repository
        var result2 = repository.findAllBy(scriptedField);                        5

        return result1;
    }
1 define the ScriptedField that calculates the age of a person.
2 when using a Query, add the scripted field to the query.
3 when adding a scripted field to a Query, an additional source filter is needed to also retrieve the normal fields from the document source.
4 get the data where the Person entities now have the values set in their age property.
5 when using the repository, all that needs to be done is adding the scripted field as method parameter.

Runtime fields

使用运行时字段时,计算的值可以在查询本身中使用。在以下代码中,用于根据给定的性别和人员的最大年龄运行查询:

When using runtime fields, the calculated value can be used in the query itself. In the following code this is used to run a query for a given gender and maximum age of persons:

    public SearchHits<Person> findWithGenderAndMaxAge(String gender, Integer maxAge) {

        var runtimeField = new RuntimeField("age", "long", """                    1
                                Instant currentDate = Instant.ofEpochMilli(new Date().getTime());
                                Instant startDate = doc['birth-date'].value.toInstant();
                                emit (ChronoUnit.DAYS.between(startDate, currentDate) / 365);
                """);

        // variant 1 : use a direct query
        var query = CriteriaQuery.builder(Criteria
                        .where("gender").is(gender)
                        .and("age").lessThanEqual(maxAge))
                .withRuntimeFields(List.of(runtimeField))                         2
                .withFields("age")                                                3
                .withSourceFilter(FetchSourceFilter.of(b -> b.withIncludes("*"))) 4
                .build();

        var result1 = operations.search(query, Person.class);                     5

        // variant 2: use the repository                                          6
        var result2 = repository.findByGenderAndAgeLessThanEqual(gender, maxAge, runtimeField);

        return result1;
    }
}
1 define the runtime field that calculates the age of a person. // see [role="bare"]https://asciidoctor.org/docs/user-manual/#builtin-attributes for builtin attributes.
2 when using Query, add the runtime field.
3 when adding a scripted field to a Query, an additional field parameter is needed to have the calculated value returned.
4 when adding a scripted field to a Query, an additional source filter is needed to also retrieve the normal fields from the document source.
5 get the data filtered with the query and where the returned entites have the age property set.
6 when using the repository, all that needs to be done is adding the runtime field as method parameter.

除了在查询中定义运行时字段外,还可以通过将 @Mapping 注释的 runtimeFieldsPath 属性设置为指向包含运行时字段定义的 JSON 文件来在索引中定义它们。

In addition to define a runtime fields on a query, they can also be defined in the index by setting the runtimeFieldsPath property of the @Mapping annotation to point to a JSON file that contains the runtime field definitions.