Spring Data Commons 的分页机制提供对数据进行高效分页和滚动的能力。通过使用 Pageable
接口和 ScrollingPage
接口,开发人员可以获取分区数据、查询总记录数,并在数据集合中进行滚动,从而实现可扩展和高效的数据访问。这种机制对于处理大型数据集和按需提供数据特别有用。
Scrolling
滚动是迭代处理较大结果集区块的一种更细粒度的处理方式。滚动包含稳定的排序、滚动类型(基于偏移或基于键集的滚动)以及结果限制。你可以使用属性名称定义简单的排序表达式,并使用 xref:repositories/query-methods-details.adoc#repositories.limit-query-result[Top
或 First
关键字通过查询派生定义静态结果限制。你可以联接表达式以将多个条件收集到一个表达式中。
Scrolling is a more fine-grained approach to iterate through larger results set chunks.
Scrolling consists of a stable sort, a scroll type (Offset- or Keyset-based scrolling) and result limiting.
You can define simple sorting expressions by using property names and define static result limiting using the Top
or First
keyword through query derivation.
You can concatenate expressions to collect multiple criteria into one expression.
滚动查询返回一个 Window<T>
,它允许获取滚动位置以继续获取下一个 Window<T>
直到您的应用程序使用整个查询结果。类似于通过获取下一批结果使用 Java Iterator<List<…>>
消费查询结果,查询结果滚动允许您通过 Window.positionAt(…)
访问 ScrollPosition
。
Scroll queries return a Window<T>
that allows obtaining the scroll position to resume to obtain the next Window<T>
until your application has consumed the entire query result.
Similar to consuming a Java Iterator<List<…>>
by obtaining the next batch of results, query result scrolling lets you access the a ScrollPosition
through Window.positionAt(…)
.
Window<User> users = repository.findFirst10ByLastnameOrderByFirstname("Doe", ScrollPosition.offset());
do {
for (User u : users) {
// consume the user
}
// obtain the next Scroll
users = repository.findFirst10ByLastnameOrderByFirstname("Doe", users.positionAt(users.size() - 1));
} while (!users.isEmpty() && users.hasNext());
WindowIterator
提供了一个实用工具,可通过消除检查是否存在下一个 Window
和应用 ScrollPosition
的需求来简化跨 Window
的滚动。
WindowIterator
provides a utility to simplify scrolling across Window`s by removing the need to check for the presence of a next `Window
and applying the ScrollPosition
.
WindowIterator<User> users = WindowIterator.of(position -> repository.findFirst10ByLastnameOrderByFirstname("Doe", position))
.startingAt(OffsetScrollPosition.initial());
while (users.hasNext()) {
User u = users.next();
// consume the user
}
Scrolling using Offset
偏移量滚动类似于分页,它使用偏移量计数器跳过一定数目的结果,并让数据源仅从给定偏移量开始返回结果。此简单机制可避免向客户端应用程序发送较大的结果。但是,在服务器返回结果之前,大多数数据库都需要实现完整的查询结果。
Offset scrolling uses similar to pagination, an Offset counter to skip a number of results and let the data source only return results beginning at the given Offset. This simple mechanism avoids large results being sent to the client application. However, most databases require materializing the full query result before your server can return the results.
OffsetScrollPosition
with Repository Query Methodsinterface UserRepository extends Repository<User, Long> {
Window<User> findFirst10ByLastnameOrderByFirstname(String lastname, OffsetScrollPosition position);
}
WindowIterator<User> users = WindowIterator.of(position -> repository.findFirst10ByLastnameOrderByFirstname("Doe", position))
.startingAt(OffsetScrollPosition.initial()); 1
1 | Start from the initial offset at position 0 . |
Scrolling using Keyset-Filtering
基于偏移量的方法需要大多数数据库在服务器返回结果之前实现全部结果的物化。因此,当客户端只看到请求结果的一部分时,服务器需要构建全部结果,这会导致额外的负载。
Offset-based requires most databases require materializing the entire result before your server can return the results. So while the client only sees the portion of the requested results, your server needs to build the full result, which causes additional load.
键集过滤方法通过利用数据库的内置功能来检索结果子集,旨在降低单个查询的计算和 I/O 需求。此方法维护一组键用于通过查询中传入键来恢复滚动,有效修正筛选条件。
Keyset-Filtering approaches result subset retrieval by leveraging built-in capabilities of your database aiming to reduce the computation and I/O requirements for individual queries. This approach maintains a set of keys to resume scrolling by passing keys into the query, effectively amending your filter criteria.
键集过滤的核心思想是使用稳定的排序顺序开始检索结果。一旦想要滚动到下一个块,就可以获取用于重建排序结果中位置的 ScrollPosition
。ScrollPosition
捕获当前 Window
中最后一个实体的键集。要运行查询,重建会重写条件子句以包括所有排序字段和主键,以便数据库可以利用潜在的索引来运行查询。数据库只需要根据给定的键集位置构建一个更小的结果,而无需完全实现一个较大的结果,然后跳过一些结果直到达到特定的偏移量。
The core idea of Keyset-Filtering is to start retrieving results using a stable sorting order.
Once you want to scroll to the next chunk, you obtain a ScrollPosition
that is used to reconstruct the position within the sorted result.
The ScrollPosition
captures the keyset of the last entity within the current Window
.
To run the query, reconstruction rewrites the criteria clause to include all sort fields and the primary key so that the database can leverage potential indexes to run the query.
The database needs only constructing a much smaller result from the given keyset position without the need to fully materialize a large result and then skipping results until reaching a particular offset.
键集过滤需要键集属性(用于排序的属性)为非空。此限制适用于由于比较运算符对存储特定的 null
值处理以及由于需要针对索引来源运行查询。针对空属性的键集过滤会导致意外的结果。
Keyset-Filtering requires the keyset properties (those used for sorting) to be non-nullable.
This limitation applies due to the store specific null
value handling of comparison operators as well as the need to run queries against an indexed source.
Keyset-Filtering on nullable properties will lead to unexpected results.
KeysetScrollPosition
with Repository Query Methodsinterface UserRepository extends Repository<User, Long> {
Window<User> findFirst10ByLastnameOrderByFirstname(String lastname, KeysetScrollPosition position);
}
WindowIterator<User> users = WindowIterator.of(position -> repository.findFirst10ByLastnameOrderByFirstname("Doe", position))
.startingAt(ScrollPosition.keyset()); 1
1 | Start at the very beginning and do not apply additional filtering. |
当数据库包含与排序字段相匹配的索引时,键集过滤效果最佳,因此静态排序效果很好。应用键集过滤的滚动查询需要将排序顺序中使用的属性通过查询返回,而这些属性必须映射在返回的实体中。
Keyset-Filtering works best when your database contains an index that matches the sort fields, hence a static sort works well. Scroll queries applying Keyset-Filtering require to the properties used in the sort order to be returned by the query, and these must be mapped in the returned entity.
可以使用接口和 DTO 投影,但是确保包括已对其进行排序的所有属性,以避免键集提取失败。
You can use interface and DTO projections, however make sure to include all properties that you’ve sorted by to avoid keyset extraction failures.
在指定 Sort
顺序时,包含与查询相关的排序属性就足够了;如果您不想确保唯一查询结果,就不需要这样做。键集查询机制会通过包括主键(或复合主键的任何剩余部分)来修正排序顺序,确保每个查询结果都是唯一的。
When specifying your Sort
order, it is sufficient to include sort properties relevant to your query;
You do not need to ensure unique query results if you do not want to.
The keyset query mechanism amends your sort order by including the primary key (or any remainder of composite primary keys) to ensure each query result is unique.