Resetting Offsets

当应用程序启动时,每个已分配分区中的初始位置取决于两个属性 startOffsetresetOffsets。如果 resetOffsetsfalse,则正常的 Kafka 消费者 link:https://kafka.apache.org/documentation/#consumerconfigs_auto.offset.reset[auto.offset.reset 语义适用。也就是说,如果绑定消费者的分区没有已提交的偏移量,则位置为 earliestlatest。默认情况下,具有显式 group 的绑定使用 earliest,匿名的绑定(没有 group)使用 latest。可以通过设置 startOffset 绑定属性来覆盖这些默认值。当使用特定的 group 第一次启动绑定时,不会有已提交的偏移量。偏移量不存在的另一个条件是偏移量已过期。对于现代代理程序(自 2.1 起)和默认的代理程序属性,偏移量在最后一个成员离开组后 7 天后过期。有关详细信息,请参阅 link:https://kafka.apache.org/documentation/#brokerconfigs_offsets.retention.minutes[offsets.retention.minutes 代理程序属性。

When an application starts, the initial position in each assigned partition depends on two properties startOffset and resetOffsets. If resetOffsets is false, normal Kafka consumer auto.offset.reset semantics apply. i.e. If there is no committed offset for a partition for the binding’s consumer group, the position is earliest or latest. By default, bindings with an explicit group use earliest, and anonymous bindings (with no group) use latest. These defaults can be overridden by setting the startOffset binding property. There will be no committed offset(s) the first time the binding is started with a particular group. The other condition where no committed offset exists is if the offset has been expired. With modern brokers (since 2.1), and default broker properties, the offsets are expired 7 days after the last member leaves the group. See the offsets.retention.minutes broker property for more information.

resetOffsetstrue 时,绑定应用与代理上没有已提交偏移量时应用的语义类似,就像此绑定从未从该主题使用过一样;也就是说,将忽略任何当前已提交的偏移量。

When resetOffsets is true, the binder applies similar semantics to those that apply when there is no committed offset on the broker, as if this binding has never consumed from the topic; i.e. any current committed offset is ignored.

以下是可能使用它的两个用例。

Following are two use cases when this might be used.

  1. Consuming from a compacted topic containing key/value pairs. Set resetOffsets to true and startOffset to earliest; the binding will perform a seekToBeginning on all newly assigned partitions.

  2. Consuming from a topic containing events, where you are only interested in events that occur while this binding is running. Set resetOffsets to true and startOffset to latest; the binding will perform a seekToEnd on all newly assigned partitions.

如果在初始分配后重新平衡,则仅在初始分配期间未分配任何新分配的分区上执行查找。

If a rebalance occurs after the initial assignment, the seeks will only be performed on any newly assigned partitions that were not assigned during the initial assignment.

有关主题偏移的更多控制,请参阅 rebalance listener;当提供一个侦听器时,resetOffsets 不应设置为 true,否则,这将导致错误。

For more control over topic offsets, see rebalance listener; when a listener is provided, resetOffsets should not be set to true, otherwise, that will cause an error.