Dev Services for Kafka

如果存在任何与 Kafka 相关的扩展(例如 quarkus-messaging-kafka),Dev Services for Kafka 会在 dev 模式和运行测试时自动启动一个 Kafka 代理。因此,您无需手动启动代理。应用程序会自动配置。

If any Kafka-related extension is present (e.g. quarkus-messaging-kafka), Dev Services for Kafka automatically starts a Kafka broker in dev mode and when running tests. So, you don’t have to start a broker manually. The application is configured automatically.

由于启动 Kafka 代理可能需要较长时间,因此 Dev Services for Kafka 使用 Redpanda,一个在约 1 秒内启动的与 Kafka 兼容的代理。

Because starting a Kafka broker can be long, Dev Services for Kafka uses Redpanda, a Kafka compatible broker which starts in ~1 second.

Enabling / Disabling Dev Services for Kafka

除非满足以下条件,否则 Dev Services for Kafka 会被自动启用:

Dev Services for Kafka is automatically enabled unless:

  • quarkus.kafka.devservices.enabled is set to false

  • the kafka.bootstrap.servers is configured

  • all the Reactive Messaging Kafka channels have the bootstrap.servers attribute set

Dev Services for Kafka 依赖于 Docker 来启动代理。如果您的环境不支持 Docker,您需要手动启动代理,或连接到已运行的代理。您可以使用 kafka.bootstrap.servers 来配置代理地址。

Dev Services for Kafka relies on Docker to start the broker. If your environment does not support Docker, you will need to start the broker manually, or connect to an already running broker. You can configure the broker address using kafka.bootstrap.servers.

Shared broker

大多数情况下,您需要在应用程序之间共享代理。Dev Services for Kafka 为您的多个,在 dev 模式下运行的 Quarkus 应用程序实现了共享单个代理的 service discovery 机制。

Most of the time you need to share the broker between applications. Dev Services for Kafka implements a service discovery mechanism for your multiple Quarkus applications running in dev mode to share a single broker.

Dev Services for Kafka 用 quarkus-dev-service-kafka 标签启动容器,用于标识容器。

Dev Services for Kafka starts the container with the quarkus-dev-service-kafka label which is used to identify the container.

如果您需要多个(共享)代理,您可以配置 quarkus.kafka.devservices.service-name 属性并指示代理名称。它将查找具有相同值的容器,或在找不到容器的情况下启动一个新容器。默认服务名称为 kafka

If you need multiple (shared) brokers, you can configure the quarkus.kafka.devservices.service-name attribute and indicate the broker name. It looks for a container with the same value, or starts a new one if none can be found. The default service name is kafka.

在 dev 模式下默认启用共享,但在测试模式下禁用共享。您可以使用 quarkus.kafka.devservices.shared=false 禁用共享。

Sharing is enabled by default in dev mode, but disabled in test mode. You can disable the sharing with quarkus.kafka.devservices.shared=false.

Setting the port

默认情况下,Dev Services for Kafka 选择一个随机端口并配置应用程序。您可以通过配置 quarkus.kafka.devservices.port 属性来设置端口。

By default, Dev Services for Kafka picks a random port and configures the application. You can set the port by configuring the quarkus.kafka.devservices.port property.

请注意,Kafka 通告地址会使用所选端口自动配置。

Note that the Kafka advertised address is automatically configured with the chosen port.

Configuring the image

Dev Services for Kafka 支持 Redpanda, kafka-nativeStrimzi(处于 Kraft模式)映像。

Dev Services for Kafka supports Redpanda, kafka-native and Strimzi (in Kraft mode) images.

*Redpanda*是一个与 Kafka 兼容的事件流式传输平台。由于它提供快速启动时间,因此 dev services 默认使用 `vectorized/redpanda`的 Redpanda 映像。您可以从 [role="bare"][role="bare"]https://hub.docker.com/r/vectorized/redpanda中选择任何版本。

Redpanda is a Kafka compatible event streaming platform. Because it provides a fast startup times, dev services defaults to Redpanda images from vectorized/redpanda. You can select any version from [role="bare"]https://hub.docker.com/r/vectorized/redpanda.

*kafka-native*提供使用 Quarkus 和 GraalVM 编译为原生二进制文件的 Apache Kafka 发行版的映像。同时仍然是 experimental,它提供了非常快速启动时间,并且空间占用小。

kafka-native provides images of standard Apache Kafka distribution compiled to native binary using Quarkus and GraalVM. While still being experimental, it provides very fast startup times with small footprint.

可以使用配置映像类型

Image type can be configured using

quarkus.kafka.devservices.provider=kafka-native

*Strimzi*为在 Kubernetes 上运行 Apache Kafka 提供容器映像和操作符。虽然 Strimzi 针对 Kubernetes 进行了优化,但这些映像在经典容器环境中也能完美运行。Strimzi 容器映像在 JVM 上运行“真正的”Kafka 代理,启动速度较慢。

Strimzi provides container images and Operators for running Apache Kafka on Kubernetes. While Strimzi is optimized for Kubernetes, the images work perfectly in classic container environments. Strimzi container images run "genuine" Kafka broker on JVM, which is slower to start.

quarkus.kafka.devservices.provider=strimzi

对于 Strimzi,您可以从 [role="bare"][role="bare"]https://quay.io/repository/strimzi-test-container/test-container?tab=tags中选择具有 Kraft 支持(2.8.1 及更高版本)的任何 Kafka 版本映像。

For Strimzi, you can select any image with a Kafka version which has Kraft support (2.8.1 and higher) from [role="bare"]https://quay.io/repository/strimzi-test-container/test-container?tab=tags

quarkus.kafka.devservices.image-name=quay.io/strimzi-test-container/test-container:0.106.0-kafka-3.7.0

Configuring Kafka topics

您可以将 Dev Services for Kafka 配置为在代理启动后创建主题。主题使用给定数量的分区和 1 个副本创建。

You can configure the Dev Services for Kafka to create topics once the broker is started. Topics are created with given number of partitions and 1 replica.

以下示例创建一个名为 `test`且有 3 个分区的主题,以及一个名为 `messages`且有 2 个分区的第二个主题。

The following example creates a topic named test with 3 partitions, and a second topic named messages with 2 partitions.

quarkus.kafka.devservices.topic-partitions.test=3
quarkus.kafka.devservices.topic-partitions.messages=2

如果已经存在具有指定名称的主题,则将跳过创建过程,而不会尝试将现有主题重新分区为其他数量的分区。

If a topic already exists with the given name, the creation is skipped, without trying to re-partition the existing topic to a different number of partitions.

您可以使用 `quarkus.kafka.devservices.topic-partitions-timeout`来配置主题创建中使用的 Kafka 管理客户端调用的超时时间,其默认值为 2 秒。

You can configure timeout for Kafka admin client calls used in topic creation using quarkus.kafka.devservices.topic-partitions-timeout, it defaults to 2 seconds.

Transactional and Idempotent producers support

默认情况下,Redpanda 代理配置为启用事务和幂等性功能。您可以使用以下方法禁用它们:

By default, the Redpanda broker is configured to enable transactions and idempotence features. You can disable those using:

quarkus.kafka.devservices.redpanda.transaction-enabled=false

Redpanda 事务不支持完全一次处理。

Redpanda transactions does not support exactly-once processing.