Dev Services for Kafka
如果存在任何与 Kafka 相关的扩展(例如 quarkus-messaging-kafka
),Dev Services for Kafka 会在 dev 模式和运行测试时自动启动一个 Kafka 代理。因此,您无需手动启动代理。应用程序会自动配置。
If any Kafka-related extension is present (e.g. quarkus-messaging-kafka
), Dev Services for Kafka automatically starts a Kafka broker in dev mode and when running tests.
So, you don’t have to start a broker manually.
The application is configured automatically.
由于启动 Kafka 代理可能需要较长时间,因此 Dev Services for Kafka 使用 Redpanda,一个在约 1 秒内启动的与 Kafka 兼容的代理。
Because starting a Kafka broker can be long, Dev Services for Kafka uses Redpanda, a Kafka compatible broker which starts in ~1 second.
Enabling / Disabling Dev Services for Kafka
除非满足以下条件,否则 Dev Services for Kafka 会被自动启用:
Dev Services for Kafka is automatically enabled unless:
-
quarkus.kafka.devservices.enabled
is set tofalse
-
the
kafka.bootstrap.servers
is configured -
all the Reactive Messaging Kafka channels have the
bootstrap.servers
attribute set
Dev Services for Kafka 依赖于 Docker 来启动代理。如果您的环境不支持 Docker,您需要手动启动代理,或连接到已运行的代理。您可以使用 kafka.bootstrap.servers
来配置代理地址。
Dev Services for Kafka relies on Docker to start the broker.
If your environment does not support Docker, you will need to start the broker manually, or connect to an already running broker.
You can configure the broker address using kafka.bootstrap.servers
.
Shared broker
大多数情况下,您需要在应用程序之间共享代理。Dev Services for Kafka 为您的多个,在 dev 模式下运行的 Quarkus 应用程序实现了共享单个代理的 service discovery 机制。
Most of the time you need to share the broker between applications. Dev Services for Kafka implements a service discovery mechanism for your multiple Quarkus applications running in dev mode to share a single broker.
Dev Services for Kafka 用 |
Dev Services for Kafka starts the container with the |
如果您需要多个(共享)代理,您可以配置 quarkus.kafka.devservices.service-name
属性并指示代理名称。它将查找具有相同值的容器,或在找不到容器的情况下启动一个新容器。默认服务名称为 kafka
。
If you need multiple (shared) brokers, you can configure the quarkus.kafka.devservices.service-name
attribute and indicate the broker name.
It looks for a container with the same value, or starts a new one if none can be found.
The default service name is kafka
.
在 dev 模式下默认启用共享,但在测试模式下禁用共享。您可以使用 quarkus.kafka.devservices.shared=false
禁用共享。
Sharing is enabled by default in dev mode, but disabled in test mode.
You can disable the sharing with quarkus.kafka.devservices.shared=false
.
Setting the port
默认情况下,Dev Services for Kafka 选择一个随机端口并配置应用程序。您可以通过配置 quarkus.kafka.devservices.port
属性来设置端口。
By default, Dev Services for Kafka picks a random port and configures the application.
You can set the port by configuring the quarkus.kafka.devservices.port
property.
请注意,Kafka 通告地址会使用所选端口自动配置。
Note that the Kafka advertised address is automatically configured with the chosen port.
Configuring the image
Dev Services for Kafka 支持 Redpanda, kafka-native和 Strimzi(处于 Kraft模式)映像。
Dev Services for Kafka supports Redpanda, kafka-native and Strimzi (in Kraft mode) images.
*Redpanda*是一个与 Kafka 兼容的事件流式传输平台。由于它提供快速启动时间,因此 dev services 默认使用 `vectorized/redpanda`的 Redpanda 映像。您可以从 [role="bare"][role="bare"]https://hub.docker.com/r/vectorized/redpanda中选择任何版本。
Redpanda is a Kafka compatible event streaming platform.
Because it provides a fast startup times, dev services defaults to Redpanda images from vectorized/redpanda
.
You can select any version from [role="bare"]https://hub.docker.com/r/vectorized/redpanda.
*kafka-native*提供使用 Quarkus 和 GraalVM 编译为原生二进制文件的 Apache Kafka 发行版的映像。同时仍然是 experimental,它提供了非常快速启动时间,并且空间占用小。
kafka-native provides images of standard Apache Kafka distribution compiled to native binary using Quarkus and GraalVM. While still being experimental, it provides very fast startup times with small footprint.
可以使用配置映像类型
Image type can be configured using
quarkus.kafka.devservices.provider=kafka-native
*Strimzi*为在 Kubernetes 上运行 Apache Kafka 提供容器映像和操作符。虽然 Strimzi 针对 Kubernetes 进行了优化,但这些映像在经典容器环境中也能完美运行。Strimzi 容器映像在 JVM 上运行“真正的”Kafka 代理,启动速度较慢。
Strimzi provides container images and Operators for running Apache Kafka on Kubernetes. While Strimzi is optimized for Kubernetes, the images work perfectly in classic container environments. Strimzi container images run "genuine" Kafka broker on JVM, which is slower to start.
quarkus.kafka.devservices.provider=strimzi
对于 Strimzi,您可以从 [role="bare"][role="bare"]https://quay.io/repository/strimzi-test-container/test-container?tab=tags中选择具有 Kraft 支持(2.8.1 及更高版本)的任何 Kafka 版本映像。
For Strimzi, you can select any image with a Kafka version which has Kraft support (2.8.1 and higher) from [role="bare"]https://quay.io/repository/strimzi-test-container/test-container?tab=tags
quarkus.kafka.devservices.image-name=quay.io/strimzi-test-container/test-container:0.106.0-kafka-3.7.0
Configuring Kafka topics
您可以将 Dev Services for Kafka 配置为在代理启动后创建主题。主题使用给定数量的分区和 1 个副本创建。
You can configure the Dev Services for Kafka to create topics once the broker is started. Topics are created with given number of partitions and 1 replica.
以下示例创建一个名为 `test`且有 3 个分区的主题,以及一个名为 `messages`且有 2 个分区的第二个主题。
The following example creates a topic named test
with 3 partitions, and a second topic named messages
with 2 partitions.
quarkus.kafka.devservices.topic-partitions.test=3
quarkus.kafka.devservices.topic-partitions.messages=2
如果已经存在具有指定名称的主题,则将跳过创建过程,而不会尝试将现有主题重新分区为其他数量的分区。
If a topic already exists with the given name, the creation is skipped, without trying to re-partition the existing topic to a different number of partitions.
您可以使用 `quarkus.kafka.devservices.topic-partitions-timeout`来配置主题创建中使用的 Kafka 管理客户端调用的超时时间,其默认值为 2 秒。
You can configure timeout for Kafka admin client calls used in topic creation using quarkus.kafka.devservices.topic-partitions-timeout
, it defaults to 2 seconds.
Transactional and Idempotent producers support
默认情况下,Redpanda 代理配置为启用事务和幂等性功能。您可以使用以下方法禁用它们:
By default, the Redpanda broker is configured to enable transactions and idempotence features. You can disable those using:
quarkus.kafka.devservices.redpanda.transaction-enabled=false
Redpanda 事务不支持完全一次处理。 |
Redpanda transactions does not support exactly-once processing. |