Using OpenTelemetry Tracing

本指南解释了 Quarkus 应用程序如何利用 OpenTelemetry (OTel) 为交互式 Web 应用程序提供分布式跟踪。

This guide explains how your Quarkus application can utilize OpenTelemetry (OTel) to provide distributed tracing for interactive web applications.

Prerequisites

Unresolved directive in opentelemetry-tracing.adoc - include::{includes}/prerequisites.adoc[]

Architecture

本指南中，我们将创建一个简单的 REST 应用程序来演示分布式追踪。

In this guide, we create a straightforward REST application to demonstrate distributed tracing.

Solution

我们建议你按照后续章节中的说明，逐步创建应用程序。但是，你可以直接跳到已完成的示例。

We recommend that you follow the instructions in the next sections and create the application step by step. However, you can skip right to the completed example.

克隆 Git 存储库： git clone {quickstarts-clone-url}，或下载 {quickstarts-archive-url}[存档]。

Clone the Git repository: git clone {quickstarts-clone-url}, or download an {quickstarts-archive-url}[archive].

解决方案位于 opentelemetry-quickstart directory。

The solution is located in the opentelemetry-quickstart directory.

Creating the Maven project

首先，我们需要一个新项目。使用以下命令创建一个新项目：

First, we need a new project. Create a new project with the following command:

Unresolved directive in opentelemetry-tracing.adoc - include::{includes}/devtools/create-app.adoc[]

此命令会生成 Maven 项目并导入 quarkus-opentelemetry 扩展，其中包括默认的 OpenTelemetry 支持以及用于 OTLP 的 gRPC span 导出器。

This command generates the Maven project and imports the quarkus-opentelemetry extension, which includes the default OpenTelemetry support, and a gRPC span exporter for OTLP.

如果 Quarkus 项目已配置，则可以通过在项目基本目录中运行以下命令来将 quarkus-opentelemetry 扩展添加到项目中：

If you already have your Quarkus project configured, you can add the quarkus-opentelemetry extension to your project by running the following command in your project base directory:

Unresolved directive in opentelemetry-tracing.adoc - include::{includes}/devtools/extension-add.adoc[]

这会将以下内容添加到构建文件中：

This will add the following to your build file:

pom.xml
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-opentelemetry</artifactId>
</dependency>
build.gradle
implementation("io.quarkus:quarkus-opentelemetry")

Examine the Jakarta REST resource

使用以下内容创建 src/main/java/org/acme/opentelemetry/TracedResource.java 文件：

Create a src/main/java/org/acme/opentelemetry/TracedResource.java file with the following content:

package org.acme.opentelemetry;

import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
import org.jboss.logging.Logger;

@Path("/hello")
public class TracedResource {

    private static final Logger LOG = Logger.getLogger(TracedResource.class);

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        LOG.info("hello");
        return "hello";
    }
}

请注意，应用程序中不包含任何跟踪特定代码。默认情况下，发送到此端点的请求将在无需任何所需代码更改的情况下被跟踪。

Notice that there is no tracing specific code included in the application. By default, requests sent to this endpoint will be traced without any required code changes.

Create the configuration

Unresolved directive in opentelemetry-tracing.adoc - include::{includes}/opentelemetry-config.adoc[]

Run the application

首先我们需要启动一个系统来可视化 OpenTelemetry 数据。我们有 2 个选项：

First we need to start a system to visualise the OpenTelemetry data. We have 2 options:

  • Start an all-in-one Grafana OTel LGTM system for traces and metrics.

  • Jaeger system just for traces.

Grafana OTel LGTM option

此功能包含 Quarkus Dev 服务，其中包括用于可视化数据的 Grafana、用于存储日志的 Loki、用于存储跟踪的 Tempo 以及用于存储指标的 Prometheus。还提供和 OTel 收集器来接收数据。

This features a Quarkus Dev service including a Grafana for visualizing data, Loki to store logs, Tempo to store traces and Prometheus to store metrics. Also provides and OTel collector to receive the data.

Jaeger to see traces option

配置并启动 OpenTelemetry Collector 以接收、处理和将遥测数据导出到 Jaeger，该数据将显示捕获的跟踪。

Configure and start the OpenTelemetry Collector to receive, process and export telemetry data to Jaeger that will display the captured traces.

一体化 Jaeger 包括 Jaeger 代理、OTel 收集器和查询服务/UI。您无需安装单独收集器。您可以直接将跟踪数据发送到 Jaeger（在此处启用 OTLP 接收器后，参见此 blog entry 了解更多详细信息）。

Jaeger-all-in-one includes the Jaeger agent, an OTel collector, and the query service/UI. You do not need to install a separated collector. You can directly send the trace data to Jaeger (after enabling OTLP receivers there, see e.g. this blog entry for details).

通过以下 docker-compose.yml 文件启动 OpenTelemetry 收集器和 Jaeger 系统，您可以通过 docker-compose up -d 启动此文件：

Start the OpenTelemetry Collector and Jaeger system via the following docker-compose.yml file that you can launch via docker-compose up -d:

version: "2"
services:

  # Jaeger
  jaeger-all-in-one:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686" # Jaeger UI
      - "14268:14268" # Receive legacy OpenTracing traces, optional
      - "4317:4317"   # OTLP gRPC receiver
      - "4318:4318"   # OTLP HTTP receiver, not yet used by Quarkus, optional
      - "14250:14250" # Receive from external otel-collector, optional
    environment:
      - COLLECTOR_OTLP_ENABLED=true

您应删除不需要的可选端口。

You should remove the optional ports you don’t need them.

Start the application

现在我们已准备好运行我们的应用程序。如果使用 application.properties 配置跟踪器：

Now we are ready to run our application. If using application.properties to configure the tracer:

Unresolved directive in opentelemetry-tracing.adoc - include::{includes}/devtools/dev.adoc[]

或者通过 JVM 参数配置 OTLP gRPC 端点：

or if configuring the OTLP gRPC endpoint via JVM arguments:

include::{includes}/devtools/dev.adoc[]:!dev-additional-parameters:

Unresolved directive in opentelemetry-tracing.adoc - include::{includes}/devtools/dev.adoc[] :!dev-additional-parameters:

当 OpenTelemetry 收集器、Jaeger 系统和应用程序正在运行时，可以向提供的端点发出请求：

With the OpenTelemetry Collector, the Jaeger system and the application running, you can make a request to the provided endpoint:

$ curl http://localhost:8080/hello
hello

当提交第一个请求时，您将能够在日志中看到跟踪信息：

When the first request has been submitted, you will be able to see the tracing information in the logs:

10:49:02 INFO  traceId=, parentId=, spanId=, sampled= [io.quarkus] (main) Installed features: [cdi, opentelemetry, resteasy-client, resteasy, smallrye-context-propagation, vertx]
10:49:03 INFO  traceId=17ceb8429b9f25b0b879fa1503259456, parentId=3125c8bee75b7ad6, spanId=58ce77c86dd23457, sampled=true [or.ac.op.TracedResource] (executor-thread-1) hello
10:49:03 INFO  traceId=ad23acd6d9a4ed3d1de07866a52fa2df, parentId=, spanId=df13f5b45cf4d1e2, sampled=true [or.ac.op.TracedResource] (executor-thread-0) hello

然后访问 Jaeger UI 以查看跟踪信息。

Then visit the Jaeger UI to see the tracing information.

按 CTRL+C 或键入 q 停止应用程序。

Hit CTRL+C or type q to stop the application.

JDBC

JDBC instrumentation 会为您的应用程序执行的每个 JDBC 查询添加一个 span，要启用它，请将以下依赖项添加到您的构建文件：

The JDBC instrumentation will add a span for each JDBC queries done by your application, to enable it, add the following dependency to your build file:

pom.xml
<dependency>
    <groupId>io.opentelemetry.instrumentation</groupId>
    <artifactId>opentelemetry-jdbc</artifactId>
</dependency>
build.gradle
implementation("io.opentelemetry.instrumentation:opentelemetry-jdbc")

由于它使用专用 JDBC 数据源包装器，因此您必须为数据源启用遥测：

As it uses a dedicated JDBC datasource wrapper, you must enable telemetry for your datasource:

# enable tracing
quarkus.datasource.jdbc.telemetry=true

# configure datasource
quarkus.datasource.db-kind=postgresql
quarkus.datasource.jdbc.url=jdbc:postgresql://localhost:5432/mydatabase

Additional configuration

某些用例将需要 OpenTelemetry 的自定义配置。这些章节将概述正确配置它所需的步骤。

Some use cases will require custom configuration of OpenTelemetry. These sections will outline what is necessary to properly configure it.

ID Generator

OpenTelemetry 扩展将默认情况下在创建跟踪和跨度标识符时使用随机 ID Generator。

The OpenTelemetry extension will use by default a random ID Generator when creating the trace and span identifier.

某些特定于供应商的协议需要自定义 ID 生成器，可通过创建生成器来覆盖默认值。OpenTelemetry 扩展将检测 `IdGenerator`CDI bean，并在配置跟踪生成器时使用该 bean。

Some vendor-specific protocols need a custom ID Generator, you can override the default one by creating a producer. The OpenTelemetry extension will detect the IdGenerator CDI bean and will use it when configuring the tracer producer.

@Singleton
public class CustomConfiguration {

    /** Creates a custom IdGenerator for OpenTelemetry */
    @Produces
    @Singleton
    public IdGenerator idGenerator() {
        return AwsXrayIdGenerator.getInstance();
    }
}

Propagators

OpenTelemetry 通过 propagators跨越式地传播关注点，该关注点将共享一个底层的 `Context`以在分布式事务的生命周期中存储状态和访问数据。

OpenTelemetry propagates cross-cutting concerns through propagators that will share an underlying Context for storing state and accessing data across the lifespan of a distributed transaction.

默认情况下，OpenTelemetry 扩展启用 W3C Trace Context和 W3C Baggage传播器，但是，您可以通过设置 OpenTelemetry Configuration Reference中描述的 `propagators`配置来选择任何受支持的 OpenTelemetry 传播器。

By default, the OpenTelemetry extension enables the W3C Trace Context and the W3C Baggage propagators, you can however choose any of the supported OpenTelemetry propagators by setting the propagators config that is described in the OpenTelemetry Configuration Reference.

Additional Propagators

  • The b3, b3multi, jaeger and ottrace propagators will need the trace-propagators extension to be added as a dependency to your project.

pom.xml
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-extension-trace-propagators</artifactId>
</dependency>
build.gradle
implementation("io.opentelemetry:opentelemetry-extension-trace-propagators")

  • The xray propagator will need the aws extension to be added as a dependency to your project.

pom.xml
<dependency>
    <groupId>io.opentelemetry.contrib</groupId>
    <artifactId>opentelemetry-aws-xray-propagator</artifactId>
</dependency>
build.gradle
implementation("io.opentelemetry.contrib:opentelemetry-aws-xray-propagator")

Customise Propagator

若要自定义传播报头，您可以实现 `TextMapPropagatorCustomizer`接口。例如，可将其用于限制 OpenTelemetry 跟踪报头传播并防止机密数据被发送到第三方系统。

To customise the propagation header you can implement the TextMapPropagatorCustomizer interface. This can be used, as an example, to restrict propagation of OpenTelemetry trace headers and prevent potentially sensitive data to be sent to third party systems.

/**
 * /**
 * Meant to be implemented by a CDI bean that provides arbitrary customization for the TextMapPropagator
 * that are to be registered with OpenTelemetry
 */
public interface TextMapPropagatorCustomizer {

    TextMapPropagator customize(Context context);

    interface Context {
        TextMapPropagator propagator();

        ConfigProperties otelConfigProperties();
    }
}

Resource

请参阅主 OpenTelemetry Guide resources 部分。

See the main OpenTelemetry Guide resources section.

End User attributes

启用时，Quarkus 将 OpenTelemetry 最终用户属性作为跨度属性添加。在启用此功能之前，请验证 Quarkus Security 扩展是否存在且已配置。有关 Quarkus Security 的更多信息，请参阅 Quarkus Security overview。

When enabled, Quarkus adds OpenTelemetry End User attributes as Span attributes. Before you enable this feature, verify that Quarkus Security extension is present and configured. More information about the Quarkus Security can be found in the Quarkus Security overview.

只有在已尽力进行身份验证的情况下才会添加属性。是否将最终用户属性作为跨度属性添加取决于 Quarkus 应用程序的身份验证和授权配置。如果您在身份验证之前创建自定义跨度，Quarkus 无法向其添加最终用户属性。身份验证完成后，Quarkus 只能将属性添加到当前跨度。关于自定义跨度的另一个重要事项是用于传播 Quarkus `SecurityIdentity`的活动 CDI 请求上下文。原则上，当在创建自定义跨度之前已为您激活 CDI 请求上下文时，Quarkus 能够添加最终用户属性。

The attributes are only added when authentication has already happened on a best-efforts basis. Whether the End User attributes are added as Span attributes depends on authentication and authorization configuration of your Quarkus application. If you create custom Spans prior to the authentication, Quarkus cannot add the End User attributes to them. Quarkus is only able to add the attributes to the Span that is current after the authentication has been finished. Another important consideration regarding custom Spans is active CDI request context that is used to propagate Quarkus SecurityIdentity. In principle, Quarkus is able to add the End User attributes when the CDI request context has been activated for you before the custom Spans are created.

quarkus.otel.traces.eusp.enabled=true 1
quarkus.http.auth.proactive=true 2
1 Enable the End User Attributes feature so that the SecurityIdentity principal and roles are added as Span attributes. The End User attributes are personally identifiable information, therefore make sure you want to export them before you enable this feature.
2 Optionally enable proactive authentication. The best possible results are achieved when proactive authentication is enabled because the authentication happens sooner. A good way to determine whether proactive authentication should be enabled in your Quarkus application is to read the Quarkus Proactive authentication guide.

当使用自定义 Jakarta REST SecurityContexts时，不支持此功能。

This feature is not supported when a custom Jakarta REST SecurityContexts is used.

Sampler

sampler决定是否应放弃或转发跟踪，它有效地管理噪声，并通过限制发送给收集器的收集的跟踪数来减少开销。

A sampler decides whether a trace should be discarded or forwarded, effectively managing noise and reducing overhead by limiting the number of collected traces sent to the collector.

Quarkus 配备了 built-in sampler，您还可以选择创建自定义抽样器。

Quarkus comes equipped with a built-in sampler, and you also have the option to create your custom sampler.

若要使用内置抽样器，您可以通过设置 OpenTelemetry Configuration Reference中详述的所需抽样器参数来对其进行配置。例如，您可以将抽样器配置为保留 50% 的跟踪：

To use the built-in sampler, you can configure it by setting the desired sampler parameters as detailed in the OpenTelemetry Configuration Reference. As an example, you can configure the sampler to retain 50% of the traces:

# build time property only:
quarkus.otel.traces.sampler=traceidratio
# Runtime property:
quarkus.otel.traces.sampler.arg=0.5

抽样器的一个有趣用例是根据此示例在运行时激活和停用跟踪导出：

An interesting use case for the sampler is to activate and deactivate tracing export at runtime, acording to this example:

# build time property only:
quarkus.otel.traces.sampler=traceidratio
# On (default). All traces are exported:
quarkus.otel.traces.sampler.arg=1.0
# Off. No traces are exported:
quarkus.otel.traces.sampler.arg=0.0

Quarkus 3.0 在配置方面引入了重大变更。

Quarkus 3.0 introduced breaking changes on the configuration.

抽样器相关的属性名称和值更改以符合最新的 Java OpenTelemetry SDK。在过渡期间，可以将新配置值设置为旧属性，因为我们正在映射 quarkus.opentelemetry.tracer.sampler→ quarkus.otel.traces.sampler。

Sampler related property names and values change to comply with the latest Java OpenTelemetry SDK. During a transition period it will be possible to set the new configuration values in the old property because we are mapping quarkus.opentelemetry.tracer.sampler → quarkus.otel.traces.sampler.

如果采样器基于父级，则无需设置，现在已删除的属性 quarkus.opentelemetry.tracer.sampler.parent-based。

If the sampler is parent based, there is no need to set, the now dropped property, quarkus.opentelemetry.tracer.sampler.parent-based.

您现在需要在 quarkus.opentelemetry.tracer.sampler 上设置的值为：

The values you need to set on quarkus.opentelemetry.tracer.sampler are now:

Old Sampler config value New Sampler config value New Sampler config value (Parent based)

on

always_on

parentbased_always_on

off

always_off

parentbased_always_off

ratio

traceidratio

parentbased_traceidratio

如果您需要使用自定义采样器，现在有 2 种不同的方法：

If you need to use a custom sampler there are now 2 different ways:

Sampler CDI Producer

您可以创建一个采样器 CDI 制造者。Quarkus OpenTelemetry 扩展将检测 Sampler CDI bean，并在配置 Tracer 时使用它。

You can create a sampler CDI producer. The Quarkus OpenTelemetry extension will detect the Sampler CDI bean and will use it when configuring the Tracer.

@Singleton
public class CustomConfiguration {

    /** Creates a custom sampler for OpenTelemetry */
    @Produces
    @Singleton
    public Sampler sampler() {
        return JaegerRemoteSampler.builder()
        .setServiceName("my-service")
        .build();
    }
}

OTel Sampler SPI

这将使用 OTel 自动配置提供的 SPI 钩子。您可以创建简单的采样器类：

This will use the SPI hooks available with the OTel Autoconfiguration. You can create a simple Sampler class:

public class CustomSPISampler implements Sampler {
    @Override
    public SamplingResult shouldSample(Context context,
            String s,
            String s1,
            SpanKind spanKind,
            Attributes attributes,
            List<LinkData> list) {
        // Do some sampling here
        return Sampler.alwaysOn().shouldSample(context, s, s1, spanKind, attributes, list);
    }

    @Override
    public String getDescription() {
        return "custom-spi-sampler-description";
    }
}

然后是采样器提供程序：

Then a Sampler Provider:

public class CustomSPISamplerProvider implements ConfigurableSamplerProvider {
    @Override
    public Sampler createSampler(ConfigProperties configProperties) {
        return new CustomSPISampler();
    }

    @Override
    public String getName() {
        return "custom-spi-sampler";
    }
}

在 resources/META-INF/services 上使用名称 io.opentelemetry.sdk.autoconfigure.spi.traces.ConfigurableSamplerProvider 编写 SPI 加载器文本文件，其中包含 CustomSPISamplerProvider 类的完全限定名称。

Write the SPI loader text file at resources/META-INF/services with name io.opentelemetry.sdk.autoconfigure.spi.traces.ConfigurableSamplerProvider containing the full qualified name of the CustomSPISamplerProvider class.

然后在配置中激活：

Then activate on the configuration:

quarkus.otel.traces.sampler=custom-spi-sampler

如您所见，CDI 更易于使用。

As you can see, CDI is much simpler to work with.

Additional instrumentation

某些 Quarkus 扩展将需要额外的代码来确保跟踪传播到后续执行。这些部分将概述跨进程边界传播跟踪所需的条件。

Some Quarkus extensions will require additional code to ensure traces are propagated to subsequent execution. These sections will outline what is necessary to propagate traces across process boundaries.

本节中记录的检测已通过 Quarkus 测试，并且可在标准模式和原生模式下工作。

The instrumentation documented in this section has been tested with Quarkus and works in both standard and native mode.

CDI

使用 io.opentelemetry.instrumentation.annotations.WithSpan 注释对任何 CDI 感知 bean 中的方法进行注释将创建一个新 Span，并与当前 Trace 上下文建立任何所需的关系。

Annotating a method in any CDI aware bean with the io.opentelemetry.instrumentation.annotations.WithSpan annotation will create a new Span and establish any required relationships with the current Trace context.

使用 io.opentelemetry.instrumentation.annotations.AddingSpanAttributes 对任何 CDI 感知 bean 中的方法进行注释不会创建新 Span，而是将带注释的方法参数添加到当前 Span 中的属性。

Annotating a method in any CDI aware bean with the io.opentelemetry.instrumentation.annotations.AddingSpanAttributes will not create a new span but will add annotated method parameters to attributes in the current span.

如果某个方法错误地使用 @AddingSpanAttributes 和 @WithSpan 注释进行了注释，那么 @WithSpan 注释将具有优先权。

If a method is annotated by mistake with @AddingSpanAttributes and @WithSpan annotations, the @WithSpan annotation will take precedence.

方法参数可以使用 io.opentelemetry.instrumentation.annotations.SpanAttribute 注释进行注释，以指示哪些方法参数应成为 Span 的一部分。参数名称也可以自定义。

Method parameters can be annotated with the io.opentelemetry.instrumentation.annotations.SpanAttribute annotation to indicate which method parameters should be part of the span. The parameter name can be customized as well.

示例：

Example:

@ApplicationScoped
class SpanBean {
    @WithSpan
    void span() {

    }

    @WithSpan("name")
    void spanName() {

    }

    @WithSpan(kind = SERVER)
    void spanKind() {

    }

    @WithSpan
    void spanArgs(@SpanAttribute(value = "arg") String arg) {

    }

    @AddingSpanAttributes
    void addArgumentToExistingSpan(@SpanAttribute(value = "arg") String arg) {

    }
}

Available OpenTelemetry CDI injections

根据 MicroProfile Telemetry Tracing 规范，Quarkus 支持下列类的 CDI 注入：

As per MicroProfile Telemetry Tracing specification, Quarkus supports the CDI injections of the following classes:

  • io.opentelemetry.api.OpenTelemetry

  • io.opentelemetry.api.trace.Tracer

  • io.opentelemetry.api.trace.Span

  • io.opentelemetry.api.baggage.Baggage

您可以在任何启用了 CDI 的 bean 中注入这些类。例如，Tracer 对于启动自定义 Span 特别有用：

You can inject these classes in any CDI enabled bean. For instance, the Tracer is particularly useful to start custom spans:

@Inject
Tracer tracer;

...

public void tracedWork() {
    Span span = tracer.spanBuilder("My custom span")
        .setAttribute("attr", "attr.value")
        .setParent(Context.current().with(Span.current()))
        .setSpanKind(SpanKind.INTERNAL)
        .startSpan();

    // traced work

    span.end();
}

Quarkus Messaging - Kafka

在使用针对 Kafka 的 Quarkus 消息传递扩展时，我们可以将跨度传播到 Kafka 记录中，方法是：

When using the Quarkus Messaging extension for Kafka, we are able to propagate the span into the Kafka Record with:

TracingMetadata tm = TracingMetadata.withPrevious(Context.current());
Message out = Message.of(...).withMetadata(tm);

以上内容创建了一个 TracingMetadata 对象，我们可以将其添加到正在生成的 Message，该对象会检索 OpenTelemetry Context 以提取用于传播的当前跨度。

The above creates a TracingMetadata object we can add to the Message being produced, which retrieves the OpenTelemetry Context to extract the current span for propagation.

Quarkus Security Events

Quarkus 支持将 Security events 导出为 OpenTelemetry Span 事件。

Quarkus supports exporting of the Security events as OpenTelemetry Span events.

quarkus.otel.security-events.enabled=true 1
1 Export Quarkus Security events as OpenTelemetry Span events.

Exporters

请参阅 OpenTelemetry Guide exporters 主部分。

See the main OpenTelemetry Guide exporters section.

Quarkus core extensions instrumented with OpenTelemetry tracing

Disable parts of the automatic tracing

通过将 quarkus.otel.instrument.* 属性设置为 false，可以禁用自动跟踪工具部分。

Automatic tracing instrumentation parts can be disabled by setting quarkus.otel.instrument.* properties to false.

示例：

Examples:

quarkus.otel.instrument.grpc=false
quarkus.otel.instrument.messaging=false
quarkus.otel.instrument.resteasy-client=false
quarkus.otel.instrument.rest=false
quarkus.otel.instrument.resteasy=false

OpenTelemetry Configuration Reference

请参阅 OpenTelemetry Guide configuration 主参考。

See the main OpenTelemetry Guide configuration reference.