Observability Dev Services with Grafana OTel LGTM

include::_attributes.adoc[]:iokays-category: quarkus:iokays-path: modules/ROOT/pages/observability-devservices-lgtm.adoc:categories: observability,devservices,telemetry,metrics,tracing,logging, opentelemetry, micrometer, prometheus, tempo, loki, grafana:summary: 有关如何使用 Grafana Otel LGTM 的说明:topics: observability,grafana,lgtm,otlp,opentelemetry,devservices,micrometer:extensions: io.quarkus:quarkus-observability-devservices

OTel-LGTMall-in-one Docker 映像,其中包含 OpenTelemetry 的 OTLP 作为将度量标准、跟踪和日志记录数据传输到 OpenTelemetry Collector 的协议,然后将信号数据存储到 Prometheus(度量标准)、 Tempo(跟踪)和 Loki(日志),仅供 Grafana 可视化。Quarkus Observability 使用它来提供 Grafana OTel LGTM Dev 资源。

OTel-LGTM is all-in-one Docker image containing OpenTelemetry’s OTLP as the protocol to transport metrics, tracing and logging data to an OpenTelemetry Collector which then stores signals data into Prometheus (metrics), Tempo (traces) and Loki (logs), only to have it visualized by Grafana. It’s used by Quarkus Observability to provide the Grafana OTel LGTM Dev Resource.

Configuring your project

将 Quarkus Grafana OTel LGTM 接收器(数据去向)扩展添加到您的构建文件中:

Add the Quarkus Grafana OTel LGTM sink (where data goes) extension to your build file:

pom.xml
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-observability-devservices-lgtm</artifactId>
    <scope>provided</scope>
</dependency>
build.gradle
implementation("quarkus-observability-devservices-lgtm")

Metrics

如果您需要指标,请将 Micrometer OTLP 注册表添加到您的构建文件中:

If you need metrics, add the Micrometer OTLP registry to your build file:

pom.xml
<dependency>
    <groupId>io.quarkiverse.micrometer.registry</groupId>
    <artifactId>quarkus-micrometer-registry-otlp</artifactId>
</dependency>
build.gradle
implementation("io.quarkiverse.micrometer.registry:quarkus-micrometer-registry-otlp")

在使用 MicroMeter’s Quarkiverse OTLP 注册表将度量标准推送到 Grafana OTel LGTM 时,quarkus.micrometer.export.otlp.url 属性会自动设置为 OTel 收集器端点,从 docker 容器外部可以看到。

When using the MicroMeter’s Quarkiverse OTLP registry to push metrics to Grafana OTel LGTM, the quarkus.micrometer.export.otlp.url property is automatically set to OTel collector endpoint as seen from the outside of the docker container.

Tracing

对于跟踪,请将 quarkus-opentelemetry 扩展添加到您的构建文件:

For Tracing add the quarkus-opentelemetry extension to your build file:

pom.xml
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-opentelemetry</artifactId>
</dependency>
build.gradle
implementation("io.quarkus:quarkus-opentelemetry")

quarkus.otel.exporter.otlp.endpoint 属性会自动设置为 OTel 收集器端点,从 docker 容器外部可以看到。

The quarkus.otel.exporter.otlp.endpoint property is automatically set to OTel collector endpoint as seen from the outside of the docker container.

quarkus.otel.exporter.otlp.protocol 设置为 http/protobuf

The quarkus.otel.exporter.otlp.protocol is set to http/protobuf.

Access Grafana

在开发模式下启动您的应用程序后:

Once you start your app in dev mode:

Unresolved directive in observability-devservices-lgtm.adoc - include::{includes}/devtools/dev.adoc[]

您会看到这样的日志条目:

You will see a log entry like this:

[io.qu.ob.de.ObservabilityDevServiceProcessor] (build-35) Dev Service Lgtm started, config: {grafana.endpoint=http://localhost:42797, quarkus.otel.exporter.otlp.endpoint=http://localhost:34711, otel-collector.url=localhost:34711, quarkus.micrometer.export.otlp.url=http://localhost:34711/v1/metrics, quarkus.otel.exporter.otlp.protocol=http/protobuf}

请记住,Grafana 是通过临时端口访问的,因此您需要检查日志以查看正在使用哪个端口。在此示例中,Grafana 端点为 grafana.endpoint=http://localhost:42797

Remember that Grafana is accessible in an ephemeral port, so you need to check the logs to see which port is being used. In this example, the grafana endpoint is grafana.endpoint=http://localhost:42797.

如果您错过了消息,您始终可以使用此 Docker 命令检查端口:

If you miss the message you can always check the port with this Docker command:

docker ps | grep grafana

Additional configuration

此扩展将配置您的 quarkus-opentelemetryquarkus-micrometer-registry-otlp 扩展,以便将数据发送到与 Grafana OTel LGTM 映像捆绑的 OTel 收集器。

This extension will configure your quarkus-opentelemetry and quarkus-micrometer-registry-otlp extensions to send data to the OTel Collector bundled with the Grafana OTel LGTM image.

如果您不想使用 Dev Services(例如查找和重新使用现有的运行容器等)的麻烦,您可以简单地禁用 Dev Services 并仅启用 Dev Resource 使用:

If you don’t want all the hassle with Dev Services (e.g. lookup and re-use of existing running containers, etc) you can simply disable Dev Services and enable just Dev Resource usage:

quarkus.observability.enabled=false
quarkus.observability.dev-resources=true

Tests

对于测试中最不“自动魔幻”的使用,简单地禁用两者(Dev Resource 默认情况下已禁用):

And for the least 'auto-magical' usage in the tests, simply disable both (Dev Resources are already disabled by default):

quarkus.observability.enabled=false

然后在测试中明确地将 LGTM Dev Resource 列为 @WithTestResource 资源:

And then explicitly list LGTM Dev Resource in the test as a @WithTestResource resource:

@QuarkusTest
@WithTestResource(LgtmResource.class)
@TestProfile(QuarkusTestResourceTestProfile.class)
public class LgtmLifecycleTest extends LgtmTestBase {
}

Testing full Grafana OTel LGTM stack - example

使用现有的 Quarkus MicroMeter OTLP 注册表

Use existing Quarkus MicroMeter OTLP registry

pom.xml
<dependency>
    <groupId>io.quarkiverse.micrometer.registry</groupId>
    <artifactId>quarkus-micrometer-registry-otlp</artifactId>
</dependency>
build.gradle
implementation("io.quarkiverse.micrometer.registry:quarkus-micrometer-registry-otlp")

只需将 Meter 注册表注入到您的代码中即可 - 它会定期将指标推送到 Grafana LGTM 的 OTLP HTTP 端点。

Simply inject the Meter registry into your code — it will periodically push metrics to Grafana LGTM’s OTLP HTTP endpoint.

@Path("/api")
public class SimpleEndpoint {
    private static final Logger log = Logger.getLogger(SimpleEndpoint.class);

    @Inject
    MeterRegistry registry;

    @PostConstruct
    public void start() {
        Gauge.builder("xvalue", arr, a -> arr[0])
                .baseUnit("X")
                .description("Some random x")
                .tag("my_key", "x")
                .register(registry);
    }

    // ...
}

然后您可以在其中检查 Grafana 的数据源 API 以获取现有的指标数据。

Where you can then check Grafana’s datasource API for existing metrics data.

public class LgtmTestBase {

    @ConfigProperty(name = "grafana.endpoint")
    String endpoint; // NOTE -- injected Grafana endpoint!

    @Test
    public void testTracing() {
        String response = RestAssured.get("/api/poke?f=100").body().asString();
        System.out.println(response);
        GrafanaClient client = new GrafanaClient(endpoint, "admin", "admin");
        Awaitility.await().atMost(61, TimeUnit.SECONDS).until(
                client::user,
                u -> "admin".equals(u.login));
        Awaitility.await().atMost(61, TimeUnit.SECONDS).until(
                () -> client.query("xvalue_X"),
                result -> !result.data.result.isEmpty());
    }

}

// simple Grafana HTTP client

public class GrafanaClient {
    private static final ObjectMapper MAPPER = new ObjectMapper();

    private final String url;
    private final String username;
    private final String password;

    public GrafanaClient(String url, String username, String password) {
        this.url = url;
        this.username = username;
        this.password = password;
    }

    private <T> void handle(
            String path,
            Function<HttpRequest.Builder, HttpRequest.Builder> method,
            HttpResponse.BodyHandler<T> bodyHandler,
            BiConsumer<HttpResponse<T>, T> consumer) {
        try {
            String credentials = username + ":" + password;
            String encodedCredentials = Base64.getEncoder().encodeToString(credentials.getBytes());

            HttpClient httpClient = HttpClient.newHttpClient();
            HttpRequest.Builder builder = HttpRequest.newBuilder()
                    .uri(URI.create(url + path))
                    .header("Authorization", "Basic " + encodedCredentials);
            HttpRequest request = method.apply(builder).build();

            HttpResponse<T> response = httpClient.send(request, bodyHandler);
            int code = response.statusCode();
            if (code < 200 || code > 299) {
                throw new IllegalStateException("Bad response: " + code + " >> " + response.body());
            }
            consumer.accept(response, response.body());
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new RuntimeException(e);
        }
    }

    public User user() {
        AtomicReference<User> ref = new AtomicReference<>();
        handle(
                "/api/user",
                HttpRequest.Builder::GET,
                HttpResponse.BodyHandlers.ofString(),
                (r, b) -> {
                    try {
                        User user = MAPPER.readValue(b, User.class);
                        ref.set(user);
                    } catch (JsonProcessingException e) {
                        throw new UncheckedIOException(e);
                    }
                });
        return ref.get();
    }

    public QueryResult query(String query) {
        AtomicReference<QueryResult> ref = new AtomicReference<>();
        handle(
                "/api/datasources/proxy/1/api/v1/query?query=" + query,
                HttpRequest.Builder::GET,
                HttpResponse.BodyHandlers.ofString(),
                (r, b) -> {
                    try {
                        QueryResult result = MAPPER.readValue(b, QueryResult.class);
                        ref.set(result);
                    } catch (JsonProcessingException e) {
                        throw new UncheckedIOException(e);
                    }
                });
        return ref.get();
    }
}