Grafana LGTM Dev Service

A service providing a Grafana LGTM observability platform based on OpenTelemetry for development and testing purposes. It consists of:

It works with Spring Boot libraries that support OpenTelemetry, including:

This Dev Service is mutually exclusive with other OpenTelemetry dev services. If more than one is active at the same time, the application will fail at startup with a clear error message. Disable all but one by setting arconia.dev.services.<name>.enabled=false.

Dependencies

First, you need to add the Dev Service dependency to your project.

  • Gradle

  • Maven

dependencies {
  testAndDevelopmentOnly "io.arconia:arconia-dev-services-lgtm"
}
<dependency>
    <groupId>io.arconia</groupId>
    <artifactId>arconia-dev-services-lgtm</artifactId>
    <scope>runtime</scope>
    <optional>true</optional>
</dependency>

You can optionally include the Spring Boot DevTools dependency to enable live reload of your application during development.

  • Gradle

  • Maven

dependencies {
  developmentOnly "org.springframework.boot:spring-boot-devtools"
}
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-devtools</artifactId>
    <scope>runtime</scope>
    <optional>true</optional>
</dependency>

When you use the Spring Boot DevTools in your project and run the application in dev mode, Arconia keeps the Dev Services running while you make changes to your code instead of restarting them with the application. This allows you to see the changes in real-time without having to restart the Dev Services. Changes to the configuration of a Dev Service take effect once you stop and start the application again.

Running the Application

When using the Arconia Dev Services, you can keep running your application as you normally would. The Dev Services will automatically start when you run your application.

  • Gradle

  • Maven

  • CLI

./gradlew bootRun
./mvnw spring-boot:run
arconia dev
Unlike the lower-level Testcontainers support in Spring Boot, Arconia doesn’t require special tasks to run your application when using Dev Services (./gradlew bootTestRun or ./mvnw spring-boot:test-run) nor requires you to define a separate @SpringBootApplication class for configuring Testcontainers.

Your integration tests based on @SpringBootTest will automatically use the Dev Services without any additional configuration.

Test slices (e.g. @DataJdbcTest) load only a subset of the auto-configurations, so they don’t pick up Dev Services automatically. You can import the Dev Service explicitly. For example, for the PostgreSQL Dev Service:

@DataJdbcTest
@AutoConfigureTestDatabase(replace = Replace.NONE)
@Import(PostgresqlDevServicesAutoConfiguration.class)
class CustomerRepositoryTests {
    ...
}

By default, when running the application in dev mode, multiple applications running simultaneously use the same Grafana LGTM Dev Service: the application connects to a running Grafana LGTM Dev Service started by another application if available, or else starts its own that other applications can discover in turn. To always start a container for the application alone, set the arconia.dev.services.lgtm.reuse-strategy property to none. Learn more about Reusing Dev Services.

Accessing Grafana

The application logs will show you the URL where you can access the Grafana UI.

...i.a.d.s.startup  : Dev Service 'lgtm' is ready in container <container-id> (grafana/otel-lgtm, reuse-strategy: framework)
...i.a.d.s.startup  : Dev Service 'lgtm' links: Grafana: http://localhost:<port>, OTLP/HTTP: http://localhost:<port>, OTLP/gRPC: http://localhost:<port>, Prometheus: http://localhost:<port>, Tempo MCP: http://localhost:<port>/api/mcp

The application exports logs, metrics, and traces to the OpenTelemetry Collector of the Dev Service via OTLP, using the HTTP/Protobuf format by default. The collector stores them in Loki, Prometheus, and Tempo, and Grafana comes with a data source for each of them.

Telemetry How it reaches the Dev Service

Traces and logs

Exported by the OpenTelemetry SDK, as configured by Arconia OpenTelemetry.

Metrics instrumented with Micrometer

Exported by the Micrometer OTLP registry included in the Arconia OpenTelemetry starter, or by the OpenTelemetry SDK when the Micrometer Metrics Bridge is enabled instead.

Metrics instrumented with the OpenTelemetry API

Exported by the OpenTelemetry SDK.

Prometheus stores the metrics it receives via OTLP under names translated to its own conventions: for example, jvm.memory.used becomes jvm_memory_used_bytes, and the service.name and service.instance.id resource attributes become the job and instance labels. The Prometheus link in the application logs opens its UI, where you can check the metrics received and their names before using them in a dashboard.

Exploring Telemetry

Besides the dashboards, Grafana offers two ways to look at the telemetry of your application:

  • The "Drilldown" apps in the Grafana menu let you browse metrics, logs, and traces by service without writing any query.

  • The "Explore" page lets you query each data source directly, with PromQL for Prometheus, LogQL for Loki, and TraceQL for Tempo.

The data sources are linked with each other: a log record links to its trace, and a trace links to its logs. Tempo also derives request rates, errors, and durations from the traces, and draws the calls between services as a service graph. These derived metrics appear about half a minute after the first requests reach the application.

Grafana Dashboards

The Dev Service provisions Grafana with dashboards for Spring Boot applications, replacing the ones bundled with the grafana/otel-lgtm image. The dashboards are picked for the application:

Dashboard Content Provisioned

JVM (jvm-micrometer.json)

CPU, system load, memory, garbage collection, threads, classes, and buffers.

When the JVM metrics follow the Micrometer conventions, the Spring Boot default.

JVM (jvm-opentelemetry.json)

Same as above.

When the JVM metrics follow the OpenTelemetry semantic conventions, that is when the Arconia OpenTelemetry Semantic Conventions module is on the classpath and its JVM conventions are enabled.

Logging (logging.json)

Log events by level, and the application logs with links to their traces.

When Logback is on the classpath, the Spring Boot default.

Each dashboard lets you select the application and its instance, listing the most recently started instance first. The "Dashboards" menu next to the selectors switches between them, keeping the selection and the time range.

Contributing Dashboards

Any Grafana dashboard JSON file under META-INF/grafana/dashboards on the classpath is provisioned as well, with no configuration needed. Place your own dashboards in the src/main/resources/META-INF/grafana/dashboards directory of your application.

src/main/resources
└── META-INF
    └── grafana
        └── dashboards
            ├── orders.json
            └── payments.json

A library can ship dashboards the same way, and they are provisioned whenever the library is on the classpath of the application. Each dashboard file needs a unique name: when two files share a name, the first one found is provisioned and the other is ignored with a warning in the logs.

Set arconia.dev.services.lgtm.dashboards.built-in.enabled to false to provision only the contributed dashboards, or arconia.dev.services.lgtm.dashboards.enabled to false to keep the dashboards bundled with the image.

To open one of the provisioned dashboards as the Grafana home page, set arconia.dev.services.lgtm.dashboards.home to its file name.

arconia:
  dev:
    services:
      lgtm:
        dashboards:
          home: orders.json
The dashboards are copied into the container when it starts. When the container is reused across application restarts (see reuse-strategy), changes to a dashboard file take effect the next time the container is started. When the application connects to a Grafana LGTM Dev Service started by another application, the dashboards are the ones provisioned by that application.

Using AI Tools

AI coding assistants can query the telemetry collected by the Dev Service through the Model Context Protocol (MCP).

The MCP server built into Tempo is enabled by the Dev Service and answers TraceQL queries over the traces. Its URL is the Tempo MCP link in the application logs. For example, to add it to Claude Code:

claude mcp add --transport http tempo http://localhost:<port>/api/mcp

The Grafana MCP server gives access to dashboards, metrics (PromQL), logs (LogQL), and traces. It runs on your machine and connects to the Grafana instance of the Dev Service, which accepts the arconia/arconia credentials. For example, to add it to Claude Code (it requires uvx):

claude mcp add grafana \
  -e GRAFANA_URL=http://localhost:<port> \
  -e GRAFANA_USERNAME=arconia \
  -e GRAFANA_PASSWORD=arconia \
  -- uvx mcp-grafana
The ports of the Dev Service are assigned dynamically. To keep the MCP configuration of your AI tool valid across application runs, fix the Grafana and Tempo ports with the arconia.dev.services.lgtm.port and arconia.dev.services.lgtm.tempo-port properties.

Customizing the Stack

The grafana/otel-lgtm image is configured through environment variables and configuration files, which you set with the environment and resources properties of the Dev Service. The most common cases follow. Refer to the image documentation for everything else.

Component Logs

The components of the stack don’t write logs by default, so the logs of a failing container say little. Turn on the logs of each component separately, or all of them with ENABLE_LOGS_ALL.

arconia:
  dev:
    services:
      lgtm:
        environment:
          ENABLE_LOGS_OTELCOL: "true"
          ENABLE_LOGS_GRAFANA: "true"

Component Configuration

Each component accepts additional command line arguments from an environment variable: PROMETHEUS_EXTRA_ARGS, LOKI_EXTRA_ARGS, TEMPO_EXTRA_ARGS, and OTELCOL_EXTRA_ARGS. For deeper changes, replace the configuration file of a component with your own, copied into the container from the classpath or the host filesystem.

arconia:
  dev:
    services:
      lgtm:
        environment:
          PROMETHEUS_EXTRA_ARGS: "--storage.tsdb.retention.time=7d"
        resources:
          - source-path: otelcol-config.yaml
            container-path: /otel-lgtm/otelcol-config.yaml
Component Configuration file in the container

OpenTelemetry Collector

/otel-lgtm/otelcol-config.yaml

Prometheus

/otel-lgtm/prometheus.yaml

Loki

/otel-lgtm/loki-config.yaml

Tempo

/otel-lgtm/tempo-config.yaml
The Dev Service sets TEMPO_EXTRA_ARGS itself to enable the Tempo MCP server. If you set the variable, include --query-frontend.mcp-server.enabled=true in its value to keep the MCP server on.

Forwarding Telemetry to Another Backend

The OpenTelemetry Collector of the Dev Service can forward the telemetry it receives to another OTLP/HTTP endpoint, next to storing it locally, which lets you compare the local stack with a hosted observability platform. The per-signal variables OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, OTEL_EXPORTER_OTLP_METRICS_ENDPOINT, and OTEL_EXPORTER_OTLP_LOGS_ENDPOINT take precedence over the general one.

arconia:
  dev:
    services:
      lgtm:
        environment:
          OTEL_EXPORTER_OTLP_ENDPOINT: https://otlp.example.com
          OTEL_EXPORTER_OTLP_HEADERS: Authorization=Basic ${OTLP_CREDENTIALS}

Keep credentials out of the configuration files you commit, for example by referencing an environment variable as above.

Grafana

Grafana is configured through its GF_* environment variables. For example, to pre-install plugins:

arconia:
  dev:
    services:
      lgtm:
        environment:
          GF_PLUGINS_PREINSTALL: grafana-clock-panel

Configuring the Dev Service

You can configure the Dev Service via configuration properties.

Property Default Description

arconia.dev.services.lgtm.enabled

true

Whether the dev service is enabled.

arconia.dev.services.lgtm.image-name

grafana/otel-lgtm

Full name of the container image used in the dev service.

arconia.dev.services.lgtm.environment

{}

Environment variables to set in the service.

arconia.dev.services.lgtm.network-aliases

[]

Additional hostnames the dev service container is reachable by on the shared network, next to the name of the dev service. Only applicable when the shared network is enabled.

arconia.dev.services.lgtm.port

0

Fixed port for exposing the Grafana port to the host. When it’s 0 (default), a random available port is assigned dynamically.

arconia.dev.services.lgtm.resources

[]

Resources from the classpath or host filesystem to copy into the container. They can be files or directories that will be copied to the specified destination path inside the container at startup and are immutable (read-only).

arconia.dev.services.lgtm.reuse-strategy

framework

Strategy for reusing a running container for the dev service instead of starting a new one. With framework, the dev service is discoverable by other applications, and the application connects to an existing dev service if available. With testcontainers, the container is kept running across application restarts, relying on the Testcontainers reusable containers feature, which must be enabled in the ~/.testcontainers.properties file; such containers are not stopped automatically and must be cleaned up manually. Only applicable in dev mode.

arconia.dev.services.lgtm.startup-timeout

120s

Maximum waiting time for the service to start, counted from the container start.

arconia.dev.services.lgtm.volumes

[]

Files or directories to mount from the host filesystem into the container. They are mounted at the specified destination path inside the container at startup and are mutable (read-write). Changes in either the host or the container will be immediately reflected in the other.

arconia.dev.services.lgtm.otlp-grpc-port

0

Fixed port for exposing the OTLP gRPC port to the host. When it’s 0 (default), a random available port is assigned dynamically.

arconia.dev.services.lgtm.otlp-http-port

0

Fixed port for exposing the OTLP HTTP port to the host. When it’s 0 (default), a random available port is assigned dynamically.

arconia.dev.services.lgtm.loki-port

0

Fixed port for exposing the Loki port to the host. When it’s 0 (default), a random available port is assigned dynamically.

arconia.dev.services.lgtm.tempo-port

0

Fixed port for exposing the Tempo port to the host. When it’s 0 (default), a random available port is assigned dynamically.

arconia.dev.services.lgtm.prometheus-port

0

Fixed port for exposing the Prometheus port to the host. When it’s 0 (default), a random available port is assigned dynamically.

arconia.dev.services.lgtm.dashboards.enabled

true

Whether the dev service provisions Grafana dashboards. When disabled, Grafana keeps the dashboards bundled with the container image.

arconia.dev.services.lgtm.dashboards.home

File name of the provisioned dashboard to open as the Grafana home page, for example, orders.json. By default, Grafana opens its own home page.

arconia.dev.services.lgtm.dashboards.built-in.enabled

true

Whether the dashboards shipped with the dev service are provisioned, next to the ones contributed by the classpath.

You can enable/disable the Dev Service selectively for a specific application mode (development, test), relying on one of the profiles which are automatically configured by Arconia (see Profiles).

You can enable/disable the Dev Service for a specific test class by using the @TestPropertySource annotation or equivalent Spring testing utilities.