Metrics¶

When to Use¶

Metrics combine numeric measurements over time. Use them to monitor throughput, error ratios, latency distributions, resource use, and available capacity. They support dashboards and alerts across many operations without preserving the details of each request.

A meter creates instruments. An instrument defines what a measurement means: an increment, a current value, or a sample in a distribution. Its attributes identify groups of measurements that should be aggregated separately.

Instrument Selection¶

Choose the instrument based on the values the application can supply:

• Supply changes to counters: Counter.add(1) records one additional operation; UpDownCounter.add(-1) records one fewer active operation.
• Supply individual measurements to histograms, such as each request's duration.
• Supply current values to gauges and observable instruments. An observable counter reports a running total, not the increase since its previous callback. An observable up/down counter reports the current size, not a change in size.

An additive quantity can be meaningfully summed across instances, such as memory bytes. A ratio or temperature usually cannot. Use a gauge for non-additive values and an appropriate counter for additive values. Check the installed release for synchronous gauge support before using create_gauge.

Observable Callbacks¶

The SDK invokes observable callbacks when it collects metrics. "Asynchronous instrument" refers to this collection model, not an async def callback awaited by an event loop.

Callbacks return or yield Observation objects. Keep them cheap and thread-safe, and respect CallbackOptions.timeout_millis when doing work that can block. Prefer a cached local value to a network call. Multiple readers may invoke callbacks independently, so observing a value must not change business state.

Count, Duration, and In-Flight Work¶

Create instruments once and reuse them. This wrapper records an existing checkout operation's outcome and duration, including failures. The up/down counter tracks operations that have started but not yet finished. None of these measurements depends on trace sampling.

from collections.abc import Callable
from time import perf_counter

from opentelemetry import metrics

meter = metrics.get_meter("checkout.telemetry")
completed_operations = meter.create_counter("checkout.operations", unit="{operation}")
active_operations = meter.create_up_down_counter("checkout.active", unit="{operation}")
operation_duration = meter.create_histogram("checkout.duration", unit="s")


def run_checkout(process_checkout: Callable[[], None]) -> None:
    started_at = perf_counter()
    outcome = "error"
    active_operations.add(1)
    try:
        process_checkout()
        outcome = "success"
    finally:
        active_operations.add(-1)
        outcome_attributes = {"checkout.outcome": outcome}
        elapsed_seconds = perf_counter() - started_at
        completed_operations.add(1, outcome_attributes)
        operation_duration.record(elapsed_seconds, outcome_attributes)

Use the same attributes for matching up/down increments and decrements so they affect the same series. Here neither call has attributes. The finally block restores the active count when the operation raises; it cannot run after abrupt process termination, which the backend must handle separately.

Measure durations with a monotonic clock and use consistent units, such as s for seconds and By for bytes.

Current-Value Example¶

This observable gauge reports a cached worker utilization ratio when the SDK collects metrics:

from collections.abc import Iterable

from opentelemetry import metrics
from opentelemetry.metrics import CallbackOptions, Observation

meter = metrics.get_meter("checkout.telemetry")
cached_worker_utilization_ratio = 0.5


def observe_worker_utilization(options: CallbackOptions) -> Iterable[Observation]:
    yield Observation(cached_worker_utilization_ratio)


meter.create_observable_gauge(
    "checkout.worker.utilization", callbacks=[observe_worker_utilization], unit="1"
)

The constant illustrates a snapshot; a real application must update and read it safely across threads. The callback accepts collection options but does no blocking work, so it does not use the timeout. For a cached count or size, choose an observable counter or up/down counter rather than copying the gauge pattern.

Dimensions and Cardinality¶

Metric attributes are often called dimensions. Each distinct combination can create a separate time series; the number of combinations is its cardinality. Three operation types and two outcomes can create six series before accounting for services, instances, and histogram buckets.

Prefer attributes with limited value sets, such as route templates, operation types, approved regions, and a small outcome vocabulary. Estimate their combined series cost before adding them.

Do not use request IDs, trace IDs, customer IDs, arbitrary exception text, raw URLs, or timestamps as routine metric attributes. Put permitted request-specific details in logs or spans instead. Resource attributes also affect backend series identity; moving an unrestricted value into the resource does not reduce its cardinality.

For standard infrastructure metrics, follow the applicable semantic conventions for names and units. Give custom business metrics a stable namespace and description. Do not duplicate an integration's HTTP duration metric unless the custom metric intentionally measures different work.

Histograms and Views¶

Histograms group measurements into a distribution. Choose bucket boundaries that distinguish the latency ranges and service-level objective thresholds operators care about. An SDK view customizes how an instrument's measurements are aggregated and which attribute keys are retained. Use a view with ExplicitBucketHistogramAggregation when specific boundaries are required; verify exporter and backend support before choosing exponential histograms.

To calculate a percentile across instances, combine their distributions first. Averaging per-instance percentiles does not produce the fleet percentile. Views can also prevent unwanted dimensions, but dropping an attribute during aggregation is not permission to collect a secret.

Export Intervals and Trace Links¶

Exported aggregates have a time range. Cumulative temporality reports from the start of the aggregation period; delta temporality reports changes over successive intervals. Choose the form supported by the backend and verify how its queries handle process restarts, absent observations, and stale current values.

An exemplar attaches trace and span context to a selected measurement without using those IDs as dimensions. Record the measurement while the relevant span is active, and verify SDK configuration and backend support. An exemplar is an example, not the full aggregate, and its trace may not have been retained.

What to Verify¶

Use InMemoryMetricReader with a MeterProvider local to the test. After known work, collect metrics and inspect sums, histogram counts and sums, attributes, and units. Include successful and failed operations, and confirm that the active count returns to zero.

For observable instruments, repeated collection should report snapshots rather than repeatedly adding the same total. In the backend, verify counter rates, reset handling, and histogram queries against the chosen temporality and aggregation.