Profiles¶
When to Use¶
Profiles show which code consumes resources. Use them when metrics reveal high CPU or memory use, or a trace reveals a slow operation, but you need to identify the functions or allocations responsible. Adding a span around every function is not a substitute for profiling.
Choose a profile type that measures the resource or wait you want to investigate:
| Profile type | Question | Interpretation constraint |
|---|---|---|
| On-CPU | Which functions consume processor time? | CPU samples do not measure all wall-clock waiting |
| Off-CPU | Where do threads wait or block? | Requires profiler/runtime support for the relevant wait type |
| Heap | Where was memory still in use allocated? | Memory still in use differs from all memory allocated over time |
| Allocations | Which code allocates most? | Frequent allocations do not by themselves prove a leak |
OpenTelemetry Status and Implementation¶
The OTel profiles signal is currently alpha. Collection is provided by a profiler or agent, not by a stable Python SDK instrument comparable to a counter or tracer. Do not assume a stable ProfileProvider or a standard Python OTLP profile exporter exists.
For continuous profiling on Linux, evaluate the official OpenTelemetry eBPF profiler. Check support for the exact Python runtime, system architecture, and deployment. Verify that it can resolve stack samples to useful function names, and review its required permissions and overhead.
The profiler, Collector distribution, profile pipeline, and backend must support compatible profile formats. An OTLP endpoint that accepts traces does not necessarily accept profiles. Available profile types also depend on the profiler and runtime.
Local Python Diagnostics¶
The standard library provides a stable alternative for local investigation. These commands profile an existing checkout.py script and open its results; they do not send OTel telemetry:
Use cProfile to examine function calls and execution times. Use tracemalloc for snapshots of Python memory allocations; it does not capture every allocation made by native code. Neither tool automatically exports OTLP profiles.
Interpret async workloads carefully. Elapsed time may include waiting, and several tasks may overlap. Choose a profiler whose measurements distinguish the work you need to understand rather than treating every reported duration as CPU use.
Correlation and Safety¶
Use service and instance identifiers consistent with the application's other telemetry, and ensure timestamps can be compared. Linking profile samples to a particular trace or span requires explicit support from the profiler and runtime integration. Enabling tracing alone does not establish that link.
Profiles may expose source paths, function names, and other internal state. Limit access and retention. Measure CPU, memory, and latency overhead on representative traffic before enabling continuous collection broadly. Use a time-limited diagnostic session when permissions or overhead make continuous profiling unsuitable.
What to Verify¶
Run work with a known CPU or allocation cost. Verify that the expected functions are visible, stack samples resolve to useful names, service identity is correct, and overhead is acceptable. If trace correlation is required, demonstrate a link from a retained span to relevant samples.
Distinguish the result clearly: a local profile file proves local collection, while an OTel integration also needs evidence that the backend received and indexed the profile.