OpenTelemetry Go Compile-Time with otelc
OpenTelemetry Go Compile-Time Instrumentation v1 lets you swap go build for otelc go build and get traces, metrics, and log correlation with zero source changes. No wrappers, no SDK init, no agent at startup. A SIG formed by Alibaba and Datadog, with maintainers from Cabify. Vendor-neutral, stable.
TLDR: otelc go build injects OpenTelemetry into your binary at compile time: stdlib, dependencies, and all. It works, it costs build time, and it has gaps. This post covers what ships, what doesn’t, and where it breaks.
What is otelc?
otelc is the Go toolchain wrapper for OpenTelemetry compile-time instrumentation. It sits between go build and the compiler via -toolexec, matches packages and functions against an instrumentation rule set, injects trampolines and hooks, and ships traces, metrics, and log correlation in the resulting binary. No source changes, no runtime agent, no shared library at startup.
Install and build
otelc go build -o myapp .That is it. No annotations, no wrappers, no SDK init in your main.go source. The tool finds what it can instrument by scanning your module graph and rewrites the compiled code to emit spans and metrics on its own.
Now point the binary at a collector. Set the standard OTel env var so the auto-initialized SDK knows where to export, then run the collector with a minimal config:
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc # be explicit; default differs across SDKsexport OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317otelcol --config otelcol-config.yamlreceivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317exporters: debug: verbosity: detailedservice: pipelines: traces: receivers: [otlp] exporters: [debug]Now a service that is nothing but a net/http handler and a database/sql query. Before, a curl localhost:8080/users/42 produces no telemetry. After the one-line build swap, the same curl produces spans shaped like this (representative, trimmed):
{ "resourceSpans": [{ "scopeSpans": [{ "spans": [{ "name": "GET", "kind": "SPAN_KIND_SERVER", "attributes": { "http.request.method": "GET", "url.path": "/users/42", "http.response.status_code": 200, "network.protocol.version": "1.1" } }, { "name": "DB Query", "parentSpanId": "0000000000000000", "attributes": { "db.system.name": "postgresql", "db.query.text": "SELECT name, email FROM users WHERE id = $1", "db.operation.name": "SELECT", "server.address": "db.local" } }] }] }]}The HTTP span carries the route, method, status, protocol: all semantic conventions, no ad-hoc fields.
The SQL span sits alongside the HTTP span because QueryRowContext carries context through the database/sql stack. Whether it becomes a real parent/child link depends on the OTel rules propagating the span context. The runtime hooks are present in the matched set below; I did not decode the JSON excerpt to confirm parent/child in this prototype.
The diff between “before” and “after” for application source is zero; the build-command diff is one line: go build → otelc go build.
To verify, I ran it on a minimal net/http + database/sql (modernc.org/sqlite, in-memory) service with no otel.instrumentation.go file. After the build swap, .otelc-build/matched.json listed the rules that fired:
database/sql— Open, BeginTx, Exec/QueryContext on DB/Conn/Tx/Stmt, Commit/Rollbacknet/http— client RoundTrip, server ServeHTTPlog.Logger.Output— for slog/logrus correlationruntime— goroutine-local-storage vianewproc1init_sdk— file rule injected intomain
Three curls against /users/42 produced three HTTP spans and three SELECT spans with real OTel semantic conventions attribute names (http.request.method, http.response.status_code, url.path, db.query.text, db.system.name, db.operation.name) and a schema URL of https://opentelemetry.io/schemas/1.41.0 on the resource. Numbers from this run appear in the performance section below.
One caveat before you copy-paste: v1.0.0 was retracted the day it shipped because otelc pin wrote broken module paths into go.mod. v1.0.1 fixed it the same day. Pin v1.0.1 explicitly in your Dockerfile and CI; @latest will resolve cleanly today but the retraction is recent enough to be worth dodging on purpose.
How otelc works
The Go toolchain’s go build accepts a -toolexec flag that interposes a wrapper command around every compile and link invocation. otelc is that wrapper: for every package being built, it loads its rule set, finds matching functions in the AST, and rewrites the code before the real compiler runs. The rewritten code is what ships.
Much of the built-in instrumentation uses injected hook points and trampolines (tiny functions injected at the top of instrumented calls), invoking hooks via auto-generated //go:linkname directives (a compiler mechanism that lets the injected code call into unexported OTel internals without a public import). The trampoline isolates panics: your app keeps running if a hook blows up. It also gives every hook a uniform HookContext API regardless of the target function’s signature. The rule engine can also rewrite call sites, append arguments, inject struct fields, and expand annotated declarations; see docs/rules.md for the full set of transforms. At runtime, the only cost is whatever the rule injected plus whatever the hook itself does. No agent attaches at startup, no shared library gets loaded.
otelc commands
otelc go build # instrument + build (default)otelc go install # instrument + installotelc go test # instrument + testotelc setup # generate .otelc-build/ artifacts onceotelc pin # write otel.instrumentation.go (local workflow)otelc cleanup # remove the .otelc-build/ directoryInvocation modes
Three invocation modes exist: otelc go build, a GOFLAGS export after otelc setup, and a Dockerfile swap. otelc go build is the one to reach for by default. The Dockerfile version is the same line inside a build stage:
FROM golang:1.25 AS buildRUN go install go.opentelemetry.io/otelc/tool/cmd/[email protected]COPY . /srcWORKDIR /srcRUN otelc go build -o /out/myapp .What otelc instruments in v1
Coverage is the point. v1 ships instrumentation for the libraries you use, split across four clusters.
HTTP and RPC
net/http (client and server) and google.golang.org/grpc (client and server) are pre-instrumented. Add github.com/gin-gonic/gin and you get the same server-side spans without touching your handlers.
Data stores
database/sql covers the bulk of Go services. github.com/redis/go-redis/v9 and go.mongodb.org/mongo-driver/mongo extend that to the common NoSQL pair.
Messaging and AI
github.com/segmentio/kafka-go handles the Kafka client most new Go services reach for. github.com/openai/openai-go is a signal of where the SIG thinks the audience is going.
Kubernetes and logs
k8s.io/client-go informers cover controller workloads. log/slog and sirupsen/logrus are instrumented for log records (per the supported-libraries page). otelc can inject trace context into those records, making trace–log correlation possible without manually wiring every logger. Whether logs reach an OTel backend is a separate question and depends on the logger, handler, SDK configuration, and the export pipeline you choose.
What otelc does not instrument in v1
Echo, Fiber, gorilla/mux, Hertz. pgx, sqlx, gorm. Sarama, Franz-Kafka. The Anthropic SDK. This list is partial; anything outside the supported-libraries page needs an answer of its own. If you depend on one of these, you either write your own rule or look sideways at the vendor-backed predecessor and parallel implementations. Alibaba still ships Loongsuite-go with dozens of supported libraries including echo, fiber, gorm, sqlx, sarama, kratos, and kitex. Datadog still ships Orchestrion. The OTel SIG was formed by contributors from Alibaba, Datadog, and Quesma to unify approaches; those projects are not simple forks of this repository. They keep shipping in parallel. Read that however you want; my read is “coverage gaps are real, and the vendors kept their escape hatches.”
Performance: what the benchmarks don’t show
The official claim is “no added runtime overhead.” That phrasing specifically means no runtime agent and no attach step at startup (both true, and worth having). There is still a per-request cost: the injected hooks, span creation, metrics recording, context propagation, and attribute extraction all consume CPU and may allocate. The project has not yet published request-path benchmarks that characterize this, so for a high-traffic hot path, measure on your own code before trusting any “zero overhead” framing. From a single observed run, treat the SDK init time of sub-five-millisecond as a floor, not a guarantee (trace/meter/logger providers log their init lines on startup, all synchronous, none blocking on the exporter in my test setup).
Build-time overhead is where otelc shows its cost. -toolexec places otelc in the compiler path for package builds, so even packages without matching instrumentation can contribute per-package overhead. The project’s own Makefile exposes BENCH_MAX_OVERHEAD_PCT=150 to a benchmark/threshold Go test that fails the build if overhead exceeds 150 %, a guardrail ceiling the project enforces on itself (not a measurement of typical overhead). A clean per-package floor is harder to find. Here is what I measured on the one-package prototype (net/http + database/sql + modernc.org/sqlite, warm Go cache, three runs each):
Plain go build | otelc go build | Delta | |
|---|---|---|---|
| Median | 0.39 s | 12.17 s | +11.78 s (+3051 %) |
| Min / Max | 0.38 / 0.44 s | 11.99 / 12.39 s | — |
| Binary size | 14.1 MB | 31.2 MB | +17.1 MB (+121 %) |

The 3051 % number is what to expect when the package count is small: you are paying the SDK + OTel exporter dependency load plus per-package toolexec process spawn, almost none of which amortizes. On a real codebase with hundreds of packages, the SDK download amortizes to zero and the per-package spawn cost dominates, which is where the project’s 150 % CI ceiling lives. Issue #570 is still open and there is no public runtime SLO. For a build-critical CI path, measure on a representative commit before wiring otelc in.
Two issues the announcement did not address.
Metric cardinality. Auto-instrumentation emits more telemetry and more attributes than you may expect. Audit metric labels separately from span attributes:
- Normalize routes so unparameterized paths don’t blow up label series
- Drop high-cardinality labels in the SDK or Collector
- Configure aggregation or filtering before exporting
- Use
OTEL_GO_DISABLED_INSTRUMENTATIONSto turn off specific auto-instrumented sources
Trace sampling will reduce trace volume. It does not solve metric-series cardinality.
SDK init at startup. In my run the init was fast, but otelc does inject runtime-metric collection and an OTel pipeline initialization. Nothing in the documented behavior says slow DNS resolution would block your main() from returning; OTLP exporters batch and connect asynchronously. Do not pre-emptively redesign for this; measure on your own startup path first. Expect benign processor export timeout warnings on graceful shutdown if the collector connection drops before the final flush. The prior batch has already left the process by then.
A v1 semver release is not the same as production-proven. The official docs page still carries a NOTE that the project “is under active development and is not yet ready for production use.” The table below uses the docs wording, because that note is operationally honest.
Limitations you will hit
-
Which
gosubcommands are wrapped. The documented and tested entry points areotelc go build,otelc go install, andotelc go test. Any subcommand that drives the compiler through-toolexeccan be instrumented in principle, but the tested surface is those three. For a fast dev loop where you do not want instrumentation, theGOFLAGSmode afterotelc setupkeeps plaingo buildworking. -
Generics may expose rule limitations. Generic functions can hit rule and hook gaps, especially when instrumentation needs to inspect or replace typed parameters and return values. Type parameters are not always available in the injected code, and the public docs do not enumerate every edge case. If your database layer or hot paths are generic, test those paths explicitly before adopting. Auto-instrumentation there may be partial.
-
Restricted hook imports (for packaged rules). A packaged hook for a target like
github.com/foo/baris generally limited to importing the target library, the OTel packages, and the standard library. Any other third-party import breaks the build graph. This constraint applies to packaged instrumentation rules; if you write your own rules, the rule engine can declare additional imports as long as the target module resolves them (seedocs/instrument-guide.md). If you need custom marshalling or a proprietary logger inside a packaged hook, inline it or use a side-channel. -
Migration and coexistence. If you already instrument manually with
otelhttpor the OTel SDK, you can compose manual and zero-code instrumentation (the v1 announcement explicitly supports this). The risk is double-counting when your manual code and a compile-time rule both wrap the same call. Audit overlap before enabling a built-in rule: keep manual domain spans, and either disable the overlappingotelcinstrumentation for that surface (OTEL_GO_DISABLED_INSTRUMENTATIONScovers runtime metrics and adjacent controls; per-rule disable exists for built-ins) or remove duplicate library wrappers. This is the first question teams with existing instrumentation ask, and it is worth a dedicated migration note in your own docs. -
Go version and toolchain. The
otelcmodule requires Go 1.25.0+ (per itsgo.mod), sogo install go.opentelemetry.io/otelc/tool/cmd/otelc@latestneeds a recent toolchain. Two platform questions get confused: whereotelcitself builds (theMakefilePLATFORMSvariable lists darwin/linux amd64/arm64 plus windows amd64 for the tool binary), and whatGOOS/GOARCHtargets you can instrument (which is a separate question). Using a hostotelcbinary to instrument arbitrary cross-compile targets should be tested rather than inferred from the tool’s release matrix.go mod vendoris supported since v1 (it broke setup previously). -
First-build troubleshooting. Private VCS,
GOTOOLCHAIN=automismatches, and a stale module cache are the usual suspects when the firstotelc go buildfails. On a fresh cacheotelcalso pulls in a long tail of OTel exporter modules (seventeen direct deps on a small app), so a low-quota sandbox can run out of disk on the first build.otelc’s state snapshotting (track backup files) writes to the module directory. Runotelc setupagain after anygo.modchange, and clear yourGOCACHEif the build drifts. -
-toolexecis brittle. Romain Muller, a Datadog engineer who worked on the SIG, opened golang/go#69887 listing the mechanism’s limits: influence on object build IDs, help-output parsing, past bugs with packages likenats.go. v1 works around these with a job server andpackages.Load, but the upstream complaint that-toolexecis brittle remains valid. The proposal is still open.
otelc vs otelhttp vs OBI
The three Go OpenTelemetry paths cover different constraints. The decision table below is the short version.
| Criterion | otelc (compile-time) | OBI (eBPF) | Manual SDK (otelhttp) |
|---|---|---|---|
| Zero-code | Yes | Yes | No |
| Runtime cost | In-process hooks and SDK work | Out-of-process eBPF plus in-process SDK where applicable | In-process SDK under developer control |
| Build-time overhead | Significant | None | None |
| Stdlib coverage | Yes | Partial | No |
| Third-party coverage | Limited (see v1 list) | Broad (network protocols) | No |
| Custom domain spans | Via rules | Limited | Yes, easy |
| No rebuild possible | N/A | Yes (runtime attach) | No |
| Polyglot fleet | No | Yes (multi-language) | No |
| Production-ready | v1.0.1 — docs say not production-ready | OBI v0.10.0 (Development) | Stable |
Compile-time wins for teams that can rebuild and want zero-code coverage of stdlib and dependencies. If you cannot rebuild, or you run a polyglot fleet, eBPF is the right call on principle. Many internal services can be rebuilt more easily than teams initially assume, but signed, vendor-supplied, legacy, or tightly regulated binaries are real exceptions.
Verdict
I would pilot v1.0.1 on a non-critical service this week, with build-time and request-path benchmarks before broader rollout. The one-line build swap, the auto-context-propagation, and the slog correlation are worth the build-time tax for anything that is not on your build-critical CI path. Hold until there is a public runtime SLO and the build overhead is documented per-codebase-size rather than per-CI-ceiling. If a 90-second build becoming four minutes means a deploy SLA breach, this is not the tool for that path today.
Run otelc go build on one service. Measure the build delta, the binary size, and whether the spans carry the attributes you expect. Email me what you measure; I’m collecting data points ahead of a future comparison, and real numbers beat vendor benchmarks. On a small Go service my numbers were +3051 % build time and +121 % binary size; the spread on a real monorepo is what I want to see.
FAQ
What is otelc?
otelc is the Go toolchain wrapper for OpenTelemetry compile-time instrumentation. It scans your module graph, injects hooks and trampolines into matches via -toolexec, and ships traces, metrics, and log correlation in the resulting binary — no source changes, no runtime agent.
Is otelc production-ready?
I would pilot, not ship. v1.0.1 is a stable semver release, but the official docs page still carries a NOTE that the project is “not yet ready for production use.” The comparison table above reflects that wording.
How much does otelc slow down the Go build?
On my one-package prototype (net/http + database/sql + modernc.org/sqlite, warm cache, three runs), median build time went from 0.39 s to 12.17 s — a +3051 % delta. On a real codebase the SDK download amortizes to zero and the per-package toolexec cost dominates; the project’s own threshold ceiling is BENCH_MAX_OVERHEAD_PCT=150 in the Makefile.
otelc vs otelhttp vs OBI — when to choose which?
otelc wins when you can rebuild and want zero-code coverage of stdlib and dependencies. OBI (eBPF) wins when you cannot rebuild or you run a polyglot fleet. Manual SDK with otelhttp wins when you need full control over domain spans and are willing to maintain the wrapper code. The comparison table above covers the trade-offs in detail.
References
- Announcing v1 — OpenTelemetry blog
- SIG formation: Alibaba, Datadog, Quesma
- opentelemetry-go-compile-instrumentation (GitHub)
- Supported libraries
- Instrumentation Rules (
docs/rules.md) - Instrumentation guide (
docs/instrument-guide.md) - Benchmarking (
docs/benchmarking.md) - golang/go#69887 —
-toolexeclimits - v1.0.1 release notes
- Related on this blog: context propagation and cancellation in Go, the Go concurrency patterns reading order
Related Posts
Go Concurrency Patterns: A Practical Reading Order
A guided reading order through the Go concurrency patterns that compound: producer-consumer, generator, worker pool, pipeline, semaphore, context, and real-world examples. The shortest path to writing concurrent Go you can reason about.
Context and Cancellation in Go: Stopping Work That Shouldn't Have Started
Go context cancellation without the fluff: timeouts, request lifecycles, errgroup fan-out, and graceful shutdown — no leaked goroutines.
Go Goroutines vs C# async/await: Who Carries the Cognitive Load?
Goroutines and async/await solve the same problem from opposite assumptions. A polyglot's take on primitives vs compiler-managed patterns.