What is Orchestrion?
main-toolexec feature to intercept and modify compilation units (including application code, dependencies, and the Go standard library) before they are compiled or linked.repository·main·Indexed 20 days ago
https://github.com/datadog/orchestrionA tool for automatic compile-time instrumentation of Go applications. Orchestrion leverages the Go toolchain's -toolexec feature to automatically insert tracing and observability logic into application code, dependencies, and the Go standard library without manual boilerplate. It is vendor-agnostic, supporting providers like Datadog and OpenTelemetry via configuration in a specialized orchestrion.tool.go file.
-toolexec feature to intercept and modify compilation units (including application code, dependencies, and the Go standard library) before they are compiled or linked.Orchestrion is a tool for automatic compile-time instrumentation of Go code. It processes Go source code during the compilation phase and automatically inserts instrumentation logic.
The instrumentation is driven by the specific imports present in a file named orchestrion.tool.go located at the project's root. By including specific package paths in this file, you signal to Orchestrion which integrations should be automatically applied to your codebase.
Orchestrion intercepts the standard Go toolchain build process using the -toolexec flag. It specifically targets two toolchain invocations:
go tool compile (to instrument .go source files during compilation).go tool link (to manage link-time dependencies introduced by instrumentation).To maintain build efficiency and correctness, Orchestrion uses a job server (based on the NATS protocol) that persists for the duration of the build. This job server ensures:
golang.org/x/tools/go/packages, are centralized and performed only once per package.Orchestrion uses a process similar to Aspect-oriented Programming (AoP) to inject code. It combines:
To optimize performance, Orchestrion uses heuristics to avoid evaluating all 100+ available aspects (from dd-trace-go.v1) against every file. It filters aspects based on:
net/http) is in the package's dependency tree.//dd:span) before attempting to match an aspect.The injector performs a depth-first traversal of the Abstract Syntax Trees (ASTs), evaluating applicable join points and applying advice where they match.
Orchestrion processes configuration by recursively traversing the imports defined in orchestrion.tool.go.
github.com/DataDog/dd-trace-go/orchestrion/all will transitively enable packages like ddtrace/tracer, contrib/net/http, and contrib/database/sql.orchestrion.yml file. These YAML files act as the backbone for auto-instrumentation, defining how the codebase is modified.Orchestrion influences the Go toolchain's build cache by appending metadata to the output of intercepted -V=full invocations. This ensures that if Orchestrion's configuration or the injected dependencies change, the Go toolchain recognizes the build as different and invalidates the cache.
The appended version string follows this format:
compile version go1.23.6:orchestrion@v1.1.0-rc.1;<base64-encoded-hash>
The <base64-encoded-hash> is composed of:
packages.Load).Note: This approach results in more cache invalidations than strictly necessary because the Go toolchain currently lacks a more granular way to influence build identifiers.
compile version go1.23.6:orchestrion@v1.1.0-rc.1;<base64-encoded-hash>Orchestrion provides several capabilities for managing observability in Go applications:
-toolexec to instrument not just your application code, but also dependencies and the Go standard library.When the Go toolchain invokes go tool compile, Orchestrion performs the following steps for each package:
.go files and applies configured integrations using github.com/dave/dst to decorate the AST.//line pragmas to preserve original line information.-importcfg file to provide archives for these new dependencies.main packages, a synthetic source file is created containing import statements for all recorded link-time dependencies to ensure func init() functions are correctly registered.go tool compile with the modified source files and updated -importcfg.go tool pack to add a link.deps file to the produced .a archive, listing all implied link-time dependencies.In Orchestrion, compile-time integrations are modeled using a concept called aspects. An aspect is the combination of two components:
To ensure traces are not split across goroutine boundaries, you must propagate the trace context. //dd:span annotated functions handle context propagation based on their arguments:
context.Context: If the function accepts a context.Context argument, that context is used for trace propagation.*http.Request: If the function accepts a *http.Request argument, the request's context is used.To weave context into a child goroutine, pass the context.Context through the function call.
package demo
//dd:span
func caller(ctx context.Context) {
wait := make(chan struct{}, 1)
defer close(wait)
// Weaving the span context into the child goroutine by passing ctx
go callee(ctx, wait)
<-wait
}
//dd:span
func callee(ctx context.Context, done chan<- struct{}) {
done <- struct{}{}
}You can influence the observability data produced by Orchestrion by adding special directives directly in your Go source code.
Common directives include:
//orchestrion:ignore: Used to skip instrumentation for specific code blocks.//dd:span custom-tag:value: Used to add custom tags to spans.// Example of using a directive
//dd:span custom-tag:value
func MyFunction() {
// ...
}Because Orchestrion wraps many short-lived processes via -toolexec, it uses a persistent job server (communicating via the NATS protocol) to share state across the entire build.
Key responsibilities of the job server include:
compile and link phases.compile task results to prevent re-instrumenting and re-compiling packages that are shared between the original build and injected dependencies.