DOCSEARCHbyAlgolia
Submit Docs

Apache SkyWalking Go Documentation

repository·main·Indexed 18 days ago

https://github.com/apache/skywalking-go

An auto-instrumentation agent for Golang that provides native tracing, metrics, and logging capabilities for Go-based applications. It integrates with the Apache SkyWalking observability platform and supports distributed tracing via hybrid compilation using the -toolexec parameter. Features include a Kafka reporter, TLS/mTLS configuration for gRPC, support for Logrus and Zap logging frameworks, and manual instrumentation toolkits for custom metrics (Counter, Gauge, Histogram), logging, and tracing.

Tokens
67.3K
Snippets
208
Records
306
Agent score
57%

What's inside Apache SkyWalking Go

  1. Overview of Apache SkyWalking Go

    main
    Apache SkyWalking Go is an auto-instrumentation agent designed for Golang projects. It provides native capabilities for tracing, metrics, and logging, allowing developers to observe their Go applications within the Apache SkyWalking ecosystem without manual instrumentation of every component.

  2. Understand the Apache SkyWalking Go project structure

    main

    The Apache SkyWalking Go project is organized into several functional directories that separate the agent core, plugins, and the enhancement tools used during compilation:

    Agent and Plugins

    • agent: Contains the core files of the agent that are copied during hybrid compilation.
    • plugins: Contains framework-specific adapters.
      • core: The central module containing the Agent core and APIs. All plugins must import this module.
      • core/reporter: Handles communication with the SkyWalking backend.
      • xxx: Individual plugin directories for specific frameworks (e.g., Gin, Echo, etc.).
    • bin: Contains the compiled binary files of the Go agent.
    • log: Contains log configurations for the agent.

    Enhancement Tools (tools/go-agent)

    The tools/go-agent directory contains the enhancement program used to instrument code during the build process:

    • cmd: The agent starter.
    • config: Application registration configurations.
    • instrument: The core logic for performing code enhancement during hybrid compilation. It includes specific modules for:
      • agentcore: Enhancing SkyWalking Go code for agent core file copying.
      • api: The API for the instrumentation process.
      • entry: Enhancing the main package to ensure the Agent system starts automatically.
      • plugins: Enhancing detected frameworks (see Key Principle document for details).
      • reporter: Enhancing the reporter package to manage backend communication startup.
      • runtime: Enhancing the runtime package (see Key Principle document for details).
    • tools: Utilities to assist in building the agent.

  3. Use environment variables to override agent settings at runtime

    main

    Most configuration items in the agent follow the pattern ${xxx:config_value}. This allows you to override specific settings using system environment variables without recompiling the application.

    At runtime, the agent checks for an environment variable named xxx. If the variable is present, its value is used; if the variable is not found in the system environment, the agent falls back to the provided config_value.

    Important: Environment variable resolution happens at runtime, whereas configuration file changes applied via hybrid compilation happen at compile time.

  4. How Plugins communicate with the Agent Core

    main

    Since plugins and the Agent Core are enhanced separately, they cannot share complex custom types directly (as types would be duplicated and become inconsistent). Instead, they communicate via a global object in the runtime package.

    Communication Pattern

    1. Global Object: The Agent Core defines a global variable in the runtime package with set/get methods.
    2. Initialization: When the Agent Core is loaded, it initializes this global variable.
    3. Plugin Access: Plugins import the methods required to read the global variable and access the Agent Core's APIs.

    Data Transfer Limitation

    Because communication happens through an interface, plugins can only transfer:

    • Basic data types
    • any (interface{}) types

    When passing structured data, you must pass it as an any type and perform a type cast once the data is received by the Agent Core or the plugin.

  5. Define expected data in config/expected.yml

    main
    To validate that a plugin is working correctly, you must define an expected.yml file located in the config/ directory of your plugin module. This file contains the observable data (traces, metrics, etc.) that the plugin is expected to generate. The test runner compares the actual data produced during execution against this file to determine if the test passed.

  6. How Method Interception works in SkyWalking Go

    main

    SkyWalking Go uses method interception to enable plugin functionality via compile-time enhancement. The process follows three main stages:

    1. Finding Method: The Agent uses AST (Abstract Syntax Tree) parsing to scan .go files within target packages (identified by package information in compilation arguments) to find methods that match plugin interception requirements.
    2. Modifying Methods: The Agent modifies the target method's body to embed template code. This code executes at the same line as the first statement in the method to ensure that if an exception occurs in the original framework code, the exact location is still traceable. The embedded code includes:
      • Before execution: Passes arguments and instance information to the interceptor.
      • After execution: Uses defer to intercept result parameters after completion.
    3. Saving and Compiling: The Agent writes a delegator file containing the implementation of the before/after methods, copies necessary plugin and API code, and updates the compilation arguments to include these modified files.

  7. Understand Logging Functionalities in SkyWalking Go Agent

    main

    The logging plugin provides three core capabilities to integrate application and agent logs with SkyWalking:

    1. Agent Log Adaptation: Automatically detects the system's logging framework and integrates agent logs into it.
    2. Distributed Tracing Enhancement: Links distributed tracing information with application logs, enabling real-time visibility into logs associated with specific requests.
    3. Log Reporting: Sends both application and agent logs to the SkyWalking backend for centralized retrieval and display.

  8. Integrate Tracing information into service logs

    main

    You can automatically inject distributed tracing context into your application's logs. This allows you to correlate log entries with specific traces.

    When a goroutine contains tracing data, the agent appends a context string to the log. The format is: [${ServiceName},${ServiceInstanceName},${TraceID},${SegmentID},${SpanID}].

    If no link is present, TraceID and SegmentID output as N/A, and SpanID outputs as -1.

    {"level":"info","ts":1683641507.052247,"caller":"gin/main.go:45","msg":"test log","SW_CTX":"[Your_ApplicationName,681e4178ee7311ed864facde48001122@192.168.50.193,6f13069eee7311ed864facde48001122,6f13070cee7311ed864facde48001122,0]"}

  9. How Context Carriers work for distributed tracing

    main

    Context carriers allow you to pass tracing context between different applications.

    • Entry Spans: Use an ExtractorRef to pull the context from an incoming request (e.g., HTTP headers).
    • Exit Spans: Use an InjectorRef to write the context into an outgoing request (e.g., RPC or HTTP headers).

    Defined types:

    • type ExtractorRef func(headerKey string) (string, error)
    • type InjectorRef func(headerKey, headerValue string) error
    // create a new entry span and extract the context carrier from the request
trace.CreateEntrySpan("EntrySpan", func(headerKey string) (string, error) {
    return request.Header.Get(headerKey), nil
})

// create a new exit span and inject the context carrier into the request
trace.CreateExitSpan("ExitSpan", request.Host, func(headerKey, headerValue string) error {
	request.Header.Add(headerKey, headerValue)
	return nil
})

  10. How Hybrid Compilation works in SkyWalking Go

    main

    SkyWalking Go uses a technique called Hybrid Compilation to implement its agent functionality. Instead of traditional manual instrumentation, it leverages the Go toolchain's -toolexec flag to intercept the compilation process.

    The Mechanism

    When you run a Go command (like go build or go test) with the -toolexec flag, the Go compiler executes the specified program instead of the default toolchain commands (such as compile, asm, or link).

    SkyWalking Go Agent acts as this custom program. It intercepts the compile command to:

    1. Parse and manipulate code using AST (Abstract Syntax Tree).
    2. Generate/Copy files into the Go compiler's temporary directory.
    3. Proxy command execution to weave the modified code into the final target.

    Enhanced Components

    Through this process, the agent enhances four key areas of your application:

    • SkyWalking Go: The core agent code is dynamically copied for plugin usage.
    • Plugins: Framework-specific code is enhanced based on plugin rules.
    • Runtime: The Go runtime package is enhanced (e.g., extensions for goroutines).
    • Main: The main package is enhanced to ensure the system starts with the Agent active.

  11. How Context Propagation works in SkyWalking Go

    main

    SkyWalking Go uses an internal mechanism for context propagation (e.g., tracing context) instead of relying on the standard Go context.Context. This allows for tracing without requiring changes to the target application's method signatures.

    Context Propagation between Methods

    The Agent enhances the g structure in the runtime package (which represents internal goroutine data in Golang).

    • Mechanism: A new interface{} field is added to the g struct.
    • Access: The Agent uses go:linkname to export methods for real-time setting and getting of custom field values. This provides a shared context within a single goroutine, similar to Java's Thread Local.

    Context Propagation between Goroutines

    To maintain traces when new goroutines are spawned, the Agent intercepts the runtime.newproc1 method.

    • Mechanism: When a new goroutine is created, the Agent performs a context-copy from the parent goroutine to the new one.
    • Implementation: The Agent inserts defer code to intercept g objects before and after execution, calling a copy method to assign values to the custom fields in the new goroutine.

  12. Module import requirements for plugins

    main

    When developing a plugin, you are restricted to importing only two types of modules. Importing any other modules may cause compilation errors for the end-user.

    1. Agent core: Provides all required dependencies, including the plugin API and enhancement declaration objects. Use the relative path to github.com/apache/skywalking-go/plugins/core within the repository.
    2. Framework to be enhanced: The specific library or framework (e.g., Gin, Echo, Redis) that your plugin is designed to instrument.

    Warning: Do not import any other modules. If your plugin requires additional tools, they should be requested as additions to the agent core.