onnxruntime_go

The library previously used typed Session[T] and DynamicSession[T] structs. These are now considered deprecated.

While they remain in the library for backwards compatibility (delegating to AdvancedSession internally), all new code should use AdvancedSession directly. This provides a more flexible way to manage sessions without unnecessary type parameter association.

To use onnxruntime_go, you must have a Go installation with cgo support and a copy of the onnxruntime shared library (e.g., .so, .dll, or .dylib) corresponding to your OS and architecture.

Before initializing the environment, you must provide the path to the shared library using ort.SetSharedLibraryPath(...). While the library attempts to find onnxruntime.dll on Windows or onnxruntime.so on other systems by default, explicitly setting the path is recommended for stability.

If you are running tests and need to use a specific library (e.g., for CUDA support or on unsupported architectures), set the ONNXRUNTIME_SHARED_LIBRARY_PATH environment variable.

import ort "github.com/yalue/onnxruntime_go"

func main() {
    ort.SetSharedLibraryPath("path/to/onnxruntime.so")
    // ...
}

This library is tied to specific onnxruntime C API header versions. If you need to use a version other than the one currently supported (v1.28.0), follow these steps:

1. Update Headers: Replace onnxruntime_c_api.h and onnxruntime_ep_c_api.h with the versions from your target onnxruntime release.
2. Update Shared Library: Replace the existing shared library files (e.g., test_data/onnxruntime.dll or test_data/onnxruntime*.so) with the version you wish to use.
3. DirectML Support (Optional): If using DirectML, verify that the entries in the DummyOrtDMLAPI struct in onnxruntime_wrapper.c match the order in the official OrtDmlApi struct from dml_provider_factory.h.

Header and library files can be found in the official onnxruntime releases.

The onnxruntime_go package uses specific type constraints to ensure that generic Tensors are initialized with supported ONNX data types. When working with generic Tensor implementations, your data type must satisfy the TensorData interface.

Supported types include:

• Floats: float32, float64
• Integers: int8, uint8, int16, uint16, int32, uint32, int64, uint64
• Booleans: bool

The Session[T] and DynamicSession[in, out] types are deprecated and are maintained only for backward compatibility. They internally delegate to AdvancedSession and DynamicAdvancedSession respectively.

Recommendation:

• If using Session[T], migrate to AdvancedSession.
• If using DynamicSession[in, out], migrate to DynamicAdvancedSession.
• If using NewSessionWithONNXData, migrate to NewAdvancedSessionWithONNXData.
• If using NewDynamicSessionWithONNXData, migrate to NewDynamicAdvancedSessionWithONNXData.

Before using the library, you must initialize the internal ONNX Runtime environment. This manages the core state and memory info. Once finished, you must call DestroyEnvironment to prevent memory leaks.

If you need to specify a custom path for the shared library (onnxruntime.so or onnxruntime.dll), call SetSharedLibraryPath before calling InitializeEnvironment.

import "github.com/yalue/onnxruntime_go"

func main() {
    // Optional: Set custom path to the shared library
    onnxruntime_go.SetSharedLibraryPath("/path/to/onnxruntime.so")

    // Initialize the environment
    err := onnxruntime_go.InitializeEnvironment()
    if err != nil {
        panic(err)
    }
    // Ensure cleanup
    defer onnxruntime_go.DestroyEnvironment()

    // ... use the library ...
}

You can pass EnvironmentOption functions to InitializeEnvironment to configure the runtime during startup. Common options include setting the logging level.

Available logging levels via LoggingLevel:

• LoggingLevelVerbose
• LoggingLevelInfo
• LoggingLevelWarning
• LoggingLevelError (Default)
• LoggingLevelFatal

// Initialize with verbose logging
err := onnxruntime_go.InitializeEnvironment(
    onnxruntime_go.WithLogLevelVerbose(),
)

The onnxruntime training API was deprecated in version 1.20. Consequently, onnxruntime_go has replaced its training wrapper functions with stubs that return an error.

If your project requires the training API, you must use an older version of the library, such as:

• onnxruntime_go version v1.12.1
• onnxruntime version 1.19.2

Support for the ONNX Runtime Training API has been removed from onnxruntime_go following its deprecation in onnxruntime versions 1.19.2 and later. The last version of onnxruntime_go that supported the training API was v1.12.1.

All functions and types related to training (e.g., TrainingSession, TrainStep, OptimizerStep) will return TrainingAPIRemovedError and are no longer functional.

To access model-specific metadata (like producer information or versioning), use the metadata retrieval functions. These also create a temporary session and are computationally expensive.

Important: The returned *ModelMetadata object must be manually freed by calling its Destroy() method to avoid memory leaks.

Available methods:

• GetModelMetadata(path string): Uses default options.
• GetModelMetadataWithOptions(path string, options *SessionOptions): Uses specific SessionOptions.
• GetModelMetadataWithONNXData(data []byte): Extracts metadata from a raw byte slice.

// Example: Getting metadata and ensuring it is destroyed
metadata, err := onnxruntime_go.GetModelMetadata("model.onnx")
if err != nil {
    log.Fatal(err)
}
// Crucial: Call Destroy() when finished
defer metadata.Destroy()

fmt.Println("Metadata retrieved successfully")