hashicorp/go-plugin

The go-plugin system uses RPC (Remote Procedure Call) over the local network to facilitate communication between a host application and a plugin. This architecture provides two primary benefits:

1. Isolation: Because plugins run as separate processes, a plugin crash will not crash the host application.
2. Language Agnostic: Plugins can be implemented in any language that supports the chosen RPC protocol (e.g., gRPC), not just Go.

The system consists of two main components:

• Server: The plugin process itself. It implements a specific interface and serves that implementation via RPC.
• Client: The host application. It connects to the server running on localhost (typically on a random high-numbered port), invokes methods on the interface, and receives the responses.

go-plugin requires the gRPC Health Checking Service to be registered on your server. You must explicitly set the status of the component named "plugin" to SERVING. Failure to do this may cause the host to abruptly restart your plugin process.

health = HealthServicer()
health.set("plugin", health_pb2.HealthCheckResponse.ServingStatus.Value('SERVING'))
health_pb2_grpc.add_HealthServicer_to_server(health, server)

Create and start a gRPC server to host your service implementation. You can listen on a TCP address or a Unix domain socket. Note that go-plugin assumes connections are local and reliable; do not serve plugins across a network.

Example using Python:

# Make the server
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))

# Add our service
kv_pb2_grpc.add_KVServicer_to_server(KVServicer(), server)

# Listen on a port
server.add_insecure_port(':1234')

# Start
server.start()

To implement a plugin using gRPC, you must first define your service and message structures in a .proto file. This file serves as the contract between the host (client) and the plugin (server).

syntax = "proto3";
package proto;

message GreetResponse {
    string message = 1;
}

message Empty {}

service GreeterService {
    rpc Greet(Empty) returns (GreetResponse);
}

To use the go-plugin system, follow these five high-level steps:

1. Define Interfaces: Choose the Go interfaces you want to expose for plugins.
2. Implement Client and Server: For each interface, create both a client and a server implementation that communicates via net/rpc or gRPC (or both).
3. Create a Plugin Implementation: Implement the plugin.Plugin interface, which defines how to create the RPC client/server for your specific plugin type.
4. Serve the Plugin (Plugin Author): In the plugin's main function, call plugin.Serve to start the RPC server.
5. Consume the Plugin (Plugin User): Use plugin.Client to launch the plugin subprocess and request the interface implementation over RPC.

Detailed examples are available in the examples/ directory of the repository.

If you are building a plugin in a language other than Go, you must implement a handshake so the go-plugin host can connect to your process. The handshake must be sent as a single line of data to stdout followed by a newline character (\n).

The handshake format is:

CORE-PROTOCOL-VERSION|APP-PROTOCOL-VERSION|NETWORK-TYPE|NETWORK-ADDR|PROTOCOL

Fields:

• CORE-PROTOCOL-VERSION: Must be 1.
• APP-PROTOCOL-VERSION: The version of the application data protocol (defined by your specific application).
• NETWORK-TYPE: Either unix or tcp.
• NETWORK-ADDR: The path to the Unix socket (for unix) or an IP address (for tcp).
• PROTOCOL: The communication protocol, such as grpc or netrpc (default for older versions).

1|3|unix|/path/to/socket|grpc

Once a client is initialized, follow these steps to access the plugin's functionality:

1. Create an RPC client using client.Client().
2. Use rpcClient.Dispense(name) to retrieve the plugin instance from the plugin map. This returns a raw interface.
3. Type-assert the raw interface into your local interface type to call its methods.

// Connect via RPC
rpcClient, err := client.Client()
if err != nil {
	log.Fatal(err)
}

// Request the plugin
raw, err := rpcClient.Dispense("greeter")
if err != nil {
	log.Fatal(err)
}

// Type assert the raw client into your interface
greeter := raw.(shared.Greeter)
fmt.Println(greeter.Greet())

Use the protoc compiler to generate the Go stubs required for both the client and the server. These stubs must be placed in a shared location accessible to both the host application and the plugin implementation.

protoc -I proto/ proto/greeter.proto --go_out=plugins=grpc:proto

To write a plugin in a language other than Go, you must implement a standard gRPC server for the protocol buffers service defined by the host application. For example, if the host defines a KV service, your implementation must provide the Get and Put RPC methods as defined in the .proto file.

class KVServicer(kv_pb2_grpc.KVServicer):
    """Implementation of KV service."""

    def Get(self, request, context):
        filename = "kv_" + request.key
        with open(filename, 'r') as f:
            result = kv_pb2.GetResponse()
            result.value = f.read()
            return result

    def Put(self, request, context):
        filename = "kv_" + request.key
        value = "{0}\n\nWritten from plugin-python".format(request.value)
        with open(filename, 'w') as f:
            f.write(value)

        return kv_pb2.Empty()