DOCSEARCHbyAlgolia
Submit Docs

Polaris Documentation

website·Indexed 30 days ago

https://harshadmanglani.github.io/polaris

A Go library for creating, storing, and running workflows. Polaris manages the sequence of steps and state recovery, allowing developers to define topologies of connected builders that consume and produce data.

Tokens
2.4K
Snippets
13
Records
16
Agent score
75%

What's inside Polaris

  1. Understand Polaris workflow orchestration

    Polaris is a workflow orchestrator used to create, store, and run workflows. It simplifies the management of long-running processes by breaking them into chunks for asynchronous processing.

    Key automated behaviors include:

    • Sequence Management: Polaris automatically determines the sequence of steps and identifies which steps can be executed concurrently.
    • Automatic Pausing: Workflows pause automatically when there is no new data to move the process forward, freeing up CPU resources.
    • State Persistence: When a workflow waits for an event, Polaris stores the current state and pauses logical execution. Once the event is received, the state is recovered and execution resumes.

  2. Polaris performance and use cases

    Polaris has low runtime overhead because execution graphs are generated pre-runtime; the orchestrator simply uses the graph and available data to trigger builders.

    It is best suited for:

    • Multi-step workflows where steps depend on data from previous steps.
    • Executions spanning one or multiple request scopes.
    • Systems using reusable components combined in various ways.
    • Workflows that require the ability to pause, resume, or restart.

  3. Execute a workflow using the Polaris Executor

    The polaris.Executor allows you to run a registered workflow. You can configure Before and After hooks to intercept builder execution and production of data.

    Workflows can be executed in two modes:

    • Sequential: Executes builders one after another.
    • Parallel: Executes builders concurrently (note: this does not guarantee strict parallelism).
    executor := polaris.Executor{
  Before: func(builder reflect.Type, delta []IData) {
    fmt.Printf("Builder %s is about to be run with new data %v\n", builder, delta)
  },
  After: func(builder reflect.Type, produced IData) {
    fmt.Printf("Builder %s produced %s\n", builder, produced)
  },
}

// Sequentially execute builders
response, err := executor.Sequential(workflowKey, workflowId, dataDelta)

// Concurrently execute builders
response, err := executor.Parallel(workflowKey, workflowId, dataDelta)

  5. Install Polaris in a new Go project

    To add Polaris to a new project, create a project directory, initialize it as a Go module, and then install the Polaris package using go get.
    mkdir goproject
cd goproject
go mod init
go get github.com/harshadmanglani/polaris

  6. Define a workflow using the IWorkflow interface

    A workflow in Polaris is a specification and container for a topology of connected builders that generate final data. To create a workflow, implement the IWorkflow interface, which requires a GetWorkflowMeta() method. The WorkflowMeta must define:

    1. Builders: A list of IBuilder instances representing the units of work.
    2. Target Data: The name of the Data being generated; once this data is produced, the workflow is considered complete (though it can be re-opened with new data).

    While Polaris can determine the execution order automatically, defining builders sequentially in the list is recommended for readability.

    type IWorkflow interface {
	GetWorkflowMeta() WorkflowMeta
}

type CabRideWorkflow struct {
}

func (cr CabRideWorkflow) GetWorkflowMeta() WorkflowMeta {
	return WorkflowMeta{
		Builders: []IBuilder{
			UserInitiation{},
			CabbieMatching{},
			CabbieArrivalAtSource{},
			CabDepartureFromSource{},
			CabArrivalAtDest{},
			UserPayment{},
			RideEnds{},
		},
		TargetData: WorkflowTerminated{},
	}
}

  7. Install Go prerequisite for Polaris

    Polaris requires the Go programming language to be installed. The documentation was produced using Go 1.21. You can verify your current Go installation and version by running the go version command in your terminal.
    go version

  8. Implement a custom data store for Polaris

    Polaris workflows require a database for storage. To use a specific key-value store or RDBMS, you must implement the IDataStore interface and register it using the polaris.InitRegistry method.
    type IDataStore interface {
  Write(key string, value interface{})
  Read(key string) (interface{}, bool)
}

// After implementing the interface:
polaris.InitRegistry(dataStore)

  9. Execute a Polaris workflow at runtime

    To process a request using Polaris at runtime, you must generate a unique identifier for the workflow (uniqueWorkflowId) and pass the request data that matches the Consumes requirements of the target Builder. Use the executor.Sequential method to trigger the workflow and retrieve the resulting data using the .Get() method.
    func RequestHandler(w http.ResponseWriter, r *http.Request) {
    var alpha AlphaConsumes
    err := json.NewDecoder(r.Body).Decode(&alpha)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    uniqueWorkflowId := alpha.Id
    // Execute the workflow sequentially
    result := executor.Sequential("alphaWorkflowKey", uniqueWorkflowId, alpha)

    // Fetch expected data from the result
    alphaProcessed, ok := result.Get(AlphaProcessed{})
    if !ok {
        w.WriteHeader(http.StatusInternalServerError)
        return
    }

    // Process alphaProcessed
    w.WriteHeader(http.StatusOK)
}

  10. Initialize Polaris during service startup

    During the application startup phase, you must perform three primary steps: implement the IDataStore interface to initialize Polaris, register your workflows using a unique key, and configure the Executor with optional Before and After hooks for logging or monitoring builder execution.
    var dataStore polaris.IDataStore
var executor polaris.Executor

func init() {
  dataStore := SomeDataStoreImpl{}
  polaris.InitRegistry(dataStore)

  polaris.RegisterWorkflow("alphaWorkflowKey", AlphaWorkflow{})

  executor = polaris.Executor{
    Before: func(builder reflect.Type, delta []IData) {
      fmt.Printf("Builder %s is about to be run with new data %v\n", builder, delta)
    },
    After: func(builder reflect.Type, produced IData) {
      fmt.Printf("Builder %s produced %s\n", builder, produced)
    },
  }
}

  11. Create an in-memory data store for local development

    For initial setup and testing, you can use a simple in-memory data store. Note: This implementation is not suitable for production environments as data will be lost on restart.
    type mockStorage struct {
  store map[string]interface{}
}

func (ms *mockStorage) Read(key string) (interface{}, bool) {
  val, ok := ms.store[key]
  return val, ok
}

func (ms *mockStorage) Write(key string, val interface{}) {
  ms.store[key] = val
}

// Initialize and register the mock storage
mockStorage := &mockStorage{
  store: make(map[string]interface{}),
}

polaris.InitRegistry(mockStorage)

  12. Implement a Builder in Polaris

    A Builder is a unit of work in a Polaris workflow that consumes a set of data and produces a new data object. To create a Builder, you must implement the IBuilder interface, which requires two methods: GetBuilderInfo() to define the data dependencies and Process() to execute the business logic.

    Key metadata associated with a Builder includes:

    • Name: The identifier for the builder.
    • Consumes: The set of data required for the builder to execute.
    • Produces: The data output generated by the builder.
    • Optionals: Data that can optionally be consumed (e.g., used to trigger a re-run of a builder with existing data via a data-delta).
    • Access: Data the builder reads but does not rely on for execution sequencing.
    • BuilderContext: A wrapper used within the Process method to retrieve the input data.
    type IBuilder interface {
  GetBuilderInfo() BuilderInfo
  Process(BuilderContext) IData
}