SmartEngine Documentation

repository·master·Indexed 21 days ago

https://github.com/alibaba/smartengine

A lightweight, high-performance business orchestration engine for microservice architectures and approval workflows. It supports BPMN 2.0 standardization and offers two running modes: Custom Mode for embedded integration with custom storage and DataBase Mode using MyBatis for relational storage. The engine utilizes a CQRS-style API design and supports core BPMN symbols, process jumping, and multi-signature workflows.

Tokens
84.9K
Snippets
132
Records
295
Agent score
74%

What's inside SmartEngine

  1. Overview of SmartEngine

    master

    SmartEngine is a lightweight business orchestration engine widely used within Alibaba Group. It is designed for multi-service orchestration in microservice architectures, offering high performance and low storage costs for starting or triggering process instances. It is also suitable for traditional approval workflow scenarios.

    Key design principles include:

    • KISS Principle: Keeping things simple.
    • Standardization: Adhering to BPMN 2.0 specifications for a unified domain language.
    • Extensibility: Flexible support for extending parsers, behaviors, storage, and user integrations.
    • High Performance: Optimized for simple process scenarios to improve performance and reduce storage overhead.
    • Low Dependency: Designed to avoid 'JAR hell' by minimizing external dependencies.
  2. Understand the SmartEngine module structure

    master

    SmartEngine is organized as a Maven multi-module project. The repository is divided into three primary functional areas: core (the engine kernel), extension (pluggable components like storage and retry mechanisms), and ecology (tools and ecosystem components like designers).

    Module Hierarchy:

    • core: The central engine logic.
    • extension/storage/: Contains storage implementations (e.g., storage-common, storage-mysql, storage-custom).
    • extension/retry/: Contains retry logic implementations (e.g., retry-common, retry-mysql).
    • ecology/: Contains modeling and designer tools.
  3. How variables enter the SmartEngine context

    master

    Variables in SmartEngine are used for data flow between process nodes, gateway condition evaluation, and carrying context via external signals. They enter the engine through three primary entry points via a request map:

    1. Process Start: Passed in the request map during the initial start command.
    2. Signals: Passed in the request map when triggering a signal to advance the process.
    3. Task Operations: Passed in the request map during task completion or claim operations.

    Once received, these maps enter the ExecutionContext, and the VariableCommandService or VariablePersister determines if they should be persisted.

  4. Use BPMN files as semantic test standards

    master

    BPMN files in the repository serve as the "semantic gold standard" for the engine. When developing new behaviors or parsers, you should create corresponding BPMN files and tests to ensure semantic correctness.

    Key BPMN resource directories:

    • core/src/test/resources/process-def/*
    • extension/storage/storage-custom/src/test/resources/*.xml
    • extension/storage/storage-mysql/src/test/resources/*.bpmn20.xml
  5. Use ExtensionBinding to mount custom implementations

    master

    SmartEngine allows you to mount custom logic using the @ExtensionBinding annotation. This mechanism uses a combination of a group and a bindKey to map your implementation to specific engine components. During startup, the AnnotationScanner scans the classpath to build the ExtensionContainer.

    Key fields for @ExtensionBinding:

    • group: The extension category (defined in ExtensionConstant).
    • bindKey: The identifier for the binding (e.g., a BPMN element type, an attribute name, or a behavior name).
    • priority: The execution priority for the same bindKey (higher values take precedence).

    Common extension groups include:

    • element-parser
    • attribute-parser
    • activity-behavior
    • common
    • SERVICE
    // Example conceptual usage of ExtensionBinding
    @ExtensionBinding(group = "activity-behavior", bindKey = "myCustomBehavior", priority = 100)
    public class MyCustomBehavior implements ActivityBehavior {
        // implementation
    }
  6. Optimize cross-partition queries using User Task Index tables

    master

    To avoid expensive full-partition scans when querying by assigneeId (which does not include the process_instance_id partition key), use a dedicated, non-partitioned index table: se_user_task_index.

    How it works

    • Purpose: Stores only active/pending tasks to keep the table size small and query performance high.
    • Data Redundancy: Redundantly stores common query fields (e.g., process_definition_type, domain_code, title) to avoid JOINs back to the partitioned main tables.
    • Lifecycle Management:
      • Insert: When TaskAssignee.insert() is called, insert a record into the index table.
      • Delete: When a task is completed, canceled, or the assignee is deleted, remove the corresponding record from the index table.
      • Update: When task metadata (title, priority, etc.) changes, update the index table.

    Schema Implementation

    PostgreSQL (with Partial Indexing)

    CREATE TABLE se_user_task_index (
      id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
      tenant_id varchar(64),
      assignee_id varchar(255) NOT NULL,
      assignee_type varchar(128) NOT NULL DEFAULT 'user',
      task_instance_id bigint NOT NULL,
      process_instance_id bigint NOT NULL,
      process_definition_type varchar(255),
      domain_code varchar(64),
      extra jsonb,
      task_status varchar(64) NOT NULL,
      task_gmt_modified timestamp(6),
      title varchar(255),
      priority int DEFAULT 500,
      CONSTRAINT uk_user_task_idx UNIQUE (tenant_id, assignee_id, task_instance_id)
    );
    
    -- Optimized partial index for pending tasks
    CREATE INDEX idx_user_task_pending ON se_user_task_index
      (tenant_id, assignee_id, assignee_type, task_status)
      WHERE task_status = 'pending';

    MySQL

    CREATE TABLE se_user_task_index (
      id bigint unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
      tenant_id varchar(64),
      assignee_id varchar(255) NOT NULL,
      assignee_type varchar(128) NOT NULL DEFAULT 'user',
      task_instance_id bigint NOT NULL,
      process_instance_id bigint NOT NULL,
      process_definition_type varchar(255),
      domain_code varchar(64),
      extra json,
      task_status varchar(64) NOT NULL,
      task_gmt_modified datetime(6),
      title varchar(255),
      priority int DEFAULT 500,
      UNIQUE KEY uk_user_task_idx (tenant_id, assignee_id, task_instance_id)
    );
    
    -- MySQL uses standard composite index as it lacks partial indexes
    CREATE INDEX idx_user_task_pending ON se_user_task_index
      (tenant_id, assignee_id, assignee_type, task_status);
    -- PostgreSQL Partial Index Example
    CREATE INDEX idx_user_task_pending ON se_user_task_index
      (tenant_id, assignee_id, assignee_type, task_status)
      WHERE task_status = 'pending';
  7. Integrate SmartEngine without Spring Boot

    master
    SmartEngine is a pure Java component and does not require Spring Boot. While Spring Boot makes it easier to configure DataSource, MyBatis, and bean scanning, you can use the core engine in any Java environment. The project includes examples of both pure JUnit tests and Spring XML configurations.
  8. Replace the deprecated LockStrategy with robust concurrency patterns

    master

    The LockStrategy interface is marked as Deprecated. It is no longer recommended for ensuring core correctness because its lock granularity is often too fine and it cannot guarantee correctness in clustered environments where distributed locks or transaction locks are required.

    Instead of LockStrategy, use one of the following approaches:

    • DataBase Mode: Rely on database-level row locks or optimistic locking.
    • Custom Mode: Implement an "Idempotent Advancement + De-duplication" pattern to ensure that parallel branches do not trigger duplicate side effects.
  9. Choose between Setter Injection and @ExtensionBinding for Monitoring

    master

    SmartEngine provides two mechanisms for extending components for monitoring:

    1. Setter Injection: Use this for components that implement specific interfaces and have setter methods in ProcessEngineConfiguration. This is simple, direct, and provides high control.

      • Examples: DelegationExecutor, ListenerExecutor, ExceptionProcessor, VariablePersister, ExpressionEvaluator.
    2. @ExtensionBinding: Use this for behavior classes and internal logic that do not have setters. This mechanism uses package scanning to automatically discover and bind implementations.

      • Examples: TransitionBehavior (for gateway decisions), ActivityBehavior implementations, ElementParser implementations, and Service implementations.

    Note on Static Utilities: Static tools like ExpressionUtil cannot be injected directly. To monitor them, you must wrap the underlying component they call (e.g., wrap ExpressionEvaluator to monitor ExpressionUtil).

  10. Implement custom storage for SmartEngine

    master

    To deeply integrate SmartEngine into a business system (e.g., using your own database, multi-tenant schemas, adding business fields, or implementing read/write splitting), you must implement the Storage interface.

    Each Storage interface implementation corresponds to a specific domain object family:

    • Process / Execution / Activity
    • Task / Assignee
    • Variable
    • Deployment / Definition
    • Notification / Supervision (Enhanced)

    Interfaces can be found in the package: core/.../instance/storage/*.

  11. How the Fluent Query API works

    master

    SmartEngine uses a Fluent Query API (chainable query API) inspired by Activiti/Flowable's Query<Q, T> pattern and MyBatis Plus's conditional filtering.

    Queries are constructed by calling an entry method on the SmartEngine instance, chaining various filter methods, and concluding with a terminal operation to execute the query.

    Query Hierarchy:

    • Query<Q, T>: Base interface providing pagination, tenant filtering, and terminal operations.
    • ProcessBoundQuery<Q, T>: A shared interface for queries related to a process (e.g., TaskQuery, SupervisionQuery, NotificationQuery). It provides common methods like processInstanceId and sorting.
    • ProcessInstanceQuery: A separate branch for querying process instances directly.
    // Example of the chainable pattern
    List<TaskInstance> tasks = smartEngine.createTaskQuery()
        .taskAssignee("user001")
        .taskStatus(TaskInstanceConstant.PENDING)
        .orderByCreateTime().desc()
        .listPage(0, 10);
  12. Core Engine Kernel (core)

    master

    The core/ module is the engine's heart. It is designed to be highly portable and has minimal dependencies to avoid 'JAR hell'. Its responsibilities include:

    • BPMN Parsing: Handling BPMN file parsing.
    • Execution Model: Managing process, execution, activity, and token entities.
    • Behavior Implementation: Implementing task types like serviceTask, receiveTask, and gateway.
    • Public Service Interfaces: Providing the primary entry points via SmartEngine and Command/QueryService.
    • Extension Mechanism: Managing plugins via ExtensionBinding, AnnotationScanner, and ExtensionContainer.
    • Utilities: Handling constants, tools, and exceptions via ExceptionProcessor.