SnailJob Documentation

repository·master·Indexed 21 days ago

https://github.com/aizuda/snail-job

A distributed task retry and scheduling platform designed for high reliability and scalability. SnailJob ensures business consistency in distributed systems through replayable and controllable retry mechanisms, supporting second-level precision scheduling and task orchestration. It provides official client implementations for Java, Python, and Go, and supports multiple database engines including MySQL, PostgreSQL, Oracle XE, SQL Server, MariaDB, DM8, and Kingbase.

Tokens
3.6K
Snippets
12
Records
16
Agent score
72%

What's inside SnailJob

  1. Overview of SnailJob

    master

    SnailJob is a flexible, reliable, and high-performance distributed task retry and task scheduling platform. It is designed to improve the consistency of distributed business systems through replayable and controllable task retries.

    Key features include:

    • Distributed Task Retry: Replayable and controllable mechanism to ensure business consistency.
    • High-Performance Scheduling: Supports second-level precision, interruptible tasks, and task orchestration.
    • Scalability: Uses a partition-based mode to achieve high scalability and fault tolerance.
    • Management Features: Includes comprehensive permission management, powerful alarm monitoring, and a user-friendly interface.
  2. Build the project with custom frontend changes

    master

    After making modifications to the code within the frontend directory, you must ensure that the Maven build process includes these changes.

    1. Locate the skipFrontend option in your Maven configuration.
    2. Ensure skipFrontend is not checked (it must be set to false or disabled).
    3. Run the standard Maven packaging command to build the project with the integrated frontend.
  3. How to integrate a custom frontend UI

    master

    To integrate a custom frontend interface into the snail-job-server-ui project, you must follow a specific directory structure where the frontend source code resides within the Maven project structure. The process involves creating a dedicated frontend directory, cloning the base administration UI, making your customizations, and ensuring Maven is configured to include the frontend during the build process.

    snail-job-server-ui/
    └─ src/
       └─ main/
          └─ frontend/  # New frontend directory
             └─ snail-job-admin/  # Cloned frontend code
  4. How to integrate a custom frontend

    master

    To integrate a custom frontend interface into the Snail-Job project, you must place your frontend source code within the snail-job-server-ui module's directory structure. This allows the Maven build process to include your custom UI in the final application package.

    Follow these steps:

    1. Prepare the directory: Create a frontend directory inside snail-job-server-ui/src/main/.
    2. Clone the base UI: Pull the official snail-job-admin source code into that directory to use as a starting point.
    3. Customize: Modify the files within the cloned directory to meet your requirements.
    4. Build: Ensure the Maven configuration is set to include the frontend before running the build command.
    snail-job-server-ui/
    └─ src/
       └─ main/
          └─ frontend/  # New frontend directory
             └─ snail-job-admin/  # Cloned frontend code
  5. Clone the snail-job-admin frontend code

    master

    To set up the base frontend code, navigate to the src/main/frontend directory within the snail-job-server-ui module and clone the official repository.

    Note: Ensure you are executing the command inside the frontend directory so the code is cloned directly into the expected path.

    cd src/main/frontend
    git clone https://gitee.com/opensnail/snail-job-admin.git
  6. How to integrate a custom frontend interface

    master

    To integrate a custom frontend into the snail-job-server-ui project, you must place your frontend source code within a specific directory structure so that the Maven build process can include it. The process involves creating a frontend directory, cloning the base administration UI, making your modifications, and ensuring the Maven build is configured to include the frontend assets.

    # 1. Create the directory structure
    # snail-job-server-ui/src/main/frontend/
    
    # 2. Clone the base admin UI into that directory
    cd src/main/frontend
    git clone https://gitee.com/opensnail/snail-job-admin.git
  7. Deploy SnailJob database environments using Docker Compose

    master

    SnailJob provides a docker-compose.yaml configuration to quickly spin up various database engines for testing or development. The configuration includes support for MySQL, PostgreSQL, Oracle XE, SQL Server, MariaDB, DM8 (Dameng), and Kingbase. Each service is pre-configured with the necessary initialization scripts to set up the snail_job schema.

    To use this, ensure you have Docker and Docker Compose installed, then run:

    docker compose up -d
  8. Initialize DM8 and Kingbase database images

    master

    DM8 and Kingbase are not pulled from standard registries in the compose file and require manual loading of local image files.

    DM8 Setup

    1. Download the package: wget https://download.dameng.com/eco/dm8/dm8_20240613_x86_rh6_64_rq_ent_8.1.3.140_pack5.tar
    2. Load the image: docker load -i dm8_20240613_x86_rh6_64_rq_ent_8.1.3.140_pack5.tar
    3. Execute schema creation: docker compose exec dm8 bash -c "/opt/dmdbms/bin/disql SYSDBA/SYSDBA_dm001 \ /tmp/schema.sql"`

    Kingbase Setup

    Download the appropriate tarball for your architecture:

    • x86_64: https://kingbase.oss-cn-beijing.aliyuncs.com/KESV8R3/V009R001C001B0025-安装包-docker/x86_64/kdb_x86_64_V009R001C001B0025.tar
    • aarch64: https://kingbase.oss-cn-beijing.aliyuncs.com/KESV8R3/V009R001C001B0025-安装包-docker/aarch64/kdb_aarch64_V009R001C001B0025.tar

    Then load and execute:

    1. docker load -i kdb_x86_64_V009R001C001B0025.tar
    2. docker compose exec kingbase bash -c "ksql -Uroot -d test -f /tmp/schema.sql"
    # DM8 Example
    # wget https://download.dameng.com/eco/dm8/dm8_20240613_x86_rh6_64_rq_ent_8.1.3.140_pack5.tar
    # docker load -i dm8_20240613_x86_rh6_64_rq_ent_8.1.3.140_pack5.tar
    # docker compose exec dm8 bash -c "/opt/dmdbms/bin/disql SYSDBA/SYSDBA_dm001 \` /tmp/schema.sql"
    
    # Kingbase Example
    # docker load -i kdb_x86_64_V009R001C001B0025.tar
    # docker compose exec kingbase bash -c "ksql -Uroot -d test -f /tmp/schema.sql"
  9. SnailJob Client Repositories

    master

    SnailJob provides official client implementations for various programming languages to integrate task scheduling and retries into your applications:

    • Java: snail-job-demo (Example program)
    • Python: snail-job-python (Python client)
    • Go: snail-job-go (Go client)
    - [snail-job-demo (Java客户端示例程序)](https://gitee.com/opensnail/snail-job-demo.git)
    - [snail-job-python (Python客户端)](https://gitee.com/opensnail/snail-job-python.git)
    - [snail-job-go (Go客户端)](https://gitee.com/opensnail/snail-job-go.git)