Qinglong Timed Task Management Platform

repository·develop·Indexed 12 days ago

https://github.com/whyour/qinglong

A timed task management platform supporting Python3, JavaScript, Shell, and TypeScript. It features a web interface for managing scripts, environment variables, and logs, with deployment options via Docker, Kubernetes, and npm. Version 2.21.0-16 includes a multi-process architecture using Node.js cluster to separate HTTP and gRPC services.

Tokens
17.9K
Snippets
62
Records
80
Agent score
97%

What's inside Qinglong

  1. Overview of Qinglong features

    develop

    Qinglong is a timed task management platform that supports multiple scripting languages and provides a web-based interface for management.

    Key Capabilities:

    • Scripting Support: Execute tasks written in python3, javaScript, shell, and typescript.
    • Online Management: Manage scripts, environment variables, and configuration files directly through the UI.
    • Task Monitoring: View task logs online and set tasks with second-level precision.
    • Notifications: Supports system-level notifications.
    • User Experience: Includes dark mode support and is optimized for mobile browser operation.
  2. Overview of Qinglong Architecture

    develop

    Qinglong is a full-stack TypeScript timed task management platform. It consists of a frontend admin panel (React/Umi Max), a backend application (Express/Sequelize/SQLite), and a shell runtime for task execution.

    Core Components:

    • src/: Frontend admin panel.
    • back/: Backend application logic, API, and services.
    • shell/: Runtime shell scripts for task execution and environment preloading.
    • data/: Local runtime data (SQLite DB, scripts, logs, configs, etc.).
    • static/: Built frontend and backend artifacts.
    • docker/: Docker configuration and entrypoints.
  3. Backend Architecture: API Routes and Services

    develop

    The backend follows a pattern where API routes are thin wrappers around business logic contained in Services.

    API Routes (back/api/index.ts): Routes should validate input, retrieve the appropriate service from typedi's Container, and return a { code, data, message } response. Modules include user.ts, env.ts, cron.ts, script.ts, etc.

    Services (back/services/*): Services contain the core business logic. For example:

    • cron.ts: Manages cron jobs, logs, and the scheduler client.
    • env.ts: Manages environment variables.
    • script.ts: Manages script files.
    • subscription.ts: Manages repository pulls.

    Pattern: To change backend behavior, find the API route first, then locate the corresponding service.

  4. Backend and Frontend Coding Conventions

    develop

    Follow these patterns to maintain code quality:

    Backend Conventions

    • Logic Placement: Prefer adding business logic to services, not route files.
    • Dependency Injection: Use typedi services consistently.
    • Configuration: Use existing config paths from back/config/index.ts.
    • Response Format: Always return API responses in the { code, data, message } shape.
    • Data Layer: Preserve the current SQLite/Sequelize style.

    Frontend Conventions

    • Patterns: Follow existing Umi/React/Ant Design patterns.
    • Routing: Keep route/menu changes in src/layouts/defaultProps.tsx.
    • Helpers: Use existing request/WebSocket helpers.
    • i18n: Add or update locale strings for all visible UI text.
    • Styles: Keep page-specific styles near the page component.
  5. Backend Architecture: Entry Point and Loaders

    develop

    The backend entry point is back/app.ts. It is responsible for creating the Express app, initializing the database in the primary process, and starting both gRPC and HTTP workers using Node cluster.

    Loaders: The application uses a loader pattern to initialize various subsystems. Key loaders include:

    • back/loaders/db.ts: Syncs Sequelize models.
    • back/loaders/express.ts: Configures HTTP middleware (CORS, Helmet, JWT, etc.) and mounts routes via back/api/index.ts.
    • back/loaders/initTask.ts: Initializes scheduled tasks.
    • back/loaders/depInjector.ts: Registers dependency injection bindings.
  6. High-Level Runtime Flow

    develop

    Understanding how data and execution flow through the system:

    Web Request Flow: Browser $\rightarrow$ src/pages/* $\rightarrow$ src/utils/http.tsx $\rightarrow$ /api/* $\rightarrow$ back/api/* $\rightarrow$ back/services/* $\rightarrow$ back/data/* (Sequelize models) $\rightarrow$ data/db/database.sqlite

    Task Execution Flow: Cron/task execution $\rightarrow$ back/services/cron.ts $\rightarrow$ shell/task.sh or shell/otask.sh $\rightarrow$ data/scripts/* $\rightarrow$ data/log/*

    Production Asset Serving: Frontend assets in static/dist/* are served by back/loaders/express.ts.

  7. Backend Architecture: Data Models and Configuration

    develop

    The backend uses Sequelize with SQLite for data persistence.

    Data Storage:

    • Database location: data/db/database.sqlite (configured in back/data/index.ts).
    • Common models: cron.ts, env.ts, subscription.ts, system.ts.

    Configuration (back/config/index.ts): This is the central runtime configuration. It reads .env and defines critical paths. Do not hardcode paths; use the keys defined here:

    • dataPath: Runtime data root.
    • scriptPath: User scripts.
    • dbPath: SQLite database location.
    • logPath: Task logs.
  8. Frontend Architecture: Layout and Routes

    develop

    The frontend is built with Umi Max and Ant Design.

    Routing and Layout:

    • src/layouts/defaultProps.tsx: Defines the main route/menu list. Update this file to add new pages to the sidebar.
    • src/app.ts: Handles app initialization, including locale loading and applying QlBaseUrl as the public path.

    Major Pages:

    • src/pages/crontab: Timed task management.
    • src/pages/subscription: Subscription management.
    • src/pages/env: Environment variables.
    • src/pages/script: Script management.
  9. Start and Build the Project

    develop

    Use pnpm to manage the development and build processes.

    Development:

    pnpm start

    This command runs both the backend (start:back via nodemon) and the frontend (start:front via max dev) simultaneously.

    Production Builds:

    • Build Backend: pnpm run build:back (outputs to static/build)
    • Build Frontend: pnpm run build:front (outputs to static/dist)
    pnpm start
    pnpm run build:back
    pnpm run build:front
  10. How to add or modify a Backend API

    develop

    To implement a new backend feature or modify an existing one, follow this workflow to ensure consistency with the project's architecture:

    1. Define the Route: Add or update the route in back/api/<module>.ts.
    2. Implement Logic: Add or update the business logic in back/services/<module>.ts.
    3. Update Persistence: If the change requires new data storage, update the model in back/data/<module>.ts.
    4. Add Validation: Use celebrate or Joi near the route definition to validate incoming requests.
    5. Connect Frontend: Update the caller in src/pages/** or src/utils/** to consume the new API.
  11. Set up Qinglong for development

    develop

    To develop on Qinglong, clone the repository, set up your environment variables, and use pnpm to manage dependencies.

    1. Clone the repo: git clone https://github.com/whyour/qinglong.git
    2. Navigate to the directory: cd qinglong
    3. Copy the environment template: cp .env.example .env
    4. Install pnpm globally (recommended version 8.3.1): npm install -g pnpm@8.3.1
    5. Install dependencies and start: pnpm install pnpm start

    Once started, access the platform at http://127.0.0.1:5700.

    git clone https://github.com/whyour/qinglong.git
    cd qinglong
    cp .env.example .env
    npm install -g pnpm@8.3.1
    pnpm install
    pnpm start
  12. Modify Login, Auth, or Security settings

    develop

    To change authentication or security logic, focus on these areas:

    • Backend API/Service: back/api/user.ts, back/services/user.ts
    • Auth Logic: back/shared/auth.ts, back/shared/store.ts, back/token.ts
    • Middleware/Loaders: back/loaders/express.ts
    • Frontend: src/pages/login/index.tsx, src/pages/initialization/index.tsx

    Warning: Exercise extreme caution when modifying JWT behavior, open API token behavior, first-run initialization, or platform-specific session limits.