Hitokoto API

repository·master·Indexed 20 days ago

https://github.com/hitokoto-osc/hitokoto-api

An extensible framework based on Teng-koa for providing 'Hitokoto' (one-sentence) quotes and content. Version 1.7.1 supports request statistics, JS callbacks, multi-process execution, and integrates with Redis for data storage. It includes features for filtering sentences by category and length, as well as wrappers for Netease SDK DJ details with built-in caching.

Tokens
7.9K
Snippets
29
Records
35
Agent score
68%

What's inside hitokoto-api

  1. Run Hitokoto API using Docker

    master

    You can run the Hitokoto API in a Docker container.

    Note on Networking: The default command uses --network host, which means the container shares the host's network stack. Ensure port 8000 is not already in use on your host machine. You must have Redis pre-installed on your host.

    Docker Run Command: Map your local data directory to /usr/src/app/data inside the container to persist configurations and logs.

    docker run \
    -v /path/to/your/data/dir:/usr/src/app/data \
    --network host \
    hitokoto/api
  2. Install and run Hitokoto API locally

    master

    To run the Hitokoto API on your local machine, ensure you have Node.js (>=16.x) and yarn installed.

    Important Requirements:

    • This project uses Yarn v2. You must update your Yarn version to v1.22.4 or higher before proceeding.
    • The project does not support npm, cnpm, or pnpm for dependency management.
    • Redis must be installed and running as an external dependency.

    Setup Steps:

    1. Clone the repository: git clone https://github.com/hitokoto-osc/hitokoto-api.git your_workdir
    2. Enter the directory: cd your_workdir
    3. Install production dependencies: yarn workspaces focus --production
    4. Initialize configuration: Copy config.example.yml to ./data/config.yml and modify it as needed.
    5. Start the application: yarn start
    git clone https://github.com/hitokoto-osc/hitokoto-api.git your_workdir
    cd your_workdir
    yarn workspaces focus --production
    cp config.example.yml ./data/config.yml
    yarn start
  3. Configure volume mounting for persistence

    master
    The data directory is used for storing configuration files and saving logs (日记) that require persistence. When running the API within a container, you must mount this directory to your host system's file system to ensure data is not lost when the container is restarted or removed.
  4. Handle service shutdown and exit signals

    master

    The Hitokoto API service implements graceful shutdown mechanisms to ensure child processes and HTTP workers are terminated correctly when the main process receives exit signals.

    Supported Signals:

    • SIGINT (Ctrl + C)
    • SIGTERM (Termination request)
    • uncaughtException (Unhandled errors)
    • exit (Process exit event)

    Shutdown Procedure:

    1. The service catches the signal.
    2. notifyChildProcessesExit() is called, which sends SIGTERM to all registered child processes and kills all workers in the WorkersBridge.
    3. The service logs the shutdown status.
    4. The process exits (with code 0 for graceful shutdowns or 1 for errors/uncaught exceptions).

    Error Telemetry: If telemetry:error is enabled in the configuration and the service is not in dev mode, uncaught exceptions are captured via CaptureUncaughtException(err) and sent to Sentry before the service shuts down.

  5. Understand the service initialization lifecycle

    master

    When the service starts, it follows a specific sequence of asynchronous steps within the start() function. If any step fails, the service logs the error and exits with code 1.

    1. Environment Check: preStart.check() validates the environment.
    2. Program Updates: preStart.checkProgramUpdates() checks for available updates.
    3. Sentence Updates: updateSentencesTask() runs a task to refresh sentence data.
    4. Process Registration: registerProcesses() spawns child processes based on the environment (Dev vs Prod).
    5. Worker Pool: startWorkersPool() initializes the HTTP server workers.

    Process Management:

    • In Development mode (opts.dev), only processes marked as isDev are spawned.
    • In Production mode, only processes marked as isProd are spawned.
    • The service uses staticProcess().spawnProcess() to manage child processes and ProcessInteract to handle communication via receivers.
  6. Initialize and run the Hitokoto API service

    master

    The core.js file serves as the main entrypoint for the Hitokoto API service. It orchestrates the initialization sequence, which includes environment checks, configuration loading, task execution, child process registration, and starting the HTTP worker pool.

    To run the service, you typically execute this file via Node.js. The service supports a dev mode which affects which processes are spawned and how logging is handled.

    node core.js
  7. Configure Sentences A/B Switcher Redis databases

    master

    The SentencesABSwitcher extension uses nconf to determine which Redis database IDs to use for A/B testing. You can configure these via the sentences_ab_switcher namespace. If not provided, it defaults to database 1 for slot 'a' and database 2 for slot 'b'.

    Configuration keys:

    • sentences_ab_switcher:a: The Redis database ID for partition 'a'.
    • sentences_ab_switcher:b: The Redis database ID for partition 'b'.
    • redis:password: The password for the Redis connection.
  8. Configure hitokoto_api ports and volumes

    master

    The hitokoto_api service exposes port 8000 on the host. It also uses a volume to persist application data.

    • Ports: Maps host port 8000 to container port 8000.
    • Volumes: Maps the local directory ./etc/api to /usr/src/app/data inside the container for data persistence.
    services:
      hitokoto_api:
        ports:
          - 8000:8000
        volumes: 
          - ./etc/api:/usr/src/app/data
  9. Configure the Hitokoto API with ecosystem.config.js

    master

    The project uses an ecosystem.config.js file (compatible with PM2/Alinode) to manage the deployment and runtime environment of the @hitokoto/api.v1 application. You can define the entry point, execution mode, and environment-specific variables such as PORT and NODE_ENV within the apps array.

    module.exports = {
      apps: [
        {
          name: '@hitokoto/api.v1',
          script: './core.js',
          watch: false,
          ignore_watch: ['./data/logs'],
          interpreter: 'yarn', // absolute path to yarn ; default is node
          interpreter_args: 'start',
          exec_mode: 'fork',
          cwd: '', // the directory from which your app will be launched
          args: '', // string containing all arguments passed via CLI to script
          env_production: {
            PORT: 8000,
            NODE_ENV: 'production',
            ENABLE_NODE_LOG: 'YES',
            NODE_LOG_DIR: '~/alinode_logs',
          },
        },
      ],
    }
  10. Configure PM2 deployment with ecosystem.config.js

    master

    The project uses an ecosystem.config.js file for managing application instances via PM2. This configuration defines how the @hitokoto/api.v1 service is launched, watched, and environment-configured.

    Key configuration options for the application instance include:

    • name: The identifier for the process (set to @hitokoto/api.v1).
    • script: The entry point file (set to ./core.js).
    • watch: Boolean to enable/disable file watching for restarts (set to false).
    • ignore_watch: An array of paths to exclude from the watch process (e.g., ./data/logs).
    • interpreter: The command used to run the script (set to yarn).
    • interpreter_args: Arguments passed to the interpreter (set to start).
    • exec_mode: The execution mode (e.g., fork).
    • cwd: The current working directory from which the app is launched.
    • args: A string of arguments passed directly to the script via CLI.
    • env_production: An object containing environment variables used when running in production mode, such as PORT and NODE_ENV.
    module.exports = {
      apps: [
        {
          name: '@hitokoto/api.v1',
          script: './core.js',
          watch: false,
          ignore_watch: ['./data/logs'],
          interpreter: 'yarn',
          interpreter_args: 'start',
          exec_mode: 'fork',
          cwd: '',
          args: '',
          env_production: {
            PORT: 8000,
            NODE_ENV: 'production',
          },
        },
      ],
    }
  11. Configure the hitokoto_api service environment variables

    master

    The hitokoto_api service uses several environment variables to configure its behavior, including API identity, host routing, and Redis connectivity.

    Key environment variables:

    • url: The base URL for the service.
    • api_name: The unique identifier for the API instance.
    • requests.hosts: A string representation of a list containing allowed hosts (e.g., "['v1.hitokoto.cn']").
    • redis.host: The hostname of the Redis instance (defaults to redis when using this Docker Compose setup).
    • redis.port: The port used to connect to Redis.
    • NODE_ENV: Set to production by default.
    environment:
      NODE_ENV: production
      url: https://v1.hitokoto.cn
      api_name: sh-01-X23Hwoc
      requests.hosts: "['v1.hitokoto.cn']"
      redis.host: redis
      redis.port: 6379
  12. Configure Redis settings in Docker Compose

    master

    The redis service is used as the data store for the API. To configure it, you must provide a local redis.conf file at the specified volume path.

    Important: Ensure ./etc/redis.conf exists on your host machine before starting the containers, as the service uses it to initialize the redis-server via the command instruction.

    redis:
        networks:
          - hitokoto_api
        image: redis
        restart: unless-stopped
        container_name: redis
        hostname: redis
        volumes:
          - ./etc/redis.conf:/etc/redis/redis.conf # Ensure this file exists on the host
          - ./data/redis:/data
        command: redis-server /etc/redis/redis.conf