DOCSEARCHbyAlgolia
Submit Docs

Pentaract Documentation

repository·main·Indexed 23 days ago

https://github.com/dominux/pentaract

A lightweight cloud storage system that uses Telegram as the underlying storage backend. Pentaract provides a REST API and UI to manage files and folders without consuming local server disk space, overcoming Telegram's file size limits by chunking data. It features a Rust-based server, a Solid-based UI, and a PostgreSQL database, supporting role-based access control (Viewer, Can edit, Admin) and scalable storage workers.

Tokens
6.8K
Snippets
17
Records
49
Agent score
83%

What's inside Pentaract

  1. How Pentaract storage and workers work

    main

    Pentaract is a cloud storage system that uses Telegram as the underlying storage layer, meaning it does not use your server's filesystem or paid cloud providers.

    Storages

    Every "storage" acts as a separate filesystem (similar to a drive volume). Each storage is mapped to a specific Telegram channel where the data is actually stored. You can perform standard filesystem operations: upload, download, create folders, get info, and delete.

    Storage Workers

    Storage workers are Telegram bots used to facilitate the upload and download of files via the Telegram API.

    Overcoming Telegram Limitations

    • RPM (Requests Per Minute) Limits: Telegram limits how many requests a single bot can make. To increase throughput, you can create additional storage workers (up to 20 bots per user).
    • File Size Limits: While the Telegram API limits individual file downloads to 20 MB, Pentaract overcomes this by dividing uploaded files into chunks. These chunks are saved separately in Telegram and reassembled during download, allowing for files much larger than 20 MB (e.g., >10 GB).

  2. Manage storage access roles

    main

    Pentaract allows you to manage access to your storages by granting specific roles to other users. The available roles are:

    • Viewer: Read-only access.
    • Can edit: Permission to modify files/folders.
    • Admin: Full administrative control.

    You can grant, change, or revoke (delete) access for these users within the platform.

  4. Install dependencies for the vite-template-solid project

    main

    To set up the project, run your preferred package manager's install command. While the template includes a pnpm-lock.yaml (maintained via pnpm up -Lri), you can safely remove it and use npm, pnpm, or yarn.

    $ npm install # or pnpm install or yarn install

  5. Build Pentaract from source

    main

    To build and run Pentaract from source, you need Cargo, Node.js, pnpm, and Postgres installed on your system.

    1. Build the Server (Rust):

      git clone git@github.com:Dominux/Pentaract.git
cd Pentaract/pentaract
cargo build --release

      Copy the resulting binary to your application directory.

    2. Build the UI (Node.js):

      cd ../ui
pnpm run build

      Copy the contents of the dist folder to your application's ui directory.

    3. Run: Set your environment variables (refer to the .env.example file) and execute the binary:

      ./pentaract

  7. Build the application for production

    main

    Build the app for production using npm run build. This command:

    • Bundles Solid in production mode.
    • Optimizes the build for performance.
    • Minifies the output.
    • Generates filenames with hashes for cache busting.
    • Outputs the final files to the dist folder.
    npm run build

  8. Install Pentaract via Docker Compose (Recommended)

    main

    The recommended way to run Pentaract is using Docker Compose with a pre-built image. This setup includes the Pentaract application and a Postgres database instance.

    1. Create a directory for the app:
    mkdir pentaract
    1. Create a docker-compose.yml file in that directory with the following content:
    version: "3.9"

volumes:
  pentaract-db-volume:
    name: pentaract-db-volume

services:
  pentaract:
    container_name: pentaract
    image: thedominux/pentaract
    env_file:
      - .env
    ports:
      - ${PORT}:8000
    restart: unless-stopped
    depends_on:
      - db

  db:
    container_name: pentaract_db
    image: postgres:15.0-alpine
    environment:
      POSTGRES_USER: ${DATABASE_USER}
      POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
    restart: unless-stopped
    volumes:
      - pentaract-db-volume:/var/lib/postgresql/data
    1. Create a .env file in the same directory. You must set SUPERUSER_EMAIL, SUPERUSER_PASS, and SECRET_KEY.
    2. Start the application:
    docker compose up -d

    Access the app at http://localhost:8000.

  9. Pentaract Application Startup Lifecycle

    main

    The Pentaract application initializes by loading configuration, setting up tracing/logging, and preparing the database environment. The startup sequence follows these steps:

    1. Configuration: Loads settings via Config::new().
    2. Logging: Initializes tracing_subscriber with default filters (pentaract=debug,tower_http=debug,axum::rejection=trace) if no environment variable is provided.
    3. Database Setup:
      • Creates the database using create_db.
      • Establishes a connection pool via get_pool.
      • Initializes the database schema via init_db.
      • Creates a superuser via create_superuser.
    4. Storage Manager: Spawns a background task running StorageManager to handle ClientMessage processing via an MPSC channel.
    5. Server: Builds and runs an axum-based server using Server::build_server and server.run(&addr).

  10. Communication schemas between Clients and Storage Managers

    main

    Pentaract uses asynchronous message passing via tokio channels to facilitate communication between clients and storage managers. The communication flow is structured around two primary message types:

    1. ClientMessage: Sent from the client to the storage manager. It contains a StorageManagerSender (a oneshot::Sender) used to send a response back to the client, and ClientData representing the requested operation.
    2. StorageManagerMessage: Sent from the storage manager back to the client via the oneshot channel provided in the ClientMessage. It contains StorageManagerData which carries the result of the operation.

    Client Data Types (ClientData)

    • UploadFile(UploadFileData): Initiates a file upload. Requires file_id (Uuid), user_id (Uuid), and the raw file_data (Box<[u8]>).
    • DownloadFile(DownloadFileData): Initiates a file download. Requires file_id (Uuid), storage_id (Uuid), and user_id (Uuid).

    Storage Manager Response Types (StorageManagerData)

    • UploadFile(PentaractResult<()>): Returns the result of an upload operation.
    • DownloadFile(PentaractResult<Vec<u8>>): Returns the result of a download operation, containing the file bytes on success.

  11. Configure Pentaract environment variables

    main

    Pentaract requires several environment variables for operation. If using Docker Compose, place these in a .env file.

    Required Security Settings:

    • SUPERUSER_EMAIL: The email for the superuser.
    • SUPERUSER_PASS: The password for the superuser.
    • SECRET_KEY: A long, randomly generated sequence. You can generate one using openssl rand -hex 32.

    Database Configuration:

    • DATABASE_USER: Postgres username.
    • DATABASE_PASSWORD: Postgres password.
    • DATABASE_NAME: Postgres database name.
    • DATABASE_HOST: Postgres host (e.g., db when using Docker Compose).
    • DATABASE_PORT: Postgres port (usually 5432).

    Application Settings:

    • PORT: The port the app will listen on.
    • WORKERS: Number of workers.
    • CHANNEL_CAPACITY: Capacity for channels.
    • ACCESS_TOKEN_EXPIRE_IN_SECS: Expiration for access tokens.
    • REFRESH_TOKEN_EXPIRE_IN_DAYS: Expiration for refresh tokens.
    • TELEGRAM_API_BASE_URL: Base URL for Telegram API (default: https://api.telegram.org).
    PORT=8000
WORKERS=4
CHANNEL_CAPACITY=32
SUPERUSER_EMAIL=<YOUR-EMAIL>
SUPERUSER_PASS=<YOUR-PASSWORD>
ACCESS_TOKEN_EXPIRE_IN_SECS=1800
REFRESH_TOKEN_EXPIRE_IN_DAYS=14
SECRET_KEY=<YOUR-SECRET-KEY>
TELEGRAM_API_BASE_URL=https://api.telegram.org

DATABASE_USER=pentaract
DATABASE_PASSWORD=pentaract
DATABASE_NAME=pentaract
DATABASE_HOST=db
DATABASE_PORT=5432

  12. Deploy Pentaract using Docker Compose

    main

    Pentaract can be deployed using Docker Compose, which orchestrates the application service and a PostgreSQL database. The setup uses a custom network (pentaract-network) and a named volume (pentaract-db-volume) for persistent database storage.

    To run the deployment, ensure you have a .env file configured with the required environment variables, then use docker compose up.

    # Basic structure of the docker-compose.yml
services:
  pentaract:
    image: pentaract
    build: .
    env_file:
      - .env
    ports:
      - ${PORT}:8000
    depends_on:
      - db

  db:
    image: postgres:15.0-alpine
    environment:
      POSTGRES_USER: ${DATABASE_USER}
      POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
    volumes:
      - pentaract-db-volume:/var/lib/postgresql/data