FPBrowser2API Automation Framework

repository·github·Indexed 42 days ago

https://github.com/lakysir/fpbrowser2api

An automation framework that wraps AI video and image generation sites—such as Sora, Google Veo, and ChatGPT—into schedulable APIs. It utilizes fingerprint browsers (specifically RoxyBrowser) and Playwright/CDP to execute automation logic within real browser contexts, avoiding protocol reverse-engineering. Features include a management backend for task and project configuration, and support for models like seedance-2, veo-3-1, and gpt-image2.

Tokens
17.8K
Snippets
46
Records
71
Agent score
55%

What's inside FPBrowser2API

  1. Understand why image tasks use the `/v1/videos` endpoint

    github
    Image generation tasks are routed through the /v1/videos endpoint to maintain compatibility with NewAPI's asynchronous video channels and upstream calling patterns. When an image task completes, the response will include an image_url and a generic url field.
  2. Configure RoxyBrowser for FPBrowser2API

    github

    FPBrowser2API relies on RoxyBrowser to provide isolated browser environments. Follow this two-step configuration process:

    1. RoxyBrowser Setup

    1. Download and log in to RoxyBrowser.
    2. Create a Workspace, Project, and Browser Windows.
    3. Configure proxies, account info, and target site login states for each window.
    4. Enable the Local API or LAN API.
    5. Note your API Address, API Token, workspaceId, and window dirId.

    2. FPBrowser2API Admin Configuration

    Navigate to the management backend and configure the following:

    • Project Management: Create a local project.
    • Browser Server: Add RoxyBrowser details:
      • vendor: roxy
      • lan_addr: Your RoxyBrowser API base URL
      • access_key: Your RoxyBrowser token (can be empty)
    • Space: Enter your RoxyBrowser workspaceId.
    • Sync Windows: Use the backend to sync windows (this calls RoxyBrowser's GET /browser/list_v3 and GET /browser/detail).
    • Task Types: Create task types and bind them to specific windows using appropriate processors (e.g., veo_workflow, dreamina_workflow, gpt_workflow, sora_gen_video, grok_workflow).
  3. Quick Start: Install and Run FPBrowser2API

    github

    Follow these steps to set up the environment and start the service locally.

    1. Create a Virtual Environment

    cd fpbrowser2api
    python -m venv venv

    2. Activate the Environment

    Windows:

    venv\Scripts\activate

    Linux / macOS:

    source venv/bin/activate

    3. Install Dependencies and Start

    pip install -r requirements.txt
    python main.py

    Default Service Details:

    • URL: http://127.0.0.1:8002
    • Default Admin Credentials:
      • Username: admin
      • Password: admin
    cd fpbrowser2api
    python -m venv venv
    venv\Scripts\activate
    pip install -r requirements.txt
    python main.py
  4. Workflow for Seedance 2, Nana Banana, and VEO 3.1 Video APIs

    github

    The video generation APIs follow an asynchronous NewAPI-style pattern consisting of two steps: submitting a task and polling for the result.

    Step 1: Submit a Task

    Send a POST request to /v1/videos with your model and prompt. The response will contain a task_id.

    Step 2: Query Task Status

    Use the task_id to poll the status via a GET request to /v1/videos/{task_id}.

    Task Statuses

    statusMeaning
    queuedTask is in the queue
    in_progressTask is currently generating; continue polling
    completedTask finished. Retrieve the result from video_url, image_url, or url
    failedTask failed. Check error.message for details
    # 1. Submit Task
    curl -X POST https://xxx.xxx.xxx/v1/videos \
      -H "Authorization: Bearer api-key" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "veo-3-1",
        "prompt": "an astronaut surfing on a glowing ocean at night, cinematic camera movement",
        "duration": 8,
        "aspect_ratio": "16:9"
      }'
    
    # 2. Query Task (using the returned task_id)
    curl https://xxx.xxx.xxx/v1/videos/8c7e4b0e6f3b4b0a9e0a0d8f4f0c1234 \
      -H "Authorization: Bearer api-key"
  5. Manage FPBrowser2API Service with Scripts

    github

    Use the provided service scripts to control the background process.

    Linux / macOS / Git Bash

    cd fpbrowser2api
    chmod +x ./fpbrowser2api_service.sh
    ./fpbrowser2api_service.sh start
    ./fpbrowser2api_service.sh status
    ./fpbrowser2api_service.sh restart
    ./fpbrowser2api_service.sh stop

    Windows PowerShell

    cd fpbrowser2api
    powershell -ExecutionPolicy Bypass -File .\fpbrowser2api_service.ps1 start
    powershell -ExecutionPolicy Bypass -File .\fpbrowser2api_service.ps1 status
    powershell -ExecutionPolicy Bypass -File .\fpbrowser2api_service.ps1 restart
    powershell -ExecutionPolicy Bypass -File .\fpbrowser2api_service.ps1 stop
  6. How task handlers and quota refreshers work

    github

    The system uses a dynamic registry pattern to decouple task types from their execution logic. For every task_type, the frontend can select specific handlers for two distinct lifecycle stages:

    1. Task Creation (create_task_handler): Defines how a task is submitted to the system. The registry maps a string key to a CreateTaskHandler function.
    2. Quota Refreshing (refresh_quota_handler): Defines how the system checks the remaining balance/credits for a specific window or account. The registry maps a string key to a RefreshQuotaHandler function.

    To add new functionality, you implement a new handler function and register it in the CREATE_TASK_HANDLERS or REFRESH_QUOTA_HANDLERS dictionaries. This allows adding new capabilities without changing the database schema.

    # Example of the registry structure
    CREATE_TASK_HANDLERS = {
        "default_submit": ("Description", create_task__default_submit),
        # ...
    }
    
    REFRESH_QUOTA_HANDLERS = {
        "noop": ("Description", refresh_quota__noop),
        # ...
    }
  7. How the Browser Extension Bridge works

    github

    The Browser Extension Bridge allows the server to delegate tasks to a browser extension running inside a fingerprint browser. Instead of using Playwright or CDP to control the page directly (which can be detected), the server forwards tasks via WebSockets to an extension.

    Workflow:

    1. Registration: The extension connects to the server via WebSocket and registers using a space_id and window_key.
    2. Configuration: The server can annotate target URLs with configuration (like space_id and window_key) in the URL hash. The extension's content script reads these and stores them locally.
    3. Execution: The server submits a task via the bridge. The extension executes the task within the browser context and reports progress and results back via the WebSocket connection.
  8. Implement a custom site executor using FingerprintBrowserAutomationBase

    github

    To automate specific websites (e.g., PayPal, Sora, Veo), you should create a subclass of FingerprintBrowserAutomationBase. This base class provides reusable, high-level primitives for page-level automation, such as safe clicking, text input, and human-like interactions. You should use these primitives to compose your business logic and selectors rather than writing raw Playwright commands, which reduces errors and repetition.

    Design Principles:

    • Methods are generally "best-effort" and return a bool indicating success. For critical actions, check the return value or use methods with raise_on_fail=True.
    • Use step_log to record human-readable operation steps for frontend display.
    • The class handles common Playwright complexities like locator differences and fallback click mechanisms.
    from src.services.browser_automation_base import FingerprintBrowserAutomationBase
    
    class MySiteExecutor(FingerprintBrowserAutomationBase):
        async def run_business_flow(self):
            # Use base primitives to implement site-specific logic
            await self.goto("https://example.com")
            await self.fill_first(["#username", "#email"], "my_user_id", label="Login ID")
            await self.safe_click(["button.submit"], label="Submit Button")
  9. Understand Admin API authorization and permissions

    github

    The application implements a middleware-based authorization system for administrative operations.

    Authorization Rules

    • Write Operations: POST, PUT, PATCH, and DELETE requests to paths starting with /api/admin require a valid Bearer token.
    • Authentication: Tokens are validated against admin.active_admin_tokens. If the token is missing or invalid, the API returns 401 Unauthorized.
    • Role-Based Access Control (RBAC): Even with a valid token, the user must have the is_admin flag set to True in the database. If not, the API returns 403 Forbidden with the message "仅管理员可执行此操作" (Only admins can perform this operation).

    Exceptions (Bypass Paths)

    Certain paths do not require admin authorization, including:

    • Login/Logout: /api/admin/login, /api/login, /api/admin/logout, /api/logout.
    • Password changes: /api/admin/change-password.
    • Specific non-admin write patterns (e.g., certain task-type window configurations, AI agent chats, and network capture controls).
  10. Understand the AI Browser Agent action schema

    github

    The AI agent communicates by returning a JSON object representing its next step. When building tasks or interpreting agent behavior, use the following supported action types:

    ActionDescription
    fillFills an input field. Requires ref and value_key (to reference data) or value.
    selectSelects an option in a dropdown. Requires ref and value.
    clickClicks a specific element identified by ref.
    check / uncheckToggles a checkbox or radio button.
    pressPresses a keyboard key (e.g., Enter, Tab, Escape).
    shortcutTriggers a keyboard shortcut (e.g., F12).
    open_devtools_networkOpens the Network log panel and starts capturing logs.
    waitPauses execution for a specified number of milliseconds (ms).
    gotoNavigates the browser to a specific url.
    fetch_jsonCalls an API via page_fetch_json using method, url, headers, and json.
    fetchCalls an API via page_fetch_tx to read text response.
    evaluateExecutes arbitrary JavaScript in the page context via evaluate.
    doneSignals task completion. Requires a summary in Chinese.
    need_humanSignals that manual intervention is required (e.g., CAPTCHA, SMS). Requires a summary in Chinese.
  11. Provide callbacks for progress and logging

    github

    When calling upscale_video_to_1080p, you can provide two types of callbacks to monitor the operation:

    1. log_cb (LogCB): A callable that accepts a single string (str) -> None. Used for receiving status logs.
    2. progress_cb (ProgressCB): An async callable that accepts an integer progress value and an optional detail dictionary (int, Optional[Dict[str, Any]]) -> Awaitable[None].

    Progress Stages:

    • upscale_1080p_start: Initial stage.
    • upscale_1080p_processing: Task submitted, waiting for result.
    • upscale_1080p_done: Successfully completed.

    Example Progress Callback:

    async def my_progress_callback(progress: int, detail: dict | None):
        print(f"Progress: {progress}% - Details: {detail}")
  12. Run the FPBrowser2API service

    github

    To start the FPBrowser2API service, navigate to the project directory and execute the main.py script using Python. The service is powered by uvicorn and uses configuration settings (host and port) defined in the project's core configuration module.

    cd fpbrowser2api
    python main.py