DOCSEARCHbyAlgolia
Submit Docs

Marble Documentation

repository·main·Indexed 20 days ago

https://github.com/checkmarble/marble

An open-source fraud and compliance monitoring platform for AML screening, transaction monitoring, and automated investigations. Marble provides core capabilities including customer and company screening against sanctions and PEP lists, a unified case manager investigation suite, AI-driven automation for rule building, and an audit trail for detection programs. The platform is composed of a Go-based Backend API and Worker, and a TypeScript/Remix frontend, utilizing PostgreSQL, Redis, and Elasticsearch.

Tokens
9.7K
Snippets
21
Records
46
Agent score
69%

What's inside Marble

  1. Overview of Marble features

    main

    Marble is an open-source fraud and compliance monitoring platform designed for Transaction Monitoring, AML Screening, and Case investigation.

    Core Capabilities

    • Transaction Monitoring: Real-time or post-trade monitoring based on a custom data model that integrates with core banking and 3rd party tools.
    • Customer and Company Screening: Screening against sanctions, PEP (Politically Exposed Persons), and adverse media lists.
    • Continuous Monitoring: Automated, ongoing checks of customers against updated lists.
    • Investigation Suite: A unified case manager for exploring, annotating, and acting on alerts.
    • AI Automation: AI-driven assistance for rule building, detection optimization, and investigation tasks.
    • Reporting & BI: Embedded analytics and direct database access for BI tools.
    • Audit Trail: Searchable and unalterable logs for detection programs, workflows, and case actions.
    • Governance & Security: Includes RBAC (Role-based Access Control), SSO via OpenID Connect, IP whitelisting, and SOC 2 Type II compliance (Enterprise features).

  2. Understand Marble versioning and compatibility

    main

    Marble app versions follow the vX.Y.Z format. A single Marble app release corresponds to a specific compatible pair of backend and frontend executable versions.

    Important: The Marble app version number (e.g., v1.6.0) does not necessarily match the backend or frontend version numbers. The patch version (Z) or occasionally the minor version (Y) may differ between the app release and its constituent components.

  3. How Marble's core components work together

    main

    Marble is composed of three primary services that handle the application logic, background processing, and user interface:

    1. Backend API (Go + Gin): Acts as the REST API server, managing business logic and database interactions.
    2. Worker (Go): Handles background tasks, scheduled jobs, and data processing.
    3. Frontend (TypeScript + Remix): Provides the web interface for user interactions and integrates with the API.

    Supporting Services

    • PostgreSQL (v16+): The primary datastore and job queue.
    • Redis: Used for caching. While currently optional, it is highly recommended for feature stability.
    • Elasticsearch + Motiva: Required specifically for sanctions screening features.
    • Blob storage & Firebase: Required for blob storage (GCS / S3 / Azure) and Firebase-based authentication.

  4. Understand Marble Deployment Architecture

    main

    The Marble platform consists of three primary components:

    1. Marble API: The main backend service handling REST API requests. Requires direct database access.
    2. Marble Worker: A background task processor that uses the same container image as the API. It handles scheduled jobs and tasks.
    3. Marble Frontend: The browser-based user interface that communicates with the API.

    Deployment Options:

    • Server/VM: Using a reverse proxy (like nginx) for SSL termination and request routing.
    • Managed Containers: Using services like GCP Cloud Run or AWS ECS with auto-scaling.
    • Kubernetes: Deploying as containers with a service mesh and ingress controller.

  5. Deploy and Run Marble Frontend

    main

    The frontend is a Remix application served via Node.js.

    1. Build the Frontend

    # cd /opt/marble/src/marble/front
# pnpm install --frozen-lockfile
# pnpm --filter=app-builder run build
# pnpm --filter=app-builder --prod deploy /opt/marble/front/
# cp -a packages/app-builder/build /opt/marble/front/
# chown -R marble: /opt/marble/front

    2. Configure the Frontend

    Copy the example configuration to /etc/marble/front.conf and set:

    • NODE_ENV: Must be production.
    • SESSION_SECRET: A random, high-entropy string.
    • MARBLE_API_URL: The internal HTTP base the frontend uses to reach the API.

    3. Systemd Service Definition

    Create /etc/systemd/system/marble-front.service:

    [Unit]
Description=Marble frontend

[Service]
Type=simple
Restart=always
EnvironmentFile=/etc/marble/front.conf
WorkingDirectory=/opt/marble/front
User=marble
Group=marble
ExecStart=/opt/marble/front/node_modules/@remix-run/serve/dist/cli.js ./build/server/index.js

[Install]
WantedBy=multi-user.target

    4. Start and Verify

    # systemctl daemon-reload
# systemctl enable --now marble-front

# curl localhost:3000/healthcheck
# OK

  6. Configure Marble development environment variables

    main

    Marble uses .env files for configuration. For local development, use .env.dev.example as a template to create your .env.dev file.

    Important Note on Variable Inheritance: Docker Compose does not automatically inherit all variables from your .env files into the containers. Only specific variables defined in the Compose file are passed through. If you add new variables (e.g., to point file storage to a custom S3 bucket), you must also update the docker-compose-dev.yaml file to explicitly pass those variables to the relevant container.

  7. Configure environment variables for initial setup

    main

    Before starting Marble for the first time, you must configure the following environment variables to define the initial organization and administrator. These variables are used only during the first startup and can be removed once the initial setup is complete.

    Required variables:

    • CREATE_ORG_NAME: The name of the organization to be created.
    • CREATE_ORG_ADMIN_EMAIL: The email address for the initial administrator user.
    # Example configuration in .env
CREATE_ORG_NAME=MyFirstOrg
CREATE_ORG_ADMIN_EMAIL=admin@example.com

  8. Best practices for offloading storage classes and lifecycle rules

    main

    When configuring your storage bucket for offloaded decision data, follow these guidelines to avoid performance and cost issues:

    1. Use Standard Storage Classes: Use the "standard" object class. Do not move these objects to "archive" or "cold" storage classes (like S3 Glacier), as the per-operation cost to retrieve them for AI review or auditing greatly exceeds the storage savings.
    2. Lifecycle Rules: You may use lifecycle rules to delete objects after a certain period. Deleting objects under {bucket}/offloading/decision_rules is safe and will not cause application errors.
    3. Prefix-based Management: Objects are stored using decision status prefixes:
      • offloading/decision_rules/error
      • offloading/decision_rules/hit
      • offloading/decision_rules/no_hit

    This structure allows you to configure different lifecycle rules (e.g., different retention periods) for hits versus non-hits using your blob storage provider's prefix-based rules.

  9. Trigger ingestion by creating an `upload_logs` record

    main

    After uploading your file to the storage bucket, you must manually insert a record into the upload_logs database table to signal the system to begin processing the file.

    SQL Command: Execute an INSERT statement with the following structure:

    INSERT INTO upload_logs (org_id, user_id, file_name, status, table_name)
VALUES ({org_id}, {user_id},'{file_name}','pending','{table_name}');

    Required Values:

    • {org_id}: Your organization's ID.
    • {user_id}: The ID of the user performing the upload. (Find this via SELECT user_id, email FROM users;)
    • {file_name}: The full path of the file in the storage bucket (e.g., 123/customers/1662631200.csv).
    • {table_name}: The name of the target table.
    • status: Must be set to 'pending' to initiate the process.

  10. Delete a field from the Marble data model

    main

    To delete a field, you must perform manual operations in your Marble PostgreSQL database.

    Warning: Do not remove fields used in a scenario without also deleting the scenario first, as this may cause crashes.

    Follow these steps:

    1. Find the field ID: Join data_model_tables and data_model_fields to find the field_id for a specific table name.
    2. Remove links: Check data_model_links to ensure the field is not a parent_field_id or child_field_id. If links exist, delete them using their id.
    3. Delete from data model: Delete the record from data_model_fields using the field_id.
    4. Drop the column: Locate the table in the schema org-{orgName}.{yourTableName} and use ALTER TABLE ... DROP COLUMN.
    -- 1. Find the field ID
select *
from data_model_tables as t
inner join data_model_fields as f on (f.table_id=t.id)
where t.name = 'your_table_name';

-- 2. Check for links
select *
from data_model_links
where parent_field_id='field_id' or child_field_id='field_id';

-- If links found, delete them
delete from data_model_links where id='link_id';

-- 3. Delete from data model
delete from data_model_fields where id='field_id';

-- 4. Drop the actual field in the table
alter table org-{orgName}.{yourTableName} drop column {yourFieldName};

  11. Quick start installation with Docker Compose

    main

    To get a sense of Marble quickly, you can perform a basic installation using Docker Compose. This method requires cloning the repository, setting up an environment file, and running the development compose file.

    Prerequisites

    • Docker and Docker Compose
    • Git
    • 4GB RAM minimum
    • 10GB disk space

    Installation Steps

    1. Clone the repository and enter the directory.
    2. Copy the example environment file to .env.dev.
    3. Start Marble using the docker-compose-dev.yaml file.

    For full deployment options, refer to the official Installation guide.

    # 1. Clone the repository
git clone <repository-url>
cd marble

# 2. Copy example environment file
cp .env.dev.example .env.dev

# 3. Start Marble
docker compose -f docker-compose-dev.yaml --env-file .env.dev.example up

  12. Set up authentication for first login

    main

    Marble supports two methods for the initial login.

    Option A: Social Sign-In (Recommended)

    1. Configure Firebase authentication providers in your project (this works out of the box when using the Firebase emulator).
    2. Sign in using the email address specified in CREATE_ORG_ADMIN_EMAIL.

    Option B: Email/Password

    1. On the sign-in page, click the link: "Is it your first connection? Sign up".
    2. Create an account using the email address from CREATE_ORG_ADMIN_EMAIL and a secure password.
    3. Verify your email:
      • In development: Check the Firebase emulator logs for the verification link, or set the password directly via the auth emulator interface (default: http://localhost:4000).
      • In production: Check your email inbox for the verification link.