DOCSEARCHbyAlgolia
Submit Docs

Midaz Core Banking Platform

repository·develop·Indexed 19 days ago

https://github.com/lerianstudio/midaz

A source-available, composable core banking platform built around a double-entry ledger. Midaz provides essential banking primitives including onboarding, multi-asset transactions, and real-time fraud prevention via Tracer. The platform features a unified Ledger binary managing Onboarding, Transaction, CRM, and Fees domains, supporting multi-tenancy and hexagonal architecture. It includes a shared infrastructure stack with PostgreSQL, MongoDB, Valkey, and RabbitMQ, with optional HashiCorp Vault integration for CRM envelope encryption.

Tokens
115K
Snippets
192
Records
352
Agent score
65%

What's inside Midaz

  1. Overview of Midaz Architecture and Services

    develop

    Midaz is a composable core banking platform built around a double-entry ledger. It consists of two primary Go-based units and a supporting infrastructure stack.

    Core Units

    • Ledger: A unified binary handling onboarding, transactions, CRM, and in-process fee calculation. It uses PostgreSQL and MongoDB for storage.
    • Tracer: A real-time transaction validation and fraud prevention engine using a CEL rule engine and hash-chained immutable audit trails. It uses PostgreSQL for storage.

    Infrastructure Stack

    Midaz utilizes a docker-compose stack including:

    • PostgreSQL 17 (Primary/Replica)
    • MongoDB (Replica Set)
    • RabbitMQ (Event-driven processing)
    • Valkey
    • Grafana/OpenTelemetry (Observability)

  2. Overview of the Midaz Ledger

    develop

    The Ledger is a unified Go binary that serves as the primary deploy unit for the Midaz platform. It runs on port :3002 and integrates four core domains into a single process without gRPC overhead, allowing for high-performance in-process calls between domains.

    Core Domains

    • Onboarding: Manages the resource hierarchy: Organization → Ledger → Assets/Portfolios/Segments → Accounts.
    • Transaction: Handles double-entry postings, balances, and the transaction lifecycle (commit, cancel, revert).
    • CRM: Manages holders and instruments, including PII encryption and search tokens.
    • Fees: A fee engine applied during the transaction creation process.

    Key Architectural Features

    • Unified Deployment: All domains scale together on a single port.
    • Multi-tenancy: Supports tenant-isolated database resolution via lib-commons, controlled by the MULTI_TENANT_ENABLED environment variable (defaults to off).
    • Hexagonal + CQRS Pattern: Uses a command/query separation where HTTP handlers call use cases, which interact with repository interfaces and adapters. Domain models are centralized in pkg/mmodel.

  3. Overview of Tracer Streaming Events

    develop

    The tracer component (components/tracer) emits streaming events using lib-streaming v1.6.2. Events are formatted as CloudEvents 1.0 in binary mode and are transmitted over Kafka/Redpanda.

    Key Technical Details

    • Producer: github.com/LerianStudio/lib-streaming v1.6.2
    • CloudEvents Source (ce-source): lerian.midaz.tracer
    • Emission Posture: All events are emitted via pkgStreaming.EmitImportant. This is a best-effort, synchronous operation performed at the post-commit stage. A failure to emit will log a warning but will not fail the primary request. The database write is the source of truth for durability.
    • Tenant Isolation: Does not use organizationId or ledgerId. Isolation is handled via the ce-tenantid header, resolved using pkgStreaming.ResolveTenantID(ctx).
    • Master Toggle: Streaming is controlled by the STREAMING_ENABLED environment variable (defaults to false). If disabled or if STREAMING_BROKERS is empty, a NoopEmitter is used.

  4. Overview of CRM Streaming Events

    develop

    The CRM component (embedded in the ledger binary) emits streaming events via Kafka using the CloudEvents 1.0 specification in binary mode.

    Key Characteristics:

    • Producer: github.com/LerianStudio/lib-streaming v1.4.0.
    • Source (ce-source): lerian.midaz.ledger (the process-wide source).
    • Routing: Consumers identify CRM events via the crm_ topic segment and holder or instrument resource types.
    • Reliability: Events are emitted via pkgStreaming.EmitImportant. This is a best-effort, synchronous emit at the post-commit stage. A failure to emit logs a warning but does not fail the HTTP request; durability is tied to the database write.
    • Configuration: Streaming is disabled by default. Enable it using the STREAMING_ENABLED=true environment variable. If STREAMING_BROKERS is empty or no events are registered, a NoopEmitter is injected.

  5. What is Tracer?

    develop

    Tracer is a real-time transaction validation and fraud prevention API designed for financial systems. It acts as an independent validation layer that decouples fraud prevention and spending controls from your core business logic. It provides instant decisions—ALLOW, DENY, or REVIEW—based on rule expressions and hierarchical spending limits.

    Key characteristics:

    • Low Latency: Designed for sub-100ms decisions (P99 < 80ms).
    • Rule Engine: Uses Google's Common Expression Language (CEL) for type-safe, sandboxed logic.
    • Multi-scope Limits: Supports spending controls at the Account, Segment, or Portfolio level.
    • Audit-ready: Maintains immutable audit trails for SOX/GLBA compliance.

  6. Understand the Branch Review Audit & Remediation Campaign

    develop

    The feat/monorepo-consolidation branch underwent a rigorous audit to identify and remediate code-review findings. The campaign follows a strict classification system to distinguish between real issues and false positives.

    Audit Verdict Summary:

    • Total Findings: 68
    • Real Issues: 54
    • Partially Valid: 10
    • False Positives: 4

    Guiding Principle: Avoid over-engineering. Prioritize lean, correct fixes over edge-case 'gold-plating'.

  7. Identify Midaz Deploy Units and Ports

    develop

    Midaz consists of specific deployable units. Note that many logical services are folded into a single binary to avoid network hops (no gRPC is used for in-process communication within the ledger).

    Deploy unitPortDescription
    components/ledger3002Unified binary serving onboarding, transaction, CRM (holders/instruments), and fees.
    components/tracer4020Separate service for transaction validation and fraud prevention (CEL rules, spending limits, audit trail).
    components/infra-Docker-compose for shared infrastructure (PostgreSQL, MongoDB, Valkey, RabbitMQ, otel-lgtm).

    Note on CRM and Fees: These are not standalone deploy units. crm is a package tree imported by the ledger binary, and fees are embedded within the ledger package structure.

  8. Key Capabilities of the Midaz Engine

    develop

    The Midaz platform provides several core financial engineering capabilities:

    • Double-entry engine: Ensures every credit has a matching debit.
    • Multi-asset support: Handles transactions across different currencies with automatic rate conversion.
    • Complex transactions: Supports n:n operations (multiple sources to multiple destinations).
    • Immutable records: All transactions are permanently recorded for auditability.
    • Async processing: Uses RabbitMQ for event-driven transaction handling.
    • Optimistic-concurrency balances: Uses version-based concurrency control for balance updates.
    • Hexagonal + CQRS Architecture: Isolates domain logic from adapters and separates commands from queries.
    • OpenAPI: Provides RESTful endpoints with generated OpenAPI specifications.

  9. Core concepts of Tracer validation

    develop

    To use Tracer effectively, you must understand its five core components:

    1. Validation Request: The input payload containing a unique Request ID (for idempotency), transaction data (type, amount, currency, timestamp), account context (ID, type, status), optional contexts (segment, portfolio, merchant), and custom metadata.
    2. Rules Engine: Uses CEL expressions to evaluate transaction data. Rules can be dynamic and updated without redeployment.
    3. Spending Limits: Hierarchical controls applied at different scopes:
      • Account-level: Per-user limits.
      • Segment-level: Group-based limits (e.g., VIP users).
      • Portfolio-level: Organization-wide caps.
    4. Decision Flow: The logical progression from Transaction $\rightarrow$ Rules Evaluation $\rightarrow$ Limit Check $\rightarrow$ Decision.
    5. Audit Trail: An immutable record containing request/response payloads, rule results, decision rationale, and correlation IDs.

  10. Apply correct log levels

    develop

    Log levels must be used strictly to ensure production logs remain scannable and meaningful.

    LevelUsageExamples
    DebugPer-request/message detailSQL queries, cache details, batch stats, entry/exit logs
    InfoSparse, one-time process milestones ONLYBoot sequence, config loaded, server started, leader elected
    WarnRecoverable issuesBusiness/validation failures, degraded-but-recoverable fallbacks
    ErrorSystem failuresInfrastructure failures requiring operator intervention

    Prohibition: Do not use Info level for per-request narration (e.g., Initiating..., Successfully...). These should be captured via spans or Debug logs.

  11. Prohibit sensitive data in telemetry signals

    develop

    No sensitive information is permitted in any telemetry signal (log lines, span attributes, metric labels, or persisted error metadata).

    Prohibited Data:

    • Financial values (amounts, balances, prices)
    • PII (Personally Identifiable Information)
    • Secrets
    • Raw payloads
    • SQL arguments

    Important Exception: The lib-streaming event bus is NOT subject to this rule. The event bus is a governed wire contract (JSONShape-locked) designed to carry domain data (e.g., Available or OnHold balances) via redaction-reviewed constructors. This rule applies strictly to observability/telemetry, not the data plane.

  12. Configure TLS for migrations via `DB_SSL_MODE`

    develop

    TLS for the migration runner is controlled exclusively via the DB_SSL_MODE environment variable.

    Important Security Note: The migration runner does not implement the same in-code TLS enforcement, error classification, or telemetry used by the Tracer application itself. It relies entirely on the standard sslmode parameter of the underlying migration tool. For production deployments, it is recommended to set DB_SSL_MODE=require.