Mangle Documentation

repository·main·Indexed 25 days ago

https://github.com/google/mangle

A deductive database programming language based on Datalog, extended with aggregation, recursion, and temporal reasoning. Implemented as a Go library (mangle-go) for embedding in applications, Mangle supports complex data querying, domain modeling, and knowledge graphs. It includes tools like mgwhy for explaining derivations and scinfo for inspecting simplecolumn files.

Tokens
18.5K
Snippets
65
Records
108
Agent score
76%

What's inside Mangle

  1. Introduction to Mangle Datalog

    main

    Mangle Datalog is a declarative logic programming language and deductive database system. It allows users to represent data as facts (logical statements similar to table rows) and define rules to compute new facts from existing ones.

    Key features include:

    • Declarative Syntax: Specify what you want rather than how to compute it.
    • Structured Data: Support for complex types like lists, maps, and records (structs).
    • Modularity: Programs can be split into modules.
    • Type Safety: Benefits from static type checking and type inference.
    • Advanced Querying: Supports aggregation and function calls.
  2. Overview of Mangle language

    main

    Mangle is a programming language for deductive database programming, extending Datalog with features like aggregation, function calls, and optional type-checking. It is designed to represent and query data from multiple sources in a uniform way and can model domain knowledge using n-ary relations and structured data.

    Key capabilities include:

    • Simple Queries: Select-project-join relational queries.
    • Aggregation: Grouping and performing calculations (e.g., fn:Count()).
    • Recursive Queries: Defining rules that reference themselves to traverse graphs or hierarchies.
    • Knowledge Graphs: Supporting arbitrary n-ary relations and property graphs.
    • Temporal Knowledge Graphs: Associating facts with time intervals to query when truths are valid.
  3. Understand the Mangle data model (Positive Datalog)

    main

    Mangle uses a 'positive datalog' language based on a logical data model. The core concepts are:

    • Facts: The fundamental unit of data. All data is represented as facts.
    • Base Facts (Extensional Relations): The initial set of facts that already exist in your system.
    • Derived Facts (Intensional Relations): New facts created by applying rules to base facts.
    • Evaluation: The process of applying rules to derive new facts.
    • Knowledge Base: The complete set of both base and derived facts.

    In this model, rules can only add facts; they can never remove them. This allows you to organize data freely without being constrained by the original source structure.

  4. Understand provenance modes: Simple vs Full

    main

    Mangle provides two modes for provenance, depending on whether you need to explain transforms or prioritize performance.

    Simple Mode (-mode=simple)

    • Mechanism: Post-hoc backward-chaining against the final fact store.
    • Pros: No engine changes required; zero cost during the actual evaluation phase.
    • Capabilities: Handles positive Datalog, equality/inequality, stratified negation, and recursion.
    • Limitations: Cannot explain let- or do- transforms; these nodes are flagged as partial.

    Full Mode (-mode=full)

    • Mechanism: Uses a DerivationRecorder to capture events during evaluation.
    • Pros: Captures everything in Simple mode plus transforms.
    • Capabilities:
      • let-transforms: One proof node per row.
      • do-transforms (aggregation): One proof node per group, including group-by keys and input facts.
    • Cost: One callback per derivation during evaluation.

    Negation Handling

    In both modes, Mangle uses stratified evaluation. For a negated atom !p(a), the explainer checks the final fact store:

    • If p(a) is absent, the premise is satisfied via an absence_leaf.
    • If p(a) is present, the rule cannot fire and the explainer backtracks. Note: Simple mode uses 'bare absence' and does not witness why candidate rules failed to derive the atom.
  5. Understand the relationship between Datalog and Relational Algebra

    main
    Non-recursive Datalog programs can be translated into Relational Algebra expressions. Relational algebra provides the mathematical foundation for SQL operations such as selection (filtering), projection, and joining. This connection allows Datalog logic to be executed by relational database management systems.
  6. Define Temporal Facts with Intervals

    main

    You can track when facts are true by adding a time interval using the @[start, end] syntax.

    Special Bounds:

    • _: Unbounded (beginning of time or ongoing into the future).
    • now: The current evaluation time.

    Date Semantics: Dates without time (e.g., 2024-01-01) are interpreted as midnight (00:00:00) at the start of that day. Mangle uses inclusive intervals [start, end]. To include the entirety of a day, use the midnight of the following day as the end bound.

    # Alice was on engineering from Jan 2020 to June 2023
    team_member(/alice, /engineering)@[2020-01-01, 2023-06-15].
    
    # Bob joined engineering in 2019 and is still there
    team_member(/bob, /engineering)@[2019-06-01, _].
    
    # Something that happened at a specific moment
    login(/alice)@[2024-03-15T10:30:00].
    
    # Currently active (from 2024 until now)
    active(/alice)@[2024-01-01, now].
    
    # Something happening right now
    logged_in(/bob)@[now].
  7. Integrate Mangle with gRPC

    main

    Mangle's API design is independent of the transport layer. You can wrap Mangle queries in a gRPC service. The service implementation should translate the wire format (e.g., Protobuf messages) into a Mangle query or program for evaluation.

    // gRPC service definition
    message ByAvRequest { repeated Availability availability = 1; }
    message ByAvReply { repeated Volunteer reply = 1; }
    service VolunteerQuery {
      rpc GetByMatchingAvailability (ByAvRequest) returns (ByAvReply) {}
    }
  8. Define Extensional Databases using Predicates

    main

    In Mangle, you can define an 'extensional' database by listing facts directly in the source. Each fact is associated with a predicate (which acts like a table). Use constant symbols (e.g., /v/1, /monday) as identifiers to represent specific entities or values.

    # Define entities
    volunteer(/v/1).
    
    # Define properties/facts
    volunteer_name(/v/1, "Aisha Salehi").
    volunteer_time_available(/v/1, /monday, /afternoon).
    volunteer_skill(/v/1, /skill/admin).
  9. Set up a local environment for Mangle documentation

    main

    To build the Mangle documentation locally, create and activate a Python virtual environment, then install sphinx and the required dependencies from the documentation directory's requirements.txt file.

    python -m venv manglereadthedocs
    . manglereadthedocs/bin/activate
    pip install -U sphinx
    READTHEDOCS=<path to readthedocs dir>
    pip install -r ${READTHEDOCS}/requirements.txt
  10. Handle absence of values using negation

    main

    Because Datalog primarily deals with positive information, rows that do not meet a criteria (e.g., a project with zero developers) will not appear in a standard aggregation result. To include these 'zero' cases, you must use negation (!).

    To identify entities that lack a specific property:

    1. Define a helper table that identifies entities with the property.
    2. Define a second table that selects all entities from a base set and uses ! to negate the helper table.

    Constraint: Every variable mentioned in the head of a rule must be mentioned in at least one subquery that is not negated.

    Example pattern for finding projects without developers:

    project_with_developers(ProjectID) ⟸
      project_assignment(ProjectID, _, /software_development, _).
    
    project_without_developers(ProjectID) ⟸
      project_name(ProjectID, _).
      !project_with_developers(ProjectID)
  11. Rectify Datalog rules for translation

    main

    Before translating a Datalog program, rules must be rectified. Rectification involves rewriting rules that mention constants in the head or repeat variables by introducing fresh variables and explicit equations.

    Example Transformation: Input rule:

    knows(X, X) :- person(X).

    Rectified rule:

    knows(X, Y) :- person(X), Y = X.

    Once rectified, the relational expression for an intensional predicate is the union of all its rules, processed inductively from level 0 (predicates referring only to extensional predicates) upwards.