MQL Query Language and Toolset

repository·main·Indexed 19 days ago

https://github.com/mondoohq/mql

A query language and toolset for interacting with diverse infrastructure targets, including cloud providers, network devices, and local hosts. It includes the mqlx library for evaluating expressions and querying connected infrastructure, a provider-scaffold tool for developing new providers, and a specialized Ansible provider for static security and compliance analysis of playbooks and project directories.

Tokens
92.2K
Snippets
298
Records
418
Agent score
64%

What's inside MQL

  1. Use the OS provider for cnquery/cnspec

    main

    The OS provider allows you to access and query operating systems via cnquery or cnspec. It supports a wide range of platforms including Linux, macOS, and Windows.

    To interact with an operating system, you can use one of the following connectors:

    • Local OS: For querying the machine running the MQL command.
    • SSH: For querying remote Linux/macOS systems via SSH.
    • WinRM: For querying remote Windows systems via WinRM.
    • Docker: For querying containers.
  2. Use the Proc Filesystem for Linux provider

    main

    The procfs package provides helpers to parse structured information from the Linux /proc filesystem. Unlike standard human-readable tools like sysctl or top, this implementation is designed specifically for machine readers to consume structured data.

    A key feature of this implementation is that it does not require direct local system access; it is designed to work via remote connections, making it suitable for querying infrastructure remotely.

  3. Use the CloudFormation Provider for static analysis

    main
    The CloudFormation Provider allows you to perform static analysis on AWS CloudFormation and SAM templates. It parses YAML and JSON templates locally, meaning it does not require AWS credentials or API access to function. You can use it via an interactive shell or by running one-shot queries from the command line.
  4. Analyze Ansible infrastructure as code with the Ansible Provider

    main

    The Ansible provider allows for static security and compliance verification of Ansible code using cnquery. It performs entirely static analysis without executing anything against an inventory.

    You can target the provider in two ways:

    1. Single Playbook File: Analyzes specific plays, tasks, handlers, and variables within that file.
    2. Ansible Project Directory: Analyzes the entire codebase, including playbooks, roles (tasks, handlers, defaults, variables, metadata, dependencies), static inventory, host/group variables, Galaxy requirements, ansible.cfg, and vault-encrypted files.
    # To analyze a single playbook
    mql shell ansible providers/ansible/play/testdata/play_cert_validation.yaml
    
    # To analyze an entire project directory
    mql shell ansible ./my-ansible-project
  5. Use the Certificate Resource

    main

    The Certificate Resource in MQL allows you to inspect and validate SSL/TLS certificates using three primary methods:

    1. Load from file: Load a specific certificate file from the local filesystem and perform tests against it.
    2. Load OS certificate bundle: Access the operating system's trusted root certificate bundle to validate certificate chains.
    3. Load from live HTTP connection: Fetch and inspect the certificate presented by a live HTTP/HTTPS endpoint.
  6. Understand OpenStack Asset URLs and metadata

    main

    Each connection to OpenStack produces exactly one asset representing the Keystone-scoped project.

    • Asset URL Format: technology=openstack/project=<project-uuid>
    • Platform: openstack-project
    • Family: ["openstack"]
    technology=openstack/project=<project-uuid>
  7. Understand Elixir and Erlang lock file formats for SBOM

    main

    MQL parses Elixir and Erlang lock files to identify dependencies. Because these files use language-specific term syntax rather than standard formats like JSON, MQL uses regex-based extraction to identify package names and versions.

    Elixir mix.lock format

    Entries are represented as a map of package names to tuples.

    Entry structure: "name": {:hex, :name, "version", "hash", build_tools, deps, "repo", "outer_hash"}

    Example:

    %{ 
      "jason": {:hex, :jason, "1.4.1", "af1chabc...", [:mix], [], "hexpm", "fdfhash..."}
    }

    Erlang rebar.lock format

    Entries are represented as a list of package tuples.

    Entry structure: {<<"name">>, {pkg, <<"name">>, <<"version">>, <<"hash">>}, level}

    Example:

    [{<<"cowboy">>, {pkg, <<"cowboy">>, <<"2.10.0">>, <<"hash...">>}, 0}]

    Package Identification

    Both ecosystems use the Hex PURL scheme for identification:

    • PURL Pattern: pkg:hex/<name>@<version>
    # Elixir mix.lock example
    "jason": {:hex, :jason, "1.4.1", "af1chabc...", [:mix], [], "hexpm", "fdfhash..."}
    
    # Erlang rebar.lock example
    {<<"cowboy">>, {pkg, <<"cowboy">>, <<"2.10.0">>, <<"hash...">>}, 0}
  8. How mqlx works: Expression mode vs Asset mode

    main

    The mqlx package provides a high-level Go API for embedding the MQL engine in two distinct modes:

    1. Expression Mode: Used for evaluating MQL queries against arbitrary data you already have in memory (e.g., an event, a request, or a finding). It requires zero infrastructure setup, no providers, and no subprocesses. It uses an in-process core runtime.
    2. Asset Mode: Used to query actual infrastructure assets. You connect to an asset (via Connect or ConnectLocal), compile a query, and then execute that query against the connected asset to retrieve structured data.

    Core Abstractions

    TypeRoleLifetime / reuse
    EnvManages feature flags, provider schemas, and the expression runtimeOne per process; concurrent-safe
    ConnRepresents a connection to a specific assetMany queries per Conn; concurrent-safe
    QueryA compiled, immutable MQL queryEvaluate against any Conn or in expression mode
    ResultThe output of a single evaluationProvides Value(), Decode(), Err(), and Raw()

    Relationship Model

    • An Env is the root. It creates Conn objects or compiles Query objects.
    • A Query is asset-independent. You can compile a query once using Env.Compile and then run it multiple times against different connections using Query.EvalOn(ctx, conn, ...).
    // Expression Mode Example
    env, _ := mqlx.NewEnv()
    q, _ := env.Compile("props.count > 3", mqlx.WithProps(map[string]any{"count": 0}))
    res, _ := q.Eval(ctx, mqlx.WithPropValues(map[string]any{"count": 5}))
    
    // Asset Mode Example
    conn, _ := env.ConnectLocal(ctx)
    res, _ := conn.Query(ctx, "users { name uid }")
  9. Catalog Jenkins plugins using MQL

    main

    MQL can catalog Jenkins plugins by scanning directories for .jpi or .hpi files (which are ZIP archives). The cataloger extracts metadata from the META-INF/MANIFEST.MF file within each archive to identify installed plugin versions and their dependencies. This is used for supply chain security and vulnerability management.

    By default, the parser looks in standard Jenkins plugin directories, but you can specify a custom path using the jenkins.packages resource.

    mql run os -c "jenkins.packages(path: '/var/lib/jenkins/plugins') { list { name version } }"
  10. Use Chocolatey package resources in MQL

    main

    MQL provides two new resources for managing Windows software inventory via Chocolatey: chocolatey.packages (for listing) and chocolatey.package (for individual package details). These resources allow you to inspect installed packages, versions, dependencies, and pin status.

    Because these resources parse .nuspec XML files directly, they work in offline scans, container images, and remote connections without requiring the choco.exe CLI to be present.

    mql run os -c "chocolatey.packages { list { name version pinned } }"
  11. How cross-asset traversal and root resolution work

    main

    MQL supports in-query cross-asset traversal using a typed-root foundation. This allows queries to chain across different asset types by declaring a root asset and then traversing its fields.

    Key Mechanics

    • Root Declaration: The connection itself declares its root (the runtime authority).
    • Field Declaration: Fields declare the target root by name (a forward reference). The declaring provider only needs to know the name of the root, not its full schema.
    • Resolution: Chaining is statically known because the root name is declared in the field, even if the schema is not immediately available.

    Error Handling and Lifecycle

    • Missing Schemas: If a schema is absent during compile-time, the system degrades gracefully and issues a warning. Member checking is deferred to runtime.
    • Namespace Rules: Cross-calls must be declared. Undeclared global-namespace cross-calls are deprecated (starting in v14) and will stop resolving in v15.
    • Secrets: Secrets used during traversal must be explicit. There is no implicit --env support inside a traversal; secrets must be provided via an explicit source (such as the existing vault/conf.Credentials mechanism) keyed to the target asset's identity.
  12. Understand Go SBOM cataloging and dependency files

    main

    The cnspec SBOM Cataloger for Go identifies dependencies by parsing standard Go ecosystem files. This allows for Software Bill of Materials (SBOM) generation in local, remote (SSH/WinRM), or containerized environments.

    Key Files Parsed

    • go.mod: Declares the module path, minimum Go version, and dependency requirements. It distinguishes between direct and transitive dependencies using the // indirect comment.
    • go.sum: A checksum database containing cryptographic hashes (h1: prefix) for dependencies to ensure integrity.
    • vendor/modules.txt: Used when go mod vendor is active; it lists all vendored modules and uses ## explicit to identify direct dependencies.
    • Go Binaries (Future Phase): Compiled binaries can be analyzed using debug.BuildInfo to extract the dependency tree without source code.