Apache Ranger Documentation

repository·master·Indexed 21 days ago

https://github.com/apache/ranger

A framework for centralized security administration in distributed environments, providing authorization and user synchronization for services like HDFS, Hive, HBase, and Kafka. It supports Attribute-Based Access Control (ABAC) via the RangerMultiSourceUserStoreRetriever and includes a microservices-based Audit Server architecture for ingesting and dispatching audit events to Solr, HDFS, S3, and Azure.

Tokens
41.8K
Snippets
96
Records
178
Agent score
77%

What's inside Apache Ranger

  1. Overview of Apache Ranger Python Clients

    master

    The apache-ranger package provides typed helpers for interacting with various Apache Ranger services. The following clients are supported:

    • RangerClient: Manages Ranger Admin APIs (service definitions, services, policies, roles, security zones, plugin info, and policy delta maintenance).
    • RangerUserMgmtClient: Manages user, group, and user-group APIs.
    • RangerKMSClient: Manages Ranger KMS key management APIs.
    • RangerGdsClient: Manages Governed Data Sharing (GDS) APIs (datasets, projects, datashares, shared resources, and GDS policies).
    • RangerPDPClient: Manages Policy Decision Point (PDP) authorization APIs (single/batch authorization and effective resource permissions).
  2. Overview of Apache Ranger capabilities

    master

    Apache Ranger is an open-source authorization framework designed for data processing services. It provides a centralized security administration model that allows you to manage authorization tasks through a central UI or via REST APIs.

    Key capabilities include:

    • Fine-grained authorization: Manage specific actions and operations across 20+ different data processing services from a single administration tool.
    • Standardized authorization: Provides a consistent authorization method across diverse data processing environments.
    • Multiple authorization models: Supports Role-Based Access Control (RBAC), Attribute-Based Access Control (ABAC), and Tag-Based Access Control (TBAC).
    • Centralized auditing: Tracks and audits user access and administrative security actions across all integrated services.
  3. Understand the Ranger Security Model and Trust Boundaries

    master

    Apache Ranger operates as a distributed Policy Decision Point (PDP) and Policy Enforcement Point (PEP) system. It is important to understand the following architectural security assumptions:

    • Plugin Trust: Ranger treats every plugin (PEP) as fully trusted once authenticated. It does not make cross-node integrity claims to keep runtime overhead low. If a host running a plugin is compromised, the attacker can bypass enforcement locally.
    • Authentication vs. Authorization: Ranger provides authorization, not authentication. It assumes the principal (user/service) has already been authenticated by an upstream layer (e.g., Kerberos or LDAP). Ranger cannot defend against a spoofed principal if the upstream authentication is compromised.
    • Policy Distribution & Revocation: Policies are distributed from the Admin to plugins via a periodic pull. Because plugins use a local cache when the Admin is unreachable, revocation is not instantaneous. A revoked permission may persist until the next successful policy pull. Operators must size the pull interval based on data sensitivity.
    • The 'Deny' is not a Sandbox: Ranger only protects code paths that explicitly consult the Ranger plugin. If a guarded service has a code path that bypasses the plugin, that path is not protected.
    • Admin as Root of Trust: The top-level administrator is the root of authority. The model does not protect against a malicious omnipotent administrator.
  4. Configure Authentication for Ranger Clients

    master

    Authentication methods depend on the target Ranger service configuration:

    • Basic Auth: Pass a (username, password) tuple to RangerClient.
    • Kerberos/SPNEGO: Pass requests_kerberos.HTTPKerberosAuth() to the client constructor (requires requests-kerberos package).
    • KMS Hadoop Simple Auth: Use HadoopSimpleAuth("user") for RangerKMSClient.
    • PDP Trusted-Header/JWT: Pass the configured trusted caller header or Authorization: Bearer <token> via the headers argument in RangerPDPClient.
    • Custom Headers/Params: Use headers or query_params in the client constructor.
    # Kerberos Example
    from requests_kerberos import HTTPKerberosAuth
    from apache_ranger.client.ranger_client import RangerClient
    
    ranger = RangerClient("https://ranger.example.com:6182", HTTPKerberosAuth())
  5. Configure the Ozone action-matcher feature flag

    master

    The Ozone action-matcher feature flag is controlled at runtime via the ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition property in ranger-admin-site.xml.

    In the Docker environment, this is mapped from FF_ENABLE_OZONE_ACTION_MATCHES_CONDITION in scripts/admin/ranger-admin-install-<db>.properties.

    First-time setup

    Set FF_ENABLE_OZONE_ACTION_MATCHES_CONDITION=true in your install properties file, then bring up the services.

    Changing the flag later

    Option 1: Recreate Admin (Recommended for fresh setup) Edit the property in scripts/admin/ranger-admin-install-<db>.properties and run:

    docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-ozone.yml up -d --build --force-recreate ranger

    Option 2: Edit live configuration Edit the value directly in the container's ranger-admin-site.xml and restart the Admin container. Note that install.properties is only re-read during the initial setup process, not on a simple restart.

    # Verify Ozone configuration via API
    curl -s -u admin:rangerR0cks! http://localhost:6080/service/plugins/definitions/name/ozone \
      | python3 -c "import json,sys; d=json.load(sys.stdin); print('options', d.get('options')); print('conditions', [c['name'] for c in d.get('policyConditions',[])])"
  6. Format Java code blocks and braces

    master

    Braces

    • Always use braces for if, else, for, do, and while statements, even if the body is empty or contains a single statement.

    Nonempty blocks (K&R/Egyptian style)

    • No line break before the opening brace {.
    • Line break immediately after the opening brace {.
    • No empty line after the opening brace.
    • Line break before the closing brace }.
    • Line break after the closing brace } only if it terminates a statement or a body (method, constructor, or class). Do not add a line break if the brace is followed by else or a comma.
  7. Define resource paths for Arrays and Maps in Ranger Policies

    master

    When defining permissions for nested JSON structures in Apache Ranger policies, use the following syntax for the field resource:

    • Arrays: You must use an asterisk * to indicate that all elements in the array should be considered.
      • Example: store.book[*]price < 100 or store.book.*.price < 100.
    • Maps: Use standard dot . notation.
      • Example: store.bicycle.color.
  8. Understand Apache Ranger Authorization API concepts

    master

    The Apache Ranger Authorization API allows applications to programmatically request access decisions for users attempting to perform actions on resources.

    Key concepts include:

    • User: The actor performing the action, identified by a unique name. Users can have groups, roles, and custom attributes (e.g., department) used in policy evaluation.
    • Resource: The object being accessed, identified by a name in the format resource-type:resource-value (e.g., path:/etc/config, table:db.table, object:s3a://bucket/file). Resources can have attributes and sub-resources (e.g., columns within a table).
    • Action: The operation being performed (e.g., QUERY, LIST, CREATE). Note that the action in the request is primarily for auditing and does not drive the decision; the decision is driven by the requested Permissions.
    • Permission: The specific privilege required for an action (e.g., select, insert, read, write).
    • Context: Metadata about the request used for policy decisions, such as serviceName, accessTime, clientIpAddress, and additionalInfo (e.g., clusterName).
    • Decision: The outcome of the request, which is either ALLOWED or DENIED.
    • Row Filter: A mechanism for row-level security. If a policy defines a row filter, the response includes a filterExpr that the caller must apply to the data to ensure the user only sees authorized rows.
    • Data Mask: A mechanism for column-level security. The response can include a dataMask (containing maskType and maskedValue) that the caller must apply to sensitive data to transform it (e.g., masking a credit card number).
  9. Understand SampleApp Authorization Abstractions

    master

    The SampleApp demonstrates pluggable authorization using the following components:

    • IAuthorizer: The core authorization interface. It defines the method to authorize read, write, or execute access to a specific file path.
    • DefaultAuthorizer: A basic implementation of IAuthorizer that defaults to authorizing all access requests.
    • RangerAuthorizer: An implementation of IAuthorizer that performs authorization by checking against policies defined in Apache Ranger. It is configured to use a service instance (like an HDFS service) where the 'path' is treated as the resource and supports read, write, and execute access types.
  10. Identify Ranger caller roles and trust boundaries

    master

    Because Ranger is a distributed system, different actors have different trust levels and responsibilities:

    RoleDescriptionTrust Level
    Security AdministratorAuthors policies, manages users/roles, and views audits via Admin UI/REST API.Trusted for the instance
    Delegated AdministratorA group owner authorized to manage a specific subset of resources.Trusted only for their delegated scope
    Deployed Plugin (PEP)Code running inside a data service that downloads policies and enforces them.Fully Trusted once authenticated
    End UserThe principal (user/service) requesting access to guarded data.Untrusted; their identity is provided to the PEP by the host service
    Identity SourceThe authority (LDAP/AD/Unix) providing user/group membership.Trusted authority
    AuditorCan view policies and access audits but cannot author them.Limited (Read-only)
    Key AdminSpecifically manages Ranger KMS keys.Limited to KMS

    Important Note on Authentication: Ranger does not authenticate the end user at access time. The host data service (e.g., HDFS, Hive) is responsible for establishing the principal's identity (e.g., via Kerberos). Ranger then authorizes the identity presented by the host service.

  11. Understand the Ranger Audit Server Architecture

    master

    The Ranger Audit Server is a microservices-based system designed to ingest, transport, and dispatch audit events. The data flow follows this pattern:

    1. Ranger Plugins send audit events via REST API to the Audit Server.
    2. The Audit Server (ranger-audit-ingestor) acts as a producer, receiving events and producing them to a Kafka topic.
    3. Dispatcher Services act as consumers, reading from Kafka and writing to various destinations:
      • Solr Dispatcher: Indexes audits into Solr.
      • HDFS Dispatcher: Writes audits to HDFS, S3, or Azure.
      • Other Dispatchers: The architecture allows for Nth dispatchers to be added for new destinations.
    ┌─────────────────────┐
    │  Ranger Plugins     │
    │  (HDFS, Hive, etc.) │
    └──────────┬──────────┘
               │ REST API
               ▼
    ┌─────────────────────┐
    │ Audit Server        │  Port 7081
    │ (Producer)          │
    └──────────┬──────────┘
               │ Kafka
               ▼
        ┌──────────────┐
        │    Kafka     │
        │   (Topic)    │
        └──────┬───────┘
               │
          ┌────┴────┬──────┬─────────┐
          │         │      │         │
          ▼         ▼      ▼         ▼
    ┌──────────┐ ┌──────────┐ ┌──────────┐     ┌──────────┐
    │  Solr    │ │  HDFS    │ │  New     │ ... │   Nth    │
    │ Dispatcher │ │ Dispatcher │ │ Dispatcher │     │ Dispatcher │
    └────┬─────┘ └────┬─────┘ └────┬─────┘     └────┬─────┘
         │            │            │                 │
         ▼            ▼            ▼                 ▼
    ┌─────────┐  ┌──────────┐ ┌──────────┐     ┌──────────┐
    │  Solr   │  │   HDFS   │ │   New    │     │   Nth    │
    │ (Index) │  │ (Storage) │ │(Dest)    │     │ (Dest)    │
    └─────────┘  └──────────┘ └──────────┘     └──────────┘