Redis OM Python

repository·main·Indexed 23 days ago

https://github.com/redis/redis-om-python

An object-mapping library for Redis that enables modeling, validating, and querying data using modern Python patterns. It supports HashModel for flat key-value storage and JsonModel/EmbeddedJsonModel for nested structures via RedisJSON. Key features include automatic indexing with RediSearch, full-text search, complex logical queries, and a CLI for managing indexes and migrations. Version 1.1.0.

Tokens
22.8K
Snippets
66
Records
106
Agent score
79%

What's inside redis-om-python

  1. Core features of Redis OM Python

    main

    Redis OM Python provides several key capabilities for working with Redis:

    • Hash & JSON Models: Automatic serialization for storing data as Redis Hashes or JSON documents.
    • Powerful Queries: Django-like ORM syntax leveraging RediSearch for complex queries.
    • Pydantic Validation: Full Pydantic v2 support for data validation and type safety.
    • Async & Sync: Support for both synchronous (redis_om) and asynchronous (aredis_om) APIs.
  2. Choosing between HashModel and JsonModel

    main

    Redis OM provides two primary model types for object mapping. The choice depends on your data structure requirements:

    • HashModel: Use this for standard, flat data structures. It stores data in Redis Hashes. It does not support container types (Sets, Lists, Dictionaries, or other Redis OM/Pydantic models).
    • JsonModel: Use this if you need to embed models within other models (e.g., a Customer containing a list of Order models) or if you need to use container types. It uses Pydantic's JSON serialization to store data in Redis.
  3. Use embedded models with JsonModel

    main

    If you are using JsonModel, you can embed one model within another.

    Requirements for embedded models:

    1. The child model must have embedded = True in its Meta class.
    2. They are stored as nested JSON within the parent document.
    3. They can have their own indexed fields, which are included in the parent's index.
    4. They are not separately queryable; you must query through the parent model.
    from redis_om import JsonModel, Field
    
    
    class Address(JsonModel):
        street: str
        city: str = Field(index=True)
        zipcode: str
        country: str = "USA"
    
        class Meta:
            embedded = True
    
    
    class Customer(JsonModel, index=True):
        name: str
        age: int
        address: Address
  4. Use Redis OM asynchronously with asyncio

    main

    If your FastAPI application is built using asyncio, you should use the aredis_om module instead of redis_om.

    While the modules are nearly identical, aredis_om returns coroutines for its methods, meaning you must use the await keyword when calling them.

    Key differences:

    • Import: Use from aredis_om import HashModel, ...
    • Execution: Use await model.save(), await model.get(pk), etc.

    This is essential for non-blocking I/O in high-concurrency FastAPI applications.

    from aredis_om import HashModel, NotFoundError, get_redis_connection
    
    class Customer(HashModel):
        first_name: str
        # ... other fields
    
    @app.post("/customer")
    async def save_customer(customer: Customer):
        # Use await with aredis_om
        return await customer.save()
    
    @app.get("/customer/{pk}")
    async def get_customer(pk: str):
        try:
            return await Customer.get(pk)
        except NotFoundError:
            raise HTTPException(status_code=404, detail="Customer not found")
  5. Handle datetime field changes in Redis OM 1.1.0+

    main

    In Redis OM Python 1.1.0, timestamp handling was tightened. This affects how existing data is read back:

    • datetime.datetime values are read as UTC-aware datetime values.
    • datetime.date values are stored as midnight UTC.

    Warning for datetime.date: If you have existing datetime.date data written in a non-UTC environment before version 1.1.0, those records may load as a different calendar day after upgrading. If your application relies on date equality queries, you should create a custom data migration to re-save these fields to normalize them to UTC midnight.

  6. How Datetime Field Indexing works in Redis OM 1.0

    main

    In Redis OM 1.0, datetime fields are indexed as NUMERIC (Unix timestamps) instead of TAG (ISO strings). This change enables powerful new capabilities that were previously unavailable or inefficient:

    • Range Queries: Use comparison operators (e.g., >, <, >=, <=) on datetime fields.
    • Sorting: Use .sort_by() on datetime fields.
    • Between Queries: Perform queries within a specific time range.

    Data Format Change:

    • Before: Stored as ISO strings (e.g., "2023-12-01T14:30:22.123456").
    • After: Stored as Unix timestamps (e.g., 1701435022).

    Note: When migrating existing data, you must run the schema migration and the data migration commands to convert existing ISO strings to timestamps.

    # Range queries and sorting now work natively
    users = await User.find(User.created_at > datetime.now() - timedelta(days=7)).sort_by('created_at').all()
    
    # Between queries
    start = datetime(2023, 1, 1)
    end = datetime(2023, 12, 31)
    users = await User.find(
        (User.created_at >= start) & (User.created_at <= end)
    ).all()
  7. Constraints on List and Tuple indexing

    main

    When indexing List or Tuple fields in a JsonModel, there are two critical constraints:

    1. Elements must be strings: You cannot index a list of integers or other types. If you need to index non-string elements, you must use a different approach or store them without indexing (Error E12).
    2. No Full-Text Search: List and Tuple fields are indexed as TAG fields. They support exact matching but do not support the full_text_search=True option (Error E13).

    Error E12: List and tuple fields can only contain strings. Error E13: List and tuple fields cannot be indexed for full-text search.

    from typing import List
    from redis_om import JsonModel, Field
    
    # This works - list of strings
    class Article(JsonModel):
        tags: List[str] = Field(index=True)
    
    # This does NOT work - list of integers (Raises E12)
    class Article(JsonModel):
        scores: List[int] = Field(index=True)
    
    # This does NOT work - list with full-text search (Raises E13)
    class Article(JsonModel):
        tags: List[str] = Field(index=True, full_text_search=True)
  8. Validate data with Pydantic in Redis OM

    main

    Redis OM uses Pydantic for runtime validation. Since every Redis OM model is also a Pydantic model, you can use advanced types like EmailStr or constrained types (e.g., regex patterns, integer ranges) to ensure data integrity. Validation occurs during object instantiation and when calling .save() if field values were changed on an existing instance.

    from pydantic import EmailStr
    from redis_om import HashModel
    
    class Customer(HashModel):
        first_name: str
        email: EmailStr  # Ensures valid email format
        age: int          # Ensures integer type
  9. Define a HashModel

    main

    To create a model that is saved as a Redis hash, subclass HashModel. A HashModel is both a Redis OM model (providing methods to save data) and a Pydantic model (providing data validation).

    Constraints:

    • HashModel does not support list, set, or mapping (like dict) types because Redis hashes cannot contain these structures.
    • For fields requiring lists, sets, mappings, or embedded models, use JsonModel instead.

    Fields are defined using Python type annotations, which are used for Pydantic validation, serialization to Redis, and deserialization from Redis.

    import datetime
    from redis_om import HashModel
    
    
    class Customer(HashModel):
        first_name: str
        last_name: str
        email: str
        join_date: datetime.date
        age: int
        bio: str
  10. Create abstract models for shared configuration

    main

    You can create abstract models to gather shared configuration (like a global key prefix) for your application's models. Abstract models must subclass both ABC (from abc) and either HashModel or JsonModel. You cannot instantiate abstract models directly.

    When a subclass is created from a base class with a Meta object, Redis OM copies the parent's fields into the child's Meta object. A subclass can override specific fields in its Meta class without redefining the entire object.

    from abc import ABC
    from redis_om import HashModel, get_redis_connection
    
    
    redis = get_redis_connection(port=6380)
    other_redis = get_redis_connection(port=6381)
    
    
    class BaseModel(HashModel, ABC):
        class Meta:
            global_key_prefix = "customer-dashboard"
            database = redis
    
    
    class Customer(BaseModel):
        first_name: str
        last_name: str
    
        class Meta:
            database = other_redis
    
    
    print(Customer.global_key_prefix)
    # > "customer-dashboard"  # Inherited from BaseModel
  11. Supported types for multi-value indexed fields

    main

    When marking a field with index=True, only List and Tuple types are supported for multi-value indexing. Using other subscripted types like Dict[str, str] will trigger error E4.

    Error E4: Only lists and tuples are supported for multi-value fields.

  12. Validate data using Pydantic in Redis OM models

    main

    Every Redis OM model is also a Pydantic model. You can use standard Pydantic type annotations and validators (like EmailStr, Pattern, etc.) to ensure data integrity. If you attempt to instantiate a model with invalid data, a pydantic.ValidationError will be raised.

    By default, Redis OM will attempt to coerce input values to the correct type (e.g., parsing a valid ISO date string into a datetime.date object).

    import datetime
    from typing import Optional
    from redis_om import HashModel
    from pydantic import ValidationError
    
    class Customer(HashModel):
        first_name: str
        last_name: str
        email: str
        join_date: datetime.date
        age: int
        bio: Optional[str] = "Super dope"
    
    try:
        Customer(
            first_name="Andrew",
            last_name="Brookins",
            email="a@example.com",
            join_date="not a date!",  # This will trigger ValidationError
            age=38
        )
    except ValidationError as e:
        print(e)