DOCSEARCHbyAlgolia
Submit Docs

django-celery-results

repository·main·Indexed 21 days ago

https://github.com/celery/django-celery-results

An extension that enables Celery to store task results using the Django ORM or Django's cache framework. It provides the TaskResult model, allowing developers to query task results and metadata directly from their Django database using standard model queries. Supports result backends 'django-db' and 'django-cache', and includes options for extended task information and execution tracking.

Tokens
2.4K
Snippets
13
Records
16
Agent score
73%

What's inside django-celery-results

  1. What is django-celery-results?

    main
    django-celery-results provides Celery result backends that utilize the Django ORM or Django's cache framework. This allows you to store Celery task results and metadata directly within your Django application's database or cache, making them easily accessible via Django models or the cache interface.

  2. How django-celery-results works

    main
    This extension allows you to store Celery task results using the Django ORM. It introduces a single model, django_celery_results.models.TaskResult, which is used to persist task results in your database. Because it is a standard Django model, you can query the task results table using the Django ORM just like any other model in your application.

  3. Install the development version from GitHub

    main

    To install the latest snapshot (master branch) of django-celery-results directly from GitHub using pip, use the following command:

    $ pip install https://github.com/celery/django-celery-results/zipball/master#egg=django-celery-results

  5. Inject arbitrary metadata into TaskResult.meta

    main

    To store custom data in the TaskResult.meta field using django-celery-results, you must manipulate the task_instance.request.meta attribute within your task.

    Important Distinction:

    • The meta argument passed to task_instance.update_state() is not stored in TaskResult.meta. Instead, it is treated as the current task result and is stored in TaskResult.result.
    • To ensure data persists in TaskResult.meta both during progress updates and after task completion, assign your dictionary to task_instance.request.meta.
    from celery import Celery

app = Celery('hello', broker='amqp://guest@localhost//')

@app.task(bind=True)
def hello(task_instance):
    # This stores data in TaskResult.meta
    task_instance.request.meta = {'some_key': 'some_value'}
    
    # This stores 'Task current result' in TaskResult.result
    task_instance.update_state(
        state='PROGRESS',
        meta='Task current result'
    )
    
    return 'hello world'

  7. Configure django-celery-results in Django

    main

    To integrate the library with your Django project, follow these three steps:

    1. Add 'django_celery_results' to your INSTALLED_APPS in settings.py. Note that the module name uses underscores, not dashes.
    2. Run database migrations to create the necessary Celery database tables.
    3. Configure Celery to use the django-db or django-cache backend.
    # 1. settings.py
INSTALLED_APPS = (
    ...,
    'django_celery_results',
)

  8. Install django-celery-results from source

    main

    To install from a downloaded source tarball, follow these steps:

    1. Extract the archive.
    2. Navigate to the directory.
    3. Build and install using setup.py.

    Note: If you are not using a virtualenv, you must execute the install command as a privileged user.

    $ tar xvfz django-celery-results-0.0.0.tar.gz
$ cd django-celery-results-0.0.0
$ python setup.py build
# python setup.py install

  9. Run the project with Docker Compose

    main

    To run the project in a containerized environment (which starts a PostgreSQL database and runs tests using tox), execute the following command in the project root directory. Ensure Docker and Docker Compose are installed.

    $ docker-compose up --build

  10. Configure Celery result backends

    main

    You can choose between a database backend or a cache backend in your Celery settings.

    Database Backend

    Use django-db to store results in the Django database.

    Cache Backend

    Use django-cache to store results in the Django cache. You can also point to a specific cache defined in your Django CACHES setting by using its key (e.g., 'default').

    # Use the database backend
CELERY_RESULT_BACKEND = 'django-db'

# Use the cache backend
CELERY_CACHE_BACKEND = 'django-cache'

# Use a specific Django cache defined in CACHES
CELERY_CACHE_BACKEND = 'default'

# Example Django CACHES configuration
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
        'LOCATION': 'my_cache_table',
    }
}

  11. Enable extended task information and execution tracking

    main

    By default, Celery results contain basic information. You can enable additional metadata and timing information using the following settings:

    • CELERY_RESULT_EXTENDED = True: Includes extended information about your tasks.
    • CELERY_TASK_TRACK_STARTED = True: Tracks the execution duration by recording when a task starts (date_started) versus when it is created (date_created).
    CELERY_RESULT_EXTENDED = True
CELERY_TASK_TRACK_STARTED = True

  12. Run django-celery-results with Docker Compose

    main

    You can run the django-celery-results development environment using Docker Compose. The setup includes a PostgreSQL database service and an application service.

    Configuration is managed via environment variables, allowing you to customize the database credentials and version. The application service connects to the database using the alias dcr-postgres on the celery network.

    docker-compose up