DOCSEARCHbyAlgolia
Submit Docs

django-celery-beat Documentation

repository·main·Indexed 23 days ago

https://github.com/celery/django-celery-beat

An extension for Celery that allows storing periodic task schedules in a Django database, enabling management via the Django Admin interface. It provides the DatabaseScheduler to replace static configuration files and supports both IntervalSchedule and CrontabSchedule for task timing.

Tokens
1.7K
Snippets
5
Records
10
Agent score
34%

What's inside django-celery-beat

  1. How django-celery-beat models work

    main

    The extension uses several models to manage scheduling:

    • PeriodicTask: Defines a single task to be run. It is associated with either an IntervalSchedule or a CrontabSchedule.
    • IntervalSchedule: Defines a frequency (e.g., every X seconds).
    • CrontabSchedule: Defines a schedule using cron-like fields (minute, hour, etc.).
    • PeriodicTasks: An index model used to track schedule changes.

    Important: If you perform bulk updates on PeriodicTask objects, you must manually call PeriodicTasks.update_changed() to notify the celery beat service to reload the schedule from the database.

    from django_celery_beat.models import PeriodicTasks

# Call this after bulk updates to trigger a schedule reload
PeriodicTasks.update_changed()

  2. Install django-celery-beat

    main

    You can install django-celery-beat via PyPI using pip. After installation, you must add 'django_celery_beat' to your Django INSTALLED_APPS and run the migrations.

    $ pip install --upgrade django-celery-beat

# In your Django project
$ python manage.py migrate django_celery_beat
    # Add to settings.py
INSTALLED_APPS = [
    ...,
    'django_celery_beat',
]

  3. Run Celery Beat with the Database Scheduler

    main

    To use database-backed periodic tasks, you must run both a Celery worker and the Celery beat service using the DatabaseScheduler.

    1. Start the worker: celery -A [project-name] worker --loglevel=info

    2. Start the beat service: Use the -S flag to specify the django scheduler, or point directly to the DatabaseScheduler class.

    Note: Running both worker and beat in a single command is recommended for development environments only.

  5. Run django-celery-beat with Docker Compose

    main

    The project provides a docker-compose.yml file to orchestrate a full environment including Django, Celery Beat (using the DatabaseScheduler), RabbitMQ, and PostgreSQL.

    To run the services, ensure you have Docker and Docker Compose installed, then use the standard compose command. The configuration uses environment variables for port mapping and broker connectivity, providing sensible defaults if they are not set.

  6. Handle Time Zone changes in periodic tasks

    main

    If you change your Django TIME_ZONE setting, existing periodic task schedules will still be based on the old timezone. To fix this, you must reset the last_run_at field for all tasks and update the change counter.

    Warning: This resets the state as if the tasks have never run before.

    from django_celery_beat.models import PeriodicTask, PeriodicTasks

# Reset last run time and update the change counter
PeriodicTask.objects.all().update(last_run_at=None)
PeriodicTasks.update_changed()

  7. Create a crontab-based periodic task

    main

    To run a task based on a cron expression, create a CrontabSchedule object specifying minute, hour, day_of_week, day_of_month, and month_of_year. You can also specify a timezone using zoneinfo.ZoneInfo.

    from django_celery_beat.models import CrontabSchedule, PeriodicTask
import zoneinfo

# 1. Create the crontab schedule
schedule, _ = CrontabSchedule.objects.get_or_create(
    minute='30',
    hour='*',
    day_of_week='*',
    day_of_month='*',
    month_of_year='*',
    timezone=zoneinfo.ZoneInfo('Canada/Pacific')
)

# 2. Create the task
PeriodicTask.objects.create(
    crontab=schedule,
    name='Importing contacts',
    task='proj.tasks.import_contacts',
)

  8. Create an interval-based periodic task

    main

    To run a task at a specific interval (e.g., every 10 seconds), you must first create an IntervalSchedule object and then link it to a PeriodicTask.

    Available periods for IntervalSchedule include:

    • IntervalSchedule.DAYS
    • IntervalSchedule.HOURS
    • IntervalSchedule.MINUTES
    • IntervalSchedule.SECONDS
    • IntervalSchedule.MICROSECONDS

    When creating PeriodicTask, arguments (args) and keyword arguments (kwargs) must be JSON serialized.

    from django_celery_beat.models import PeriodicTask, IntervalSchedule
import json
from datetime import datetime, timedelta

# 1. Create the schedule
schedule, created = IntervalSchedule.objects.get_or_create(
    every=10,
    period=IntervalSchedule.SECONDS,
)

# 2. Create the task
PeriodicTask.objects.create(
    interval=schedule,
    name='Importing contacts',
    task='proj.tasks.import_contacts',
    args=json.dumps(['arg1', 'arg2']),
    kwargs=json.dumps({'be_careful': True}),
    expires=datetime.utcnow() + timedelta(seconds=30)
)

  9. Configure django-celery-beat environment variables

    main

    The Docker Compose setup uses several environment variables to configure service connectivity and ports. If these are not provided, the following defaults are used:

    Django Service

    • DJANGO_HOST: The host for the Django service (default: 127.0.0.1).
    • DJANGO_PORT: The host port for the Django service (default: 58000).

    RabbitMQ Service

    • RABBITMQ_HOST: The hostname for the RabbitMQ broker (default: rabbit).
    • RABBITMQ_PORT: The port for the RabbitMQ broker (default: 5672).
    • RABBITMQ_USER: The RabbitMQ username (default: guest).
    • RABBITMQ_PASSWORD: The RabbitMQ password (default: guest).

  10. Configure the celery-beat service command

    main

    The celery-beat service is configured to use the DatabaseScheduler provided by django-celery-beat. This allows periodic tasks to be managed via the Django database.

    The default command executed within the container is:

    python3 -m celery -A mysite beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

    Note: mysite is a placeholder for your Django project name.