django-rq

To run rqworker as a background service on Ubuntu, create a systemd service file at /etc/systemd/system/rqworker.service.

In the ExecStart directive, ensure you point to your virtual environment's python binary and use manage.py rqworker followed by the queues you wish to process (e.g., high, default, low).

After creating the file, enable and start the service using systemctl.

[Unit]
Description=Django-RQ Worker
After=network.target

[Service]
WorkingDirectory=<<path_to_your_project_folder>>
ExecStart=/home/ubuntu/.virtualenv/<<your_virtualenv>>/bin/python \ <<path_to_your_project_folder>>/manage.py \ rqworker high default low

[Install]
WantedBy=multi-user.target

To deploy django-rq on Heroku:

1. Ensure django-rq is in your requirements.txt (use pip freeze > requirements.txt).
2. Update your Procfile to include a worker process that runs python <your_app_name>/manage.py rqworker <queues>.
3. Commit and deploy your changes.
4. Scale the worker process using the Heroku CLI.

# requirements.txt
django-rq

# Procfile
web: gunicorn --pythonpath="$PWD/your_app_name" config.wsgi:application
worker: python your_app_name/manage.py rqworker high default low

Built-in Scheduler

To use RQ 1.2.0+ built-in scheduling, use enqueue_at and start your worker with the --with-scheduler flag.

RQ's CronScheduler

For cron-style scheduling, create a configuration file and run rqcron.

Example Cron Config:

from rq import cron
from myapp.tasks import send_report

cron.register(send_report, queue_name='default', cron='0 9 * * *')

Commands:

• Start scheduler worker: python manage.py rqworker --with-scheduler
• Run cron: python manage.py rqcron cron_config.py

from datetime import datetime
from django_rq.queues import get_queue

queue = get_queue('default')
job = queue.enqueue_at(datetime(2020, 10, 10), func)

Django-RQ automatically integrates with the Django admin interface. Once installed, you can access the dashboard at /admin/django_rq/dashboard/ to monitor queue statistics, browse job registries (scheduled, started, finished, failed, deferred), manage workers, and view Prometheus metrics (if prometheus_client is installed).

To disable the dashboard link in the admin sidebar, set RQ_SHOW_ADMIN_LINK = False in your settings.py.

If you prefer not to use the Django admin interface for monitoring, you can include the Django-RQ views at a custom URL prefix in your urls.py.

# urls.py
from django.urls import path, include

urlpatterns += [
    path('django-rq/', include('django_rq.urls'))
]

To execute the integration tests for the django-rq sample project, run the provided test script using Python.

python _test.py

To expose Prometheus-compatible metrics at /django-rq/metrics/, install prometheus_client or the extra django-rq[prometheus].

If you need to access the metrics endpoint via other HTTP clients, define RQ_API_TOKEN in your settings and use it as a Bearer token in the Authorization header.

Django-RQ allows you to control when a job is actually sent to Redis relative to Django's database transactions. This is controlled via the COMMIT_MODE setting in your RQ configuration dictionary.

Supported modes:

• on_db_commit (default): The job is enqueued only after the current database transaction successfully commits. This prevents jobs from running if the database transaction rolls back.
• auto: The job is enqueued immediately when enqueue is called.
• request_finished: The job is enqueued when the Django request/response cycle finishes (legacy behavior).

Note: The AUTOCOMMIT setting is deprecated; use COMMIT_MODE instead.

The job_detail view provides a deep dive into a specific job's lifecycle. It exposes:

• Job Metadata: Function name and serialized data.
• Results: Access to job results via result_detail using a specific result_id.
• Error Information: If a job failed, exc_info contains the exception details.
• Dependency Graph: Lists of dependencies (jobs this job depends on) and dependents (jobs that depend on this job).