django-q2

name	django-q2
description	Use when adding, changing, testing, or debugging Django Q2 background tasks, scheduled jobs, qcluster workers, Redis broker configuration, or ORM broker fallback in this generated Django SaaS app.

Django Q2 Background Jobs

Use this before touching task enqueueing, schedules, worker deployment, Q_CLUSTER, or code imported by Django Q2 workers.

Project Defaults

• The dependency is django-q2; the Python import path is django_q.
• django_q is in INSTALLED_APPS; its migrations provide task result, schedule, and broker models.
• Q_CLUSTER lives in the generated Django settings module and uses Redis through REDIS_URL by default.
• Q_CLUSTER["name"] comes from Q_CLUSTER_NAME, which defaults to <LOCAL_INSTANCE_ID>-q. Keep web and worker processes on the same value.
• The worker process is uv run --no-sync python manage.py qcluster; the local workers Docker Compose service and deployment APP_PROCESS_TYPE=worker path use that command.
• Redis also backs the default Django cache. If you remove Redis as the broker, review cache, health check, Docker, deployment, and documentation references separately.

Mental Model

• Web code calls async_task(...) or creates Schedule rows.
• The broker stores queued task packages until a qcluster process reserves work.
• Worker processes execute importable Python functions and save results or failures.
• The scheduler runs inside the cluster. Schedules are database rows; they do nothing unless qcluster is running.

Adding Tasks

1. Put the task function in the app that owns the behavior, usually apps/core/tasks.py for shared product work.
2. Keep the function importable at module import time. Do not rely on request objects, local closures, or process-local state.
3. Pass durable identifiers such as primary keys, not model instances, open files, connections, or large payloads.
4. Make the task idempotent. Redis does not give exactly-once execution guarantees, and receipt-based brokers can re-run work.
5. If the task depends on a just-saved database row, enqueue it from transaction.on_commit(...).

# apps/core/tasks.py
def send_welcome_email(user_id: int) -> None:
    from django.contrib.auth import get_user_model

    user = get_user_model().objects.get(pk=user_id)
    ...

from django.db import transaction
from django_q.tasks import async_task

transaction.on_commit(
    lambda: async_task("apps.core.tasks.send_welcome_email", user.pk)
)

Use q_options when Django Q2 options would collide with task kwargs:

async_task(
    "apps.core.tasks.rebuild_report",
    report_id,
    q_options={"timeout": 300, "group": "reports"},
)

Scheduling Work

Prefer named, idempotent schedules created by a migration, admin action, or setup command. Avoid creating schedules unconditionally at import time or app startup.

from django_q.models import Schedule

Schedule.objects.get_or_create(
    name="clear-expired-sessions",
    defaults={
        "func": "django.core.management.call_command",
        "args": "'clearsessions'",
        "schedule_type": Schedule.HOURLY,
    },
)

Use Schedule.objects.get_or_create(name=..., defaults={...}) when seeding schedules so repeated setup does not duplicate jobs. Cron schedules require the optional croniter dependency; do not use Schedule.CRON unless the generated project includes it.

Missed schedules catch up by default. Set Q_CLUSTER["catch_up"] = False when a job should run once after downtime instead of replaying every missed interval.

Broker Choices

Redis Broker

This template should normally use Redis:

Q_CLUSTER = {
    "name": "...",
    "timeout": 3600,
    "workers": 4,
    "redis": REDIS_URL,
    "ALT_CLUSTERS": {},
}

Redis is fast and matches local Docker, deployment workers, and the default cache configuration. The default Redis broker does not support delivery receipts. If a worker host dies catastrophically while executing a task, the in-flight package can be lost; if task code raises, Django Q2 records a failure. Use idempotent task design, explicit retries in task code where needed, and monitoring for failures.

When two same-slug clones intentionally share one Redis service, set unique LOCAL_INSTANCE_ID, CACHE_KEY_PREFIX, and Q_CLUSTER_NAME values per clone. Do not let two unrelated repos share the same cluster name.

ORM Broker

Use the Django database broker only for low-throughput deployments, local simplicity, or environments where Redis is unavailable:

Q_CLUSTER = {
    "name": "...",
    "timeout": 3600,
    "retry": 4800,
    "workers": 4,
    "max_attempts": 2,
    "orm": "default",
    "ALT_CLUSTERS": {},
}

When switching to ORM:

• Remove the "redis" broker key; configure one broker per cluster unless you intentionally use custom clusters.
• Run migrations for django_q. If the broker uses a non-default database alias, run migrations with --database <alias>.
• Increase "poll" above the default 0.2 seconds, for example "poll": 2.0, when you need lower database polling pressure and can tolerate higher queue pickup latency.
• The ORM broker enables the Queued Tasks admin table.
• Review Redis-dependent cache, health check, Docker, and deployment settings separately. Schedules are always database rows; the broker setting controls queued task packages, not the schedule table.

Testing

• Test task business logic by calling the function directly.
• Test enqueueing with synchronous mode:
  • per call: async_task("apps.core.tasks.fn", arg, sync=True)
  • per test: override Q_CLUSTER["sync"] = True
• For worker/broker integration, run uv run --no-sync python manage.py qcluster in a separate process and wait for result(task_id, 200) or a similar bounded wait; do not rely on arbitrary sleeps.
• Use pytest.mark.django_db(transaction=True) when a real worker process must observe committed database rows.

Debugging Checklist

• Is a qcluster process running with the same settings module, SECRET_KEY, broker URL, and cluster name as the web process?
• Can the worker import the dotted task path?
• Did database migrations run, including django_q migrations?
• Is Redis reachable from both web and worker containers, or is the ORM broker polling the expected database?
• Did a scheduled task duplicate because setup created another Schedule row with no stable name?
• Did downtime trigger schedule catch-up?
• Is task failure visible in Django admin, logs, or the configured error reporter?

References

• Official docs: https://django-q2.readthedocs.io/en/master/
• Upstream repo: https://github.com/django-q2/django-q2