메뉴
Async job processing patterns for background tasks, Celery workflows, task scheduling, retry strategies, and distributed task execution. Use when implementing background job processing, task queues, or scheduled task systems.
기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.
설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
직접 명령은 검토 Prompt를 거치지 않습니다. 실행하기 전에 소스를 확인하세요.
npx skills add https://github.com/yonatangross/orchestkit --skill async-jobs명령은 한 줄로 유지됩니다. 복사하기 전에 가로로 스크롤해 전체 내용을 확인하세요.
로컬 사본을 원하시나요? SkillsMP에서 현재 제공할 수 있는 파일을 다운로드하세요.
Chain patterns for multi-phase pipelines: MCP detection, handoff files, checkpoint-resume, worktree agents, CronCreate monitoring. Use when building or debugging a pipeline skill.
OrchestKit doctor for health diagnostics across manifest integrity, hook configuration, skill validation, agent frontmatter, MCP server connectivity, CC version compatibility, and permission rules. Reports issues with severity levels and auto-remediation suggestions. Validates component counts, detects orphaned entries, and checks CC version matrix compliance. Use when diagnosing plugin health, troubleshooting configuration issues, or running pre-release checks.
Inspects the OrchestKit telemetry pipeline for the current project — lists all known telemetry files with write counts, sizes, schema status, growth trend, and orphan detection. Use when verifying the observability pipeline is healthy, debugging a missing writer, or auditing which files have schema locks vs. which are drift-vulnerable. Read-only — never modifies telemetry files.
SKILL.md 표시 중
| name | async-jobs |
| license | MIT |
| compatibility | Claude Code 2.1.220+. |
| description | Async job processing patterns for background tasks, Celery workflows, task scheduling, retry strategies, and distributed task execution. Use when implementing background job processing, task queues, or scheduled task systems. |
| tags | ["async","jobs","celery","background-tasks","scheduling","queues"] |
| context | fork |
| agent | python-performance-engineer |
| version | 2.0.0 |
| author |
| OrchestKit |
| user-invocable | false |
| disable-model-invocation | true |
| complexity | medium |
| persuasion-type | reference |
| metadata | {"category":"workflow-automation"} |
| allowed-tools | ["Read","Glob","Grep","WebFetch","WebSearch"] |
Background task processing with Celery, ARQ, Redis and Temporal. This skill is a wrapper, not a manual: Celery and ARQ document their own product well, so what lives here is our delta, the thresholds, working config, ordering constraints and tool-choice rules we picked. Product mechanics are linked, not restated.
Start with Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/references/ork-delta.md").
| Topic | Where our part lives |
|---|---|
| Configuration | references/celery-config.md, rules/jobs-task-queue.md |
| Task Routing | references/ork-delta.md (queue taxonomy, prefetch tiers, Redis priority) |
| Canvas Workflows | rules/celery-canvas.md |
| Retry Strategies | references/ork-delta.md (backoff cap, idempotency layers, lock TTLs) |
| Scheduling | rules/jobs-scheduling.md, references/ork-delta.md (beat process model) |
| Monitoring | references/ork-delta.md (alert thresholds, histogram buckets) |
| Result Backends | rules/jobs-monitoring.md, references/ork-delta.md (return contract) |
| ARQ Patterns | rules/jobs-task-queue.md, references/ork-delta.md (budgets, pool ownership) |
| Temporal Workflows | rules/temporal-workflows.md |
| Temporal Activities | rules/temporal-activities.md |
10 topic areas, 6 rule files in rules/, house delta in references/ork-delta.md.
@app.task(bind=True, max_retries=3, default_retry_delay=60)
def process_payment(self, order_id: str):
try:
return gateway.charge(order_id)
except TransientError as exc:
raise self.retry(exc=exc, countdown=2 ** self.request.retries * 60)
Load more examples: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/references/quick-start-examples.md") for Celery
retry task and ARQ/FastAPI integration patterns.
Fetch these when you need product mechanics. The right-hand column is the part we keep, because it is a house threshold, a working config or an ordering constraint that upstream cannot know.
| Topic | First-party source | House subset stays in |
|---|---|---|
| Celery settings, serializers, time limits, worker flags | https://docs.celeryq.dev/en/stable/userguide/configuration.html and .../optimizing.html | references/celery-config.md, rules/jobs-task-queue.md |
| Queue declarations, router classes, Redis priority mechanics | https://docs.celeryq.dev/en/stable/userguide/routing.html | references/ork-delta.md |
| chain / group / chord / signature semantics | https://docs.celeryq.dev/en/stable/userguide/canvas.html | rules/celery-canvas.md keeps the house canvas subset. Its si()-in-chords guidance is UNVERIFIED and contested: confirm the argument-passing behaviour against the upstream canvas page before relying on it |
autoretry_for, retry_backoff, Reject, task base classes | https://docs.celeryq.dev/en/stable/userguide/tasks.html | references/ork-delta.md, rules/jobs-task-queue.md |
| Beat schedules, crontab syntax, DatabaseScheduler | https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html and https://django-celery-beat.readthedocs.io/en/latest/ | rules/jobs-scheduling.md keeps our beat_schedule shapes; references/ork-delta.md keeps the process model |
Flower flags, inspect, signal names | https://docs.celeryq.dev/en/stable/userguide/monitoring.html, https://docs.celeryq.dev/en/stable/userguide/signals.html, https://flower.readthedocs.io/en/latest/config.html | references/ork-delta.md |
Result backend, AsyncResult, custom states |
Load: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/references/celery-config.md").
| Decision | Recommendation |
|---|---|
| Serializer | JSON (never pickle) |
| Ack mode | Late ack (task_acks_late=True) |
| Prefetch | 1 for fair, 4-8 for throughput |
| Time limit | soft < hard (540 / 600) |
| Timezone | UTC always |
| Decision | Recommendation |
|---|---|
| Queue count | 5: critical / high / default / low / bulk |
| Priority levels | 0-9, with all four Redis priority switches set together |
| Worker assignment | Dedicated worker per queue |
| Prefetch | 1 critical, 2 high, 4 default, 8 low/bulk |
| Routing | Router class once past 5 routing rules |
Load: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/rules/celery-canvas.md").
| Decision | Recommendation |
|---|---|
| Sequential | Chain with s() |
| Parallel | Group for independent tasks |
| Fan-in | Chord (all header tasks must succeed for the body to run) |
| Ignore input | Use si() immutable signature |
| Error in chain | Reject stops the chain, retry continues it |
| Partial failures | Return an error dict from chord header tasks |
| Decision | Recommendation |
|---|---|
| Retry delay | Exponential backoff, jitter on, capped at 600s |
| Max retries | 3-5 for transient, 0 for permanent |
| Idempotency | Redis marker (86400s TTL) plus the vendor idempotency key |
| Failed tasks | DLQ for manual review |
| Singleton | Redis lock with a TTL longer than the hard time limit |
Load: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/rules/jobs-scheduling.md").
| Decision | Recommendation |
|---|---|
| Schedule type | Crontab for time-based, float interval for frequency |
| Dynamic | DatabaseScheduler (django-celery-beat) |
| Overlap | Redis lock, 3600s default and 7200s for long jobs |
| Beat process | Separate process; embedded --beat is development only |
| Timezone | UTC always |
| Decision | Recommendation |
|---|---|
| Dashboard | Flower with persistent storage |
| Metrics | Prometheus wired to task_prerun / task_postrun / task_failure |
| Health | Broker reachable, at least one worker, queue depths |
| Alerting | critical > 100, default > 5000, workers < 1 |
| Autoscale | Queue depth > 500 |
Load: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/rules/jobs-monitoring.md").
| Decision | Recommendation |
|---|---|
| Status storage | Redis result backend, status and small JSON only |
| Large results | S3 or database, task returns a reference dict |
| Progress | Custom states with update_state() |
| Result query | AsyncResult with state checks |
Load: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/rules/jobs-task-queue.md").
| Decision | Recommendation |
|---|---|
| Simple async | ARQ (native async), max_jobs=10, job_timeout=300 |
| Pool ownership | FastAPI lifespan, never a per-request create_pool |
| Complex workflows | Celery (chains, chords, DLQ, per-task rate limits) |
| In-process quick | FastAPI BackgroundTasks, under 30s, non-critical only |
| LLM workflows | LangGraph, not Celery |
Load: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/references/quick-start-examples.md") for the full tool
comparison table (ARQ, Celery, RQ, Dramatiq, FastAPI BackgroundTasks).
Load details: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/references/anti-patterns.md") for the full list.
Key rules: never run long tasks in request handlers, never block on results inside tasks, never store large results in Redis, always use idempotency for retried tasks.
Load: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/rules/temporal-workflows.md").
| Decision | Recommendation |
|---|---|
| Workflow ID | Business-meaningful, idempotent |
| Determinism | Use workflow.random(), workflow.now() |
| I/O | Always via activities, never directly |
Load: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/rules/temporal-activities.md").
| Decision | Recommendation |
|---|---|
| Activity timeout | start_to_close for most cases |
| Error handling | Non-retryable for business errors |
| Testing | WorkflowEnvironment.start_local() for integration tests |
ork:python-backend - FastAPI, asyncio, SQLAlchemy patternsork:langgraph - LangGraph workflow patterns (use for LLM workflows, not Celery)ork:distributed-systems - Resilience patterns, circuit breakersork:monitoring-observability - Metrics and alertingLoad details: Read("${CLAUDE_PLUGIN_ROOT}/skills/async-jobs/references/capability-details.md") for the keyword index
and problem-to-capability mapping.
| https://docs.celeryq.dev/en/stable/userguide/configuration.html |
rules/jobs-monitoring.md keeps our status endpoints and update_state() usage |
Per-task rate_limit, control.rate_limit, Redis Lua | https://docs.celeryq.dev/en/stable/userguide/workers.html, https://redis.io/docs/latest/develop/programmability/eval-intro/ | references/ork-delta.md |
ARQ WorkerSettings, enqueue_job, _defer_by / _defer_until, Job status | https://arq-docs.helpmanual.io/ | rules/jobs-task-queue.md keeps the worker skeleton; references/ork-delta.md keeps the budgets |
| FastAPI lifespan and dependency wiring | https://fastapi.tiangolo.com/advanced/events/ | references/ork-delta.md |
Distributed locks with SET NX EX | https://redis.io/docs/latest/commands/set/ | references/ork-delta.md |