| name | bunqueue-dev |
| description | Internal skill for contributing to bunqueue - architecture, testing, code conventions, and development workflow |
| disable-model-invocation | false |
| user-invocable | false |
| allowed-tools | Read, Grep, Glob, Bash, Edit, Write |
bunqueue Development Guide
You are working on bunqueue, a high-performance job queue for Bun with SQLite persistence.
Architecture
bunqueue uses a sharded priority queue architecture:
- Shards: Auto-detected from CPU cores (power of 2, max 64). Jobs are assigned via
fnv1aHash(queue) & SHARD_MASK
- Persistence: SQLite in WAL mode with a 10ms WriteBuffer for batching writes
- Transport: TCP (msgpack) on port 6789, HTTP on port 6790
- Two modes: Embedded (in-process) and TCP (client-server)
Request Flow
- PUSH: Client -> TcpPool -> TcpServer -> QueueManager -> Shard -> PriorityQueue -> WriteBuffer -> SQLite
- PULL: Client -> TcpServer -> QueueManager -> Shard -> PriorityQueue.pop()
- ACK: Client -> TcpServer -> AckBatcher -> Shard.complete() -> jobResults (LRU)
- FAIL: Client -> TcpServer -> Shard.fail() -> retry (backoff) OR -> DLQ
Directory Structure
src/
cli/ # CLI interface
client/ # SDK (Queue, Worker, FlowProducer, Bunqueue)
queue/ # Queue with DLQ, stall detection
worker/ # Worker with heartbeat, ack batching
tcp/ # Connection pool, reconnection
workflow/ # Workflow Engine (Workflow DSL, Engine, Executor, Store)
domain/ # Pure business logic
queue/ # Shard, PriorityQueue, DlqShard, UniqueKeyManager
application/ # Use cases and managers
operations/ # push, pull, ack, query, queueControl
infrastructure/ # Persistence, server, scheduler, backup
shared/ # Utilities (hash, lock, lru, skipList, minHeap)
Workflow Engine
Located in src/client/workflow/. Pure consumer layer on bunqueue (no core modifications).
- workflow.ts — Fluent DSL:
.step(), .branch(), .path(), .waitFor()
- engine.ts — Public facade: register, start, signal, getExecution, close
- executor.ts — Core logic: step execution, branching, compensation, signals
- store.ts — SQLite persistence for execution state (workflow_executions table)
- types.ts — StepContext, Execution, StepJobData, EngineOptions, etc.
- Export:
import { Workflow, Engine } from 'bunqueue/workflow'
Code Conventions
- MAX 300 lines per file - split if larger
- One concern per file (Single Responsibility)
- Export only what's needed
- Lock hierarchy:
jobIndex -> completedJobs -> shards[N] -> processingShards[N]
Testing (MANDATORY before any commit)
bun test
bun scripts/tcp/run-all-tests.ts
bun scripts/embedded/run-all-tests.ts
All three must pass. No exceptions.
Bug Fixing Process
NEVER fix a bug directly. Always:
- Write a test that reproduces the bug
- Launch subagents to fix the bug
- Prove the fix with a passing test
Publishing
After every commit:
- Bump version in
package.json
- Update changelog in
docs/src/content/docs/changelog.md
git push origin mainbun publish
