activejob-design-patterns

name	activejob-design-patterns
description	ActiveJob architecture patterns: retryable vs permanent error handling, transaction safety, external API integration, async enqueuing, internal batching, and operation-based idempotency. Auto-activates when writing background jobs, handling job errors, or implementing idempotent operations. Trigger keywords: ActiveJob, perform_later, background job, retry, idempotent, operation_id, job error handling, fatal non-fatal.
license	MIT
metadata	{"version":"2.1.0","hermes":{"tags":["Rails","ActiveJob","Background-jobs","Architecture"],"related_skills":["activerecord-transaction-boundary-optimization"]}}

ActiveJob Design Patterns

Retryable vs Permanent Error Handling

First decision for any job: will retrying help?

• Retryable: transient issues — network errors, timeouts, temporary DB locks. Wrap mutations in a transaction, re-raise the error (or use retry_on with backoff).
• Permanent (no retry): unrecoverable issues — validation failures, audit discrepancies. Log the error, do NOT re-raise (or use discard_on).

(Earlier versions of this skill called these "Fatal"/"Non-Fatal" — inverted from the industry convention where fatal means unrecoverable-don't-retry. Use Retryable/Permanent; ActiveJob's own retry_on/discard_on DSL maps to them directly and adds attempt limits and backoff that hand-rolled raise lacks.)

def perform(entity_id)
  ActiveRecord::Base.transaction do
    create_record       # keep external API calls OUT of this block —
    fetch_data          # if fetch_data is a network call, move it above
    create_content_records
  end
rescue StandardError => e
  Rails.logger.error("[JOB] FAILED - entity_id: #{entity_id}, error: #{e.class.name}: #{e.message}")
  raise  # Remove `raise` for permanent-error jobs
end

External API Integration

No transaction wrapper — API calls cannot be rolled back. Update state AFTER API success. Notification failures after successful API calls are non-fatal.

A re-raise means a retry, and a retry re-runs every step — an external call that moves money (or sends email, or charges a card) must carry its own guard: a persisted marker checked before the call (below), an API idempotency key, or the operation-id pattern from the Idempotency section. Update-state-after-success alone does NOT make the call retry-safe.

class TransferAndSend < ApplicationJob
  def perform(record_id)
    @record = Record.find(record_id)
    validate_eligibility
    transfer_funds
    send_notification
  rescue StandardError => e
    @record.update(error_message: e.message)
    raise
  end

  private

  def transfer_funds
    return if @record.transfer_id.present?  # retry guard — a re-run must not transfer twice
    transfer = CreateTransfer.perform_now(...)  # pass an idempotency key if the API takes one
    @record.update(transfer_id: transfer.id)  # Update AFTER success
  end

  def send_notification
    NotifyJob.perform_now(@record)
  rescue StandardError => e
    Rails.logger.error("[JOB] Notification FAILED (API succeeded) - #{e.message}")
  end
end

Async Enqueuing After Transaction Commits

perform_later inside transactions causes race conditions — workers may execute before commit. Enqueue after the transaction block:

ActiveRecord::Base.transaction do
  create_record
end
PublishJob.perform_later(@record.id) if @record.present?

If this code can itself run inside a caller's transaction (service object under a controller transaction), "after the block" is still before the OUTER commit — use an after_commit callback (or ActiveRecord.after_all_transactions_commit) for the race-proof form.

Single-Job Internal Batching

For bulk operations, use one job with internal batching instead of many separate jobs. When multiple jobs share a file, create per-job copies to prevent race conditions on cleanup.

class BulkProcessJob < ApplicationJob
  def perform(job_id:, item_ids:, params:)
    failed_batches = []
    item_ids.each_slice(50).with_index do |batch_ids, batch_index|
      ActiveRecord::Base.transaction do
        process_batch(batch_ids: batch_ids)
      end
    rescue StandardError => e
      failed_batches << batch_index + 1
      Rails.logger.error "Batch #{batch_index + 1} failed: #{e.message}"
    end
    raise "#{failed_batches.size} batches failed: #{failed_batches.join(', ')}" if failed_batches.any?
  end
end

A log-only rescue makes partial failure look like success — collect failures and raise (or persist a summary) at the end so retries and monitors see them. Batch size is workload-dependent; tune it, and skip manual GC.start unless profiling shows it helps.

Idempotency: Operation-Based Tracking

Entity-based idempotency (e.g., checking cloned_from_id) prevents multiple intentional runs. Operation-based idempotency uses unique operation_id + cache to allow intentional re-runs while preventing retry duplicates.

Scenario	Entity-based	Operation-based
Clone template -> A, B, C	BREAKS (one clone only)	WORKS (different operation_ids)
Retry failed clone	Creates duplicate	Returns cached result

Cache Check Before Processing (narrows the retry-duplicate window)

Check cache before any work begins. This does NOT prevent concurrent time-of-check races — two simultaneous deliveries of the same operation_id both miss the read and both execute. For true once-only semantics, take an atomic reserve first: Rails.cache.write(key, ..., unless_exist: true) as the gate, a DB unique constraint, or an advisory lock. Also note the cache must be a SHARED store (Redis/Memcached) — per-process :memory_store no-ops this pattern across workers — and an expired TTL (1 hour below) reopens the duplicate window for late retries.

def clone_to(dest, operation_id: nil)
  if operation_id.present?
    cache_key = "clone_op:#{operation_id}:entity:#{id}:#{dest.id}"
    cached = Rails.cache.read(cache_key)
    if cached&.dig(:result_id)
      existing = self.class.find_by(id: cached[:result_id])
      return OpenStruct.new(success?: true, result: existing) if existing
    end
  end

  # ... proceed with work ...

  if operation_id.present? && result&.id
    Rails.cache.write(cache_key, { result_id: result.id }, expires_in: 1.hour)
  end
end

Propagate Operation ID Through Job Chain

Add operation_id as the last parameter with nil default for backward compatibility. Generate with @operation_id = operation_id || SecureRandom.uuid and pass to all nested jobs.