Proletarian

The Queue Worker can be configured with several specialized functions to manage job lifecycles:

• :proletarian/retry-strategy-fn: Defines how many times a job should be retried and the delay between attempts.
• :proletarian/failed-job-fn: A callback triggered when a job has exhausted all retries and is marked as permanently failed.
• :proletarian/on-polling-error: A handler for errors occurring during the queue polling process, allowing the worker to remain resilient.
• :proletarian/install-jvm-shutdown-hook?: When enabled, ensures a graceful shutdown (e.g., on Ctrl-C) and prints a summary of job outcomes.

Queue Worker

A process that works off a named queue. It uses worker threads to pick up jobs and execute them.

• Default Queue: :proletarian/default.
• Parallelism: Controlled by the number of worker threads in the worker and the number of worker processes running across different machines.
• Polling: Workers poll the queue at a configurable interval when empty.

Job Handler

The logic executed by the worker. It receives the job-type and the payload. It is responsible for dispatching the work based on the type.

The Job Handler is the function passed to create-queue-worker. It is invoked whenever a job is ready. It must be an arity-2 function that accepts job-type (a keyword) and payload (the data passed during enqueuing).

A common pattern is to use a defmulti to dispatch different job types to specific implementation functions.

(ns your-app.handlers)

;; Define the dispatcher
(defmulti handle-job!
  (fn [job-type _payload] job-type))

;; Implement specific handlers
(defmethod handle-job! ::confirmation-email
  [job-type {:keys [email]}]
  ;; Logic to send email
  )

Proletarian guarantees that every job will be run at least once. It uses PostgreSQL/MySQL transactions to ensure that a job is only moved to the archive table once it has finished successfully.

In rare failure scenarios (e.g., the database goes offline exactly when a job finishes but before the transaction to archive it completes), the job may be picked up and run again when the worker restarts.

Requirement: Because of this guarantee, your job handlers must be idempotent or your business logic must tolerate the same job being executed more than once.

Proletarian guarantees at-least-once delivery. Jobs are only removed from the queue after successful completion and archival within the same database transaction.

Warning: In rare failure scenarios (e.g., a database crash during a commit), a job may execute multiple times. Therefore, all job handler functions must be idempotent.

The shape of the data passed to your job handler depends on the :proletarian/handler-fn-mode configuration:

• :default mode: The handler receives two arguments: (job-type, payload).
• :advanced mode: The handler receives the full job map containing metadata such as :job-id, :attempts, :enqueued-at, etc.

Proletarian uses Clojure multimethods to route jobs to their appropriate handlers based on the :job-type.

To handle a specific job type, you must implement the proletarian.job/handle-job! multimethod.

Pattern: Use a namespaced keyword for the :job-type. A common convention is to use the double-colon shorthand :: for the current namespace. For example, if your namespace is :example-a.enqueue-jobs, using ::echo as the job type will result in the keyword :example-a.enqueue-jobs/echo being used to dispatch the job.

To run the provided examples or develop locally, you must install the example database which creates a proletarian database. Use the following make commands to manage the database state.

# Install example database (creates 'proletarian' database)
make examples.db.install

# Uninstall example database
make examples.db.uninstall

# Recreate database (uninstall then install)
make examples.db.recreate

Use the following commands for testing and development:

Testing

# Run all tests via make
make test

# Run all tests directly with Clojure CLI
clojure -M:test -m kaocha.runner --config-file test/tests.edn

REPL Development

# Start REPL with all necessary aliases
make repl

# Or with Clojure CLI
clj -M:dev:test:examples

Building

make jar
make deploy
make mvn.install