Honeydew Documentation

When a worker crashes, Honeydew uses a failure mode to determine what happens to the job. You can select a failure mode when calling Honeydew.start_queue/3.

Available failure modes:

• Honeydew.FailureMode.Abandon: The job is simply forgotten.
• Honeydew.FailureMode.Move: The job is removed from the original queue and placed on a different queue.
• Honeydew.FailureMode.Retry: The job is re-attempted on its original queue a specified number of times. After the final failure, it triggers another failure mode.

Honeydew provides several queue implementations depending on your requirements for persistence, replication, and performance. If no queue is explicitly specified, Honeydew defaults to an in-memory Mnesia store.

Available Queue Types

• Mnesia Queue: Highly configurable. Supports replication across multiple nodes, disk persistence via dets, and various safety modes ("access contexts").
• ErlangQueue: A fast FIFO queue implemented using the :queue and Map modules. It is in-memory and does not survive node crashes.
• Ecto Poll Queue: An Ecto-backed queue that supports auto-enqueuing jobs when a new row is inserted into a database table. It is suitable for environments using replicated databases (like CockroachDB or Postgres).

Honeydew workers can be stateless or initialized with state by implementing the init/1 callback.

Important constraints:

• Immutability: Worker state is immutable. To change the state, you must cause the worker to crash so that Honeydew restarts it.
• Return Value: The init/1 function must return {:ok, state}.
• Error Handling: If init/1 returns anything else or raises an error, Honeydew will trigger the failed_init/0 callback (if implemented). If failed_init/0 is not implemented, Honeydew will attempt to re-initialize the worker after a five-second delay.

defmodule MyWorker do
  # Implement init/1 to provide state
  def init(args) do
    {:ok, %{my_state: "some_value"}}
  end
end

When a job completes successfully, Honeydew executes the handle_success/2 function from the selected success mode module on the queue's node. This is typically used for monitoring and telemetry.

To calculate performance metrics, you can use the following fields available on the job:

• :enqueued_at
• :started_at
• :completed_at

Select a success mode using Honeydew.start_queue/3.

Honeydew is a pluggable job queue and worker pool for Elixir focused on at-least-once execution. To get started, define a worker module with a function to execute, start a queue, and start workers assigned to that queue. You can then enqueue jobs using Honeydew.async/3.

defmodule MyWorker do
  def do_a_thing do
    IO.puts "doing a thing!"
  end
end

:ok = Honeydew.start_queue(:my_queue)
:ok = Honeydew.start_workers(:my_queue, MyWorker)

:do_a_thing |> Honeydew.async(:my_queue)

# => "doing a thing!"

You can integrate Honeydew into your application's supervision tree by calling Honeydew.start_queue/1 and Honeydew.start_workers/2.

When calling Honeydew.start_workers/2, the second argument is a tuple containing your worker module and a list of arguments. These arguments are passed directly to the worker's init/1 callback to establish state.

defmodule App do
  def start do
    Honeydew.start_queue(:my_queue)
    Honeydew.start_workers(:my_queue, {Worker, ['127.0.0.1', 8087]})
  end
end

To implement a custom queue, use the existing queue modules as a reference. When designing your implementation, consider the following architectural questions:

• Where exactly does the queue state live?
• Is the queue process(es) the location where jobs reside?
• Is the queue a stateless connector for an external broker?
• Or is it a hybrid approach?