Unified action surface
The problem
Section titled “The problem”Every engine implements "retry" and "cancel" differently:
- Celery -
task.retry()from inside the task, or re-apply withapp.send_task. - RQ -
Queue.requeue(job_id), orjob.requeue(). - Dramatiq - no first-class retry API; middleware-based.
- Huey - retries are configured per-task; no ad-hoc retry surface.
- arq - no retry API; re-enqueue manually.
- taskiq - per-task retry decorators.
z4j exposes one command vocabulary, but only when an adapter can honor the specific command without reconstructing redacted inputs or pretending a broker operation succeeded.
The z4j answer
Section titled “The z4j answer”Four common verbs exist, gated by each adapter's advertised capabilities:
| Verb | Semantics |
|---|---|
retry |
Re-enqueue only when the engine can recover the authoritative inputs or the operator supplies complete replacements. |
cancel |
Invoke an engine primitive that covers the adapter's documented pending/running contract. |
bulk_retry |
Retry an explicit brain-selected set only on adapters that implement it safely. |
purge_queue |
Drop pending messages only where the adapter can measure and target the requested queue. |
Capability dispatch
Section titled “Capability dispatch”Each engine adapter advertises its capability tokens in the hello frame.
When z4j dispatches a verb:
- Look up the target agent's capability map.
- If advertised, send a
commandframe and let the adapter call the engine API. - If absent, refuse the command and hide the dashboard control.
The brain stores redacted task inputs, so it does not rebuild an executable payload from its task or event rows. Retry-with-different-inputs supplies both complete replacement collections explicitly.
Audit semantics
Section titled “Audit semantics”Every action writes an audit row with the requested command and outcome.
See API § tasks for exact endpoints.
Failure behavior
Section titled “Failure behavior”- If the agent is offline, the command row is written and stays
pending, and the caller gets503 agent_offline. Nothing is held for reconnect: the timeout sweeper closes the row at its deadline (Z4J_COMMAND_TIMEOUT_SECONDS, default 60). Re-issue the action once the agent is back. - If the command times out (60s), the action records
error: timeoutand the audit log captures the failure. The task state is not modified. - A mutation timeout can be indeterminate when the broker call may still finish; the adapter reports that state instead of claiming a clean failure.
What we don't do
Section titled “What we don't do”- No blanket retry policy - the unified action surface is operator-driven. Automatic retries happen only where you arm an automation rule with a
retryaction, which is admin-gated, circuit-broken, and audited. For per-task backoff and attempt limits, use the engine's native retry configuration. - No side-effect-safety guarantees - retrying a task that already half-ran is the user's call. z4j does not introspect idempotency.