Redaction

Where it runs

Queue adapters scrub the task fields they support inside the worker process, before building the event frame. The brain then scrubs every inbound event again before persistence as defence in depth. This is best-effort filtering, not a guarantee that the brain never receives cleartext: anything an adapter does not scrub arrives at the brain before the second pass examines it.

Data entered directly into the brain is outside this pipeline. In particular, schedule args and kwargs are accepted, stored, and returned without redaction. Do not put credentials or other secrets in schedule arguments.

The canonical built-in catalog is z4j_core.redaction.patterns.

Default key-name patterns

Key patterns use case-insensitive re.fullmatch. A matching key causes its entire value to become [REDACTED]:

• Passwords: password, password_confirmation, password_hash, passwd, pwd, old_password, new_password.
• Secrets: secret, secrets, client_secret / clientSecret, webhook_secret / webhookSecret, hmac_secret / hmacSecret.
• Tokens: token, access_token / accessToken, refresh_token / refreshToken, id_token / idToken, api_token / apiToken, bearer_token / bearerToken, session_token / sessionToken.
• API keys: api_key, apiKey, apikey, x-api-key, x_api_key.
• Authorization and cookies: authorization, auth, credentials, cookie, set-cookie, set_cookie.
• Personal identifiers: ssn, social_security, social_security_number, credit_card, credit_card_number, card_number, cvv, cvc.

The spelling api-key is not a built-in match. Neither are longer names such as password_hint, user_password, or last_password_reset; full-match means the complete key must match one of the patterns above.

Default value patterns

Value patterns use case-insensitive re.search. One match replaces the whole scalar, not only the matching substring. The defaults cover:

• Three-segment JWT-shaped strings beginning eyJ.
• Bearer followed by at least 16 letters, digits, ., _, or -. The Authorization: prefix is not required. Standard-base64 tokens containing +, /, or = in the first 16 characters are not covered by this rule.
• Stripe prefixes sk_live_, sk_test_, pk_live_, pk_test_, whsec_, rk_live_, and rk_test_ only when at least 24 consecutive alphanumeric characters follow. Shortened examples such as sk_live_abc123 do not match.
• Slack xoxb-, xoxp-, and xoxa- tokens and incoming-webhook URLs; GitHub ghp_, gho_, ghs_, and github_pat_ tokens; AWS AKIA and ASIA access-key IDs; Google AIza keys; and SendGrid API keys, subject to the exact lengths in the source catalog.
• SK followed by 32 hexadecimal characters. This is a Twilio API Key SID, an identifier, not the secret. Twilio Auth Tokens and API Key Secrets are unprefixed and have no default value-pattern coverage; rely on a matching field name.
• Credential-bearing database URLs using bare schemes: postgres://, postgresql://, mysql://, mariadb://, mongodb://, mongodb+srv://, redis://, rediss://, amqp://, and amqps://. SQLAlchemy driver-qualified URLs such as postgresql+psycopg:// and postgresql+asyncpg:// are not covered.
• RSA, EC, DSA, OPENSSH, ENCRYPTED, and unqualified PRIVATE KEY BEGIN lines. -----BEGIN PGP PRIVATE KEY BLOCK----- and SSH2 headers do not match.
• Email addresses with dotted domains and US SSN-shaped values.

An email address anywhere in an error message or traceback replaces the entire message or traceback with [REDACTED]; it is not reduced to an in-place placeholder. user@localhost does not match the email rule.

Customization and its current limit

The Config model accepts redaction_extra_key_patterns, redaction_extra_value_patterns, and redaction_defaults_enabled, including their Django and Flask nested-setting spellings. The current runtime does not pass those resolved fields into automatically discovered queue adapters. They are therefore accepted configuration that has no redaction effect; do not rely on them for a security boundary.

For Celery, RQ, and Dramatiq tasks, use the adapter's z4j_meta decorator to force-redact named kwargs (and, where supported, keep only an allowlist):

@z4j_meta(redact_kwargs=["api-key", "password_hint"])
def send_message(*, api_key: str, password_hint: str) -> None:
    ...

Bare installations can inject a configured engine directly into a queue adapter before passing that adapter to install_agent:

from z4j_core.redaction import RedactionConfig, RedactionEngine
from z4j_celery import CeleryEngineAdapter

redaction = RedactionEngine(
    RedactionConfig(
        extra_key_patterns=(r"api-key", r"password_hint"),
        extra_value_patterns=(r"postgresql\+psycopg://[^\s]+",),
    )
)
engine = CeleryEngineAdapter(celery_app=celery_app, redaction=redaction)

There is no per-project redaction setting in the brain.

Task, exception, and traceback fields

Task names, queue names, routing keys, and worker names have no exemption from value matching. Ordinary names survive; an email-, credentialed-URL-, or SSN-shaped substring replaces the whole name. This includes ordinary Celery node names such as [email protected], which look like email addresses.

Celery, RQ, and Dramatiq keep an exception class in a separate field. TaskIQ and Huey combine ClassName: message; a matching value therefore removes the class name as well. Those two adapters do not scrub that exception string in the worker process, so it reaches the brain's second pass first.

A traceback is scrubbed as one scalar, not line by line. Only value patterns apply to its text. One matching value removes the whole traceback, including file, line, and function names. Key-name rules do not recognize text such as password = "..." inside a traceback, so it remains unless the value itself matches a value pattern.

Size behavior

• The default redaction engine keeps the first 8192 UTF-8 bytes of a scalar and appends [...N more bytes truncated]. Shorter strings are not tagged. Z4J_MAX_PAYLOAD_BYTES is currently accepted by agent configuration but is not wired into discovered adapters, so it does not change this limit.
• bytes and bytearray are converted with str() and then scanned and truncated like any other scalar. They are not replaced with a length-only marker; a binary payload under a non-secret key may be recorded as its Python representation.
• There is no 2 MiB total-event truncation. A WebSocket frame above Z4J_WS_MAX_FRAME_BYTES (default 1 MiB) is undeliverable and purged by the agent transport. Long-poll uploads are HTTP requests and also face Z4J_MAX_PAYLOAD_SIZE_BYTES (default 8192 bytes); an oversized request gets HTTP 413, and a persistently oversized single frame is eventually dropped rather than shortened.

Disabling redaction

There is no working high-level setting that disables redaction. The low-level API can still do it: injecting RedactionEngine(RedactionConfig(default_patterns_enabled=False)) into an adapter disables every built-in key and value pattern. Audit any custom adapter wiring for that argument. The similarly named resolved Config field is not wired and currently changes nothing.