ADR-0034: Redis and Valkey access

• Status: Accepted (the owner asked for the implementation on 2026-10-17; the open questions were settled as the recommendations below, see “As built”)
• Date: 2026-10-17

Context

docs/design.md §5.3 sketches Redis access, and the data side already exists as stubs: the policy accepts redis targets and grants (RedisTarget, RedisGrant), the audit schema has redis.command, and limen redis connect parses its options and answers “planned for Phase 7”. No wire protocol is implemented. This ADR turns the sketch into a contract the owner can approve, and lists what is open.

The pattern from ADR-0002 and ADR-0015 holds: the worker terminates the protocol next to the target and holds the credential; the proxy only authorizes and routes; the CLI opens a loopback listener so redis-cli and client libraries work unchanged.

Decision

1. Targets. Redis OSS, Valkey, and Amazon ElastiCache (node-based replication groups, and ElastiCache Serverless). Standalone and replication-group primary endpoints are in scope. Cluster mode (slot redirection, MOVED/ASK) is not in the first cut; see “Open questions”.
2. Wire. Our own RESP2 and RESP3 codec (ADR-0012: redis-protocol is rejected as a core dependency). The codec is fuzzed, and tested against the redis crate as a dev-dependency. The worker speaks the protocol version the client negotiates with HELLO, and downgrades to RESP2 toward a target that doesn’t support RESP3.
3. Local listener. limen redis connect <resource> [--user U] [--socket] opens 127.0.0.1:<port> (or a mode-0600 unix socket in a 0700 directory) and prints a ready-to-run redis-cli command with no password. Each accepted local connection becomes one Limen session (threat model: the loopback-listener risk is already documented). The listener closes after idle (default 1 hour).
4. Client authentication. The client’s own AUTH and HELLO … AUTH are accepted with any value and discarded: the outer hop is already authenticated (mTLS plus grant). The worker never forwards client credentials.
5. Target authentication.
   • Preferred, ElastiCache: IAM (Redis OSS 7.0+ or Valkey 7.2+, in-transit encryption required). The worker sends a SigV4 token (valid 15 minutes) through AUTH/HELLO as a user created with authentication-mode Type=iam, where the user name equals the user ID.
   • Fallback, and self-hosted: an ACL user the worker holds. The password lives in Secrets Manager (ElastiCache) or in the worker’s configuration as Secret<String> (self-hosted), is never logged, and is rotated by the worker where it manages the user.
   • TLS toward the target is required for ElastiCache (verify-full against the AWS CA bundle), and optional for self-hosted (tls: true|false on the target, default true, plaintext only with an explicit flag and a console warning).
6. One ACL user per policy redis.users name (the same shape as ADR-0015): a grant names limen-ro or limen-rw, the worker connects as that user, and the target-side ACL (command categories and key patterns) is the first control. The worker’s bootstrap reconciler for ElastiCache creates and updates these users and attaches them to the user group, only for names with the limen- prefix. The prefix is reserved and other users are never touched. Reconcile is idempotent and reports drift. Self-hosted Redis is documented, not bootstrapped (the operator writes the ACL), because ACL SETUSER on someone else’s server needs an admin credential we would rather not hold.
7. Connection lifetime. ElastiCache drops IAM connections after 12 hours unless re-AUTHed. The worker re-authenticates proactively (at 11 hours, with a fresh token) and defers it while the connection is inside MULTI, a blocking command, or a Lua script, where AUTH isn’t allowed. If it can’t re-authenticate by the deadline the session ends with reason credential_expired. Sessions also end at max_session_ttl from the role, as with SSH.
8. Audit. One redis.command event per command, written before the command is forwarded; if the event can’t be written the command is refused (audit fails closed).
   • Fields: user, roles, Redis user, command name (upper-cased, plus the subcommand for CLIENT, CONFIG, ACL, XGROUP, and similar), key names, arity, start time, duration, reply type (ok, error, and the error class), and bytes in and out.
   • Values are redacted by default, and so are arguments that are not keys. Per resource and per command the owner can opt in to logging them (audit.log_values), never for the credential commands below.
   • AUTH, HELLO … AUTH, ACL SETUSER, MIGRATE … AUTH, and CONFIG SET requirepass and masterauth are always redacted, whatever the setting.
   • MULTI/EXEC: each queued command is logged when queued, and EXEC is logged with the count. EVAL/EVALSHA/FUNCTION/FCALL: the script body (or its SHA and the loaded body) is logged, KEYS are names, ARGV is redacted. Pipelining: every command in the pipeline is logged in order.
   • Key names are data too (they can hold ids or emails): audit.redact_keys: true per resource replaces them with a keyed hash so the same key is still recognisable across events.
   • Pub/Sub: SUBSCRIBE/PSUBSCRIBE are logged with the channels. Messages are never logged.
9. Policy blocks, enforced in the worker as well as in the target ACL. A grant can allow or deny commands and ACL categories, and key patterns:

   redis:
     users: [limen-ro]
     deny_commands: [FLUSHALL, FLUSHDB, CONFIG, DEBUG, MONITOR, KEYS, SHUTDOWN]   # default deny list, see below
     keys: ["session:*", "cache:*"]

   The worker checks every command against the grant (default deny for FLUSHALL, FLUSHDB, CONFIG, DEBUG, MONITOR, SHUTDOWN, REPLICAOF/SLAVEOF, MODULE, ACL, CLIENT KILL, and SCRIPT/EVAL unless granted) and refuses a denied one with -NOPERM and an auth.denied event. The target ACL is the second layer if the worker is ever wrong. Key-pattern checks need the command’s key positions, so the worker uses the COMMAND GETKEYSANDFLAGS table fetched at connect time (not a hard-coded list). A command whose keys cannot be determined is refused when the grant restricts keys.
10. Things refused outright: MONITOR and SYNC/PSYNC (they stream every other client’s traffic, or replicate the data set), CLIENT SETNAME that impersonates another session, and SELECT to a database the grant doesn’t name (default: 0, or the only database ElastiCache has).
11. Attribution. The target sees limen-ro, not the person. The worker sets CLIENT SETNAME limen:<session_id> (and CLIENT SETINFO lib-name limen) so CLIENT LIST and slow-log entries join to Limen sessions.
12. Tests and demo. Compose gets Redis and Valkey (with an ACL file) services. Tests: codec round-trips and fuzzing; redis-cli and the redis crate through the worker against both servers; redaction and always-redact cases; MULTI/pipeline/Lua audit; denied command and key pattern; re-AUTH deferral inside MULTI against a short fake IAM expiry; audit fails closed with the sink down. Demo: demos/demo-phase7.sh.

Exit criteria (from the milestone)

redis-cli works through limen redis connect, values are redacted in audit, and the ElastiCache IAM path is written. Anything that needs real ElastiCache is labelled unverified against real service: IAM token auth and its 12-hour disconnect, the user-group bootstrap, ElastiCache Serverless, and the AWS CA bundle path. Locally the IAM exchange is tested against a stand-in that accepts a token we sign.

Dependencies

Probably none new for the codec (our own, ADR-0012). SigV4 signing is shared with Phase 6 (RDS IAM) and Phase 10 (OpenSearch). ElastiCache control-plane calls (DescribeUsers, CreateUser, ModifyUserGroup) need aws-sdk-elasticache. If approved it gets an ADR-0012 row after the usual maintenance and advisory check; it isn’t added by this proposal. redis (dev-dependency, tests only) also needs a row.

Open questions (settled; see “As built”)

1. Ordering. The sketch assumes Phase 6 lands first, because Phase 7 reuses its worker protocol plumbing (loopback listener, session start, SigV4 token helper, reconciler). Is that right, or should Phase 7 start independently and share nothing? Starting before Phase 6 means duplicating that plumbing and merging later.
2. Cluster mode. ElastiCache clusters (and Valkey Cluster) answer MOVED and ASK with node addresses the client then dials directly, which has the same shape as Kafka’s advertised-listener problem (ADR-0018). Options: (a) not supported in Phase 7, standalone and replication-group endpoints only (recommended); (b) the worker follows redirects itself and hides them from the client, which breaks multi-key commands that span slots but works for redis-cli without -c; (c) a local multi-listener. I recommend (a) now and a spike for (b) before promising it.
3. Default deny list. The list in item 9 is stricter than the sketch (which named FLUSHALL, CONFIG, MONITOR, DEBUG). Is “deny unless granted” right for KEYS (it blocks a production server) and EVAL, or should those be allowed by default and audited?
4. Key redaction default. Keys are logged in the clear by default here (they are what makes the audit useful). Do you want hashed keys as the default instead?
5. ElastiCache user bootstrap. Is worker-managed limen-* user creation in scope for Phase 7, or should the first cut only document the Terraform for the users and have the worker connect as given? The second is smaller and needs no aws-sdk-elasticache.

Consequences

• + redis-cli and client libraries work unchanged and never see a credential.
• + The same session, audit, lock, and kill machinery as SSH applies (a lock ends the Redis session).
• − Our own RESP codec is more code to own than a library, accepted in ADR-0012 because RESP is small.
• − Command and key policy in the worker is only as good as the key-position table; the target-side ACL stays the backstop.
• − Pub/Sub, blocking commands, and MULTI make the worker’s per-connection state machine the part to review most carefully.
• Threat model: no new entry point (the loopback listener is already listed), but a new data store connection, so docs/threat-model.md gets a Redis row when the code lands; this proposal adds none.

As built (2026-10-17)

The open questions were settled with the recommendations, and a few details changed while building. Where this section and the text above differ, this section is right.

1. Ordering. Phase 7 shares no code with Phase 6 (which isn’t built). It reuses the existing session path: the proxy’s limen-redis/1 ALPN (reserved since Phase 1) goes through the same code as SSH, with a Kind, and the worker’s session.rs admits redis grants next to ssh ones. SigV4 token signing lives in the Redis module and moves to a shared place when Phase 6 needs it.
2. Cluster mode is not supported: standalone and replication-group primary endpoints only. MOVED and ASK replies pass through to the client, which can’t follow them. A redirect-following design is a spike before it is promised.
3. Default deny list. FLUSHALL, FLUSHDB, CONFIG, DEBUG, SHUTDOWN, REPLICAOF/SLAVEOF, MODULE, ACL, CLIENT KILL, MIGRATE, SWAPDB, MOVE, SAVE, BGSAVE, BGREWRITEAOF, SLOWLOG, FAILOVER, CLUSTER, and scripting (EVAL*, FCALL*, SCRIPT, FUNCTION). KEYS is not on it: it is audited, and denied only for key-restricted grants. A grant opens a listed command with allow_commands, and closes more with deny_commands; an explicit deny wins. Never allowed, whatever the grant says: MONITOR, SYNC, PSYNC, REPLCONF, RESET, CLIENT REPLY, CLIENT PAUSE, CLIENT UNPAUSE. A key-restricted grant also refuses scripting, KEYS, SCAN, RANDOMKEY, FLUSHALL, FLUSHDB, SWAPDB, and SORT with BY or GET, even if listed.
4. Keys are logged in the clear by default. audit.redact_keys: true on the target replaces them with h: and 16 hex characters of SHA-256 over the resource name and the key. It is a plain hash, not a keyed one: it stops casual reading, not a guess at a likely id. Values are logged only with audit.log_values: true, and never for AUTH, HELLO, ACL, MIGRATE, CONFIG, REPLICAOF, SLAVEOF.
5. No ElastiCache user bootstrap. The worker logs in as the user the role names and does not create users or user groups, so aws-sdk-elasticache is not added. The users are created by Terraform or by hand (docs/guides/redis.md). Bootstrap can follow with Phase 6’s reconciler.
6. Credentials. A target that doesn’t use IAM gets its password from the worker’s --redis-credentials-file (LIMEN_REDIS_CREDENTIALS_FILE), a JSON map of resource to user to password, never from policy. TLS roots come from --redis-ca-file; a TLS target without it is refused. TLS 1.2 and 1.3 are accepted toward the target.
7. Audit timing. The redis.command event is written before the command is forwarded, so it cannot hold the reply, the duration, or the byte counts. The session’s session.end has the totals. A denied command writes auth.denied (action: redis) instead of redis.command. HELLO is audited, then forwarded without its AUTH and SETNAME; AUTH and CLIENT SETNAME are answered by the worker.
8. Pub/Sub. After SUBSCRIBE, PSUBSCRIBE, or SSUBSCRIBE the worker stops matching replies to commands and passes frames through; RESP3 push and attribute frames always pass through. A command refused in that state is answered at once.
9. Limits. One request or reply may take 256 MiB in memory. A bulk string over 512 MiB, nesting past 64 levels, or a line over 64 KiB ends the session. SELECT is limited to database 0 unless the grant lists databases; COPY ... DB is refused.
10. Re-login for IAM runs at 11 hours, only when no command is in flight and the connection is not inside MULTI or subscribed; it retries every 5 seconds and ends the session with credential_expired at 11 hours 55 minutes.
11. Dependencies. The redis crate is not a dev-dependency: sessions are tested against an in-process fake server, and the demo runs real Redis 7.4 and Valkey 8.0. aws-credential-types is a new direct dependency (it was already in the tree through aws-config), for the worker’s AWS credentials.
12. Not verified. Everything toward real ElastiCache: IAM token acceptance (the signature matches an independent implementation, not the service), the 12-hour disconnect, TLS with the AWS bundle, Serverless caches (ResourceType=ServerlessCache isn’t sent), and Valkey IAM. redis-cli was not available where this was built, so the demo uses a small RESP client when it is missing.
13. Review changes (2026-10-17, after the scenario tests). Audit events for a pipelined batch are written together, so one group fsync covers the batch (before: a 60,000-command pipeline took 672 s; after: about 2 s). The control plane’s audit ingest appends a batch together for the same reason (before: about 57 events/s, so a busy session’s backlog took minutes to drain; after: 60,000 events in about 6 s). A refused command inside MULTI poisons the transaction on the server, so EXEC aborts. Denials while subscribed are answered at once. The TLS handshake to the target has a timeout. The new redis_user field shows as the session’s login in the console. tls and audit are left out of a target’s snapshot JSON when they have their defaults, so proxies and workers from before Phase 7, which reject unknown fields, still read a policy that holds a plain Redis resource. A grant that uses the new grant fields (keys, allow_commands, …) still needs the components upgraded first: control plane, then proxies, then workers.
14. Known costs. One command at a time pays one audit fsync. The spool now syncs when an event arrives instead of on a 10 ms timer; the scenario test measured 8.7 ms per sequential command in the dev stack (the fsync alone is about 2.6 ms on that VM; before the change it was 10 to 15 ms). The pipeline of 60,000 commands takes about 1.6 s.
15. Independent review (2026-10-17). A second reviewer read the PR without the author’s context; confirmed findings were fixed. Key positions of commands with subcommands (OBJECT, XINFO, XGROUP, MEMORY) come from the subcommand entries in COMMAND INFO, since the container entry has none; an unknown command or subcommand has unknown keys, so a key-restricted grant refuses it. (Item 9 says COMMAND GETKEYSANDFLAGS; the code uses COMMAND INFO.) A grant covers only the databases it names, and the worker tracks SELECT (including inside pipelines); MOVE and COPY ... DB n are checked against the database they name. XREAD/XREADGROUP options are read as the server reads them. CLIENT LIST, CLIENT UNBLOCK are default-denied; CLIENT TRACKING, PUBSUB CHANNELS, and keyspace-notification subscriptions are refused to key-restricted grants. Denial reasons no longer contain the key (they said it in the clear even with redact_keys). Audit events list at most 100 keys or arguments. UNSUBSCRIBE is treated like SUBSCRIBE; before a SUBSCRIBE is sent, replies owed for earlier commands are delivered, so a pipelined refusal is no longer lost. A client that closes its writing side still gets its replies (up to 10 s). A command refused because audit failed inside MULTI now poisons the transaction as a policy refusal does. IAM re-login retries a failed signing until its deadline. The worker bounds opening the target connection to 25 s and no longer shows the endpoint or account in the error the user sees. IPv6 endpoints work with TLS. Of the original design text, these were not built and are not planned: audit fields for duration, reply type, and bytes (item 7), the EXEC count, the script body, and the channel names of SUBSCRIBE. The console’s session page lists a Redis session’s commands, projected from redis.command and the Redis auth.denied events into the existing db_statements table (status allowed or denied, no new table). Known and left: a request is held up to three times in memory (about 750 MiB at the 256 MiB cap) per session; RESP3 attribute frames nested inside an aggregate are miscounted; a failed SELECT on the server is not noticed by the worker’s database tracking.

Amendment: concise role selectors (2026-10-23)

redis.users names the target ACL username directly; selectors such as read and write need no Limen prefix. The ElastiCache user-bootstrap reconciler proposed above was not built, so the Redis names are not reserved for Limen-managed users. Prefixed examples in the original proposal are historical, not a current requirement.