feat: add Redis Cluster mode support (GCP Memorystore, AWS ElastiCache)

[pedrojreis]

Overview

Adds opt-in Redis Cluster support to every service component. Standalone
Redis remains the default — existing deployments require zero configuration
changes and behave exactly as before.

Validated in production against Google Cloud Memorystore in cluster mode with
TLS and CA-certificate verification.

---

Motivation

The service previously constructed Redis connections with inline
new IORedis({ ... }) calls in four separate modules, each hardcoded to
standalone mode. Connecting to a clustered Redis (GCP Memorystore cluster, AWS
ElastiCache cluster) was impossible: the client would only ever reach a single
shard and fail with MOVED/CROSSSLOT errors under load.

This PR centralizes connection creation behind a single factory and teaches
every component to speak the Redis Cluster protocol when asked.

---

What's new

🔌 Cluster mode (opt-in, auto-detected)

Enable it either explicitly or implicitly:

# Explicit
USE_REDIS_CLUSTER=true
REDIS_HOST=node1.example.com:6379

# Auto-detected — a comma in REDIS_HOST turns on cluster mode
REDIS_HOST=node1:6379,node2:6379,node3:6379

🔐 TLS with CA-certificate validation

REDIS_TLS=true
REDIS_CA=/etc/redis-tls/ca.crt   # PEM file → full cert verification

When REDIS_CA is set it takes precedence and enables validated TLS.
REDIS_TLS=true on its own keeps the previous rejectUnauthorized: false
behaviour for backward compatibility.

🧩 BullMQ cluster-safety

Queue, Worker and QueueEvents receive a {codeapi} hash-tag prefix in cluster
mode so all BullMQ keys map to a single hash slot (a hard requirement for BullMQ
on Redis Cluster). Standalone deployments keep their existing key layout — no
migration needed.

---

New environment variables

Variable	Default	Description
USE_REDIS_CLUSTER	false	Force cluster mode. Also auto-enabled when REDIS_HOST contains a comma.
REDIS_CA	(unset)	Path to a PEM CA-cert file. Enables TLS with full certificate validation; takes precedence over REDIS_TLS.

Existing variables are unchanged and fully backward-compatible:
REDIS_HOST, REDIS_PORT, REDIS_PASSWORD, REDIS_TLS,
REDIS_USE_ALTERNATIVE_DNS_LOOKUP, REDIS_KEEP_ALIVE_MS.

---

Implementation

service/src/redis-connection.ts (new — single source of truth)

Export	Responsibility
createRedisConnection(overrides)	Returns Redis | Cluster based on env; each caller passes its own retry / readyCheck overrides
isClusterMode()	USE_REDIS_CLUSTER=true or comma in REDIS_HOST
parseRedisNodes()	Parses REDIS_HOST into [{ host, port }] startup nodes
buildTlsOptions()	REDIS_CA → { ca } (validated); else REDIS_TLS=true → { rejectUnauthorized: false }; else no TLS
bullmqPrefix()	'{codeapi}' in cluster mode, undefined otherwise

Refactored clients

All four inline new IORedis({ ... }) blocks now call createRedisConnection():

• queue.ts — shared BullMQ connection + prefix: bullmqPrefix() on Queue / QueueEvents
• workers.ts — prefix: bullmqPrefix() on both Worker instances
• egress-ledger.ts — mutation-connection pool made cluster-safe (Cluster has no .duplicate(), so a fresh createRedisConnection() is used instead)
• tool-call-server.ts, file-server.ts — session-state clients

service/src/service/replay-state.ts

scanKeys() is now cluster-aware. ioredis.Cluster has no top-level
scanStream, so in cluster mode the helper fans out across every master node
via cluster.nodes('master') and streams SCAN on each. Masters own disjoint
hash-slot ranges, so results never overlap. This fixes the runtime crash:

TypeError: <client>.scanStream is not a function

service/src/config.ts

Adds the USE_REDIS_CLUSTER flag to the parsed env.

Helm chart (helm/codeapi/)

New values.yaml surface:

redis:
  cluster:
    enabled: false
    nodes: ""                        # "host1:6379,host2:6379,host3:6379"
  tls:
    enabled: false
    caSecretName: ""                 # Secret holding the CA cert
    caKey: "ca"
    caMountPath: /etc/redis-tls/ca.crt
  useAlternativeDnsLookup: false     # required for GCP Memorystore cluster TLS

New _helpers.tpl templates — codeapi.redis.clusterEnabled,
codeapi.redis.tlsEnv, codeapi.redis.caVolume, codeapi.redis.caVolumeMount
— are wired into all five component Deployments, including mounting the CA cert
from a Secret into each pod.

service/.env.example

Documents every new variable with inline guidance.

---

Tests

New service/src/redis-connection.test.ts — 18 unit tests, no live Redis required:

Suite	Coverage
parseRedisNodes	single host, embedded port, comma list, whitespace trimming, default fallback
isClusterMode	explicit flag, comma auto-detect, standalone
buildTlsOptions	no TLS, REDIS_TLS only, REDIS_CA file read, CA precedence over REDIS_TLS, missing CA file
bullmqPrefix	standalone, cluster via flag, cluster via comma host

✓ 18 pass   redis-connection.test.ts
✓ 35 pass   egress-ledger / egress-gateway / replay-state (unchanged, still green)

---

Backward compatibility

• ✅ Standalone is the default — no env changes for existing deployments.
• ✅ REDIS_TLS=true without REDIS_CA keeps the prior rejectUnauthorized: false behaviour.
• ✅ BullMQ key prefixes are added only in cluster mode; standalone key layout is untouched.
• ✅ No breaking changes to any existing environment variable.

---

How to verify

cd service
bun test src/redis-connection.test.ts     # 18/18 pass

# render the Helm chart in cluster mode
helm template codeapi helm/codeapi \
  --set redis.enabled=false \
  --set redis.cluster.enabled=true \
  --set redis.cluster.nodes="n1:6379\,n2:6379\,n3:6379" \
  --set redis.tls.enabled=true \
  --set redis.tls.caSecretName=my-memorystore-secret

[CLAassistant]

All committers have signed the CLA.

[danny-avila]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 418e509eb3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[pedrojreis]

@danny-avila all issues should be fixed