History Cache

In delta mode a read has no message list to return: it reconstructs one by replaying ancestor writes back to the nearest _DeltaSnapshot (see Snapshot Cadence). That walk is the materialization cost. The history cache stores the reconstructed histories so the next read pays a lookup instead of a walk, and it wraps the raw saver on the delta path.

Where the cache sits

CachedHistorySaver overrides exactly one behaviour, get_delta_channel_history / aget_delta_channel_history. Everything else — tuple reads, writes, listing, copy, delete, prune — is delegated to the inner saver unchanged. The sync path shares one process-local memory cache and recreates it when max_entries or the derived key prefix changes, because entries under a stale prefix would be unreachable and no longer covered by thread purges.

redis on the sync path is rejected outright: database.checkpoint_cache.type 'redis' is not supported on the sync checkpointer path (TUI/embedded); use 'memory'.

Entry shape

An entry is a DeltaChannelHistory-shaped dict, exactly what the raw saver would return for that checkpoint and channel:

Reads are copy-on-read: writes comes back as a fresh list, while seed is shared by reference and never mutated in place.

Immutability: why there is no invalidation

A checkpoint’s delta history is a pure function of its sealed ancestor chain — the LangGraph contract excludes the target’s own pending writes, parent links are fixed at creation, and an ancestor’s writes are sealed once its child exists. Entries keyed by thread, namespace, checkpoint id, and channel are therefore immutable.

That is why the whole policy is performance-only and safe to differ per worker.

Keys and namespace

make_history_key(key_prefix, thread_id, checkpoint_ns, checkpoint_id, channel) builds:

{key_prefix}:{thread_id}:{sha256(checkpoint_ns \x00 checkpoint_id \x00 channel)[:24]}

Compose instead of walk

Real runs create several checkpoints per super-step and only some are ever materialized as targets, so a target’s direct parent is usually an unwarmed intermediate checkpoint. Composing only one level deep therefore measured zero cache hits on a 500-step sqlite run. The wrapper composes instead:

The sync path mirrors this recursion and shares the same cache and depth budget. Cold composition never calls the inner saver’s history method: it either recurses or walks itself, so the fallback walk is the only place the raw saver is consulted.

Configuration

The cache is built with the checkpointer, once per process. langgraph_runtime() constructs the checkpointer from the startup AppConfig snapshot (backend/app/gateway/deps.py:463), and the delta saver is wrapped in CachedHistorySaver around whatever make_checkpoint_cache() returned at that moment (backend/packages/harness/deerflow/runtime/checkpointer/async_provider.py:250-254). Nothing re-reads this section at runtime, so every field below keeps its startup value for the process’s lifetime.

database:
  checkpoint_channel_mode: delta
  checkpoint_cache:
    type: memory            # memory | redis (redis is Gateway/async only)
    max_entries: 128        # 0 disables the cache
    redis_url: null         # or DEER_FLOW_CHECKPOINT_CACHE_REDIS_URL / REDIS_URL
    ttl_seconds: 86400      # redis leak safety net; purge itself is immediate
    key_prefix: ""          # default: hash of the database identity

Only the redis URL has environment fallbacks; nothing else here is read from the environment, and $VAR interpolation inside config values is handled centrally before this section is parsed.

max_entries: 0 disables the cache uniformly for both types: the factory yields a disabled memory backend (enabled is max_entries > 0), so the wrapper never needs a None check. A zero-entry cache must behave exactly like the raw inner saver — same digests, hits == 0 — and that parity is pinned by test. The memory backend rejects negatives with max_entries must be >= 0. Eviction is strict LRU: hits and writes both refresh recency and the oldest entries are dropped while over capacity, with a cumulative evictions counter. A one-entry cache thrashes on every read but stays correct.

The redis client is imported lazily; without the optional extra, constructing a redis backend raises redis is required for the redis checkpoint cache backend. Install it with: uv sync --extra redis. An unrecognised type raises Unknown checkpoint cache type: {config.type!r}.

Purge and lifecycle

The rationale is data lifecycle: source-of-truth removal must not leave residual history payloads behind in the cache. It is never a correctness requirement.

Redis failure behaviour

Every redis failure is performance-only. The source of truth has already been updated by the time the cache is touched, so an outage costs hits, never availability:

The TTL is a leak safety net, not a correctness mechanism, because entries are immutable and no read depends on them. ttl_seconds: 0 opts out of expiry entirely, so orphaned keys then rely on the redis maxmemory policy alone. A failed purge is the only way a deleted thread’s history lingers, and only until the TTL.

Stats

CachedHistorySaver.stats() merges the backend’s counters with the wrapper’s own composition counters. It is the only cache surface: no gateway endpoint or health check reports cache state.

A steady-state process should show compose_hits growing while full_walks stays low; a persistently high full_walks means composition keeps hitting cold chains, and hits pinned at 0 on a warm process means lookups never reach entries — usually a disabled cache or a namespace that does not match the entries’ prefix, since the prefix is derived from the database identity. See Troubleshooting.

Compiled accessor-graph cache

Delta reads must go through the thread-state accessor, whose compiled graph is cached per assistant, channel mode, snapshot cadence, and app config. That cache has its own size cap:

This cap applies in both modes and is unrelated to the history cache, which only wraps delta savers. It exposes no counters; it is bounded by size only.

Measuring the cache

cache_effect_ms (state cold minus warm p50) is the decision metric for the production accessor-cache defaults, and it comes out of the production checkpoint benchmark, which also reports state_cold_ms, state_warm_p50_ms, state_warm_p95_ms, db_bytes, and per-limit history read times. The channel benchmark reports warm_read_ms, cold_read_ms, and reducer_replay_ms with delta/full ratios. Timing thresholds are not CI gates. See Operating Checkpoints.

Related

• Snapshot Cadence — the walk this cache memoizes.
• Channel Modes — the accessor and the mode marker.
• Observability — where these counters surface.
• Troubleshooting — redis errors on the embedded path and zero-hit caches.
• Reference — the full config-key table.