Skip to Content
DeerFlow

Reference

Every name, key, string, and test anchor behind database.checkpoint_channel_mode. This page is for looking things up, not for learning the model; start with Channel Modes if you need the concepts.

Evidence column values are repo-relative path:line references using the aliases below. A fact that appears here is traceable to one of those lines.

AliasPath
dbcfgbackend/packages/harness/deerflow/config/database_config.py
appcfgbackend/packages/harness/deerflow/config/app_config.py
reloadbackend/packages/harness/deerflow/config/reload_boundary.py
cmodebackend/packages/harness/deerflow/runtime/checkpoint_mode.py
cstatebackend/packages/harness/deerflow/runtime/checkpoint_state.py
tstatebackend/packages/harness/deerflow/agents/thread_state.py
factorybackend/packages/harness/deerflow/agents/factory.py
clientbackend/packages/harness/deerflow/client.py
workerbackend/packages/harness/deerflow/runtime/runs/worker.py
agentbackend/packages/harness/deerflow/agents/lead_agent/agent.py
goalsrcbackend/packages/harness/deerflow/runtime/goal.py
checkpoint_patchesbackend/packages/harness/deerflow/checkpoint_patches.py
syncprovbackend/packages/harness/deerflow/runtime/checkpointer/provider.py
asyncprovbackend/packages/harness/deerflow/runtime/checkpointer/async_provider.py
cacheprovbackend/packages/harness/deerflow/runtime/checkpoint_cache/provider.py
threadsbackend/app/gateway/routers/threads.py
thread_runsbackend/app/gateway/routers/thread_runs.py
runsbackend/app/gateway/routers/runs.py
servicesbackend/app/gateway/services.py
depsbackend/app/gateway/deps.py
context_usagebackend/app/gateway/context_usage.py
exampleconfig.example.yaml

The pinned dependency set referenced by the defect table is langgraph 1.2.9, langgraph-checkpoint 4.2.0, and langgraph-checkpoint-postgres 3.1.2.

Configuration keys

Every database.* key related to checkpoints, with the constraint the loader actually enforces. “Restart” means the running process captured the value at startup — frozen into process state, compiled into a channel table, or bound when a singleton such as the checkpointer and its history cache was constructed — so an edit does not take effect until that process restarts.

KeyTypeDefaultRangeRestartEvidence
database.backendLiteral["memory", "sqlite", "postgres"]"memory" (pydantic); sqlite when the key is absent from config.yamlmemory | sqlite | postgresYesdbcfg:143-146, appcfg:67-70, appcfg:519-527
database.checkpoint_channel_modeCheckpointChannelMode = Literal["full", "delta"]"full"full | deltaYes — frozen on first use; a later different value raises CheckpointModeReconfigurationErrordbcfg:59, dbcfg:147-155, cmode:39-45
database.checkpoint_delta.snapshot_frequencyint10 (DEFAULT_CHECKPOINT_SNAPSHOT_FREQUENCY)ge=1 (no upper bound)Yes — frozen alongside the modedbcfg:61, dbcfg:74-84, cmode:53-69
database.checkpoint_graph_cache.accessor_graph_maxint64ge=1No — hot-reloadable, re-read on every eviction checkdbcfg:96-99, services:1133-1138
database.checkpoint_cache.typeLiteral["memory", "redis"]"memory"memory | redis; redis is async/Gateway onlyYes — captured when the checkpointer is builtdbcfg:113-116, syncprov:181-182, asyncprov:250-254
database.checkpoint_cache.max_entriesint128ge=0; 0 disables the cache entirelyYes — captured when the checkpointer is builtdbcfg:117-121, cacheprov:76-84, deps:463
database.checkpoint_cache.redis_urlstr | NoneNoneany redis URL; when omitted, DEER_FLOW_CHECKPOINT_CACHE_REDIS_URL, then REDIS_URL, then redis://localhost:6379/0Yes — captured when the checkpointer is builtdbcfg:122-125, cacheprov:19-23
database.checkpoint_cache.ttl_secondsint86400ge=0; 0 explicitly disables expiryYes — captured when the checkpointer is builtdbcfg:126-135
database.checkpoint_cache.key_prefixstr""any string; empty derives a hash of the database identityYes — captured when the checkpointer is builtdbcfg:136-139, cacheprov:60-64
database.checkpoint_delta_snapshot_frequency (deprecated, flat)intnone — carried onto the nested keyge=1 after migrationn/adbcfg:218-254

What each key affects

  • database.checkpoint_channel_mode selects how the messages channel is stored: whole-value checkpoints (full) or a LangGraph DeltaChannel sentinel plus per-step writes (delta). It affects every read and write of a thread.
  • database.checkpoint_delta.snapshot_frequency is the cadence at which delta mode writes a full messages snapshot every N per-step writes. Higher means smaller checkpoints and slower materialization. It is ignored in full mode.
  • database.checkpoint_graph_cache.accessor_graph_max caps the compiled thread-state accessor graphs a process caches, keyed per assistant, channel mode, and snapshot cadence. Unlike the mode and cadence it is not restart-required, because a different cap only changes when the cache evicts, never graph semantics.
  • database.checkpoint_cache.* configures the delta-mode materialized-history cache only. It is performance-only and never frozen — no correctness invariant requires cross-process agreement, and results are identical with it disabled — but it is not hot-reloadable: the cache object is constructed with the checkpointer at startup (deps:463, asyncprov:250-254), inside a database section that reload registers as startup-only (reload:47), so changing any of these fields needs a restart. The memory backend is bounded by max_entries, the redis backend by ttl_seconds and the server’s own maxmemory policy.
  • database.backend selects which saver serves the threads (InMemorySaver, SQLite, or Postgres). Mode semantics live in the compiled channel table rather than in the saver, so both modes work on all three backends.

The verbatim Field(description=...) text for the two frozen keys is worth reading once, because it states the cross-process rule:

"Checkpoint representation for accumulating channels. 'full' preserves full-value message checkpoints; 'delta' uses LangGraph DeltaChannel for messages. Restart is required, and all processes sharing one checkpoint database must use the same value."
"DeltaChannel snapshot cadence: a full messages snapshot is stored every N per-step writes (higher = smaller checkpoints, slower materialization). Restart is required, and all processes sharing one checkpoint database must use the same value."

Validation

InputResultEvidence
checkpoint_channel_mode set to anything outside full/delta (for example auto)pydantic ValidationError at config loadbackend/tests/test_app_config_reload.py:118-120
snapshot_frequency set to 0 or -1pydantic ValidationError (constraint ge=1)dbcfg:76, backend/tests/test_app_config_reload.py:131-134
Deprecated flat key set to 0 or -1pydantic ValidationError after the value is carried onto the nested keybackend/tests/test_app_config_reload.py:154-157
Unknown keys under database:ignored (pydantic extra="ignore"), which is why the legacy-key shim existsdbcfg:224-227
Both the flat key and checkpoint_delta.snapshot_frequency setthe nested key wins, with a warningdbcfg:218-240

Deprecated flat key and migration warnings

The old top-level database.checkpoint_delta_snapshot_frequency is still honored by the pydantic @model_validator(mode="before") named _migrate_legacy_snapshot_frequency. It logs one of exactly three strings, all operator-visible:

ConditionVerbatim stringEvidence
Both keys set, checkpoint_delta is a dictBoth database.checkpoint_delta_snapshot_frequency (deprecated) and database.checkpoint_delta.snapshot_frequency are set; the nested key wins.dbcfg:236-238
checkpoint_delta is an already-built CheckpointDeltaConfig objectIgnoring deprecated database.checkpoint_delta_snapshot_frequency because database.checkpoint_delta is already set.dbcfg:246-248
Legacy value carried forwarddatabase.checkpoint_delta_snapshot_frequency is deprecated; use database.checkpoint_delta.snapshot_frequency instead. Carried the legacy value (%r) forward.dbcfg:250-253

Environment variables

No environment variable selects the checkpoint mode or its cadence. Both come only from config.yaml. The only mode-feature environment variables are the history-cache redis URL fallbacks (DEER_FLOW_CHECKPOINT_CACHE_REDIS_URL, then REDIS_URL, then redis://localhost:6379/0) at cacheprov:19-23. $VAR interpolation in config values is resolved centrally by AppConfig.resolve_env_variables() before DatabaseConfig is constructed, so DatabaseConfig itself does no environment processing (dbcfg:13-22, appcfg:442).

Config version

The schema change shipped with config_version 27; config.example.yaml is currently config_version: 49 and a missing version counts as 0 (example:23, appcfg:531-575). An outdated config.yaml logs:

Your config.yaml (version %d) is outdated — the latest version is %d. Run `make config-upgrade` to merge new fields into your config.

make config-upgrade merges the new key with the safe default full (appcfg:570-575, backend/tests/test_config_version.py:151-203). See Configuration for the general upgrade flow.

Constants and keys

SymbolValueWhere it livesEvidence
CheckpointChannelModeLiteral["full", "delta"]deerflow.config.database_configdbcfg:59
DEFAULT_CHECKPOINT_SNAPSHOT_FREQUENCY10deerflow.config.database_configdbcfg:61
INTERNAL_CHECKPOINT_MODE_KEY"__deerflow_checkpoint_channel_mode" (a RunnableConfig configurable key)deerflow.runtime.checkpoint_modecmode:18
CHECKPOINT_MODE_METADATA_KEY"deerflow_checkpoint_channel_mode" (a checkpoint metadata key; the value written is "delta")deerflow.runtime.checkpoint_modecmode:19, cmode:86
counters_since_delta_snapshotLangGraph’s own metadata key; a dict containing "messages" is treated as delta even without the DeerFlow markerupstream langgraph-checkpoint; detected in deerflow.runtime.checkpoint_modecmode:97-98

Marker semantics, in one place: inject_checkpoint_mode(config, mode) always sets config["configurable"]["__deerflow_checkpoint_channel_mode"]. In delta mode it additionally sets config["metadata"]["deerflow_checkpoint_channel_mode"] = "delta"; in full mode it removes that metadata key, so absence means full and pre-feature checkpoints need no migration (cmode:81-88). The snapshot cadence is deliberately not stamped into checkpoint metadata, because the mode marker contract and the full-to-delta migration semantics are unchanged by the frequency value (cmode:53-60).

Exceptions and HTTP status

ClassBaseDocstringEvidence
CheckpointModeMismatchErrorRuntimeError"Raised before a full-mode graph reads a Delta checkpoint."cmode:22-23
CheckpointModeReconfigurationErrorRuntimeError"Raised when a process attempts to hot-switch its persistence mode."cmode:26-27

Verbatim messages raised by the mode machinery and its guards:

Message (verbatim)Raised byEvidence
checkpoint_channel_mode is restart-required and cannot change in a running processfreeze_checkpoint_channel_mode → CheckpointModeReconfigurationErrorcmode:44
checkpoint_delta.snapshot_frequency is restart-required and cannot change in a running processfreeze_checkpoint_snapshot_frequency → CheckpointModeReconfigurationErrorcmode:68
snapshot frequency must be positivefreeze_checkpoint_snapshot_frequency → ValueError, when a non-positive value bypasses pydanticcmode:63-64
Thread requires delta mode; materialize and convert its checkpoints before using full mode.CheckpointModeMismatchError, from raise_if_snapshot_incompatible and raise_if_checkpoint_tuple_incompatible, and duplicated in the degraded gateway accessorcmode:123, cmode:129, services:1229
database.checkpoint_cache.type 'redis' is not supported on the sync checkpointer path (TUI/embedded); use 'memory'.ValueError in _wrap_sync_if_deltasyncprov:181-182
f"Unknown checkpoint cache type: {config.type!r}"ValueError in make_checkpoint_cachecacheprov:102
create_deerflow_agent does not support checkpoint_channel_mode='delta' with a checkpointer: persisted graphs built here bypass checkpoint mode marker injection and the fail-closed compatibility gate (see deerflow.runtime.checkpoint_mode), so a mixed-mode store would silently corrupt thread state. Use the guarded application paths (make_lead_agent or DeerFlowClient) for delta persistence; delta without a checkpointer is ephemeral and allowed.ValueError in create_deerflow_agentfactory:135-141

HTTP status mapping

Only the threads router imports these errors. It registers the gate tuple _CHECKPOINT_MODE_ERRORS = (CheckpointModeMismatchError, CheckpointModeReconfigurationError) (threads:82) and maps it as follows. Bodies are FastAPI’s default {"detail": "<string>"}.

ConditionStatusdetailEvidence
CheckpointModeMismatchError409f"Thread {thread_id}: {exc}", i.e. Thread <thread_id>: Thread requires delta mode; materialize and convert its checkpoints before using full mode.threads:105-106
CheckpointModeReconfigurationError503str(exc)threads:107

The router states the rationale: a mismatch means the thread’s persisted checkpoints conflict with the process’s frozen mode (operator-actionable, 409); a reconfiguration means the process itself is mid mode-flip (transient, 503) (threads:99-102). The 503 half is defensive — nothing in these routes can raise it. The freeze helpers are called only by Gateway startup (deps:451-452), agent:835,839, and client:230-231, and no state, history, or thread route builds an agent: a checkpoint_channel_mode edit is injected over by the run worker (worker:1018-1019, preferred at agent:831-835) and ignored until a restart, and a snapshot_frequency edit makes the next run fail with the message (worker:1450-1462) instead of returning a status.

Routes that return 409

RouteHandlerNotesEvidence
GET /api/threads/{thread_id}get_threadchecked before the 404 Thread <id> not foundthreads:1276,1291-1292,1298-1299,1303
GET /api/threads/{thread_id}/stateget_thread_statethreads:1446,1454-1455,1458-1459
POST /api/threads/{thread_id}/stateupdate_thread_statea client-supplied checkpoint_id older than the head is applied as a delta fork with no linearization — see Forks are accepted, not refusedthreads:1495,1559-1560
POST /api/threads/{thread_id}/historyget_thread_historythreads:1695,1715-1716,1907-1908
POST /api/threads/{thread_id}/branchesbranch_thread → _branch_thread_with_reservationthe detail names the source thread id; no test pins this routethreads:953,231-232,298-299,1105-1106
POST /api/threads/{thread_id}/compactcompact_thread409 is also returned when compaction is disabled; no test pins this routethreads:1399,1413-1414,1429-1430,1431-1432

Routes where the mismatch fails asynchronously

RouteImmediate responseFailure payloadEvidence
POST /api/threads/{thread_id}/runs200 run record, no pre-gate — but 500 when the thread’s run-event feed is empty, so ensure_checkpoint_history_seeded materializes the head inside start_run firstthe run record later has status: error and the mismatch text as errorthread_runs:945-960, worker:1018-1030,1044-1047, worker:1450-1462, services:1503-1520,2055
POST /api/threads/{thread_id}/runs/stream200 text/event-streaman event: error frame whose data is {"message": <mismatch text>, "name": "CheckpointModeMismatchError"}worker:1466-1472, services:2374-2378
POST /api/threads/{thread_id}/runs/wait200 JSON, never 409{"status": "error", "error": "Thread requires delta mode; materialize and convert its checkpoints before using full mode."}thread_runs:1018-1076
POST /api/runs/stream, POST /api/runs/wait (stateless)same as abovesame {"status", "error"} bodyruns:59-91

Routes with no mode mapping

RouteActual behaviourEvidence
POST /api/threads/{thread_id}/runs/regenerate/prepare500 with detail: "Failed to read latest checkpoint" — the accessor build sits outside any trythread_runs:747,749-752
same route, lineage/history scan500 with detail: "Failed to inspect checkpoint history"thread_runs:679-681
POST /api/threads/{thread_id}/runs/edit-regenerate/preparethe same two 500 pathsthread_runs:838-841, thread_runs:633-697
POST /api/threads/{thread_id}/runs, /runs/stream, /runs/wait, and the stateless /api/runs/* routes, when the thread’s run-event feed is empty and a checkpoint head exists500: ensure_checkpoint_history_seeded reads that head through the gated accessor before admission, and start_run maps only ConflictError (409) and UnsupportedStrategyError (501)services:1503-1520, services:2055, services:2126-2129
GET /api/threads/{thread_id}/token-usage200 with context_usage: null and a warning log onlycontext_usage.py:73-78, thread_runs:1872-1873
GET / PUT / DELETE /api/threads/{thread_id}/goalnot gated at all: raw checkpointer.aget_tuple / aput on channel_values["goal"]threads:1329-1383, goalsrc:442-453,468-477,480-542
DELETE /api/threads/{thread_id}, POST /search, PATCH, POST /movemetadata store only, so mode-blindthreads:706,1173,1216,1255

No reporting surface

No health, readiness, features, capabilities, or console field reports the mode or cadence. No response model, header, or event field carries it either: the only custom SSE headers are Cache-Control, Connection, X-Accel-Buffering, and Content-Location. The mode is consumed server-side only, through app.state.checkpoint_channel_mode, app.state.checkpoint_snapshot_frequency, and RunContext (deps:451-452,767-768, services:1093,1283-1312). A latent exception is that the response metadata on GET /api/threads/{thread_id} and on POST /api/threads/{thread_id}/history strips other internal keys but not the delta marker or counters_since_delta_snapshot, so those can pass through to a client (threads:1308-1315, threads:1890-1900). The SSE metadata frame carries only {"run_id", "thread_id"} (worker:1067-1074). See Observability.

Public symbols

Import paths a consumer may rely on. Everything listed here is a Python import; the route contracts are in the HTTP status mapping section above.

SymbolImport pathRoleEvidence
CheckpointChannelModedeerflow.config.database_configmode literaldbcfg:59
DEFAULT_CHECKPOINT_SNAPSHOT_FREQUENCYdeerflow.config.database_configdefault cadence, 10dbcfg:61
CheckpointDeltaConfigdeerflow.config.database_configdelta tuning model (snapshot_frequency)dbcfg:64
CheckpointGraphCacheConfigdeerflow.config.database_configgraph-cache cap model (accessor_graph_max)dbcfg:86
CheckpointCacheConfigdeerflow.config.database_confighistory-cache policy modeldbcfg:103
DatabaseConfigdeerflow.config.database_configthe whole database: sectiondbcfg:142
resolve_checkpoint_graph_cache_max(database_config, field_name, default)deerflow.config.database_configtolerant hot-reloadable cap readerdbcfg:45-57
INTERNAL_CHECKPOINT_MODE_KEYdeerflow.runtime.checkpoint_modeconfigurable keycmode:18
CHECKPOINT_MODE_METADATA_KEYdeerflow.runtime.checkpoint_modecheckpoint metadata keycmode:19
CheckpointModeMismatchErrordeerflow.runtime.checkpoint_modefail-closed cross-mode errorcmode:22
CheckpointModeReconfigurationErrordeerflow.runtime.checkpoint_moderestart-required violation errorcmode:26
frozen_checkpoint_channel_mode()deerflow.runtime.checkpoint_moderead the frozen mode or Nonecmode:34-36
freeze_checkpoint_channel_mode(mode)deerflow.runtime.checkpoint_modefreeze on first call, reject changescmode:39-45
frozen_checkpoint_snapshot_frequency()deerflow.runtime.checkpoint_moderead the frozen cadence or Nonecmode:48-50
freeze_checkpoint_snapshot_frequency(n)deerflow.runtime.checkpoint_modefreeze cadence; ValueError if non-positivecmode:53-69
resolve_checkpoint_snapshot_frequency(n=None)deerflow.runtime.checkpoint_modeexplicit → frozen → defaultcmode:72-78
inject_checkpoint_mode(config, mode)deerflow.runtime.checkpoint_modestamp the configurable key and metadata markercmode:81-88
checkpoint_metadata_uses_delta(metadata)deerflow.runtime.checkpoint_modemarker/counters detectioncmode:91-98
checkpoint_tuple_uses_delta(tuple)deerflow.runtime.checkpoint_modetuple-level detectioncmode:101-104
state_snapshot_uses_delta(snapshot)deerflow.runtime.checkpoint_modesnapshot-level detectioncmode:107-111
raise_if_snapshot_incompatible(snapshot, mode)deerflow.runtime.checkpoint_moderead gatecmode:114-123
raise_if_checkpoint_tuple_incompatible(tuple, mode)deerflow.runtime.checkpoint_moderaw-tuple gatecmode:126-129
ensure_checkpoint_mode_compatible(checkpointer, config, mode)deerflow.runtime.checkpoint_modesync pre-write gatecmode:132-140
aensure_checkpoint_mode_compatible(checkpointer, config, mode)deerflow.runtime.checkpoint_modeasync pre-write gatecmode:143-146
ThreadStatedeerflow.agents.thread_statefull-mode state schematstate:280
DeltaThreadStatedeerflow.agents.thread_statedelta schema at the default cadencetstate:396-397,423-426
delta_messages_field(snapshot_frequency=DEFAULT_CHECKPOINT_SNAPSHOT_FREQUENCY)deerflow.agents.thread_stateAnnotated[list[AnyMessage], DeltaChannel(merge_message_writes, snapshot_frequency=…)]tstate:385-390
DELTA_MESSAGES_FIELDdeerflow.agents.thread_statemodule-level default-cadence delta fieldtstate:393
merge_message_writes(state, writes)deerflow.agents.thread_statelinear-time add_messages-equivalent reducertstate:328-382
get_thread_state_schema(mode, snapshot_frequency=None)deerflow.agents.thread_stateThreadState for non-delta, cached delta schema otherwisetstate:416-419
adapt_state_schema_for_mode(schema, mode, snapshot_frequency=None)deerflow.agents.thread_statemiddleware/state schema adaptationtstate:437-440
normalize_middleware_state_schemas(middleware, mode, snapshot_frequency=None)deerflow.agents.thread_statemiddleware list with adapted state_schema copieststate:454-467
THREAD_STATE_REDUCER_FIELDSdeerflow.agents.thread_statefrozenset of reducer channels (including "messages"), used to decide Overwrite wrappingtstate:400-413, threads:1065-1066
create_deerflow_agent(..., checkpoint_channel_mode="full", checkpoint_snapshot_frequency=None, ...)deerflow.agents.factorypublic agent factory; delta plus a checkpointer is rejectedfactory:66-77, factory:135-141
CheckpointStateAccessor, build_state_mutation_graphdeerflow.runtime (package re-export)accessor choke point and mutation graphdeerflow/runtime/__init__.py:8,21-22
ThreadState, DeltaThreadState, SandboxStatedeerflow.agents (package re-export)lazily exported state schemasagents/__init__.py:14-16,40-47
CheckpointStateAccessor.bind(graph, checkpointer, *, store=None, mode="full")deerflow.runtime.checkpoint_statebind graph, saver, and modecstate:112-134
build_state_mutation_graph(as_node, mode, state_schema=None, *, snapshot_frequency=None)deerflow.runtime.checkpoint_statestate-only writer graph for Overwrite-style writescstate:39-65

In delta mode a raw saver read sees sentinels, not messages. Consumers must read state through CheckpointStateAccessor instead of calling the checkpointer directly; the accessor docstring says so explicitly.

Internal symbols

These are implementation details. Do not import them, and do not build on their names or shapes.

SymbolWhy it is not APIEvidence
_frozen_checkpoint_channel_mode, _frozen_checkpoint_snapshot_frequencymodule globals, only touched through the freeze/frozen helpers; tests monkeypatch themcmode:30-31, backend/tests/conftest.py:96-111
_delta_thread_state_schema@cache-decorated internal builder; generates DeltaThreadState_f{freq}tstate:422-434
_adapt_state_schema_for_delta@cache-decorated internal adapter; generates Delta{module_underscored}_{Name}_f{freq}tstate:443-451
_resolve_snapshot_frequencyinternal lazy-import resolvertstate:25-33
_STATE_ACCESSOR_GRAPH_CACHE_MAXin-code fallback constant 64, used only when the configured value is unusableservices:1127,1133-1138
_RawCheckpointReadAccessor, _RawCheckpointSnapshotdegraded full-mode read-only fallbackservices:1199-1263
_checkpoint_mode_http_error, _CHECKPOINT_MODE_ERRORSrouter-internal HTTP mappingthreads:82,96-107
_wrap_sync_if_deltasync provider internalsyncprov:164-192
_migrate_legacy_snapshot_frequencypydantic mode="before" validator, labelled -- Legacy key migration (not user-configured) --dbcfg:216-254
_finish_state_mutationno-op writer nodecstate:32-33

Load-bearing tests

Tests a reader can run or cite as the executable contract. All paths are under backend/tests/.

Mode gate, freezing, and configuration

  • test_checkpoint_mode.py (freeze, detection, gate) — the module named as the mode gate in backend/packages/harness/deerflow/runtime/AGENTS.md:133
  • test_checkpoint_mode.py::test_yaml_mode_change_is_rejected_when_graph_is_reconstructed
  • test_checkpoint_mode.py::test_delta_mode_accepts_plain_full_checkpoint
  • test_checkpoint_mode.py::test_gateway_runtime_rejects_mode_different_from_frozen_process
  • test_threads_checkpoint_mode.py (router-level mode behaviour)
  • test_threads_checkpoint_mode.py::test_delta_snapshot_frequency_controls_snapshot_cadence
  • test_gateway_checkpoint_mode.py (dual-mode end-to-end parity)
  • test_app_config_reload.py::test_checkpoint_channel_mode_rejects_unknown_value
  • test_config_version.py (config-version migration)

Materialization, storage shape, and migration

  • test_delta_channel_checkpointers.py::test_full_to_delta_migration_replays_on_same_thread
  • test_delta_channel_checkpointers.py::test_materialization_is_deterministic_and_message_ids_are_stable
  • test_delta_channel_checkpointers.py::test_delta_storage_shape_and_snapshot_cadence
  • test_delta_channel_checkpointers.py::test_non_delta_writers_preserve_delta_messages_and_markers
  • test_delta_channel_checkpointers.py::test_long_chain_history_survives_pagination
  • test_delta_channel_checkpointers.py::test_parallel_superstep_replay_matches_live_write_order
  • test_delta_channel_state.py::test_mode_selects_expected_state_schema
  • test_delta_channel_state.py::test_compiled_delta_graph_bakes_configured_snapshot_frequency
  • test_delta_channel_state.py::test_production_message_forms_keep_assigned_ids_across_delta_replay
  • test_delta_channel_state.py::test_delta_normalization_compiles_stable_channel_without_mutating_middleware

test_long_chain_history_survives_pagination and test_parallel_superstep_replay_matches_live_write_order assert the correct contract and trigger-skip while the pinned dependency still has the defect; they are the gates that flip live on a dependency bump.

History cache

  • test_cached_history_saver.py::test_composition_matches_full_walk_and_avoids_it
  • test_cached_history_saver.py::test_latest_config_caches_under_resolved_checkpoint_id
  • test_cached_history_saver.py::test_recursive_resolve_caches_intermediate_levels
  • test_cached_history_saver.py::test_deep_cold_chain_delegates_one_inner_walk_at_depth_limit
  • test_cached_history_saver.py::test_cold_resolve_never_delegates_to_inner_history
  • test_cached_history_saver.py::test_sync_cold_resolve_never_delegates_to_inner_history
  • test_cached_history_saver.py::test_snapshot_parent_composes_without_parent_history_lookup
  • test_cached_history_saver.py::test_root_checkpoint_history_is_empty_writes
  • test_cached_history_saver.py::test_eviction_falls_back_to_walk_but_stays_correct
  • test_cached_history_saver.py::test_sync_path_matches_async
  • test_cached_history_saver.py::test_stats_expose_composition_counters
  • test_cached_history_saver.py::test_adelete_thread_purges_only_that_threads_cache_entries
  • test_cached_history_saver.py::test_sync_delete_thread_purges_cache_entries
  • test_cached_history_saver.py::test_prune_purges_rewritten_threads_cache_entries
  • test_cached_history_saver.py::test_delete_for_runs_delegates_without_cache_purge
  • test_cached_history_saver_integration.py::test_sequential_run_composes_without_inner_walks
  • test_cached_history_saver_integration.py::test_cache_disabled_parity
  • test_cached_history_saver_integration.py::test_branch_divergence_no_cross_contamination
  • test_cached_history_saver_integration.py::test_interrupt_resume_appended_head_writes
  • test_cached_history_saver_integration.py::test_eviction_only_costs_performance
  • test_cached_history_saver_integration.py::test_rollback_supersede_does_not_pollute
  • test_cached_history_saver_integration.py::test_full_to_delta_migration_under_cache_states

Resume linearization and rollback

  • test_run_worker_delta_resume.py::test_linearizes_a_delta_resume_onto_the_head
  • test_run_worker_delta_resume.py::test_linearization_restores_all_selected_state_and_clears_newer_channels
  • test_run_worker_delta_resume.py::test_linearized_resume_preserves_selected_checkpoint_agent_binding
  • test_run_worker_delta_resume.py::test_regenerating_in_a_branched_thread_does_not_resurrect_the_old_answer
  • test_run_worker_delta_resume.py::test_run_agent_streams_from_the_linearized_delta_resume
  • test_run_worker_delta_resume.py::test_run_agent_serializes_resume_preparation_with_checkpoint_writes
  • test_run_worker_delta_resume.py::test_cancelled_delta_resume_rolls_back_to_pre_linearization_head
  • test_run_worker_delta_resume.py::test_full_mode_keeps_the_fork
  • test_run_worker_delta_resume.py::test_ordinary_run_without_a_checkpoint_selector_is_untouched
  • test_run_worker_delta_resume.py::test_selecting_the_head_is_already_linear
  • test_run_worker_delta_resume.py::test_unmaterializable_resume_state_fails_closed
  • test_run_worker_rollback.py::test_rollback_linearizes_delta_restore_onto_cancelled_head
  • test_run_worker_rollback.py::test_rollback_linearizes_delta_restore_sqlite_reopen
  • test_run_worker_rollback.py::test_rollback_restores_pre_run_pending_writes_for_delta_checkpoints
  • test_run_worker_rollback.py::test_rollback_preserves_middleware_contributed_channels[delta]
  • test_run_worker_rollback.py::test_rollback_preserves_agent_binding_for_manual_compaction[delta]
  • test_run_worker_rollback.py::test_run_agent_marks_rollback_unusable_when_capture_fails

Retention

  • test_checkpoint_retention_contract.py::test_growth_baseline_full_vs_delta
  • test_checkpoint_retention_contract.py::test_deleting_branch_ancestor_breaks_lineage_loudly
  • test_checkpoint_retention_contract.py::test_deleting_explicit_resume_target_breaks_resume
  • test_checkpoint_retention_contract.py::test_pending_writes_are_retained_state_not_garbage
  • test_checkpoint_retention_contract.py::test_leaf_duration_checkpoint_deletion_is_safe
  • test_checkpoint_retention_contract.py::test_leaf_sibling_branch_deletion_is_safe

Patches

  • test_checkpoint_patches.py::test_binop_overwrite_patch_is_active
  • test_checkpoint_patches.py::test_binop_overwrite_patch_stands_down_when_upstream_fixed

Pinned upstream defects

State at the pinned set (langgraph 1.2.9, langgraph-checkpoint 4.2.0, langgraph-checkpoint-postgres 3.1.2). Each local test asserts the correct contract and trigger-skips while the defect is present, so a later dependency bump flips it into a live gate; the buggy behaviour is never asserted as correct.

idKind / statusScenario that breaksLocal handlingPinned versionEvidence
#4380DeerFlow issue; patched locallyA replace-style write into an empty Union-typed BinaryOperatorAggregate channel persists the Overwrite wrapper; the next read crashes with TypeError: 'Overwrite' object is not subscriptableProbe-guarded patch in checkpoint_patches.py; patch tests; branch/state-update anchors in test_threads_router.pylanggraph 1.2.9checkpoint_patches:91-107, backend/tests/test_checkpoint_patches.py:31-95
#8526 (fixes upstream #8384), plus postgres #8535upstream langgraph PR; merged 2026-08-07; fixed in langgraph-checkpoint 4.2.0After switching a thread from BinaryOperatorAggregate to DeltaChannel, the first post-migration write survives the live invoke return value but disappears from every subsequent read; on postgres, with no seed the walk runs to the root and replays everythingLocal InMemorySaver patch deleted; dependency floor langgraph-checkpoint>=4.2.0,<5.0; regression gate test_delta_channel_checkpointers.py::test_full_to_delta_migration_replays_on_same_thread; a version-guarded saver patch must not be re-addedlanggraph-checkpoint 4.2.0checkpoint_patches:11-17, backend/packages/harness/pyproject.toml:49-56, CHANGELOG.md:3040-3048
#8382upstream langgraph issue; open; fix PR #8544 unmerged, no released versionDelta replay order diverges from live execution order for parallel-superstep writes: live invoke returns path-sorted [a..h], get_state replays [f,b,g,h,a,c,d,e] (a pure permutation); reproduced on memory, sqlite, and postgresTrigger-skip naming the issue; test_delta_channel_checkpointers.py::test_parallel_superstep_replay_matches_live_write_orderlanggraph-checkpoint 4.2.0test_delta_channel_checkpointers.py:419,440-460
#8448upstream langgraph issue; open; fix PR #8556 unmergedPostgresSaver.get_delta_channel_history permanently poisons the walk cursor when the target checkpoint is not in the first pagination page, so old checkpoints hydrate empty; only Postgres pages its stage-1 scanAssertion stays live on memory/sqlite (differential against an InMemorySaver oracle); Postgres trigger-skips; test_delta_channel_checkpointers.py::test_long_chain_history_survives_paginationlanggraph-checkpoint-postgres 3.1.2test_delta_channel_checkpointers.py:459
#8548upstream langgraph PR; open; fixes upstream #8443, with related issue #8551 still waiting on itDoes not replay an abandoned branch into a DeltaChannel forkWhile unmerged the local linearization cannot be removed: _linearize_delta_checkpoint_resume is retainedlanggraph 1.2.9worker:2267-2279, test_run_worker_delta_resume.py::test_linearizes_a_delta_resume_onto_the_head
#4458DeerFlow issue; the user-visible symptom of the fork bugRegenerating in a branched thread resurrected the old assistant message beside the new one after a reload; reproduced on postgres, sqlite, and the in-memory saver, and it also broke branch history when seeded rows shared one run idLinearization plus per-turn seeded run ids; test_run_worker_delta_resume.py::test_regenerating_in_a_branched_thread_does_not_resurrect_the_old_answerlanggraph 1.2.9worker:2267-2289, test_run_worker_delta_resume.py:204
#4189 item 3DeerFlow design discussionRetention and deletion of checkpoint rows; what may be deletedbackend/docs/checkpoint-retention-contract.md, backend/tests/test_checkpoint_retention_contract.py, backend/app/gateway/checkpoint_retention.py; no production trigger is wired—checkpoint_retention.py:1-40, backend/docs/checkpoint-retention-contract.md:31-77
#3859 (review)DeerFlow review referenceAn interrupted-title write must bump channel_versions["title"] and declare it in new_versions, or database savers drop the blobbackend/tests/test_run_worker_rollback.py:3501-3504 pins the shape—backend/tests/test_run_worker_rollback.py:3501-3504

See Operating Checkpoints for what the remaining patch does and how the dependency floor is enforced, and Troubleshooting for the symptoms each defect produces for a user.

Change log (June to September 2026)

Pull requests whose changelog entries concern dual-mode checkpoint storage. English entries under ## [Unreleased] accumulate toward milestone 2.1.0; the ## [2.0.0] entries are from the 2026-06-15 release. The section column names the changelog section each entry sits under.

PRChangelog sectionChange
#4516 ### ⚠ Breaking changesThe config key moved from database.checkpoint_delta_snapshot_frequency to database.checkpoint_delta.snapshot_frequency, and the default changed from 1000 to 10. The legacy top-level value is still honored with a deprecation warning and an explicit nested key wins. Threads now snapshot 100x more often in delta mode; set …snapshot_frequency: 1000 explicitly to keep the previous cadence.
#4292 ### AddedDual-mode checkpoint storage with LangGraph DeltaChannel cuts thread storage from O(N²) to near-linear for long research and coding runs.
#4638 ### AddedDelta-mode checkpoint history cache (memory/redis) with O(1) incremental composition, configured via database.checkpoint_cache.
#4437 , #4460 , #4468 ### FixedSerialize checkpoint writes with active runs, linearize delta-mode checkpoint resume, and accept the SDK’s default stream_resumable=false to avoid resume races.
#4383 ### Fixedcheckpoint: unwrap Overwrite first writes into empty channels.
#5255 ### FixedCheckpoint-retention contract suite: per-step growth baselines for both full and delta schemas, deleting a branch ancestor fails loudly with CheckpointLineageError, and pending writes are treated as retained state.
#4140 ### Securitystorage: stop persisting base64 image data in checkpoint state. Adjacent rather than mode-specific.
#5734 ### InternalRaise langgraph-checkpoint to >=4.2.0,<5.0 and langgraph-checkpoint-postgres to >=3.1.2,<3.2, and drop the InMemorySaver delta-history patch: upstream 4.2.0 fixes the dropped first write after full→delta (langgraph#8526) and postgres locates plain-value seeds (#8535). The migration contract test remains the gate.
#4395 ### Internalbench: add an isolated checkpoint channel-mode benchmark comparing full and delta across latency, storage, and replay metrics.
#5051 ### Internalbench: measure Postgres checkpoint, blob, and write storage growth in the checkpoint benchmark alongside memory and SQLite.
#2582 ## [2.0.0] → ### Fixed → #### Runtime, gateway & persistenceRollback restore checkpoint now supersedes newer checkpoints.
#3559 ## [2.0.0] → ### Fixedsubagent: isolate subagents from the parent run’s checkpointer.

No entry in this list records a change to the mode default, which has been full throughout.

The #4292 row quotes the released changelog entry verbatim. Its “near-linear” wording describes the per-step representation; retained storage still adds up to O(N²/K + N) with K = snapshot_frequency, because every periodic snapshot holds the whole growing list — see Checkpoints and Delta Channels.