Projection-cache predecessor recovery and Session-format binding (session_projcache v3-v6 → v7)
投影缓存前代恢复与 Session 格式绑定(session_projcache v3-v6 → v7)
The `session_projcache` storage domain evolved through several on-disk generations. An upgraded DSH_HOME exposed three risks: - **A v3 single-file home bricked startup after the upgrade**: the per-record layout's legacy bootstrap migrated the old whole-unit file without checking its `unit.version`, stamping the old records with the current version into the new tree; the domain layer's per-record zod validation at ope
English
Problem
The session_projcache storage domain evolved through several on-disk generations. An upgraded DSH_HOME exposed three risks:
- A v3 single-file home bricked startup after the upgrade: the per-record layout's legacy bootstrap migrated the old whole-unit file without checking its
unit.version, stamping the old records with the current version into the new tree; the domain layer's per-record zod validation at open then hit the missing now-required fields →invalid-record→ the whole domain refused to open → the plugin tree failed to load. And because the bootstrap writes before validation runs, the first boot permanently wrote the bad documents into the new tree ("poisoning") — every later boot saw a non-empty tree, never took the legacy path again, and the home stayed unusable. - A v4 per-record home lost its listing titles after the upgrade: v4 documents were silently discarded by the version-stamp check (the per-record contract), and SessionList is a zero-I/O cache-only read, so a miss served the row without projections; titles only returned as each session was individually reopened.
- A Session-format bump could reuse a fold produced under older event semantics: versions 3 through 6 did not record the Session format generation. Treating an absent generation as current would let a cache row bypass bounded historical normalization or a cardinality-changing migration.
The cache domain's own contract is "a stale or unreadable cache costs a longer tail replay, never a wrong value, never a refused load" — the hard failure and the wholesale discard each broke the first half of that contract or the product expectation.
The on-disk generations
| domain version | shipped in | layout | on-disk form | identity fields | row fields |
|---|---|---|---|---|---|
| 3 | 0.1.1-rc.2 | single | one file storages/session_projcache.json ({unit:{name,version}, global, tables}) | createdAt, cwd? | ver, seq, val |
| 4 | 0.1.2-alpha.3 | per-record | one file per session storages/session_projcache/sessions/<sessionId>.json ({version, record}) | createdAt, cwd? | same |
| 5 | 0.1.2-alpha.4 | per-record | same as v4 | + isSeeded (shipped required; now optional), inheritedEventCount (same) | same (seq numbers mean the same as v4; only type brands were added) |
| 6 | pre-v1 mainline | per-record | same as v5 | same as v5 | same |
| 7 | current | per-record | same as v5 | + formatVersion; current writes also require both lineage fields | same |
The only substantive v4→v5 difference is the two lineage identity fields; v6 changed only the write stamp. The ver/seq/val row representation is identical across these predecessor generations, and seq numbering did not change (the 2026-08-31 seq/offset brands note pins the on-disk numbers as unchanged). v3→v4 was a layout migration with identical record content. Version 7 adds the Session format generation to the cache identity because row semantics cannot be inferred from the domain stamp.
One derived shape also exists: a v3 home that ran the v5 build once (the poisoned state) — its new tree holds documents stamped 5 whose content is a v3 record (no lineage fields).
Decision
Declared read compatibility — reads tolerate vouched-for older versions, writes always stamp the current one:
-
DomainSpec.compatibleVersions(new, optional): the domain owner declares "records stored under these older versions are also readable under the current record schemas" (typically by declaring the fields old records lack as optional).defineDomainvalidates each entry as a non-negative integer below the current version;descriptorOfprojects the set onto the backendKvUnitDescriptor. -
json backend per-record reads accept version stamps in "current ∪ compatibleVersions"; anything outside the set is still discarded as foreign. The write path always stamps the current version (the first checkpoint after reading an old record naturally advances it). The
singlelayout stays exact-version. -
Legacy-bootstrap version gate (the actual bug fix): the old whole-unit file's
unit.versionmust fall inside the accepted set to be migrated; otherwise the file is left alone and the unit reads empty — stamping records the owner never vouched for turns a discardable stale cache into hard schema failures at the domain layer. -
The projcache domain declares
version: 7, compatibleVersions: [3, 4, 5, 6], and the format and lineage identity fields are optional in the stored schema so vouched-for predecessor records can open. Current writes always include all three fields. -
Identity matching is stricter than structural admission: an absent
formatVersionnever matches a current Session, so predecessor rows cannot seed a projection and refold from the authoritative log. Once the format matches,identityMatchesnormalizes absent lineage to unseeded (?? false/?? 0): exact for an unseeded session, while a seeded expectation fails the match. Poisoned v5 homes therefore boot safely, but their unbound rows are not exposed as current values. -
Schema-validation backstop:
invalidRecords: 'backup-and-skip'(declared by this domain only). A stored record that still fails to parse beyond read compatibility no longer refuses the whole domain: the domain layer calls the backend'sKvUnit.backupRecord(json per-record implementation = rename the document to<key>.json.bak.<YYYYMMDDHHmm>, bytes kept, never read again), prints the concrete failure withlogger.error(domain, table, key, destination, zod cause), and continues the open with the record absent; the next cold read rebuilds and rewrites that session's cache. The policy is an explicit per-domain declaration and the default stays fail-loud — other domains still refuse the whole load on invalid stored data, and a backend withoutbackupRecord(single layout, row stores) also falls back to fail-loud. Naming history: quarantine → backup-and-skip (user ruling: the word must carry both "back up" and "skip", sharing its root with the.baksuffix; skip-backup was rejected because the CLI--skip-Xconvention reads it as "do not back up"). For this domain it supersedes the reset/destroy recovery path of the 2026-07-28 storage recovery proposal, which stays live for authoritative and whole-medium damage. -
A predecessor title is a listing hint, not a fold shortcut: Session-list startup remains metadata/cache-only and never opens cold log bodies. The log header is authoritative; a lifecycle-matching checkpoint is a durable-prefix witness that may lag but cannot lead the log.
cachedPredecessorTitletherefore exposes only a predecessortitlerow that still passes the current title unit'sstateVersionand schema. Both adjacent Session-format edges preserve title text. The hint carries the stored title row's own watermark asasOfSeq; because a cardinality-changing log migration can remap that coordinate, the Session list labels the blockkind: 'cached'and the client never compares it with the connected Session's seqs (listing identity and cached rows). Other predecessor rows remain hidden, andhydratePrepared/coldSnapshotretain strict format identity, because normalizers can change values such asblankorlastPromptAteven when storage is physically consistent.
v3-v6 → v7 disposition
Versions 3 through 6 remain structurally readable because their record and row representations are valid inputs to the current schema. Their identities lack formatVersion, so they are deliberately unusable as current fold shortcuts. An unseeded, lifecycle-matching record may still provide its version-compatible title to zero-I/O listing; it cannot provide any authoritative seed. A cold read or live checkpoint rebuilds the values from the migrated Session log and writes a v7 record with the complete format and lineage identity. No eager value migration runs at startup; a schema-invalid accepted record follows backup-and-skip.
Upgrade matrix
| home shape | behavior after the fix |
|---|---|
| v3 single-file (not poisoned) | bootstrap migrates (3 ∈ accepted set) → boot succeeds; compatible title is list-visible, unbound fold waits for cold rebuild |
| v3 + poisoned new tree | new-tree documents parse under optional fields → boot restored; compatible title is list-visible, unbound fold waits for cold rebuild |
| v4/v5/v6 per-record | documents read structurally → missing format generation rejects the fold shortcut; compatible title is list-visible; current checkpoint rewrites v7 |
| v7 current with matching identity | cached values serve normally |
| matching-format record without lineage | unseeded caller may use it; seeded caller rejects it and refolds cold |
Alternatives considered
- Reject predecessor stamps in the storage layer: safe for projections, but prevents the guarded legacy bootstrap and loses the ability to retain a structurally sound record until an authoritative refold replaces it. Structural admission plus semantic identity rejection keeps the boot recoverable without serving an unproven value.
- Treat a missing format generation as current for every use: preserves cached values, but lets a pre-migration fold bypass the Session-format edge. Rejected because v0→v1 includes bounded historical normalizers and later edges may change event cardinality. The title-only listing hint is narrower: it does not seed a fold, and title text is invariant across the installed edges.
- Schema
.default()fills: behaviorally equivalent to optional + reader normalization, but bakes the "absent = unseeded" interpretation into the durable schema's output type; ruled for optional — the schema honestly describes every accepted on-disk shape and the interpretation lives at the consumer (user ruling, 2026-09-02). - Roll the domain version back to 4: a small diff, but breaks version monotonicity, depends on the "bootstrap skips no versions" bug itself, and drops every poisoned and healthy v5 home's cache.
Consequences
- A deployment routing this domain to the sqlite backend gets none of the tolerance: sqlite implements neither
compatibleVersionsnorbackupRecord, so behavior degrades to the old strict-version semantics (a whole-unit version mismatch still refuses withversion-mismatch; nothing loosens, nothing serves wrong values). Shipped compositions route this domain to json, so this stays a deployment-configuration risk only. - The optional format field lets predecessor records pass structural validation, but absence always fails current identity matching. Optional lineage is normalized only after the format matches; a seeded caller still refuses a lineage-less record. The per-row
verguard continues to screen every served value. backupRecordoverwrites a same-minute backup of the same key (the newer bytes win); distinct minutes and distinct keys never collide.
Testing
storage-jsonunit tests: compat-stamped reads / out-of-set discards / writes stamping current; legacy bootstrap migrating only accepted versions (including the migrated-documents-stamp-current assertion);backupRecordmove / absent read / rewrite / closed guard.storage-domainunit tests:compatibleVersions/invalidRecordsdeclaration validation; backup-and-skip falling back to fail-loud when the backend has nobackupRecord.session-projection-cacheunit tests: a matching format with absent lineage serves only an unseeded Session; a predecessor record without a format generation cannot seed a fold but may serve only its compatible title hint.- Archived-fixture recovery tests (
tests/fixtures.spec.ts+tests/fixtures/): four media archives produced by the real released builds —v3-single-unit.json(the 0.1.1-rc.2 whole-unit file),v4-session-doc.json(0.1.2-alpha.3),v5-session-doc.json(0.1.2-alpha.4),v5-lineageless-doc.json(the unguarded bootstrap's poisoned form, synthesized from the v3 record) — each opens through the real storage stack, serves only its compatible predecessor title, never serves its unbound fold, then accepts a live write that replaces it with a v7 record carrying the complete identity and fresh value. The same suite proves backup-and-skip for a schema-failing record: boot survives,.baklands, diagnostics name the failure, and a neighboring predecessor record remains rewritable.
Future bump procedure: add an older domain version to compatibleVersions only when the current stored schema can parse it, and let the owning reader decide whether its semantic identity is sufficient. A Session-format change never inherits an absent format generation. The package README requires every bump to land with archived fixtures and tests proving structural admission, semantic use or rejection, and current rewrite.
中文
问题
session_projcache 存储域演进过多代磁盘结构。升级后的 DSH_HOME 暴露了三类风险:
- v3 单文件 home 升级后启动硬失败:per-record 布局的 legacy bootstrap 迁移旧单文件时不检查其
unit.version,把旧记录原样打上当前版本戳写入新树;domain 层开域时逐条 zod 校验,旧记录缺新增必填字段 →invalid-record→ 整个域拒开 → 插件树加载失败。且 bootstrap 先写盘后校验,首次启动即把坏文档永久写入新树("投毒")——此后每次启动新树非空、连 legacy 路径都不再走,home 持续不可用。 - v4 per-record home 升级后列表丢标题:v4 文档被版本戳检查静默丢弃(per-record 契约),SessionList 是零 I/O 纯缓存读,miss 后整行不带投影;标题要等每个会话被逐个重新打开后才恢复。
- Session 格式递增可能复用按旧事件语义折叠的结果:版本 3 至 6 不记录 Session 格式代。若把缺失代视为当前代,缓存行就能绕过有界历史规范化或改变基数的迁移。
缓存域自身的契约是"过期或不可读的缓存只付出更长的尾部重放,绝不给出错值、绝不拒载"——硬失败与整体丢弃都违背该契约的前半句或产品预期。
磁盘结构代际
| domain version | 携带发布 | 布局 | 磁盘形态 | identity 字段 | 行字段 |
|---|---|---|---|---|---|
| 3 | 0.1.1-rc.2 | single | 单文件 storages/session_projcache.json({unit:{name,version}, global, tables}) | createdAt, cwd? | ver, seq, val |
| 4 | 0.1.2-alpha.3 | per-record | 每会话一份 storages/session_projcache/sessions/<sessionId>.json({version, record}) | createdAt, cwd? | 同上 |
| 5 | 0.1.2-alpha.4 | per-record | 同 v4 | + isSeeded(v5 首发必填;现为 optional)、inheritedEventCount(同前) | 同上(seq 数值语义与 v4 相同,仅类型加 brand) |
| 6 | v1 之前的 mainline | per-record | 同 v5 | 同 v5 | 同上 |
| 7 | 当前版本 | per-record | 同 v5 | + formatVersion;当前写入也要求两个 lineage 字段 | 同上 |
v4→v5 的唯一实质差异是 identity 新增两个 lineage 字段;v6 只改变写入版本戳。这些前代的行内 ver/seq/val 表示一致,seq 的数值含义未变(2026-08-31 seq/offset brands note 明确 on-disk 数值不变)。v3→v4 是布局迁移,记录内容结构一致。v7 把 Session 格式代加入缓存 identity,因为无法从 domain 版本戳推导行语义。
另有一种衍生形态:跑过一次 v5 版本的 v3 home(投毒态)——新树里存在版本戳为 5 但内容是 v3 记录(缺 lineage 字段)的文档。
决策
声明式读兼容——读容忍 owner 背书过的旧版本,写恒戳当前版本:
-
DomainSpec.compatibleVersions(新增,可选):域 owner 声明"这些旧版本的存量记录在当前记录 schema 下也可读"(典型手段:新增字段标 optional)。defineDomain校验各项为小于当前 version 的非负整数;descriptorOf透传到后端KvUnitDescriptor。 -
json 后端 per-record 读:接受"当前版本 ∪ compatibleVersions"内的版本戳,集合外照旧视为 foreign 丢弃;写路径永远戳当前版本(读到旧记录后的下一次 checkpoint 自然把它推进到当前版本)。single 布局维持 exact-version 不变。
-
legacy bootstrap 版本把关(bug 修复本体):旧单文件的
unit.version必须落在接受集合内才迁移,否则视为空 unit 留在原地——为 owner 未背书的记录打当前版本戳,会把"可丢弃的过期缓存"变成 domain 层的 schema 硬失败。 -
projcache 域声明
version: 7, compatibleVersions: [3, 4, 5, 6];存储 schema 中的格式与 lineage identity 字段均为 optional,使 owner 背书的前代记录可以打开。当前写入始终包含这三个字段。 -
identity 匹配比结构准入更严格:缺失
formatVersion的记录绝不匹配当前 Session,因此前代行不能播种投影,而会从权威日志重新折叠。格式匹配后,identityMatches才把缺失 lineage 归一化为 unseeded(?? false/?? 0):对 unseeded 会话精确,对 seeded 期望则匹配失败。v5 投毒 home 因而可以安全启动,但其未绑定行不会作为当前值暴露。 -
schema 校验兜底:
invalidRecords: 'backup-and-skip'(仅本域声明)。读兼容之外仍然解析失败的存量记录不再让整个域拒开:domain 层调用后端的KvUnit.backupRecord(json per-record 实现=把文档改名为<key>.json.bak.<YYYYMMDDHHmm>,字节留档、不再被读取),用logger.error打印具体失败信息(域名、表、键、移动去向、zod 失败原因),随后当该记录不存在继续启动;下一次冷读会重建并重写该会话的缓存。该策略是域级显式声明,缺省仍为 fail-loud——其他业务域的存量数据校验失败照旧整域拒载;后端没有backupRecord能力(single 布局、行存储)时也回退 fail-loud。命名沿革:quarantine → backup-and-skip(用户裁决:词要同时含"备份"与"跳过"两义,且与.bak后缀同源;skip-backup 因 CLI--skip-X惯例存在"不备份"反读而弃用)。对本域而言,该策略取代了 2026-07-28 存储恢复提案中 reset/destroy 的恢复途径;该提案对权威介质与整介质损坏仍然有效。 -
predecessor title 是列表 hint,而不是 fold shortcut:Session list 启动保持 metadata/cache-only,绝不打开冷 log body。log header 是权威来源;生命周期匹配的 checkpoint 是 durable prefix witness,可以落后但不能领先日志。因此
cachedPredecessorTitle只公开仍通过当前 title unitstateVersion与 schema 的 predecessortitlerow。两条相邻 Session format edge 都保留 title 文本。该 hint 的asOfSeq是存储 title row 自己的水位;因为改变日志事件数量的迁移会重新映射该坐标,Session list 把该 block 标为kind: 'cached',客户端不会拿它与建连 Session 的 seq 比较(只读面按 lifecycle 身份匹配与 cached 行)。其他 predecessor row 继续隐藏,hydratePrepared/coldSnapshot仍要求严格格式 identity,因为即使物理存储一致,normalizer 仍可能改变blank或lastPromptAt等值。
v3-v6 → v7 处置
版本 3 至 6 仍可结构化读取,因为它们的记录与行表示是当前 schema 的有效输入。其 identity 缺少 formatVersion,因此有意不能作为当前折叠捷径。unseeded 且生命周期匹配的记录仍可向零 I/O 列表提供版本兼容的 title,但不能提供任何权威 seed。冷读或实时检查点会从迁移后的 Session 日志重建值,并写入带完整格式与 lineage identity 的 v7 记录。启动时不运行 eager 值迁移;schema 校验失败的已接受记录执行 backup-and-skip。
升级矩阵
| home 形态 | 修复后行为 |
|---|---|
| v3 单文件(未投毒) | bootstrap 迁移(3 ∈ 接受集)→ 启动成功;兼容 title 在列表可见,未绑定折叠等待冷重建 |
| v3 + 投毒新树 | optional 字段让新树文档可解析 → 启动恢复;兼容 title 在列表可见,未绑定折叠等待冷重建 |
| v4/v5/v6 per-record | 文档结构化读入 → 缺格式代而拒绝 fold shortcut;兼容 title 在列表可见;当前检查点重写 v7 |
| identity 匹配的 v7 当前记录 | 正常服务缓存值 |
| 格式匹配但缺 lineage 的记录 | unseeded 调用方可以使用;seeded 调用方拒绝并回落冷折叠 |
备选方案
- 在存储层拒绝前代版本戳:对投影安全,但会阻止受保护的 legacy bootstrap,也无法保留结构完好的记录直到权威重折叠替换它。结构准入加语义 identity 拒绝既让启动可恢复,也不服务未经证明的值。
- 在所有用途中把缺失格式代视为当前代:可以保留缓存值,但让迁移前折叠绕过 Session 格式边。不采用,因为 v0→v1 含有界历史规范化,后续边还可能改变事件基数。title-only 列表 hint 更窄:它不播种 fold,而且 title 文本在已安装 edge 之间保持不变。
- schema
.default()填缺省:行为与 optional+读点归一化等价,但把"缺失=unseeded"的解释固化进 durable schema 的输出类型;拍板为 optional——schema 如实描述介质上所有被接受的形态,解释权在消费点(2026-09-02 用户裁决)。 - 域版本回退到 4:改动很小,但破坏版本单调性、依赖"bootstrap 不查版本"这个 bug 本身、且投毒态与正常 v5 home 的缓存全被丢弃。
影响
- 部署方若把本域路由到 sqlite 后端,得不到任何容忍能力:sqlite 既未实现
compatibleVersions也没有backupRecord,行为退化为原有的严格版本语义(整 unit 版本不匹配仍version-mismatch拒开;不放松、不出错值)。shipped 组合固定路由 json,此风险仅存在于部署配置层面。 - optional 格式字段允许前代记录通过结构校验,但缺失格式始终无法通过当前 identity 匹配。只有格式匹配后才归一化 optional lineage;seeded 调用方仍拒绝缺 lineage 的记录。逐行
ver守卫继续筛查每个实际服务的值。 backupRecord对同一键的同一分钟内重复备份会覆盖前一份(新字节胜出);不同分钟、不同键永不冲突。
测试
storage-json单测:compat 版本戳读入/集合外丢弃/写恒当前版本;legacy bootstrap 仅在版本被接受时迁移(含迁移后文档戳当前版本断言);backupRecord移档/读缺席/重写/封闭守卫。storage-domain单测:compatibleVersions/invalidRecords声明校验;后端无backupRecord时 backup-and-skip 回退 fail-loud。session-projection-cache单测:格式匹配且缺 lineage 的记录只服务 unseeded Session;缺格式代的前代记录不能播种 fold,但只能提供其兼容 title hint。- 归档 fixtures 独立恢复测试(
tests/fixtures.spec.ts+tests/fixtures/):真实发布物产出的四份介质存档——v3-single-unit.json(0.1.1-rc.2 整域单文件)、v4-session-doc.json(0.1.2-alpha.3)、v5-session-doc.json(0.1.2-alpha.4)、v5-lineageless-doc.json(无守卫 bootstrap 的投毒形态,由 v3 记录合成)——逐一走真实存储栈开域、只服务其兼容 predecessor title、绝不服务其未绑定折叠,随后由实时写入替换成带完整 identity 与新值的 v7 记录。同一套件还证明 schema 失败记录的 backup-and-skip:启动不失败、.bak落盘、诊断点名失败,且邻近前代记录仍可重写。
未来 bump 流程:只有当前存储 schema 能解析旧 domain 版本时,才把它加入 compatibleVersions,再由 owner reader 判断其语义 identity 是否充分。Session 格式变化绝不继承缺失的格式代。包 README 要求每次 bump 都随附归档 fixture 与测试,证明结构准入、语义使用或拒绝,以及当前重写。