DSH / Atlas
dsh-atlas repoTracking master @ b150a551b8
2026-06-16proposedarchitecture

Runtime schemas for the event vocabulary (Zod vs the merge-extensible-map pattern)

事件词汇的运行时 schema（Zod 与 merge-extensible-map 模式之辩）

The harness models its core vocabulary — content blocks, message sources, finish reasons, turn triggers, turn-end reasons, and session events — as **merge-extensible maps**: a TypeScript `interface` (e.g. `SessionEventMap`, `ContentBlockMap`) that plugins augment via declaration merging, with the public union derived as `Map[keyof Map]`. This is the repo's universal extension pattern, documented in [docs/architecture

.agents/notes/proposed/architecture/2026-06-16-typed-event-schemas.md

English

Problem

The harness models its core vocabulary — content blocks, message sources, finish reasons, turn triggers, turn-end reasons, and session events — as merge-extensible maps: a TypeScript interface (e.g. SessionEventMap, ContentBlockMap) that plugins augment via declaration merging, with the public union derived as Map[keyof Map]. This is the repo's universal extension pattern, documented in docs/architecture.md ("The same merge-extensible-map pattern is used for MessageSource, FinishReason, TurnTrigger, and TurnEndReason") and relied on by the defineTool InferArgs DSL and the assertNever exhaustiveness convention.

The pattern is compile-time only. The types vanish at runtime: there is no schema object to validate an incoming value against, parse untrusted input with, or enumerate at runtime. The session-persistence contract exposes two consequences:

  1. Persistence treats event.data as opaque JSON. The JSONL/SQLite backends JSON.stringify/JSON.parse each event verbatim; the only runtime guard is isJsonValue (round-trip serializability — rejects BigInt, functions, cycles, non-finite numbers, …), NOT structural validation. A corrupted-but-still-JSON event datum (wrong field types, missing fields) round-trips silently and is only caught later, if at all, by a consumer's switch.
  2. No runtime contract for plugin-added variants. A plugin that declaration-merges a new SessionEventMap key gets compile-time typing for its own code, but nothing validates that the values it produces match the shape it declared — at the producer, at the persistence boundary, or on reload.

This raises whether the event vocabulary should move to Zod or another runtime-schema library so durable and plugin boundaries have runtime schemas rather than erased types.

Why this is not a persistence change

It is tempting to read "use Zod for serialization" as a local change to dsh-session-persistence-jsonl/src/format.ts. It is not, for one structural reason: a plugin cannot declaration-merge a Zod schema. Declaration merging is a TypeScript compile-time mechanism; a Zod schema is a runtime value. To validate events with Zod you need a runtime registry that every event-producing package contributes its schema to (e.g. ctx.sessionEvents.register('compaction/marker', z.object({…}))), and every consumer reads from. That registry — not the persistence backend — becomes the source of truth for the vocabulary, replacing the merge-extensible interface.

So the real proposal is: replace the compile-time merge-extensible-map pattern with a runtime schema registry, repo-wide. That is a core-vocabulary redesign.

Blast radius (measured)

A migration of the event/vocabulary API to runtime schemas touches, at minimum:

  • Six merge-extensible maps (~370 LOC of core types): ContentBlockMap, MessageSourceMap, FinishReasonMap (in dsh-llm); TurnTriggerMap, TurnEndReasonMap, SessionEventMap (in dsh-session).
  • ~10 declare module augmentation sites across dsh-agent, dsh-agent-loop, dsh-shell, dsh-llm, dsh-session, dsh-session-persistence, dsh-system-prompt, dsh-tools — each would move from declaration merging to a runtime register() call.
  • The event producers — 16 session.append(...) call sites in the loop — unchanged in shape but now validated at the boundary.
  • ~7 switch-consumers that branch on these unions: deriveMessages and the package-owned invariant companion (dsh-session), BlockAssembler (dsh-llm), both LLM adapters (dsh-llm-deepseek, dsh-llm-pi-ai), and the tool schema layer (dsh-tools). The assertNever-on-closed-unions vs fall-through-on-extensible-unions convention (a documented lint rule) would need rethinking — runtime variants are not statically exhaustive.
  • The defineTool InferArgs DSL (dsh-tools), which derives zero-cast execute arg types from a compile-time schema spec — the showcase of the current approach.
  • Docs: architecture.md (the pattern is described as foundational), dev-mode invariants, and any Agent Note that references the pattern.

This is a repository-wide vocabulary redesign, not a persistence implementation detail.

Alternatives considered

A. Status quo — merge-extensible types + isJsonValue at the durable boundary

Keep the compile-time pattern. Persistence stays opaque-JSON + serializability guard. Plugins extend via declaration merging; correctness of event shape is the producer's responsibility and is enforced by TypeScript at compile time. Package-owned invariant companions check selected cross-record relationships when enabled but do not provide general runtime shape schemas.

  • Pros: zero churn; plugin extension is a one-line interface augmentation with full type inference and no runtime registration ceremony; no new runtime dependency; the defineTool DSL and assertNever exhaustiveness keep working.
  • Cons: no runtime structural validation at the persistence boundary or at plugin boundaries; a malformed-but-JSON datum is caught late.

B. Header/closed-shape validation only (schemastery), events stay opaque

Tighten only the genuinely-closed shapes that already have hand-rolled type guards — e.g. the JSONL HeaderLine guard (isHeaderLine) — using schemastery (the repo's existing schema library, already used for every plugin static Config). Leave the merge-extensible event union as-is.

  • Pros: small, fits the existing convention (schemastery, not a new lib); replaces hand-rolled guards on closed shapes with declarative schemas; no core redesign.
  • Cons: does not address event-data validation; only the fixed metadata records improve.

C. Runtime schema registry for the whole vocabulary (Zod or schemastery)

Replace the merge-extensible maps with a runtime registry the producers contribute to and the persistence/consumer paths validate against.

  • Pros: real runtime validation at the durable boundary and at plugin boundaries; one source of truth; enables generic tooling (auto-generated docs, fuzzing, wire-format checks).
  • Cons: the full blast radius above; Zod is not currently a direct dependency (only a transitive dep of @earendil-works/pi-ai) and the repo's chosen schema lib is schemastery — adopting Zod broadly is itself a dependency decision; declaration-merge ergonomics (one-line plugin extension, full inference) are replaced by runtime registration + manual type wiring; the assertNever exhaustiveness guarantee weakens (runtime variants aren't statically exhaustive).

Proposal

Defer. If runtime validation is wanted at the durable boundary, Option B (schemastery on closed header and metadata shapes) is the proportionate step within the existing convention. Option C is an architecture decision that requires its own implementation Agent Note, including a choice between Zod and schemastery.

Acceptance criteria

  • Option C proceeds only through its own implementation Agent Note, never as a persistence side effect.
  • If Option B is taken up, the closed header/metadata shapes (the JSONL isHeaderLine guard and kin) validate through schemastery in place of hand-rolled guards, with the merge-extensible maps untouched.

Risks

  • The deferral leaves event data structurally unvalidated at the durable boundary: a malformed-but-JSON datum is caught late, by a consumer's switch — the status-quo cost, accepted deliberately.
  • If Option C is ever adopted, the ergonomic loss is real: one-line declaration merging becomes runtime registration plus manual type wiring, and the assertNever static-exhaustiveness guarantee weakens.

Open questions

  • If a registry is adopted, is the library schemastery (already in the tree, already the config schema lib) or Zod (richer ecosystem, currently only transitive)? Adopting two schema libraries is a cost in itself.
  • Can a hybrid keep compile-time inference (so defineTool and plugin DX survive) while adding an optional runtime schema per variant, validated only at the persistence/wire boundary rather than on every in-process append?
  • Does the ctx.invariants service already cover enough of the runtime-shape gap when enabled that boundary validation is only needed for genuinely untrusted input (reload of an externally-modified log)?

中文

问题

harness 将其核心词汇——内容块、消息来源、结束原因、轮次触发器、轮次结束原因与会话事件——建模为 merge-extensible map：一个 TypeScript interface（如 SessionEventMap、ContentBlockMap），插件通过声明合并对其扩展，公开联合类型则以 Map[keyof Map] 派生。这是本仓库的通用扩展模式，记录在 docs/architecture.md 中（「The same merge-extensible-map pattern is used for MessageSource, FinishReason, TurnTrigger, and TurnEndReason」），defineTool 的 InferArgs DSL 和 assertNever 穷举约定都依赖于它。

该模式仅存在于编译期。类型在运行时消失：没有 schema 对象可供校验传入值、解析不可信输入或在运行时枚举变体。会话持久化约定暴露了两个后果：

  1. 持久化将 event.data 视为不透明 JSON。 JSONL/SQLite 后端对每个事件原样执行 JSON.stringify/JSON.parse；唯一的运行时守卫是 isJsonValue（往返可序列化性检查：拒绝 BigInt、函数、循环引用、非有限数等），而非结构校验。一个损坏但仍为合法 JSON 的事件数据（字段类型错误、字段缺失）会静默往返，只有在后续消费方的 switch 中才可能被捕获。
  2. 插件新增变体没有运行时约定。 一个通过声明合并添加新 SessionEventMap 键的插件，在自身代码中获得了编译期类型，但没有任何机制校验它产出的值是否符合它所声明的形状——无论是在生产者处、持久化边界处还是重新加载时。

由此引出问题：事件词汇是否应迁移到 Zod 或其他运行时 schema 库，使持久化边界和插件边界拥有运行时 schema 而非被擦除的类型。

为什么这不是一个持久化层的改动

很容易把「用 Zod 做序列化」理解为对 dsh-session-persistence-jsonl/src/format.ts 的局部修改。但它不是，原因在于一个结构性事实：插件无法对 Zod schema 进行声明合并。 声明合并是 TypeScript 编译期机制；Zod schema 是运行时值。要用 Zod 校验事件，就需要一个运行时注册表，每个产出事件的包向其贡献自己的 schema（如 ctx.sessionEvents.register('compaction/marker', z.object({…}))），每个消费方从中读取。这个注册表——而非持久化后端——将成为词汇的真源，取代 merge-extensible 接口。

因此，真正的提案是：用运行时 schema 注册表替换编译期的 merge-extensible-map 模式，范围覆盖整个仓库。 这是一次核心词汇的重新设计。

影响范围（已度量）

将事件/词汇接口迁移到运行时 schema，至少涉及：

  • 六个 merge-extensible map（约 370 行核心类型）：ContentBlockMap、MessageSourceMap、FinishReasonMap（位于 dsh-llm）；TurnTriggerMap、TurnEndReasonMap、SessionEventMap（位于 dsh-session）。
  • 约 10 处 declare module 声明增补位置，分布在 dsh-agent、dsh-agent-loop、dsh-shell、dsh-llm、dsh-session、dsh-session-persistence、dsh-system-prompt、dsh-tools 各包中——每处都将从声明合并改为运行时 register() 调用。
  • 事件生产者——agent loop（智能体循环）中 16 处 session.append(...) 调用——形状不变，但现在在边界处被校验。
  • 约 7 个 switch 消费方，对这些联合类型进行分支：deriveMessages 与包自有的不变式 companion（dsh-session）、BlockAssembler（dsh-llm）、两个 LLM（大语言模型）适配器（dsh-llm-deepseek、dsh-llm-pi-ai）以及工具 schema 层（dsh-tools）。assertNever 对封闭联合类型的穷举 vs 对可扩展联合类型的 fall-through 约定（一条已记录的 lint 规则）需要重新考量——运行时变体在静态层面不可穷举。
  • defineTool 的 InferArgs DSL（dsh-tools），它从编译期 schema 规范派生出零类型转换的 execute 参数类型——这是当前方案的标杆用例。
  • 文档：architecture.md（该模式被描述为基础性的）、开发模式不变式，以及所有引用该模式的 Agent Note。

这是一次仓库级别的词汇重新设计，而非持久化的实现细节。

曾考虑的替代方案

A. 维持现状——merge-extensible 类型 + 持久化边界处 isJsonValue

保留编译期模式。持久化继续使用不透明 JSON + 可序列化性守卫。插件通过声明合并扩展；事件 形状的正确性由生产者负责，并由 TypeScript 在编译期保证。启用包自有的不变式 companion 后，它们会检查选定的跨记录关系，但不提供通用运行时形状 schema。

  • 优点：零变动；插件扩展只需一行 interface 增补，享有完整类型推断，无需运行时注册仪式；无新运行时依赖；defineTool DSL 与 assertNever 穷举继续工作。
  • 缺点：持久化边界和插件边界处无运行时结构校验；格式错误但仍为合法 JSON 的数据被延迟捕获。

B. 仅对头部/封闭形状做校验（schemastery），事件仍为不透明

仅对那些已有手写类型守卫的真正封闭形状加以收紧——例如 JSONL 的 HeaderLine 守卫（isHeaderLine）——使用 schemastery（仓库现有的 schema 库，已用于每个插件的 static Config）。merge-extensible 事件联合类型保持不变。

  • 优点：改动小，契合现有约定（schemastery，而非新库）；用声明式 schema 替换封闭形状上的手写守卫；无核心重新设计。
  • 缺点：不解决事件数据校验问题；仅固定的元数据记录得到改善。

C. 为整个词汇建立运行时 schema 注册表（Zod 或 schemastery）

用运行时注册表替换 merge-extensible map，生产者向其贡献 schema，持久化/消费路径据此校验。

  • 优点：持久化边界和插件边界处获得真正的运行时校验；单一真源；可支撑通用工具（自动生成文档、模糊测试、协议格式（wire format）检查）。
  • 缺点：上述全部影响范围；Zod 目前不是直接依赖（仅作为 @earendil-works/pi-ai 的传递依赖），仓库选定的 schema 库是 schemastery——广泛引入 Zod 本身就是一个依赖决策；声明合并的易用性（一行插件扩展、完整推断）被运行时注册 + 手动类型接线取代；assertNever 穷举保证弱化（运行时变体在静态层面不可穷举）。

提案

推迟。如果需要在持久化边界做运行时校验，方案 B（对封闭的头部和元数据形状使用 schemastery）是现有约定下的适度步骤。方案 C 是一个架构决策，需要自己的实现 Agent Note，其中包括 Zod 与 schemastery 之间的选择。

验收标准

  • 方案 C 只能通过自己的实现 Agent Note 推进，绝不能作为持久化的附带改动。
  • 如果采纳方案 B，封闭的头部/元数据形状（JSONL 的 isHeaderLine 守卫及同类）改用 schemastery 校验，替代手写守卫，merge-extensible map 保持不动。

风险

  • 推迟意味着事件 data 在持久化边界处仍无结构校验：格式错误但仍为合法 JSON 的数据被延迟捕获，由消费方的 switch 兜底——这是现状的代价，有意接受。
  • 如果方案 C 最终被采纳，易用性的损失是真实的：一行声明合并变为运行时注册加手动类型接线，assertNever 的静态穷举保证弱化。

待解问题

  • 如果采用注册表，库选 schemastery（已在仓库中，已作为配置 schema 库）还是 Zod（生态更丰富，目前仅为传递依赖）？同时维护两个 schema 库本身就是一种成本。
  • 能否采用混合方案：保留编译期推断（使 defineTool 和插件开发体验不受影响），同时为每个变体添加可选的运行时 schema，仅在持久化/协议边界校验，而非每次进程内 append 都校验？
  • ctx.invariants 服务启用后是否已覆盖了足够多的运行时形状缺口，使得边界校验仅在面对真正不可信输入（重新加载外部修改过的日志）时才有必要？

More architecture decisions