Embedded Assistant stream consumers read compact records
内嵌 Assistant 流的消费方直接读取紧凑记录
Session format v2 embeds each model attempt's compact stream (`AssistantStreamRecord[]`: packed `text-chunks`, `reasoning-chunks`, and `tool-call-chunks` runs plus timestamped raw `chunk` records) in `assistant/message` and `assistant/attempt`. Consumers that folded those settlements called `expandAssistantStream()` first; it materializes the complete per-member array, so a consumer that needs one fact (`find` on the
English
Problem
Session format v2 embeds each model attempt's compact stream (AssistantStreamRecord[]: packed text-chunks, reasoning-chunks, and tool-call-chunks runs plus timestamped raw chunk records) in assistant/message and assistant/attempt. Consumers that folded those settlements called expandAssistantStream() first; it materializes the complete per-member array, so a consumer that needs one fact (find on the first token, the last usage chunk, a joined text, one block-end) paid O(members) allocation and time: about two objects per member on top of the compact form.
After v2 embedded streams settlement widened with the message content and Chat and Trajectory sections settled directly from it, the remaining expand consumers are the Host and client folds: Session Stats reads the first-token time per assistant/attempt and assistant/message (the projection phase of every Session open), the token meter rebuilds provider content and scans every stream for its last usage chunk (the projection unit still scans to the end), the subagent output fold joins plain text, and the Session Controller image lookup scans for block-end chunks.
Decision
@deepseek-ai/dsh-llm answers consumer questions directly from compact records; every remaining consumer folds records once with early exit.
packages/llm/llm/src/assistant-stream.ts exports record-level readers beside the accumulator and expandAssistantStream:
- Chunk rules:
isTokenDelta(non-empty text, reasoning, or Tool-call arguments fragment, or any name-bearing Tool-call delta),isVisibleChunk(non-whitespace text or reasoning, or a block start or end of any kind other than text, reasoning, or Tool call), andchunkHasVisibleText(non-whitespace text delta or completed text block). - Run readers:
runFirstTokenTimeandrunFirstVisibleTimereconstruct the first qualifying member's time fromtime0and thedtgaps and stop scanning there; a name-bearing Tool-call run yieldstime0without reading a fragment. - Stream readers:
assistantStreamFirstTokenTime,assistantStreamHasVisibleContent,assistantStreamHasVisibleText,lastAssistantStreamChunk(stream, type)(backward scan),assistantStreamChunks(stream, type),joinAssistantStreamText, andassembleAssistantStream, which feeds aBlockAssemblerone joined delta per run (assembly only concatenates, so blocks, usage, finish, and replay state equal the per-member result).RawStreamChunkTypeexcludes the delta types, so a raw-chunk lookup can never silently skip packed members.
Session Stats and Trajectory read assistantStreamFirstTokenTime from both assistant/attempt and assistant/message, retaining the Step's first token across retries. Trajectory settles content from the assembled message while reading timing independently, so reopening history retains TTFT and decoding metrics without expanding streams. Chat follows its settled-reply timing policy. The token meter reads lastAssistantStreamChunk(stream, 'usage') and assembles provider output through assembleAssistantStream; the subagent output fold appends joinAssistantStreamText; the Session Controller scans assistantStreamChunks(stream, 'block-end') for images.
expandAssistantStream keeps its strict validation and its remaining callers, which need every member or validate the stream at a durable boundary: Session restore validation, the v1-to-v2 migration validator and publication Worker replay, the reconnect baseline, and test support.
Measurements
The repo's synthetic first-open benchmark (200 turns, 127,400 released-v0 events, 500,000 streamed deltas in 1,600 compact records; five samples, median):
| Phase | Before | After |
|---|---|---|
| first-open projection | 28.0 ms | 5.9 ms |
| first-open total | 76.9 ms | 53.8 ms |
| first-open peak RSS | 137.2 MB | 94.6 MB |
| reopen projection | 17.8 ms | 6.5 ms |
Open, read, and restore phases are unchanged; the reader keeps the same first-token time by construction (the first qualifying member is the first record's first qualifying fragment, and the deltas stay ordered).
Alternatives considered
Memoize expandAssistantStream per input array. Expanding all streams once costs tens of milliseconds, but retaining the expansions costs about ten times the compact stream for the event's lifetime — a permanent version of the transient allocation the change removes. The readers remove the need for retained expansions entirely.
Keep the per-member fold. Early-exit .find still materializes the whole array first, so the allocation and O(members) time remain.
Consequences
Host and Client folds of an embedded settlement cost O(records) plus one join per run, and no consumer materializes members unless it validates at a durable boundary or needs every member. The token, visibility, and visible-text rules have one home in dsh-llm, so a record reader and the accumulator's packing rules cannot drift apart.
Publication verification (assertCurrentAssistantStreams) still replays every settlement at publish time; because it must prove content-by-chunk agreement, converting it to run-aware assembly without member materialization remains open work.
中文
问题
Session 格式 v2 将每次模型尝试的紧凑流（AssistantStreamRecord[]：打包的 text-chunks、reasoning-chunks、tool-call-chunks run 加上带时间戳的原始 chunk 记录）嵌入 assistant/message 与 assistant/attempt。折叠这些 settlement 的消费方会先调用 expandAssistantStream()；它会物化完整的逐成员数组，因此只需一个事实的消费方（find 首个 token、最后一个 usage chunk、拼接文本、一个 block-end）也要付出 O(members) 的分配与时间：在紧凑形式之上每个成员约两个对象。
在 v2 内嵌流 settlement 随消息内容扩展、Chat 与 Trajectory 区块直接由内容结算之后，剩余的 expand 消费方是 Host 与客户端折叠：Session Stats 读取每个 assistant/attempt 与 assistant/message 的首 token 时间（每次打开 Session 的 projection 阶段）、token 计量重建提供商内容并扫描每个流到最后一个 usage chunk（projection 单元仍扫描到末尾）、子代理输出折叠拼接纯文本、Session Controller 镜像查找扫描 block-end chunk。
决策
@deepseek-ai/dsh-llm 直接从紧凑记录回答消费方问题；剩余消费方对记录做一次带提前退出的折叠。
packages/llm/llm/src/assistant-stream.ts 在累加器与 expandAssistantStream 之外导出记录级读取器：
- Chunk 规则：
isTokenDelta（非空文本、reasoning 或 Tool-call 参数片段，或任何带名称的 Tool-call delta）、isVisibleChunk（非空白文本或 reasoning，或 text/reasoning/Tool call 之外的任意块开始或结束）、chunkHasVisibleText（非空白文本 delta 或完成的文本块）。 - Run 读取器：
runFirstTokenTime与runFirstVisibleTime从time0与dt间隔重建首个合格成员的时间并停止扫描；带名称的 Tool-call run 直接产出time0，不读片段。 - 流读取器：
assistantStreamFirstTokenTime、assistantStreamHasVisibleContent、assistantStreamHasVisibleText、lastAssistantStreamChunk(stream, type)（逆向扫描）、assistantStreamChunks(stream, type)、joinAssistantStreamText与assembleAssistantStream（每个 run 向BlockAssembler喂入一个拼接后的 delta；组装只做拼接，因此 blocks、usage、finish 与 replay state 与逐成员结果一致）。RawStreamChunkType排除 delta 类型，因此原始 chunk 查找不可能静默跳过打包成员。
Session Stats 与 Trajectory 从 assistant/attempt 和 assistant/message 读取 assistantStreamFirstTokenTime，跨重试保留步骤的首个 token。Trajectory 从组装后的消息结算内容，并独立读取计时，因此重新打开历史时无需展开流便能保留 TTFT 与解码指标。Chat 遵循已结算回复的计时策略。token 计量读取 lastAssistantStreamChunk(stream, 'usage') 并通过 assembleAssistantStream 组装提供商输出；子代理输出折叠追加 joinAssistantStreamText；Session Controller 用 assistantStreamChunks(stream, 'block-end') 扫描镜像。
expandAssistantStream 保留其严格校验与其余调用方（需要每个成员或在持久边界校验流）：Session 恢复校验、v1-to-v2 迁移校验器与发布 Worker 重放、重连基线、测试支撑。
测量
仓库的合成 first-open 基准（200 循环、127,400 个 released-v0 事件、1,600 条紧凑记录中的 500,000 个流式 delta；五次采样取中位数）：
| 阶段 | 之前 | 之后 |
|---|---|---|
| first-open projection | 28.0 ms | 5.9 ms |
| first-open 总计 | 76.9 ms | 53.8 ms |
| first-open 峰值 RSS | 137.2 MB | 94.6 MB |
| reopen projection | 17.8 ms | 6.5 ms |
Open、read、restore 阶段不变；读取器按构造保持相同的首 token 时间（首个合格成员即首条记录的首个合格片段，且 delta 保持有序）。
备选方案
按输入数组记忆化 expandAssistantStream。 展开全部流只需几十毫秒，但保留展开结果在事件生命周期内约花费紧凑流的十倍内存——这是本变更移除的瞬时分配的永久版本。读取器完全消除了对保留展开的需求。
保留逐成员折叠。 提前退出的 .find 仍然先物化整个数组，因此分配与 O(members) 时间仍在。
后果
Host 与客户端折叠一次内嵌结算的代价为 O(records) 加每个 run 一次拼接，且除非在持久边界校验或需要每个成员，消费方不再物化成员。token、可见性与可见文本规则在 dsh-llm 中只有一处，因此记录读取器与累加器的打包规则不可能漂移。
发布校验（assertCurrentAssistantStreams）仍在发布时重放每个 settlement；因为它必须按 chunk 证明内容一致，将其转为不入成员的 run 感知组装仍是未完成工作。