Session telemetry seam with mandatory redaction and the OTel backend
设有强制脱敏点和 OTel 后端的会话遥测 seam
Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the `session-telemetry-otlp-rfc` branch (PR #222/#231) but never reached master: the proposal exported raw sessio
English
Problem
Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the session-telemetry-otlp-rfc branch (PR #222/#231) but never reached master: the proposal exported raw session events verbatim, which legal review declined. The capture-side design (backend contract, coordinator, handoff cursor, chunk projection) was sound and reviewed; the export-side stance was the blocker.
Decision
packages/session/ (formerly telemetry/) revives the two reviewed packages under the SDK stance — the harness provides the capability, the deployment configures where records go and owns what leaves in them:
@deepseek-ai/dsh-session-telemetry— the seam.SessionTelemetrySink(emit/flush?/shutdown), the service-registeredSessionTelemetryBackendform, andSessionTelemetryCoordinatorowning capture: live adoption with cursor read-back and the per-append firehose (project →structuredClone→ redact →emit, zero I/O), buffer-free on-demand replay from the canonical log, the fixed first-chunk-per-(turn, step) projection, the liveagent/errorrelay, and live dispose-timeshutdownrecords.- The
session-telemetry/recordwaterfall — the delta over the branch version and the seam's redaction extension point. Every record passes it before reaching any backend; the seam ships NO rules of its own — the innermostnext()is a pass-through, deployments mount their rules as listeners (stacking by transformingnext()'s return value), and a throwing rule withholds the record fail-closed. Redaction applies to the exported copy only; the canonical log is never rewritten. @deepseek-ai/dsh-session-telemetry-otel— the reference backend: OTel JS SDK log pipeline (LoggerProvider→BatchLogRecordProcessor→ OTLP/HTTP exporter), configured verbatim throughexporter/processorpassthroughs.DISABLEDis the default and constructs no transport; the feedback-gated telemetry decision defines the explicitFULLandFEEDBACK_ONLYdelivery modes, which requireexporter.url, without moving the redaction or backend boundary. Buffer-free feedback replay avoids a second in-memory copy of the session prefix.
The boundary axiom holds: the harness's aspect ends at emit(). Batching, retry, queueing, and loss policy are the reporting SDK's, configured through passthroughs — delivery is best-effort (at-most-once across a crash), which the READMEs state plainly.
Alternatives considered
Implement the runtime-telemetry RFC's outbox (durable spool, per-sink cursors, at-least-once, a readCommitted persistence-seam method). Deferred, not rejected: the SDK stance makes delivery semantics the reporting SDK's territory, and the OTel SDK's own batch pipeline is the honest default. The outbox is a pure additive layer (the emit() contract does not move); revive it when a deployment states a crash-loss requirement telemetry must satisfy.
No in-process redaction point, delegating to receiver-side collector processors. Rejected — receiver-side redaction ships the secret first and scrubs it second. The waterfall puts an auditable, stackable scrubbing point before bytes leave the process; where the branch version (what PR #222 shipped) had no redaction point at all, every record now passes one.
A built-in conservative rule set as the waterfall's innermost next(). Rejected: as an SDK we cannot know which patterns are secrets in a given deployment, a shipped list invites false confidence ("redaction is on") while catching only known shapes, and false positives would corrupt exported bodies for consumers who never asked. The seam owns the mechanism; the deployment owns the policy — the innermost next() is a pass-through, and rules mount as listeners.
Map onto OTel spans (GenAI semantic conventions) instead of logs. Rejected for this revival: the branch implementation's log mapping is reviewed and shipped-shaped; the span model is lossy for forkable, interruptible sessions and belongs to a future consumer with real span queries to serve.
Full-log replay when no handoff cursor survived (re-export constructor seeds). Shipped in the first revival round, then narrowed: adoption now replays from the session's construction boundary (Session.firstLiveSeq, the constructor-seed length — a fact the session already validated but did not expose; header.seedLength cannot serve, it is the durable fork-lineage value and a resumed session's constructor seed is its full stored log). A resumed session's history already shipped from the previous process under the same id, and a fork's inherited prefix already shipped in the parent's stream — re-exporting either re-billed every resume for its full history and doubled query-time counts on OTLP backends with no native ingest dedupe. Receivers stitch fork lineage via session.parent_id + session.seed_length. What the narrowing gives up, consistently with the at-most-once stance: a resume no longer backfills records the previous process failed to deliver (telemetry unmounted then, or queued at crash) — the full replay's only real benefit, bought at the common case's expense. A deployment that states a backfill requirement needs the deferred outbox above, not replay. The boundary also swallows the synthetic turn closers SessionPersistence.load() writes when repairing a crash-interrupted log (they sit below firstLiveSeq despite never existing in the previous process) — deliberate, not incidental: exporting a synthetic closer cannot complete the remote turn whose real tail records died in the crashed process's queue, it can only make an incomplete turn look closed. The wire stream stays faithful to what the crashed process actually shipped; receivers read a never-closed turn on a resumed stream as "the previous process died inside it" (the OTel README states the rule), and a later clean shutdown marker attests only to the resumed process's exit. Threading the pre-repair boundary through load/prepare so repairs export as live events would couple three packages to un-ship that signal.
Forwarding the seam's turn-boundary flush() hint to the OTel provider's forceFlush(). Shipped in the first revival round, then removed: three distinct silent-loss paths shared the wrapper state — a dispose racing an in-flight flush (the SDK's concurrent-flush guard makes shutdown's internal drain skip), overlapping hints displacing the retained promise, and the provider's fixed 30-second flush timeout rejecting while the processor still drains. Every path exists only because the forwarding made this backend the process's second flusher against undocumented SDK internals from the upstream experimental tree; with no flush() implemented, the batch processor is the only flusher, its scheduledDelayMillis (already deployment-tunable through the processor passthrough) governs export cadence, and shutdown()'s drain is complete by construction. Reinstate only if a deployment states a turn-boundary latency requirement scheduledDelayMillis cannot meet — and then by calling the retained BatchLogRecordProcessor's own forceFlush(), never the provider's timeout-wrapped one.
Consequences
A deployment adds one cordis.yml entry with an OTLP endpoint and explicitly selects FULL to stream sessions into an OTel-compatible stack or FEEDBACK_ONLY to replay a canonical-log prefix when feedback is recorded. DISABLED is the default and constructs no reporting pipeline; removing the entry remains a silent opt-out, while the disabled mode keeps the local feedback warning. A rule-free deployment exports records exactly as captured — including any credentials embedded in file contents or command output — so a deployment crossing a trust boundary must mount session-telemetry/record listeners, and both READMEs state this plainly. Where rules are mounted, exported bodies can differ from canonical log bytes, so receivers must not treat telemetry as a byte-exact replica; the log remains the source of truth. Crash durability is explicitly out of scope until the outbox decision above is revisited.
中文
问题
每个想把 harness 会话接入可观测性体系的部署方都得手写一个会话日志消费方：订阅、生命周期交接、以及最难的脱敏——原始日志携带文件内容与命令输出，可能内嵌凭据。遥测 seam 和 OTel 后端曾在 session-telemetry-otlp-rfc 分支（PR #222/#231）上完成过一版，但从未进入 master：该提案将原始会话事件原样导出，法务评审未予通过。捕获侧设计（后端约定、coordinator、handoff 游标、分片投影）本身合理且经过评审；导出侧的立场才是阻塞点。
决策
packages/session/（原 telemetry/）以 SDK 立场复活这两个经过评审的包——harness 提供能力，部署方配置上报去向并对导出内容负责：
@deepseek-ai/dsh-session-telemetry—— seam 本体。SessionTelemetrySink（emit/flush?/shutdown）、服务注册形态的SessionTelemetryBackend、以及拥有捕获侧的SessionTelemetryCoordinator：带游标回读的实时纳管与逐 append 的 firehose（投影 →structuredClone→ 脱敏 →emit，零 I/O）、从权威日志进行的无缓冲按需回放、固定的每个（轮次、步骤）组合首分片投影、实时agent/error转发，以及实时 dispose（资源释放）时的shutdown记录。session-telemetry/recordwaterfall（瀑布式事件） —— 相对分支版本的增量，也是该 seam 的脱敏扩展点。每条记录抵达任何后端前必经此处；seam 自身不带任何规则——最内层next()原样透传，部署方以监听器挂载自己的规则（通过变换next()的返回值堆叠），抛异常的规则将该记录 fail-closed 扣下。脱敏只作用于导出副本；canonical log 永不改写。@deepseek-ai/dsh-session-telemetry-otel—— 参考后端：OTel JS SDK 日志流水线（LoggerProvider→BatchLogRecordProcessor→ OTLP/HTTP exporter），经exporter/processorpassthrough 原样配置。DISABLED是默认值，且不构造任何传输；反馈门控遥测决策定义了需显式启用的FULL与FEEDBACK_ONLY投递模式，这两种模式要求exporter.url，且不移动脱敏或后端边界。无缓冲反馈回放避免在内存中创建会话前缀的第二份副本。
边界公理保持不变：harness 的职责止于 emit()。批处理、重试、排队与丢失策略属于 reporting SDK，经 passthrough 配置——投递是尽力而为（崩溃时至多一次），两份 README 对此如实陈述。
考虑过的替代方案
实现 runtime-telemetry RFC 的 outbox（落盘 spool、每 sink 游标、at-least-once、持久化 seam 的 readCommitted 方法）。 推迟而非否决：SDK 立场使投递语义归属 reporting SDK，OTel SDK 自身的批处理流水线是诚实的默认。outbox 是纯增量层（emit() 约定不动）；待某个部署提出遥测必须满足的崩溃丢失要求时再复活。
不设进程内脱敏点，交给接收端 collector processor。 否决——接收端脱敏是先把秘密发出去再擦除。waterfall 在字节离开进程前提供一个可审计、可堆叠的擦除点；分支版本（PR #222 交付的形态）完全没有脱敏点，如今每条记录都必经该脱敏点。
在 waterfall 最内层 next() 内置一套保守规则集。 否决：作为 SDK 我们无法预知某个部署里什么模式算秘密，内置列表只覆盖已知形状却会带来「脱敏已开启」的虚假信心，且误报会破坏未提出此要求的消费方所接收的导出 body。seam 拥有机制，部署方拥有策略——最内层 next() 原样透传，规则以监听器挂载。
映射到 OTel span（GenAI 语义约定）而非日志。 本次复活否决：分支实现的日志映射已经过评审、形态可交付；span 模型对可 fork、可中断的会话有损，留给将来真正有 span 查询需求的消费方。
handoff 游标未存活时全量回放日志（重新导出构造函数种子）。 首轮复活曾交付此方案，其后收窄：接管操作现在从会话的构造边界起回放（Session.firstLiveSeq，即构造函数种子长度，这一事实会话早已校验过却未曾暴露；header.seedLength 不能胜任：它是持久保存的 fork 谱系（lineage）值，而恢复会话的构造函数种子是其完整的已存储日志）。恢复会话的历史已由上一个进程以同一 id 发出，fork 继承的前缀也已在父会话的流中发出；再次导出任何一者，都会让每次恢复为其完整历史重复付费，并在没有原生摄取去重的 OTLP 后端上使查询时的计数翻倍。接收端基于 session.parent_id + session.seed_length 拼接 fork 谱系。此次收窄放弃的内容与至多一次立场一致：恢复不再回填上一个进程未能投递的记录（彼时遥测未挂载，或崩溃时仍在队列中）——这本是全量回放唯一的真实收益，代价却由常见情形承担。提出回填要求的部署需要的是上文已推迟的 outbox，而不是回放。该边界同样吞掉 SessionPersistence.load() 修复被崩溃打断的日志时写入的合成轮次关闭事件（它们落在 firstLiveSeq 之前，尽管在上一个进程中从未存在过）。这是有意为之，而非附带效果：远端轮次的真实尾部记录已随崩溃进程的队列一同消亡，导出合成关闭事件无法补全该轮次，只会让一个未完成的轮次看起来已经关闭。导出的流忠实于崩溃进程实际发出的内容；接收端会把恢复后的流中一个从未关闭的轮次读作「上一个进程死在了该轮次之内」（OTel README 陈述了这条规则），其后干净的 shutdown 标记也只证明恢复后进程自身的退出。若为让修复以实时事件的身份导出而将修复前边界贯穿 load/prepare 传递，将使三个包相互耦合，只为抹除这一信号。
将 seam 的轮次边界 flush() 提示转发到 OTel 提供方的 forceFlush()。 首轮复活曾交付此转发，其后移除：三条不同的静默丢失路径共用同一份包装层状态——dispose 与进行中的 flush 之间的竞态（SDK 的并发 flush 防护会令 shutdown 的内部排空被跳过）、相互重叠的提示顶掉留存的 promise、以及提供方固定的 30 秒 flush 超时在批处理器仍在排空时便 reject。这些路径存在的唯一原因，是该转发让这个后端成为进程内第二个执行 flush 的组件，面对的还是上游实验性（experimental）源码树中未见诸文档的 SDK 内部行为；不实现 flush() 时，批处理器就是唯一执行 flush 的组件，其 scheduledDelayMillis（已可由部署方经 processor passthrough 调优）决定导出节奏，shutdown() 的排空从构造上就是完整的。仅当某个部署提出 scheduledDelayMillis 无法满足的轮次边界延迟要求时才恢复此转发——且届时应调用留存的 BatchLogRecordProcessor 自身的 forceFlush()，绝不调用提供方那个带超时包装的版本。
后果
部署方在 cordis.yml 加一个带 OTLP endpoint 的 Cordis 配置项，并显式选择 FULL，即可把会话流接入任何 OTel 兼容体系；选择 FEEDBACK_ONLY 则会在记录反馈时回放权威日志前缀。DISABLED 是默认值，且不构造上报流水线；删除该配置项仍是静默退出方式，而禁用模式会保留本地反馈警告。未挂载规则的部署导出的记录与捕获时完全一致，包括文件内容与命令输出中内嵌的任何凭据。因此，跨信任边界的部署必须挂载 session-telemetry/record 监听器，两个 README 对此如实陈述。挂载规则后，导出的 body 可能与 canonical log 字节不同，接收端不得把遥测当作字节精确副本；日志仍是真源。崩溃持久性在上述 outbox 决定重新审议前明确不在范围内。