Session telemetry seam with mandatory redaction and the OTel backend
设有强制脱敏点和 OTel 后端的会话遥测 seam
Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the `session-telemetry-otlp-rfc` branch (PR #222/#231) but never reached master: the proposal exported raw sessio
English
Problem
Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the session-telemetry-otlp-rfc branch (PR #222/#231) but never reached master: the proposal exported raw session events verbatim, which legal review declined. The reusable capture-side design comprised the backend contract, coordinator, handoff cursor, and session-event subscription; the export-side stance was the blocker.
Decision
packages/session/ (formerly telemetry/) revives the two reviewed packages under the SDK stance — the harness provides the capability, the deployment configures where records go and owns what leaves in them:
@deepseek-ai/dsh-session-telemetry— the seam.SessionTelemetrySink(emit/flush?/shutdown), the service-registeredSessionTelemetryBackendform, andSessionTelemetryCoordinatorown lifecycle-local capture: each new Session object starts immediately beforefirstLiveSeq, then the per-append firehose deep-copies, redacts, and hands off every event with zero I/O; re-adopting the same object resumes after its module-scope cursor. Buffer-free on-demand capture uses the same one-record-per-event mapping through an optional inclusive boundary. Ledger identity includessession.id,session.format_version, andevent.seq; live capture also relaysagent/errorand creates dispose-timeshutdownrecords.- The
session-telemetry/recordwaterfall — the delta over the branch version and the seam's redaction extension point. Every record passes it before reaching any backend; the seam ships NO rules of its own — the innermostnext()is a pass-through, deployments mount their rules as listeners (stacking by transformingnext()'s return value), and a throwing rule withholds the record fail-closed. Redaction applies to the exported copy only; the canonical log is never rewritten. @deepseek-ai/dsh-session-telemetry-otel— the reference backend: OTel JS SDK log pipeline (LoggerProvider→BatchLogRecordProcessor→ OTLP/HTTP exporter), configured verbatim throughexporter/processorpassthroughs.DISABLEDis the plugin default and constructs no transport.FEEDBACK_ONLYrequiresexporter.url; the explicit-feedback policy requires a new submission for every bounded capture, for all providers. Buffer-free feedback replay avoids a second in-memory copy of the session prefix.
The Session-log request limits supersede the SDK-only batching rule below for the OTel Session sender; capture, redaction, and handoff rationale remain applicable.
The boundary axiom holds: the harness's aspect ends at emit(). Batching, retry, queueing, and loss policy are the reporting SDK's, configured through passthroughs. Delivery is best-effort: a crash can lose queued records. Backend retry or losing the same live object's module-scope cursor can duplicate lifecycle-local rows, so receivers deduplicate on (session.id, session.format_version, event.seq).
Alternatives considered
Implement the runtime-telemetry RFC's outbox (durable spool, per-sink cursors, at-least-once, a readCommitted persistence-seam method). Deferred, not rejected: the SDK stance makes delivery semantics the reporting SDK's territory, and the OTel SDK's own batch pipeline is the honest default. The outbox is a pure additive layer (the emit() contract does not move); revive it when a deployment states a crash-loss requirement telemetry must satisfy.
No in-process redaction point, delegating to receiver-side collector processors. Rejected — receiver-side redaction ships the secret first and scrubs it second. The waterfall puts an auditable, stackable scrubbing point before bytes leave the process; where the branch version (what PR #222 shipped) had no redaction point at all, every record now passes one.
A built-in conservative rule set as the waterfall's innermost next(). Rejected: as an SDK we cannot know which patterns are secrets in a given deployment, a shipped list invites false confidence ("redaction is on") while catching only known shapes, and false positives would corrupt exported bodies for consumers who never asked. The seam owns the mechanism; the deployment owns the policy — the innermost next() is a pass-through, and rules mount as listeners.
Map onto OTel spans (GenAI semantic conventions) instead of logs. Rejected for this revival: the branch implementation's log mapping is reviewed and shipped-shaped; the span model is lossy for forkable, interruptible sessions and belongs to a future consumer with real span queries to serve.
Replay every constructor seed automatically. A fork's inherited events and a resumed log may predate the current sharing act. The coordinator defaults to firstLiveSeq for a new object; includeHistory explicitly permits a backend to capture stored context. OTel selects complete history only through new own feedback, never on construction, restoration, or HMR. Same-object cursors reduce duplicate handoff but do not guarantee historical delivery or replace the deferred durable outbox.
Forwarding the seam's turn-boundary flush() hint to the OTel provider's forceFlush(). Shipped in the first revival round, then removed: three distinct silent-loss paths shared the wrapper state — a dispose racing an in-flight flush (the SDK's concurrent-flush guard makes shutdown's internal drain skip), overlapping hints displacing the retained promise, and the provider's fixed 30-second flush timeout rejecting while the processor still drains. Every path exists only because the forwarding made this backend the process's second flusher against undocumented SDK internals from the upstream experimental tree; with no flush() implemented, the batch processor is the only flusher, its scheduledDelayMillis (already deployment-tunable through the processor passthrough) governs export cadence, and shutdown()'s drain is complete by construction. Reinstate only if a deployment states a turn-boundary latency requirement scheduledDelayMillis cannot meet — and then by calling the retained BatchLogRecordProcessor's own forceFlush(), never the provider's timeout-wrapped one.
Consequences
A deployment configures an OTLP endpoint and selects FEEDBACK_ONLY to release a complete canonical prefix at new explicit feedback. DISABLED is the plugin default (dsh-session-telemetry-otel README) and constructs no reporting pipeline; removing the entry is a silent opt-out, while disabled mode keeps the local feedback warning. SDK scheduled flush and shutdown only drain authorized batches, without capturing later records. A rule-free deployment exports each captured event body, including compact assistant streams and credentials embedded in file contents or command output, so deployments crossing a trust boundary must mount session-telemetry/record listeners. Redacted bodies can differ from canonical log bytes; the log remains the source of truth. Lost handoff cursors can cause duplicates on a later authorized capture, and crash durability remains out of scope until the outbox decision above is revisited.
中文
问题
每个想把 harness 会话接入可观测性体系的部署方都得手写一个会话日志消费方：订阅、生命周期交接、以及最难的脱敏——原始日志携带文件内容与命令输出，可能内嵌凭据。遥测 seam 和 OTel 后端曾在 session-telemetry-otlp-rfc 分支（PR #222/#231）上完成过一版，但从未进入 master：该提案将原始会话事件原样导出，法务评审未予通过。可复用的捕获侧设计包括后端约定、coordinator、handoff 游标与会话事件订阅；导出侧的立场才是阻塞点。
决策
packages/session/（原 telemetry/）以 SDK 立场复活这两个经过评审的包——harness 提供能力，部署方配置上报去向并对导出内容负责：
@deepseek-ai/dsh-session-telemetry—— seam 本体。SessionTelemetrySink（emit/flush?/shutdown）、服务注册形态的SessionTelemetryBackend与SessionTelemetryCoordinator共同拥有生命周期本地捕获：每个新的 Session 对象从firstLiveSeq之前开始，随后逐 append firehose 以零 I/O 深拷贝、脱敏并交接每个事件；重新收养同一对象时从模块作用域游标之后继续。无缓冲按需捕获使用同样的一事件一记录映射，直到可选的包含式边界。Ledger 身份包含session.id、session.format_version与event.seq；实时捕获还会转发agent/error，并创建 dispose（资源释放）时的shutdown记录。session-telemetry/recordwaterfall（瀑布式事件） —— 相对分支版本的增量，也是该 seam 的脱敏扩展点。每条记录抵达任何后端前必经此处；seam 自身不带任何规则——最内层next()原样透传，部署方以监听器挂载自己的规则（通过变换next()的返回值堆叠），抛异常的规则将该记录 fail-closed 扣下。脱敏只作用于导出副本；canonical log 永不改写。@deepseek-ai/dsh-session-telemetry-otel—— 参考后端：OTel JS SDK 日志流水线（LoggerProvider→BatchLogRecordProcessor→ OTLP/HTTP exporter），经exporter/processorpassthrough 原样配置。DISABLED是插件默认值，且不构造任何传输。FEEDBACK_ONLY要求exporter.url；显式反馈策略要求每次有界捕获都由新提交触发，适用于所有提供方。无缓冲反馈回放避免在内存中创建会话前缀的第二份副本。
Session 日志请求上限 在 OTel Session 发送器中替代下述仅由 SDK 分批的规则；采集、脱敏和交接的理由仍然适用。
边界公理保持不变：harness 的职责止于 emit()。批处理、重试、排队与丢失策略属于 reporting SDK，并经 passthrough 配置。投递是尽力而为：崩溃可能丢失已排队记录。后端重试或丢失同一 live 对象的模块作用域游标可能重复生命周期本地 row，因此接收端基于 (session.id, session.format_version, event.seq) 去重。
考虑过的替代方案
实现 runtime-telemetry RFC 的 outbox（落盘 spool、每 sink 游标、at-least-once、持久化 seam 的 readCommitted 方法）。 推迟而非否决：SDK 立场使投递语义归属 reporting SDK，OTel SDK 自身的批处理流水线是诚实的默认。outbox 是纯增量层（emit() 约定不动）；待某个部署提出遥测必须满足的崩溃丢失要求时再复活。
不设进程内脱敏点，交给接收端 collector processor。 否决——接收端脱敏是先把秘密发出去再擦除。waterfall 在字节离开进程前提供一个可审计、可堆叠的擦除点；分支版本（PR #222 交付的形态）完全没有脱敏点，如今每条记录都必经该脱敏点。
在 waterfall 最内层 next() 内置一套保守规则集。 否决：作为 SDK 我们无法预知某个部署里什么模式算秘密，内置列表只覆盖已知形状却会带来「脱敏已开启」的虚假信心，且误报会破坏未提出此要求的消费方所接收的导出 body。seam 拥有机制，部署方拥有策略——最内层 next() 原样透传，规则以监听器挂载。
映射到 OTel span（GenAI 语义约定）而非日志。 本次复活否决：分支实现的日志映射已经过评审、形态可交付；span 模型对可 fork、可中断的会话有损，留给将来真正有 span 查询需求的消费方。
自动回放每个 constructor seed。 fork 的继承事件和恢复的日志可能早于当前共享动作。协调器对新对象默认从 firstLiveSeq 开始；includeHistory 显式允许后端捕获存储的上下文。OTel 仅在新的自身反馈时选择完整历史，绝不在构造、恢复或 HMR 时捕获。同对象游标减少重复交接，但不保证历史投递，也不替代延后的持久化 outbox。
将 seam 的轮次边界 flush() 提示转发到 OTel 提供方的 forceFlush()。 首轮复活曾交付此转发，其后移除：三条不同的静默丢失路径共用同一份包装层状态——dispose 与进行中的 flush 之间的竞态（SDK 的并发 flush 防护会令 shutdown 的内部排空被跳过）、相互重叠的提示顶掉留存的 promise、以及提供方固定的 30 秒 flush 超时在批处理器仍在排空时便 reject。这些路径存在的唯一原因，是该转发让这个后端成为进程内第二个执行 flush 的组件，面对的还是上游实验性（experimental）源码树中未见诸文档的 SDK 内部行为；不实现 flush() 时，批处理器就是唯一执行 flush 的组件，其 scheduledDelayMillis（已可由部署方经 processor passthrough 调优）决定导出节奏，shutdown() 的排空从构造上就是完整的。仅当某个部署提出 scheduledDelayMillis 无法满足的轮次边界延迟要求时才恢复此转发——且届时应调用留存的 BatchLogRecordProcessor 自身的 forceFlush()，绝不调用提供方那个带超时包装的版本。
后果
部署方配置 OTLP endpoint 并选择 FEEDBACK_ONLY，即可在新的显式反馈时释放完整权威日志前缀。DISABLED 是插件默认值（dsh-session-telemetry-otel README），且不构造上报流水线；删除配置项是静默退出方式，而禁用模式保留本地反馈警告。SDK 定时刷新和关闭只排空已授权批次，不捕获后续记录。未挂载规则的部署会导出每条已捕获事件的正文，包括紧凑 assistant stream 以及文件内容或命令输出中内嵌的凭据，因此跨信任边界的部署必须挂载 session-telemetry/record 监听器。脱敏后的正文可能与权威日志字节不同；日志仍是真源。丢失交接游标可能使后续已授权捕获产生重复，崩溃持久性则在上述 outbox 决定重新审议前继续不在范围内。