MCP client auto-reconnect with bounded backoff
MCP client auto-reconnect with bounded backoff
The [MCP client](2026-07-07-mcp-client-plugin.md) connected once at plugin load. When a stdio server crashed or was killed, its registered tools stayed visible but every call failed with `Not connected` until a human edited the config (HMR) or restarted the Host — v1 explicitly deferred reconnection. Long-running hosts (ACP automation, web) cannot be bounced because a child process died, and for stdio the harness com
English
Problem
The MCP client connected once at plugin load. When a stdio server crashed or was killed, its registered tools stayed visible but every call failed with Not connected until a human edited the config (HMR) or restarted the Host — v1 explicitly deferred reconnection. Long-running hosts (ACP automation, web) cannot be bounced because a child process died, and for stdio the harness composition is the only party that can respawn it. External feedback escalated this as a real operational gap (issue #1746).
Decision
packages/mcp/mcp-client/src/connection.ts owns a per-instance connection supervisor; apply() shrinks to config resolution plus two effects (the serverName reservation and the supervisor's lifecycle). The supervisor owns the client/transport generations, the live tool registrations, and the reconnect loop.
Trigger. The supervisor arms client.onclose per generation. The SDK fires it when the stdio child exits, so a crash is observed without polling. StreamableHTTPClientTransport fires onclose only for deliberate closes — it owns its internal SSE-stream recovery and surfaces request failures per call — so HTTP servers are effectively outside supervisor restarts; the package README records that limitation.
Generations without interleaving. Each attempt builds a fresh transport and Client (the SDK binds a Protocol to one transport for life). One per-supervisor queue serializes every syncTools call — initial syncs and list_changed re-syncs across all generations — and an isCurrent fence makes stale generations inert, so no two syncs can interleave the dispose-previous/register-next swap (which would double-dispose one generation and leak another). The queue also closes a pre-existing race where two rapid list_changed notifications re-synced concurrently. The activation attempt, rather than the first queue entrant, explicitly owns strict startup registration: an early list_changed notification uses contained re-sync semantics and cannot consume failOnStartupError. Failure signals are idempotent per generation: a connect rejection racing its own transport close schedules exactly one retry. A failed attempt cannot enter backoff until both Client.close() settles and the transport reports onclose, which for stdio proves the child exited; a missing close signal stops reconnection after the SDK's bounded termination window instead of allowing two server processes to overlap. Disposal uses the same bounded close-signal barrier and reports an incomplete shutdown without ever restarting.
Bounded backoff with an outage budget. Delays double from initialDelayMs up to maxDelayMs. One outage shares maxAttempts consecutive failed attempts; exhaustion unregisters the server's tools, logs at error level, and stops until disposal or reload. A connection that survives past the stability window — maxDelayMs, derived rather than a fifth tunable, as the longest configured backoff spacing — resets the budget, so an occasionally-crashing server recovers indefinitely while a crash loop whose connects briefly succeed cannot launder its budget into a restart storm.
Config and resolution. Both transports accept reconnect { enabled, initialDelayMs, maxDelayMs, maxAttempts } with schemastery defaults (on, 500ms, 30s, 10). resolveReconnectPolicy() is the explicit resolve step: it re-judges every bound and cross-field constraint because programmatic construction may bypass Schemastery, and misconfiguration fails the plugin instance at load.
Observable states. An initial or retry-attempt failure says connection failed; an established generation ending says connection lost. Retrying logs at warn with attempt count and delay, recovery at info, final failure and disabled recovery at error. During an outage the last good generation stays registered and calls against it fail — deterministic public names mean a recovered unchanged tool list reproduces identical definitions, keeping the model-visible schema prefix stable instead of flapping. With reconnect.enabled: false a lost connection keeps the v1 manual-recovery behavior.
Disposal. Dispose flips the fence, cancels any pending timer, closes the current client, then awaits the in-flight attempt and the sync queue before unregistering — quiescence, not just a request to stop. The reconnect timer is unref'd so a waiting backoff never holds a finishing process open.
Alternatives considered
Consecutive-failure counter that resets on every successful connect. Rejected: a crash-looping server whose connects briefly succeed would reset the budget each cycle and restart forever — exactly the restart storm the failure cap exists to prevent. The uptime-gated reset distinguishes a recovered server from a looping one without new configuration.
Reuse one SDK Client across reconnects. The Protocol clears its transport on close and can technically connect again, but the SDK's own guidance is one connection per Protocol instance, and reuse carries notification handlers and negotiated capability state across server incarnations. A fresh Client per generation plus the isCurrent fence is unambiguous.
Unregister tools immediately on disconnect, re-register on recovery. Rejected: a transient outage would flap the model-visible tool list (two schema-prefix invalidations per crash) for no information gain; failing calls already signal the outage, and the swap on recovery is atomic per generation. Tools are unregistered at final failure so a permanently dead server does not leak permanently broken tools.
Route Streamable HTTP request failures into the supervisor. Rejected for now: the HTTP transport already reconnects its SSE stream with its own backoff, per-request errors do not imply a dead server, and there is no child process the harness could respawn. Transport close stays the single trigger.
Restart through Loader/HMR machinery instead of an in-plugin supervisor. Rejected: the Loader owns config-driven recomposition, not runtime health. A plugin restarting itself through the Loader would conflate config generations with connection generations and lose the per-outage budget.
Testing
Unit (tests/reconnect.spec.ts, mocked SDK): recovery swaps generations without duplication or leaks and serves post-recovery calls, diagnostics distinguish initial or retry failure from established connection loss, strict startup registration survives a pre-connect list_changed notification, failed initialization waits for the old generation's close signal and fails closed when that signal never arrives, disposal waits for the same signal with a bounded incomplete-shutdown path, the failure cap unregisters tools and stops, dispose cancels a pending backoff and quiesces an in-flight sync, a close after dispose schedules nothing, disabled mode keeps the v1 behavior, the stability window resets the budget while a crash loop exhausts it, double failure signals schedule one retry, stale generations and handlers are inert, and resolveReconnectPolicy rejects each invalid bound. E2E (tests/mcp-client.e2e.ts, keyless): the fixture server gained a crash tool that replies then exits; real-process tests prove a stdio crash recovers end to end and that unloading the plugin mid-outage stops reconnection promptly. Snapshot: deliberately none, per the original note's rationale — reconnection adds no new presentation shape, and a snapshot composition spawning a crashing server would make replays timing-dependent.
Consequences
- A crashed stdio MCP server recovers without human intervention: bounded backoff, re-discovery, atomic generation swap. Default policy retries an outage for roughly 2.5 minutes before giving up.
- Connection state is genuinely more intricate than connect-once — the partial-availability window v1 avoided now exists (registered tools failing during an outage), concentrated in one module with the invariants named.
reconnectis new config surface on both transports, and the stability window is deliberately derived frommaxDelayMs; making it independently tunable is a compatible future change.- After final failure or with reconnect disabled, the plugin stays loaded with no (or failing) tools until reload — deliberate and logged, so a chronically broken server cannot restart forever.
中文
问题
MCP 客户端在插件加载时仅连接一次。stdio 服务器崩溃或被终止后，其已注册的工具仍然可见，但每次调用均以 Not connected 失败，直到人工编辑配置触发 HMR（热模块替换）重载，或重启 Host——v1 明确推迟了重连机制。长时间运行的 Host（ACP 自动化、Web）不能因为子进程死亡就被重启；而对于 stdio 传输，harness 组合层是唯一能重新拉起子进程的一方。外部反馈将此升级为真实的运维缺口（issue #1746）。
决策
packages/mcp/mcp-client/src/connection.ts 拥有一个逐实例的连接监督器；apply() 收缩为配置解析加两个副作用（serverName 预留和监督器的生命周期）。监督器负责管理 client/transport 代、活跃的工具注册以及重连循环。
触发条件。 监督器在每一代上挂载 client.onclose。SDK 在 stdio 子进程退出时触发该回调，因此崩溃无需轮询即可感知。StreamableHTTPClientTransport 仅在主动关闭时触发 onclose——它内部拥有自己的 SSE（Server-Sent Events）流恢复机制，并将请求失败以逐调用方式暴露——因此 HTTP 服务器实际上不在监督器的重启范围内；包 README 记录了该限制。
代隔离，无交错。 每次尝试构建一个全新的 transport 和 Client（SDK 将一个 Protocol 绑定到一个 transport 上终身使用）。每个监督器内部有一个队列将所有 syncTools 调用串行化——跨所有代的初始同步和 list_changed 再同步——isCurrent 栅栏使过时的代变为惰性，从而确保不会有两次同步交错执行 dispose 上一代/注册下一代的切换（否则会对同一代执行两次 dispose 并泄漏另一代）。该队列还消除了一个先前存在的竞态：两次快速的 list_changed 通知同时触发重新同步。严格启动注册由激活尝试本身显式拥有，而非由首个入队者拥有；提前到达的 list_changed 采用故障隔离的再同步语义，不能消费 failOnStartupError。失败信号按代幂等：一次连接拒绝与其自身 transport 关闭竞态时，仅调度恰好一次重试。失败尝试只有在 Client.close() 结算且 transport 报告 onclose 后才能进入退避；对 stdio 而言，onclose 证明子进程已退出；若关闭信号始终未到，则在 SDK 的有界终止窗口结束后停止重连，而不是允许两个服务器进程重叠运行。dispose 使用同一个有界关闭信号屏障；若关停未完成则予以报告，且绝不重启。
有界退避与故障预算。 延迟从 initialDelayMs 起逐次翻倍，上限为 maxDelayMs。一次故障期间共享 maxAttempts 次连续失败尝试的预算；耗尽后注销该服务器的工具、以 error 级别记录日志并停止，直到 dispose 或重新加载。连接在存活超过稳定窗口——即 maxDelayMs，作为最长退避间隔从配置推导得出而非作为第五个独立调参项——之后重置预算；因此偶尔崩溃的服务器可无限恢复，而连接短暂成功后立即再次崩溃的循环无法将其预算洗白为重启风暴。
配置与解析。 两种传输均接受 reconnect { enabled, initialDelayMs, maxDelayMs, maxAttempts } 配置，Schemastery 默认值为（启用、500ms、30s、10）。resolveReconnectPolicy() 是显式的解析步骤：它重新校验每个边界值和跨字段约束，因为程序化构造可能绕过 Schemastery，配置错误在加载时即令插件实例失败。
可观测状态。 初始尝试或重试尝试失败时记录 connection failed，已建立的代结束时记录 connection lost；重试的 warn 日志包含尝试次数和延迟，恢复以 info 级别记录，最终失败和禁用重连时的断连以 error 级别记录。故障期间，上一个正常代保持注册，对其工具的调用返回失败——确定性公开名称意味着恢复后未变化的工具列表会复现相同的定义，保持模型可见 schema 前缀稳定而非反复抖动。设置 reconnect.enabled: false 后，断连保持 v1 的手动恢复行为。
资源释放。 dispose 翻转栅栏、取消待执行的定时器、关闭当前 client，然后等待正在进行的尝试和同步队列完成后再注销工具——完全停稳，而非仅发出停止请求。重连定时器使用 unref，因此等待中的退避不会阻止进程正常退出。
曾考虑的替代方案
连续失败计数器，每次成功连接即重置。 否决：连接短暂成功后立即崩溃的循环服务器会在每个周期重置预算并永远重启——恰恰是失败上限旨在防止的重启风暴。基于运行时间的重置能区分已恢复的服务器与循环崩溃的服务器，无需新增配置。
跨重连复用同一个 SDK Client。 Protocol 在关闭时清除其 transport，技术上可以再次连接，但 SDK 自身的指导方针是每个 Protocol 实例对应一次连接，且复用会将通知处理器和已协商的能力状态带入新的服务器实例。每代创建全新 Client 加 isCurrent 栅栏的方式无歧义。
断连时立即注销工具，恢复时重新注册。 否决：短暂故障会使模型可见工具列表抖动（每次崩溃触发两次 schema 前缀失效），而无任何信息增益；失败的调用已足以标示故障，恢复时的切换按代原子执行。工具仅在最终失败时注销，确保永久死亡的服务器不会泄漏永久失效的工具。
将 Streamable HTTP 请求失败路由到监督器。 暂不采纳：HTTP 传输已使用自己的退避机制重连其 SSE 流，逐请求错误并不意味着服务器已死，且 harness 没有可重新拉起的子进程。transport 关闭仍是唯一触发条件。
通过 Loader/HMR 机制重启，而非使用插件内监督器。 否决：Loader 负责配置驱动的重组合，而非运行时健康管理。插件通过 Loader 重启自身会混淆配置代与连接代，并丢失逐故障预算。
测试
单元测试（tests/reconnect.spec.ts，mock SDK）：恢复在不产生重复或泄漏的前提下切换代并服务恢复后的调用、诊断区分初始或重试尝试失败与已建立连接丢失、严格启动注册在连接前收到 list_changed 通知后仍然生效、初始化失败会等待旧代的关闭信号，若该信号始终未到则停止重连、dispose 同样等待同一关闭信号，并在有界等待到期时报告关停未完成、失败上限注销工具并停止、dispose 取消待执行的退避并使进行中的同步完全停稳、dispose 后的关闭不调度任何操作、禁用模式保持 v1 行为、稳定窗口重置预算而崩溃循环耗尽预算、双重失败信号仅调度一次重试、过时的代和处理器为惰性、resolveReconnectPolicy 拒绝每个无效边界值。E2E（tests/mcp-client.e2e.ts，无需密钥）：fixture 服务器新增了一个 crash 工具（先回复再退出）；真实进程测试证明 stdio 崩溃端到端恢复，以及在故障期间卸载插件能立即停止重连。快照：刻意不做，原因与原 Agent Note 相同——重连不引入新的展示形态，而在快照组合中 spawn 崩溃服务器会使回放依赖时序。
后果
- 崩溃的 stdio MCP 服务器无需人工干预即可恢复：有界退避、重新发现、原子代切换。默认策略对一次故障大约重试 2.5 分钟后放弃。
- 连接状态确实比一次性连接更复杂——v1 刻意回避的部分可用窗口现已存在（故障期间已注册工具返回失败），集中在一个模块中并命名了所有不变式。
reconnect是两种传输上的新配置表面，稳定窗口刻意从maxDelayMs推导；将其设为独立可调参数是兼容的未来变更。- 最终失败后或禁用重连时，插件保持加载状态但无（或失败的）工具，直到重新加载——行为是刻意的且有日志记录，确保长期故障的服务器不能永远重启。