DSH / Atlas
2026-09-06implementedtestingarchived 2026-09-30

Bounded retry for Python runtime dependency install

Python runtime 依赖安装的有界重试

The Python runtime lane's `Install (immutable)` step runs `pnpm install` on every target, and install-time native build downloads fetch Node headers from nodejs.org. That endpoint stalls intermittently: on 2026-09-06 the hosted `node24-macos-x64` cell failed when the `fs-ext` build's node-gyp download raised `ConnectTimeoutError` against nodejs.org after a 10-second connect timeout, aborting the immutable install. Th

English

Problem

The Python runtime lane's Install (immutable) step runs pnpm install on every target, and install-time native build downloads fetch Node headers from nodejs.org. That endpoint stalls intermittently: on 2026-09-06 the hosted node24-macos-x64 cell failed when the fs-ext build's node-gyp download raised ConnectTimeoutError against nodejs.org after a 10-second connect timeout, aborting the immutable install. The stall is external and transient; the lane previously had no recovery beyond a human job rerun.

Decision

The install step retries pnpm install --frozen-lockfile up to three attempts total with a ten-second pause between failures, running under bash on every platform (Git Bash is on the hosted Windows images). Success on any attempt ends the step immediately; a file-lock check or native-build error that would fail every attempt still fails the step after the bounded budget. This mirrors the Wine lane's documented bounded-transfer policy without pulling in a mirror, because these installs also resolve native addons whose second-download source matters.

Alternatives considered

Increase the connect or job timeout. Rejected: the observed stall is a connect timeout after 10 seconds, and retrying the whole operation with a fresh connection is the recovery the failure mode calls for; a longer timeout still fails when the endpoint is down.

Use a mirror for Node header downloads. Deferred: the Wine lane's mirror resumes its own archive; the Python runtime lane would need a per-target mirror and its own checksum authority, which the retry does not require for a transient outage.

Rerun failed jobs by hand. Rejected as the lane's standing remediation: it costs a full lane cycle and stays manual; the bounded retry absorbs the transient while a sustained outage still fails loudly.

Consequences

A transient nodejs.org stall costs at most two extra install attempts (about twenty seconds), while a deterministic install defect still fails after the budget. All targets share the same retry path, and install diagnostics remain the pnpm output captured inside the step.

中文

问题

Python runtime 通道的 Install (immutable) 步骤在每个目标上运行 pnpm install,安装期原生构建下载会从 nodejs.org 获取 Node 头文件。该端点会间歇性停滞:2026-09-06,托管的 node24-macos-x64 单元在 fs-ext 构建的 node-gyp 下载对 nodejs.org 抛出 10 秒连接超时后的 ConnectTimeoutError 时失败,使不可变安装中止。该停滞属于外部、瞬时问题;此前通道除人工重跑作业外没有任何恢复手段。

决策

安装步骤对 pnpm install --frozen-lockfile 最多重试三次(共三次尝试,失败间隔十秒),并在每个平台上以 bash 运行(托管 Windows 镜像自带 Git Bash)。任意一次成功立即结束步骤;每次尝试都会失败的文件锁检查或原生构建错误仍会在有界预算后使步骤失败。这借鉴了 Wine 通道已记录的有界传输策略,而不引入镜像,因为这些安装还会解析原生 addon,其二次下载来源同样重要。

已考虑的替代方案

提高连接或作业超时。 不采用:观察到的停滞是 10 秒后的连接超时,而对该失败模式适用的恢复是携带新连接的整体重试;端点故障时更长的超时仍然失败。

为 Node 头文件下载使用镜像。 推迟:Wine 通道的镜像只续传自己的归档;Python runtime 通道需要按目标配置镜像及自己的校验权威,而瞬态故障的重试并不需要这些。

人工重跑失败作业。 不采用为该通道的常设补救:它消耗一整个通道周期且停留在手动层面;有界重试吸收瞬态故障,持续性故障仍会响亮失败。

后果

一次 nodejs.org 瞬态停滞最多付出两次额外安装尝试(约二十秒),确定性的安装缺陷仍会在预算后失败。所有目标共享同一重试路径,安装诊断仍为步骤内捕获的 pnpm 输出。