Time-sliced Session-list work
按时间片执行 Session 列表工作
Large cached projection states can monopolize the Host while a Session list is assembled. Yielding after every row avoids that batch stall but adds thousands of event-loop round trips when a list contains many cheap rows. Row count alone is not a useful scheduling budget.
English
Problem
Large cached projection states can monopolize the Host while a Session list is assembled. Yielding after every row avoids that batch stall but adds thousands of event-loop round trips when a list contains many cheap rows. Row count alone is not a useful scheduling budget.
Decision
ApiSessionList uses one monotonic deadline for classification and summary generation. It yields only after elapsed work exhausts the configured listWorkSliceMs, then starts a new deadline after the wait. The optional deployment setting defaults to 16 positive integral milliseconds. Small lists have no forced final yield or no-op asynchronous helper. Cancellation is checked per row, after each yield, and before returning; no partial list is returned.
Every row remains synchronous. Live rows precede queued cold rows before the stable activity sort; the list promises no cross-Session atomic cut. A cold row already queued before an attachment remains cached, and Client mutation replay plus cached/sequenced precedence reconcile the newer live state. No projection data or validation is omitted.
Measurement and calibration
The production-entry benchmark measures the real Session Controller, query engine, persistence, projection registry/cache and storage domain. Persisted fixtures use generationLogPath, toHeaderLine and compressZstdFrame from the persistence owner. All content is deterministic generated data. Setup and explicit GC are outside timing; complete results and JSON remain reachable during memory sampling.
| Workload | Sessions | State observations | Request records | State JSON |
|---|---|---|---|---|
| Cheap live | 3,000 | 0 | 0 | 277,890 bytes |
| Modest persisted | 50 | 4,925 | 641 | 1,813,852 bytes |
| Heavy persisted | 300 | 300,000 | 37,575 | 111,255,440 bytes |
The cheap live case avoids disk enumeration masking scheduling cost. Persisted row counts use 50/75/100/175% strata of 100 or 1,000 generated observations. Each fresh child measures one first call and three repeats with a single fixed-cost immediate probe per call. A separate fifth call re-arms at most one pending probe to measure the worst queue delay over the complete list. Full-probe timings are not throughput evidence: per-row yielding would otherwise execute proportionally more diagnostic callbacks. Yield counts exclude post-measurement GC helpers. Peak RSS includes setup and the entire process lifetime.
Local Node 26.5.0/macOS arm64 comparisons alternate implementations and use three fresh children per workload. Repeat metrics first take each child's median, then the median across children. Queue delay takes the maximum in each separate diagnostic call, then the median across children.
| Repeated metric | Synchronous | Per-row yield | 16 ms slices |
|---|---|---|---|
| Cheap live: list + JSON | 61.60 ms | 111.65 ms | 65.68 ms |
| Cheap live: yield calls | 0 | 3,000 | 3 |
| Cheap live: maximum queue delay | 54.35 ms | 0.54 ms | 16.54 ms |
| Modest persisted: list + JSON | 19.76 ms | 18.23 ms | 18.26 ms |
| Modest persisted: yield calls | 0 | 50 | 0 |
| Heavy persisted: list + JSON | 253.22 ms | 291.36 ms | 232.18 ms |
| Heavy persisted: yield calls | 0 | 300 | 10 |
| Heavy persisted: maximum queue delay | 172.39 ms | 0.80 ms | 16.67 ms |
For cheap lists, slicing removes 99.9% of per-row yields and reduces measured repeat time by 41.2%; the synchronous reference remains slightly faster. Heavy-list wall time varies substantially on the shared host, so no universal throughput speedup is asserted. Retained heap stays about 35 MB for cheap lists and 186–188 MB for heavy lists, without a decoded-state cache.
| Raw child aggregates | Synchronous | Per-row yield | 16 ms slices |
|---|---|---|---|
| Cheap repeat ms | [63.10, 61.60, 59.08] | [108.19, 115.61, 111.65] | [65.68, 71.75, 65.17] |
| Cheap maximum queue ms | [112.69, 54.35, 49.67] | [0.54, 0.76, 0.41] | [16.41, 16.54, 41.15] |
| Heavy repeat ms | [277.06, 253.22, 236.22] | [291.36, 302.76, 282.01] | [212.35, 232.18, 304.38] |
| Heavy maximum queue ms | [297.76, 172.39, 164.17] | [0.56, 1.34, 0.80] | [16.39, 16.67, 16.76] |
Separate 4/8/16 ms experiments measured cheap-list repeat medians of 58.39/58.57/52.34 ms and heavy-list medians of 279.81/253.85/216.67 ms. Their heavy-list maximum queued-work delay medians were 5.19/8.45/16.70 ms. The 16 ms default chooses fewer round trips while retaining a short work target; deployments can select a smaller budget.
CI uses elapsed queue-delay and coarse throughput limits, not an exact batch size. The shared scale 2 and headroom 1.25 turn a 30 ms queue reference allowance into 75 ms, leaving room for one-row overshoot and scheduling variance while rejecting an unyielded heavy batch. Throughput reference allowances are 50/500/150 ms for modest/heavy/cheap, producing 125/1,250/375 ms bounds. These are safety ceilings, not claims of fine-grained non-regression. The earlier Linux Node 24.21 EPYC run measured 102.16/779.57 ms first calls for modest/heavy cases; the larger heavy allowance accounts for the observed hosted/reference ratio exceeding 2. Heap uses 240 MiB and only variance headroom, giving 300 MiB.
The existing 3,000-Session corpus lane remains the stored-corpus throughput authority. Its earlier per-row CI run measured first-call samples 3,205.3/3,166.2/3,165.1 ms and repeat samples 2,643.5/2,577.8/2,589.6 ms, within its existing calibrated limits. The projection benchmark adds a distinct cheap-live and large-state dimension rather than replacing that corpus. On the same local machine, the stored-corpus per-row/sliced first medians are 885.5/859.4 ms and repeat medians are 700.3/692.1 ms; that small difference is within ordinary run variation, with no observed throughput regression. The new three-workload file takes 23.98 s locally, including fixture setup and all nine children; its previously measured two-workload CI version took 36.80 s on EPYC Node 24.21.0.
Alternatives considered
Unconditional per-row yielding. It minimizes individual queue waits but charges an event-loop round trip for every cheap row. The many-live-Session comparison exposes that cost; a fixed number of rows would still ignore variable row cost.
Larger or smaller time slices. Smaller slices offer shorter queue waits at more scheduling cost. Larger slices reduce switching but extend the period other Host work waits. Configuration exposes this tradeoff without changing row semantics.
Caching decoded state or wire views. Decoded-state reuse retains another graph per stored row and changes schema-evaluation ownership. Wire views can depend on current runtime state. Scheduling needs neither cache and preserves all schema checks and dynamic views.
Consequences
Controlled-clock tests require cheap rows to remain in one synchronous slice, budget exhaustion and post-wait reset to govern yields, and cancellation to stop work even when no yield is due. They also preserve stable ties and attachment/removal/status interleavings. The time-based benchmark guard rejects unyielded or first-only-yield heavy work without forbidding efficient batches.
A single row, provider enumeration, GC, final sorting and serialization can exceed the target. Concurrent lists and operating-system scheduling also affect latency. The budget is not a hard global bound, and these measurements do not claim browser paint or model/network latency improvements.
中文
问题
较大的缓存投影状态会在组装 Session 列表时独占 Host。每行都让出可以避免整批停顿，却会在包含大量低开销行的列表上增加数千次事件循环往返。行数本身不是合适的调度预算。
决策
ApiSessionList 对分类和摘要生成共享一个单调时钟截止时间。仅当经过的工作时间耗尽配置的 listWorkSliceMs 时才让出，并在等待结束后建立新的截止时间。这个可选部署设置默认为 16 毫秒，要求正整数。小列表没有强制的最后一次让出，也不经过空操作异步 helper。每行、每次让出之后以及返回前检查取消；不会返回部分列表。
每行仍同步计算。live 行先于已进入队列的 cold 行，随后按活动时间稳定排序；列表不承诺跨 Session 的原子 cut。接入前已经排入队列的冷行仍是缓存行，Client 变更重放与 cached/sequenced 优先级协调较新的 live 状态。不省略任何投影数据或校验。
测量与校准
生产入口基准测量真实 Session Controller、查询引擎、持久化、投影注册表与缓存，以及 storage domain。持久化夹具使用持久化拥有方的 generationLogPath、toHeaderLine 和 compressZstdFrame。全部内容都是确定性生成数据。准备和显式 GC 不计入耗时；内存采样时完整结果与 JSON 仍可达。
| 负载 | Session 数 | 状态观测数 | 请求记录数 | 状态 JSON |
|---|---|---|---|---|
| 低开销驻留 | 3,000 | 0 | 0 | 277,890 bytes |
| 适中持久化 | 50 | 4,925 | 641 | 1,813,852 bytes |
| 大状态持久化 | 300 | 300,000 | 37,575 | 111,255,440 bytes |
低开销驻留场景避免磁盘枚举掩盖调度成本。持久化行按 100 或 1,000 条生成观测的 50/75/100/175% 分层。每个全新子进程测量一次首次调用和三次重复调用，每次只使用一个固定成本的 immediate 探针。独立的第五次调用以最多一个待处理探针反复采样，测量完整列表中的最差排队延迟。完整探针模式的耗时不作为吞吐量证据，否则逐行让出会按比例执行更多诊断回调。yield 次数不包含测量后 GC helper 的调用。峰值 RSS 包含准备阶段及整个进程生命周期。
本地比较使用 Node 26.5.0/macOS arm64，交替运行实现，每种负载使用三个全新子进程。重复指标先取每个子进程的中位数，再取子进程间的中位数。排队延迟先取独立诊断调用的最大值，再取子进程间的中位数。
| 重复读取指标 | 同步 | 逐行让出 | 16 ms 时间片 |
|---|---|---|---|
| 低开销驻留：列表与 JSON | 61.60 ms | 111.65 ms | 65.68 ms |
| 低开销驻留：yield 次数 | 0 | 3,000 | 3 |
| 低开销驻留：最大排队延迟 | 54.35 ms | 0.54 ms | 16.54 ms |
| 适中持久化：列表与 JSON | 19.76 ms | 18.23 ms | 18.26 ms |
| 适中持久化：yield 次数 | 0 | 50 | 0 |
| 大状态持久化：列表与 JSON | 253.22 ms | 291.36 ms | 232.18 ms |
| 大状态持久化：yield 次数 | 0 | 300 | 10 |
| 大状态持久化：最大排队延迟 | 172.39 ms | 0.80 ms | 16.67 ms |
在低开销列表中，时间片去掉了 99.9% 的逐行 yield，并使测得的重复耗时下降 41.2%；同步参考仍略快。共享 Host 上大状态列表的墙钟耗时波动明显，因此不宣称普遍提高吞吐量。常驻堆在低开销场景约为 35 MB，在大状态场景约为 186–188 MB，没有解码状态缓存。
| 原始子进程聚合值 | 同步 | 逐行让出 | 16 ms 时间片 |
|---|---|---|---|
| 低开销重复 ms | [63.10, 61.60, 59.08] | [108.19, 115.61, 111.65] | [65.68, 71.75, 65.17] |
| 低开销最大排队 ms | [112.69, 54.35, 49.67] | [0.54, 0.76, 0.41] | [16.41, 16.54, 41.15] |
| 大状态重复 ms | [277.06, 253.22, 236.22] | [291.36, 302.76, 282.01] | [212.35, 232.18, 304.38] |
| 大状态最大排队 ms | [297.76, 172.39, 164.17] | [0.56, 1.34, 0.80] | [16.39, 16.67, 16.76] |
独立的 4/8/16 ms 实验测得，低开销列表重复耗时中位数分别为 58.39/58.57/52.34 ms，大状态列表为 279.81/253.85/216.67 ms。大状态列表最大排队延迟的中位数分别为 5.19/8.45/16.70 ms。16 ms 默认值在保持较短工作目标的同时减少往返；部署可以选择更小的预算。
CI 使用经过时间的排队延迟与粗粒度吞吐量上限，而不是精确批大小。共享的 2 倍时间系数和 1.25 倍余量将 30 ms 排队参考余量转换为 75 ms，为单行超时和调度波动留出空间，同时拒绝没有让出的大状态批次。适中、大状态、低开销三种负载的吞吐量参考余量分别为 50/500/150 ms，得到 125/1,250/375 ms 上限。这些是安全上限，不证明细粒度不回退。先前 Linux Node 24.21 EPYC 运行测得适中和大状态场景首次调用为 102.16/779.57 ms；较大的大状态余量考虑了实际 hosted/reference 比值超过 2。堆使用 240 MiB，并只乘波动余量，得到 300 MiB。
已有的 3,000 Session 会话集合测试仍是持久化集合吞吐量的依据。先前逐行版本的 CI 首次调用样本为 3,205.3/3,166.2/3,165.1 ms，重复调用为 2,643.5/2,577.8/2,589.6 ms，均在既有校准上限内。投影基准补充低开销驻留和大状态维度，不替代该集合。在同一本地机器上，持久化集合的逐行/时间片首次中位数为 885.5/859.4 ms，重复中位数为 700.3/692.1 ms；这点差异处于普通运行波动内，未观察到吞吐量回退。新的三负载文件在本地耗时 23.98 s，包含夹具准备和全部九个子进程；先前测得的双负载 CI 版本在 EPYC Node 24.21.0 上耗时 36.80 s。
考虑过的替代方案
无条件逐行让出。 它尽量缩短单次排队等待，却对每个低开销行都收取一次事件循环往返成本。大量驻留 Session 的比较暴露了这项成本；固定行数同样忽略了行成本差异。
更大或更小的时间片。 更小的时间片缩短排队等待，但增加调度成本。更大的时间片减少切换，却延长其他 Host 工作的等待。配置显式提供这一取舍，不改变行语义。
缓存解码状态或 wire view。 解码状态复用会为每份存储行保留另一份对象图，并改变 schema 执行的所有权。wire view 可能依赖当前运行状态。调度不需要这两类缓存，且保留全部 schema 校验和动态视图。
后果
受控时钟测试要求低开销行保持在同一同步时间片内，由预算耗尽和等待后重置决定让出，并在无需让出时也能响应取消。测试还保留了稳定并列排序和接入、移除、状态变更的交错行为。按时间判断的基准保护会拒绝不让出或仅首次让出的大状态工作，同时允许高效批处理。
单行、提供方枚举、GC、最终排序和序列化都可能超出目标。并发列表和操作系统调度也会影响延迟。预算不是全局硬上限，这些测量也不宣称改善浏览器绘制或模型、网络延迟。