A pi-ai model declares its own input modalities, and undeclared means text
pi-ai 模型自行声明输入模态，未声明即为文本
Nothing in `settings.yaml` could describe a hand-declared pi-ai model as accepting images, and the adapter assumed text-only for every model the installed pi-ai catalog does not describe. Every model a deployment adds through the web UI's "add a custom provider" card is such a model, so an OpenAI-compatible gateway serving a vision model reported `inputModalities: ['text']` no matter what it actually served. The harn
Source compatibility: Chinese title uses a compatible non-canonical header
English
Problem
Nothing in settings.yaml could describe a hand-declared pi-ai model as accepting images, and the adapter assumed text-only for every model the installed pi-ai catalog does not describe. Every model a deployment adds through the web UI's "add a custom provider" card is such a model, so an OpenAI-compatible gateway serving a vision model reported inputModalities: ['text'] no matter what it actually served.
The harness treats an omitted modality as negative capability, and three admission points act on it before any request is built: model selection refuses to switch into a session that already holds images, prompt admission refuses an image, and read_image refuses to read one. Their diagnostics tell the user to select an image-capable model — advice with no reachable referent, because no configuration key could make a hand-declared model image-capable. The route was closed at the metadata, not at the capability: the request converter and every pi-ai wire protocol carry images, and llm-pi-ai's own stream guard is the only thing that would have stopped one.
The assumption was justified in the source as the adapter's real capability rather than a deployment choice, and [[2026-08-03-pi-ai-declared-provider-catalog]] recorded the same reasoning when it decided which Model fields the configuration surface would expose ("nothing reads them: … context.ts keeps only text blocks"). That justification described the DeepSeek chat-completions adapter, whose serializer genuinely rejects image blocks, and had never been true of the pi-ai route. This note supersedes that one on modalities alone; pricing stays closed there for its own, still-current reason.
Decision
Modalities resolve entry input → installed catalog entry → route defaultInput, which itself defaults to [text]. That is the chain contextWindow and maxTokens already use, field for field. pi-ai types Model.input required and per-model, so the entry field mirrors upstream directly: one route can serve a vision model beside a text-only one, and an override can correct a catalog model whose gateway serves other modalities than the catalog records. The route field spares a gateway whose undescribed models all take images from repeating itself on every entry.
The route value is a fallback, not an override — the catalog outranks it. This is the default* ordering rather than compat's, and the two are not interchangeable: compat shadows the catalog because a route-level protocol repoint invalidates the catalog's reasoning-dispatch facts wholesale, while a modality is a per-model property the catalog states accurately for the models it ships. Making the route value win would mean defaultInput: [text] silently strips images from every catalog vision model on the route — a footgun with no matching benefit, since narrowing one such model is what that model's own input is for.
Undeclared means [text], and that is the absence of a declaration rather than a guess at the endpoint. Nothing can interrogate a gateway for its modalities because no OpenAI-compatible listing endpoint reports them. The only safe floor is the modality every supported protocol certainly carries. Under-claiming refuses the image before it is attached, names the model, and has a documented configuration remedy. Over-claiming admits and persists an image before the provider can reject it. Later requests to that same incorrectly declared route will encounter the image again, although the user can select a text-only model because request assembly projects durable images to placeholders.
An entry's empty list means the same as an absent one; the route's is refused. [] describes a model that accepts nothing and could serve no request, so it states no answer and resolution continues past it. That reading is not cosmetic: the config schema materializes [] for an absent array, so treating it as "accepts nothing" would silently strip images from every catalog vision model a models list happens to name. The route value has nothing below it to answer instead, so its empty list is refused where it is written. The route's models list already resolves absent-and-empty the same way for the same reason.
No configuration surface edits input. It joins compat, reasoningEfforts, thinkingBudgets, and headers as a settings-document field, and the model-list editor stays a hand-written form over id, name, and the two capacities. This costs nothing durable because that card was already built to carry fields it does not edit: its row patch spreads the stored row before applying changes, and adoption keeps an existing row over a rediscovered candidate, so a hand-written input survives both.
The direct DeepSeek adapter owns a separate exact-model catalog. Its supported vision entry declares image input, while its text models and unlisted pass-through ids remain text-only.
Alternatives considered
- An optimistic
[text, image]default — makes the motivating case work with zero configuration, and the web form writes no modality at all, so a conservative default leaves the remedy in the settings document. Rejected because a false positive persists an image before the provider refuses it and causes repeated failure on that route. Text-only request projection provides recovery but does not make the declaration true. - A route value that overrides the catalog (
compat's ordering: entry → route → catalog) — lets a deployment that repoints a catalog route at its own gateway declare "no vision here" once. Rejected because the same sentence then silently disables every catalog vision model on a route where someone wrote it by analogy with the capacity fields, and the legitimate case is served by that model's owninput. An override would also have to be namedinputat the route, since calling itdefault*beside two genuine fallbacks would misdescribe it. - No route field at all, only the entry one — closest to upstream, which has no route-level concept. Rejected on the bulk case the product's own flow produces: "fetch available models" adopts thirty ids with no modality, and an all-vision gateway would need
inputhand-written on each. - A route-level
defaultInputwith no entry field — cannot mix modalities on one route or correct a single catalog model, leaving "split the provider across two route keys" as the only workaround, at the cost of a second permanent provider id and a duplicate entry in every model selector. - Probe the endpoint for its modalities — no OpenAI-compatible listing endpoint reports them.
- Infer from the model id (
*-vision,*-vl) — a naming convention is not a capability, and a gateway renames freely. - Keep refusing and improve the diagnostic — the message was already accurate about the state and useless about the remedy; the missing thing was the remedy.
Consequences
A vision model on a custom provider costs one line, input: [text, image], written in the settings document — or one line at the route when every model it lists takes images. That is the whole of the fix: the three admission points then admit images on it and read_image works. A deployment that writes nothing keeps exactly the behavior it had, so no existing route changes what it reports.
The image-admission gate keeps its meaning everywhere, because every modality it reads is now either recorded by the installed catalog or written by a person. Nothing claims a capability on a deployment's behalf.
A model that declares image input its endpoint does not serve is not caught locally because the claim is not verified. Prompt admission commits the user message durably before request construction, so the rejected image stays in the session log and later requests to that route can fail again. Recovery is to correct the declaration, select an image-capable route, or select a text-only route whose request projection replaces durable images with placeholders.
Testing
packages/llm/llm-pi-ai/tests/catalog.spec.ts covers each rung of the chain and both readings of an empty list at the resolver: one route mixing an undeclared model with entry-declared text-only and vision models, a route default answering an undeclared model while an entry still outranks it, a catalog vision model keeping its modalities under a narrower route default, an entry's [] inheriting rather than emptying, and the route's [] refused. A separate case re-asserts every rung end to end — a written settings section, the plugin's own registration, and ctx.llm.listModels / resolveModelInfo — so a break between the document and LlmModelInfo cannot pass.
config.spec.ts holds the schema boundary: an unknown modality refused at both levels, the empty route list accepted by the schema and refused by the namespace validator that the settings seam actually runs, and the [] materialization for an absent array that the inheritance rule depends on.
No keyless snapshot lane exercises a pi-ai route: the snapshot examples drive dsh-llm-replay, which declares modalities directly in its configuration, and a pi-ai route needs a live endpoint whose port a static cordis.yml cannot name. The admission points this change feeds are already covered there through that provider (examples/acp-agent/image.cordis.snapshot.yml and image-text-route.cordis.snapshot.yml) and are unaffected — what changed is what one adapter reports, not how a gate reads it.
中文
问题
settings.yaml 里没有任何写法能把一个手写的 pi-ai 模型描述成接受图片，而适配器对已安装 pi-ai catalog 未描述的每个模型都假定纯文本。部署通过 Web UI 的“添加自定义提供商”卡片新增的模型统统属于这一类，因此一个提供视觉模型的 OpenAI 兼容网关，无论实际提供什么，报出来的都是 inputModalities: ['text']。
Harness 把缺失的模态当作否定能力，并有三个准入点在构造任何请求之前就依据它行动：模型选择拒绝切换进已含图片的会话，prompt 准入拒绝图片，read_image 拒绝读取。它们的诊断让用户去选一个支持图片的模型——这条建议没有可达的指向对象，因为没有任何配置键能让手写模型变得支持图片。这条路封死在元数据上而非能力上：请求转换器和每种 pi-ai 线路协议都能携带图片，llm-pi-ai 自身的流前置检查才是唯一会拦下它的东西。
源码中把这一假定论证为适配器的真实能力而非部署选择，而 [[2026-08-03-pi-ai-declared-provider-catalog]] 在决定配置面公开哪些 Model 字段时记录了同一套论证（「没有任何读取方：…… context.ts 只保留文本块」）。该论证描述的是 DeepSeek chat-completions 适配器——它的序列化器确实拒绝 image block——对 pi-ai 路由从来就不成立。本 note 仅在模态一项上取代那一条；定价在那里依然因其自身、仍然成立的理由保持关闭。
决策
模态的解析顺序是：条目的 input → 已安装 catalog 条目 → 路由的 defaultInput，后者自身默认为 [text]。 这与 contextWindow、maxTokens 已经在用的链条逐字段一致。pi-ai 把 Model.input 定义为必填且按模型给出，因此条目字段直接镜像上游：一条路由可以让视觉模型与纯文本模型并存，而 override 可以修正那些网关实际提供的模态与 catalog 记录不符的模型。路由字段则让那些未被 catalog 描述的模型全都接受图片的网关，不必在每个条目上重复一遍。
路由值是回退值而非覆盖值——catalog 的优先级更高。 这采用的是 default* 的顺序而非 compat 的，两者不可互换：compat 之所以盖住 catalog，是因为路由级的协议改指会整体作废 catalog 关于推理分派的事实；而模态是按模型的属性，对 catalog 自己出货的那些模型，它记录得准确无误。让路由值获胜就意味着 defaultInput: [text] 会悄悄剥掉该路由上每一个 catalog 视觉模型的图片能力——一个没有对应收益的坑，因为收窄其中某个模型正是该模型自己的 input 要做的事。
未声明即 [text]，而这是「尚未声明」，不是对端点的猜测。 没有任何环节能询问网关的模态，因为 OpenAI 兼容列表端点不会报告它们。安全的底线是每个受支持协议都确定携带的模态。少声明会在图片附加之前拒绝、点名模型，并给出有文档的配置补救方法。多声明会先接纳并持久化图片，再由提供方拒绝。之后对同一错误声明路由的请求还会再次遇到图片，但用户可以选择纯文本模型，因为请求组装会把持久图片投影为占位符。
条目的空列表与缺省同义；路由的空列表则被拒绝。 [] 描述的是一个什么都不接受、无法服务任何请求的模型，因此不作答，解析继续往下走。这个读法不是修辞：配置 schema 会为缺省数组物化出 []，把它当作“什么都不接受”，会悄悄剥掉 models 列表恰好点到的每一个 catalog 视觉模型的图片能力。而路由值下面没有可以代为作答的层级，因此它的空列表在写入处即被拒绝。路由的 models 列表出于同样的理由，早已用同一种方式解析缺省与空。
没有任何配置界面编辑 input。 它和 compat、reasoningEfforts、thinkingBudgets、headers 一样是 settings 文档字段，而模型列表编辑器仍是一张只覆盖 id、名称和两个容量的手写表单。这不会带来持久代价，因为那张卡片本来就是按“承载自己并不编辑的字段”建造的：它的行 patch 会先展开已存储的行再应用改动，而采纳候选时已有行优先于重新发现的候选，因此手写的 input 在两条路径上都能存活。
DeepSeek 直接适配器拥有独立的精确模型目录。支持视觉的条目声明图片输入，纯文本模型和未列出的透传 ID 保持纯文本。
备选方案
- 乐观的
[text, image]默认值 —— 让触发场景无需配置即可工作，而网页表单不会写入模态，因此保守默认值会把补救方法留在 settings 文档里。否决原因是错误的肯定声明会在提供方拒绝之前持久化图片，并让该路由重复失败。纯文本请求投影提供了恢复方法，但不能让错误声明变成事实。 - 让路由值盖住 catalog（
compat的顺序：条目 → 路由 → catalog）—— 可以让把 catalog 路由改指到自家网关的部署，一句话声明「这里没有视觉能力」。被否决是因为同一句话也会在有人照着容量字段类比写下它的路由上，悄悄禁用每一个 catalog 视觉模型；而那个正当场景由该模型自己的input承担。覆盖值还必须在路由级改名叫input，因为在两个货真价实的回退值旁边把它叫作default*是名不副实。 - 完全不要路由字段，只要条目字段 —— 最贴近上游（上游没有路由级概念）。被否决的理由是产品自身流程会产生的批量场景：「获取可用模型」一次采纳三十个不带模态的 id，全是视觉模型的网关就得逐个手写
input。 - 只要路由级
defaultInput，不要条目字段 —— 无法在一条路由上混合模态，也无法修正单个 catalog 模型，唯一的变通办法只剩「把该提供方拆成两个路由键」，代价是多一个永久的 provider id 和每个模型选择器里的一项重复。 - 探测端点的模态 —— 没有任何 OpenAI 兼容的列表端点会报告它们。
- 从模型 id 推断（
*-vision、*-vl）—— 命名约定不是能力，网关可以随意改名。 - 继续拒绝，只改进诊断 —— 那条消息对状态的描述本来就准确，对补救办法则毫无用处；缺的正是补救办法。
影响
自定义提供方下的视觉模型只需在 settings 文档里写一行 input: [text, image]——或者当该路由列出的模型全都接受图片时，在路由上写一行。这就是修复的全部：三个准入点随即接纳它上面的图片，read_image 也可用。什么都不写的部署保持原有行为完全不变，因此没有任何既有路由改变它报出的内容。
图片准入门禁在各处都保住了自己的意义，因为它读到的每一个模态，如今要么由已安装 catalog 记录，要么由人写下。没有任何环节会替部署宣称一项能力。
声明了端点并不提供的图片能力时，本地无法发现该错误，因为声明不会被远端验证。prompt 准入会在请求构造前持久化用户消息，因此被拒绝的图片留在会话日志中，之后对该路由的请求可能再次失败。恢复方法是修正声明、选择支持图片的路由，或选择由请求投影把持久图片替换为占位符的纯文本路由。
测试
packages/llm/llm-pi-ai/tests/catalog.spec.ts 在 resolver 层覆盖了这条链的每一级以及空列表的两种读法：一条路由上未声明的模型与条目声明的纯文本、视觉模型并存，路由默认值为未声明的模型作答而条目仍然压过它，catalog 视觉模型在更窄的路由默认值下保住自身模态，条目的 [] 走继承而非清空，以及路由的 [] 被拒绝。另有一条用例把每一级端到端复验一遍——写入的 settings 段、插件自身的注册、以及 ctx.llm.listModels / resolveModelInfo——因此文档与 LlmModelInfo 之间若有断点无法蒙混过关。
config.spec.ts 负责 schema 边界：两个层级上的未知模态拒绝、路由空列表被 schema 接受而由 settings seam 真正运行的命名空间校验器拒绝，以及继承规则所倚赖的「缺省数组物化为 []」这一事实。
没有任何无密钥 snapshot 通道会跑 pi-ai 路由：snapshot 示例驱动的是 dsh-llm-replay，它在自己的配置里直接声明模态，而 pi-ai 路由需要一个真实端点，其端口是静态 cordis.yml 无法写出的。本次变更所供给的那些准入点已经通过该提供方在那里得到覆盖（examples/acp-agent/image.cordis.snapshot.yml 与 image-text-route.cordis.snapshot.yml）且不受影响——改变的是某个适配器报告什么，而非门禁如何读取它。