Audio attachments look simple at the interface: attach a clip and ask a question. The routing decision behind that interaction is less simple. A model endpoint may accept text while rejecting audio, and even an audio-capable endpoint may expose a different wire format.
AIClaw 3.11.1 makes that decision from the selected model’s declared capability and service protocol. When native audio is unavailable or unknown, it uses the configured speech-to-text route instead of silently sending audio bytes to a text-only request.
The routing decision
The audio preparation code enables native input only when AudioInput is explicitly marked supported and the selected protocol is Chat Completions or Gemini. WAV and MP3 attachments can follow this path; other formats go through the transcription fallback.
That is a deliberately narrow claim. A provider being compatible with a protocol does not establish that every model or endpoint accepts audio. The capability setting needs to match the actual service. Responses and other protocols are not treated as native audio routes by this version of the code.
For native input, the protocol adapters format the same audio differently:
- Chat Completions sends an
input_audiocontent part with a format and base64 data. - Gemini sends inline data with the appropriate audio MIME type.
When a route is not explicitly capable, AIClaw prepares a transcription instead. That depends on a working configured transcription service; it is not a guarantee that any arbitrary attachment will be understood.
Keep the request bounded
Native audio is subject to an 8 MiB limit per file and a 180-second request budget. For WAV files, the implementation estimates duration from the data size and byte rate. When it cannot determine duration, it conservatively charges the entire budget. This also matters for MP3 in this implementation: compressed file size is not used as a duration estimate.
When conversation history is assembled for another model call, the code walks from the newest messages backward and includes clips within the duration budget. Missing files and clips outside the budget are represented by an omission note rather than silently treated as audio the model received. A stored SHA-256 is also checked before reuse when present, so a changed local file is not silently substituted for the original attachment.
A rejection is evidence, not a reason to replay
If a service returns one of the recognized audio-specific HTTP 400 codes, the agent records a rejection against the model route only when the request contained exactly one clip. With multiple clips, the response cannot identify which clip caused the failure, so the code does not attribute it to one of them. The corresponding agent tests cover duration and fallback behavior, durable rejection, and the no-retry path. The adapter tests cover native wire formats and unsupported protocols.
Those are repository tests, not evidence that every provider has been exercised with live audio. In particular, the rejection tracking described here applies to later requests; it should not be read as an automatic replay of the failed request.
The result is a clearer boundary between capability configuration, protocol serialization, and the evidence the agent actually has. It is a small architectural distinction, but it prevents an attachment UI from promising more than the selected model route can deliver.
