A vision-capable model can reason about screenshots, diagrams, photos, and other image input alongside text. That capability is useful for debugging interfaces, reading charts, interpreting documents, and describing visual content. But a model name alone does not prove image support, and a generic URL field is not a safe upload system.
Discover capabilities from the model registry
Model support changes over time and can vary by provider route. A client should read an explicit capability such as input_modalities: ["text", "image"] from the current model descriptor. If image is absent, the interface should disable attachment controls and the server should reject image blocks before reserving provider capacity.
This server-side check is essential. A disabled button improves the experience, but API callers can bypass browser controls. The Router remains responsible for enforcing the selected model’s contract.
Use owner-bound private uploads
Allowing an arbitrary external image URL makes the server fetch attacker-controlled resources and can expose internal network behavior. A safer flow creates a short-lived signed upload target in private object storage, ties the upload record to the authenticated user, and passes only an opaque upload identifier in chat messages.
- The browser submits the filename, declared media type, and byte size.
- The server returns a short-lived signed target for a private storage path.
- The browser uploads directly to storage, avoiding a large file through the application server.
- A completion call validates the stored object before it can be referenced by chat.
- At turn time, the server verifies ownership and expiry, then creates a short-lived provider-readable URL.
Validate the bytes, not the extension
A file named diagram.png may not be a PNG. Completion should inspect the decoded media, allow only intentional formats, enforce both file-size and dimension ceilings, and normalize the image to strip metadata. Limits on images per turn and conservative token accounting protect model context and platform capacity.
Common baseline controls include JPEG, PNG, and WebP only; a 10 MiB encoded-size limit; a 4096-by-4096 dimension ceiling; and no more than four images in one request. Those are product choices rather than universal constants, but publishing them gives users predictable behavior.
Design for failure and cleanup
Uploads can be abandoned between creation and completion. A completed upload can also outlive the conversation that needed it. Records therefore need expiry timestamps, explicit deletion, and a periodic cleanup job that removes both the storage object and database state. Feature flags should allow operators to disable new vision uploads without affecting ordinary text chat.
The interface should show each attachment’s state—uploading, ready, or failed—and let the user remove it. Sending should be blocked while any attachment is unfinished. If the selected model changes to a text-only route, the client should explain why attachments are unavailable instead of silently dropping them.
Questions about multimodal chat
- Why not send a base64 image in the JSON request?
- Large base64 bodies increase application-server memory and request-size pressure. Direct private storage uploads keep the chat request bounded and make ownership and expiry explicit.
- Why count tokens for images conservatively?
- Provider tokenization can vary by resolution and model. A conservative estimate prevents an image-heavy request from bypassing admission limits before exact provider usage is known.
- Should every endpoint accept images?
- No. Each surface should advertise and enforce its own contract. A lightweight footer integration can remain text-and-tools only even when browser chat supports vision models.
Capability, storage, and UX must agree
Reliable vision support is an end-to-end feature. The registry declares capability, the browser communicates it, private storage bounds the file lifecycle, the Router validates ownership and content, and the provider adapter serializes only approved image references. Missing any one layer turns a convenient file picker into an ambiguous security and reliability problem.