Each device in your fleet exposes a single OpenAI-compatible API at http://127.0.0.1:9100/v1. Any application that works with the OpenAI API can use it by changing the base URL and API key, including coding agents like Hermes Agent, Aider and Continue, and chat interfaces like Open WebUI.
Your application sends requests to Cascadia on its own machine, which processes and routes each one to whichever device in your fleet serves the model you ask for.
The API only accepts requests from the machine it runs on, so run your application on an enrolled device.
Create an API token
Section titled “Create an API token”You need the admin or owner role.
- In the console, open Settings → API tokens.
- Enter a Label that says what the token is for, such as
prod-apporci-runner. - Choose when it expires, then select Create token.
- Copy the token now. The console shows it only once and can’t retrieve it later.
Give each application its own label. You revoke tokens by label, so tokens that share a label are revoked together.
Point your application at the fleet
Section titled “Point your application at the fleet”Set your application’s base URL to the device’s API:
http://127.0.0.1:9100/v1Send the token as a bearer token. With the OpenAI Python SDK:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:9100/v1", api_key="<TOKEN>")reply = client.chat.completions.create( model="<MODEL_ID>", messages=[{"role": "user", "content": "Hello!"}],)print(reply.choices[0].message.content)GET /v1/models lists the models your fleet serves.
Keep conversations fast
Section titled “Keep conversations fast”For multi-turn conversations, send an X-Cascadia-Session header with a stable value per conversation, such as its ID. It keeps the conversation on the same devices, so long histories don’t have to be processed again each turn.
- Always send the full conversation. The header is a routing hint, not stored state.
- Send it on every turn of a conversation, or on none.
- Values are scoped to your organization, so they can’t collide with anyone else’s.
Handle errors
Section titled “Handle errors”If generation fails partway through a streaming response, the gateway sends, in this order:
- A final chunk with
"finish_reason": "error". - An
errorevent with acode. The OpenAI SDKs raise an exception here. data: [DONE].
Discard any partial text and resend the full conversation. Don’t join partial text to a retry’s output.
A non-streaming request that fails partway is retried once on other devices automatically. If that also fails, or if no device produces a first token in time, the gateway returns HTTP 503 with a Retry-After header.
code |
Meaning | What to do |
|---|---|---|
chain_failover_exhausted |
Generation failed and no other devices could take over | Wait, then resend the full conversation |
first_token_timeout |
No device produced a first token in time | Retry after Retry-After, but give up after a few attempts |
Treat any other code as not retryable.
Revoke a token
Section titled “Revoke a token”In Settings → API tokens, find the token in the list and select Revoke. It stops working across the fleet within about a minute. This can’t be undone, so create a new token to restore access.
Removing a member from your organization also revokes every token they created.

