Skip to content
Docs
Enterprise

If the installation stops or fails, it will print an exit code naming the problem that occured.

Code Meaning What to do
10 The installer isn’t running with admin rights, the OS isn’t supported, or (Windows) it couldn’t restrict access to %ProgramData%\Cascadia Windows: run PowerShell as administrator. Ubuntu: run with sudo. Only 64-bit Ubuntu 22.04 or later is supported.
11 The enrollment credential or CA address is missing or malformed Copy the install block from Devices → Add device again and paste the whole thing.
12 The credential is expired or already used, the organization is suspended or gone, or the network failed For an expired or used credential, create a new installer in Devices → Add device and run it. For a suspended organization, contact your Cascadia representative. For a network error, run it again.
13 Downloading or verifying the release failed Usually a temporary network problem; run it again. If it keeps happening, contact your Cascadia representative.

Resuming an installation after the device is enrolled

Section titled “Resuming an installation after the device is enrolled”

Once the machine has been enrolled, the credential will expire.

To finish the install, get a new installer for the same machine:

  1. In the console, open Devices → Add device and create a new installer, the same way you did the first time. See Add a device.
  2. On the machine that stopped, run the new installer or paste its terminal command.

The machine keeps the identity it enrolled with, so it skips enrollment and finishes the remaining steps.

If the console says Nothing has enrolled yet:

  • Windows: check the install window for a message.
  • Ubuntu: open a terminal and run journalctl -u cascadia-installer. Or, attempt an installation from the terminal.

Each device checks in with the console every minute or so.

Its status on the Devices page will show one of the following:

Status Meaning
online Checked in within the last two minutes
stale Last checked in between two and ten minutes ago. Often a brief network drop or a restart in progress.
offline Hasn’t checked in for more than ten minutes
revoked Removed from the fleet. It can’t serve or rejoin without enrolling again.
  1. Check that the machine is switched on, awake and connected to the network. A laptop that has left the office goes offline until it’s back on a network that can reach the console.
  2. If it’s online elsewhere but not in the console, check the device’s log (see the start of this page).
  3. When it’s back online, select Restart on its row if it still misbehaves. Only active devices can be restarted, and a restart queued for an offline device expires after an hour.

The console warns you with Devices offline when a device has been silent for more than 24 hours, and Fleet offline when every device is.

Online, but other devices can’t reach it

Section titled “Online, but other devices can’t reach it”

The device checks in with the console, but other devices and applications can’t connect to it.

  • Windows: the installer’s firewall rules only apply on Private and Domain networks. If the device is on a network marked Public, change the network profile.
  • Check that the ports in Requirements are open in any firewall between your devices.
  • Devices on different networks or behind NAT need a relay.

If a relay shows not reachable, see Configure a relay.

Relays can’t be restarted from the console. To restart one, run sudo systemctl restart cascadia-relay on the relay host.

Message What to do
This organization is already at its limit of active relays Revoke a relay you no longer need, or select Re-enrol on an existing relay’s row to move or upgrade it.
That public address is not valid Enter a hostname or IP address without a port, for example relay.example.com or 203.0.113.10.
That public address points at a loopback, link-local, metadata or other non-routable target Use a public IP address or a public hostname that your devices can reach.
Message What to do
This credential re-enrols one existing relay, and this host does not hold its key You selected Re-enrol, which only works on the relay’s original host. Run the installer on that host, or add a new relay instead.
This organization is already at its limit of active relays Another relay enrolled after you created this installer. Revoke a relay you no longer need, then create a new installer.

These messages appear when you connect the Fleet Manager to your account. See Install the Fleet Manager for the full steps.

Message What to do
Use an https:// console URL Enter the console address starting with https://. A LAN address such as http://192.168.x.x won’t work.
Couldn’t reach the console at that URL Check the address and your internet connection. If you’re on a VPN, check that it allows the console.
This machine’s clock is too far from the console’s Turn on automatic time sync on the computer running the Fleet Manager, then try again.
This key isn’t allowed to manage the fleet In Settings → Fleet manager keys, approve the key again with Fleet manager ticked.
This key has expired / The console no longer knows this key Approve the Fleet Manager’s key again in Settings → Fleet manager keys and paste the new code.
A newer code was approved for this key Paste the latest code from Settings → Fleet manager keys.
This code was approved for a different fleet manager Copy the key from step 1 in the Fleet Manager again, and approve that one.
The console doesn’t recognize this organization The key is for a different organization or console. Check the console URL.

If the Fleet Manager shows Fleet data is out of date or Couldn’t refresh, it can’t currently reach the console. It keeps showing the last data it fetched and retries on its own.

The Fleet Manager couldn’t update itself

Section titled “The Fleet Manager couldn’t update itself”

If an in-app update fails, the Fleet Manager keeps running the version you had. Most failures, such as a download that didn’t match its checksum, a folder the Fleet Manager can’t write to, or an admin prompt that was dismissed, are fixed by downloading the latest installer from the console’s Overview page and running it yourself.

When some devices don’t take a model, the Fleet Manager’s Activity feed marks the job Failed or Partly failed and gives a reason for each device.

Reason What to do
The device is offline / The device couldn’t be reached Bring the device back online and deploy again. An offline device can take new models when it reconnects.
The device never picked up the command before it expired The device was offline or busy for too long. Check it’s online, then deploy again.
The device stayed busy past the deadline Something on the device is stuck. Restart it, then deploy again.
The download failed Check the device’s internet connection, then deploy again.
The download failed its integrity check Usually a corrupted download. Deploy again. If it keeps happening, contact your Cascadia representative.
The device refused: check its model directory The device couldn’t write the model to disk. Check it has free space, then deploy again.
The device couldn’t check its free disk space Restart the device, then deploy again.
The device rejected the request’s signature Reconnect the Fleet Manager to your account so its key is current.
The device hit an unexpected error Restart the device. If it happens again, contact your Cascadia representative.

If a device is short of space, free up disk space by deleting models it no longer serves.

A model is degraded when one of its pipelines is missing a stage, usually because a device serving that stage went offline. Requests go to the copies that still serve. Bring the missing device back online, or deploy the model to other devices.

No pre-exported shards published for this model yet means Cascadia hasn’t published this model in a form devices can download. Choose another model from the catalog.

Devices update themselves. If the Activity feed says The device couldn’t apply the update or The revert couldn’t be verified, restart the device and roll the release out again. See Update your fleet to find devices that are behind.

The gateway returns errors in the same format as the OpenAI API, so the OpenAI SDKs raise them as exceptions. The code field says what went wrong. For errors partway through a response, see Handle errors.

HTTP status code What to do
401 missing_token, malformed_token Send the token in an Authorization: Bearer <token> header.
401 invalid_token, token_invalid The token isn’t one the fleet recognizes. Check you copied the whole token, or create a new one.
401 token_expired Create a new token in Settings → API tokens.
401 token_revoked The token was revoked. Create a new one under a different label.
400 malformed_body The request body isn’t valid. Check it against the OpenAI chat completions format.
404 model_not_found No device serves that model. Check the model name, or deploy it.
503 at_capacity, backend_busy Every device serving the model is busy. Retry after the Retry-After header.
503 backend_unavailable, no_alternative_peer No device serving the model is available. Check that devices serving it are online.
503 maintenance_mode The device is temporarily not taking requests. Retry after the Retry-After header.
Message What to do
That label has been revoked Revoking a token revokes its label for good. Choose a different label.
A token already uses that label Give the new token a label of its own.
Message What to do
Your session has expired Sign in again.
Your role does not allow this action Ask an owner or admin to do it, or to change your role. See Team and roles.
Your email address is not confirmed yet Open the confirmation link in the email we sent, then sign in again.
This invitation is no longer valid Ask an admin in your organization to invite you again.
An organization needs at least one owner Make someone else an owner before you change or remove the last one.
This organization is suspended Contact your Cascadia representative to restore service.

If you can’t fix a problem here, contact your Cascadia representative. To help them, include:

  • The names of the affected devices and their status in the console
  • The message you saw, or the reason in the Activity feed
  • The installer’s exit code, if the problem is with an install
  • The device’s recent log (see the start of this page)
  • What you expected to happen

Remove private prompts and other sensitive data from logs before you share them.