Skip to content
Docs
Enterprise

New devices are installed with a small model that runs on the CPU. To serve a larger model, or to use the GPU or NPU, you can deploy a model from the Fleet Manager.

  1. In the Fleet Manager, open Fleet → Models and choose a model from the catalog.
  2. Choose how to place it:
    • Whole model: every chosen device serves its own full copy. Use this when the model fits in one device’s memory.
    • Sharded: the model is split into stages across an ordered list of devices that work together as one pipeline. Use this when the model is too large for one device.
  3. Choose the devices and start the deploy.

Each device downloads its part and switches over. Fleet → Activity shows the progress of each device and whether the deploy succeeded.

To deploy to a single device, expand its row in Fleet → Devices.

Models a device no longer serves stay on its disk. To remove them, expand the device’s row in Fleet → Devices, select the idle models and delete them. A device never deletes the model it’s serving.

When the Fleet Manager is connected to your console account, it can’t yet replace the model a device is already serving, edit or move an existing deployment, or place models automatically.

Contact your Cascadia representative if you need one of these.