Spot Instances

Spot instances are in beta. Expect higher risk of bugs.

Understanding Priority

When you deploy a spot instance, you first set a bid price.

Storage prices are billed at the standard rate, and are added atop the compute price that you bid. If you bid a price for $0.35/hr with 200GB of storage at $0.00005/GB/hr, then your total cost will be $0.36/hr when the workload is running and $0.01/hr when outbid.

The bid price you set is per-virtual machine, not per GPU. We prioritize workloads for max revenue, not the per-GPU price. For instance, if there are these bids for an 8x H100 machine:

$3.00/hr for 1x H100 ($3.00/hr per H100)
$4.00/hr for 8x H100 ($0.50/hr per H100)

Then the 8x H100 will be prioritized because the overall revenue is higher, even though it has a lower per-GPU price.

However, if there are these bids for a 3x A6000 machine:

$3.00/hr for 1x H100
$1.10/hr for 1x H100
$4.00/hr for 8x H100

Then the first and second VMs will win the instance, because the combined revenue of those two VMs ($4.10/hr) is greater than the revenue of the third VM ($3.00/hr).

Note that these are for illustrative purposes only. Each host has a minimum bid price set per GPU, typically around 50% of the on-demand price.

Because we optimize for aggregate revenue and not per-unit revenue, we expect consumer cards and older enterprise GPUs may have great price-to-performance, since there will be less bidders.

Deploying Spot Instances

First, deploy an on-demand VM and click it in the "Your Servers" page to convert it to a spot instance. Set the following fields in your API request:

price_type: spot
price: [your bid]

Managing Interruptible Instances

Interruptible instances will appear within your dashboard like the image above.

Notes

Interruptible instances are only available on GPU instances, and storage costs are billed at the standard storage price set by the host. There is no "spot storage" tier, so to speak.

Last updated 9 months ago

Was this helpful?

Understanding Priority

When you deploy a spot instance, you first set a bid price.

The bid price you set is per-virtual machine, not per GPU. We prioritize workloads for max revenue, not the per-GPU price. For instance, if there are these bids for an 8x H100 machine:

$3.00/hr for 1x H100 ($3.00/hr per H100)

$4.00/hr for 8x H100 ($0.50/hr per H100)

Then the 8x H100 will be prioritized because the overall revenue is higher, even though it has a lower per-GPU price.

However, if there are these bids for a 3x A6000 machine:

$3.00/hr for 1x H100

$1.10/hr for 1x H100

$4.00/hr for 8x H100

Then the first and second VMs will win the instance, because the combined revenue of those two VMs ($4.10/hr) is greater than the revenue of the third VM ($3.00/hr).

Note that these are for illustrative purposes only. Each host has a minimum bid price set per GPU, typically around 50% of the on-demand price.

Because we optimize for aggregate revenue and not per-unit revenue, we expect consumer cards and older enterprise GPUs may have great price-to-performance, since there will be less bidders.