Spot vs On‑Demand GPU instances differ primarily in cost and reliability. Spot offers deep discounts but can be reclaimed at short notice; on‑demand provides stable capacity at higher fixed pricing.
This post may contain affiliate/partner links. If you buy or sign up through them, we may earn a commission at no extra cost to you.
Last updated: September 28, 2026
What are spot GPU instances versus on‑demand GPU instances?
On‑demand GPU instances are cloud GPU resources available at a fixed price with guaranteed availability for as long as you pay, making them ideal for production, latency‑sensitive, or stateful workloads.
Spot GPU instances (also called preemptible or interruptible) are the same underlying hardware offered at significant discounts—often 40‑80% off on‑demand—but the provider can reclaim capacity with seconds to minutes’ notice, making them suitable only for fault‑tolerant workloads. In exchange for cheaper compute, you accept the interruption risk. ([computingpower.org](https://computingpower.org/providers/spot-vs-on-demand-gpu-pricing))
How much can you save with spot GPU compared to on‑demand?
Typical spot discounts range between 30% and 80%, though real‑world gaps vary by provider and GPU model:
| Provider / Source | GPU Type | On‑Demand | Spot | Saving |
|---|---|---|---|---|
| ComputingPower.org | A100 | $3.22 | $0.97 | ~70% |
| DigitalOcean (public preview) | GPU Droplet | Fixed rate | Variable, ≤ on‑demand | Varies daily |
| Spheron Blog | H100 SXM5 | $2.54 | $1.03 | ~59% |
| Cloud‑GPUs.com (median pricing) | H100 SXM | $3.29 | $2.04 | ~38% |
These numbers show two patterns: hyperscaler spot rates typically land around 30‑40% of on‑demand list price, while smaller marketplaces like Vast.ai or RunPod may offer lowest absolute spot rates—but with weaker availability guarantees. ([computingpower.org](https://computingpower.org/providers/spot-vs-on-demand-gpu-pricing))
Why the spot discount exists: interrupted capacity versus cost savings
Cloud providers sell unused capacity at lower prices to avoid idle hardware. That discount comes with the risk of preemption—you could lose compute mid‑job. Hyperscalers typically give 30‑120 seconds notice, whereas marketplaces may abruptly kill instances when outbid. ([computingpower.org](https://computingpower.org/providers/spot-vs-on-demand-gpu-pricing))
Because spot pricing floats, there are days when spot can even exceed reservation rates—spot is variable, reservation is fixed. Large fleets often favor reserved commitments over spot for reliability, even if savings are slightly lower. ([computingpower.org](https://computingpower.org/providers/spot-vs-on-demand-gpu-pricing))
When should you choose spot vs when to stick with on‑demand?
Spot GPU works best for:
- Batch model training with frequent checkpoints
- Hyperparameter sweeps or large-scale experimentation
- Offline inference pipelines or preprocessing tasks
- Stateless workloads that can restart gracefully
These workloads tolerate interruption and capitalize on deep cost savings. ([docs.digitalocean.com](https://docs.digitalocean.com/products/droplets/concepts/spot-vs-on-demand/))
On‑demand GPU is better for:
- Production inference APIs or latency‑sensitive services
- Stateful processing requiring continuous GPU access
- Workloads with strict run‑time deadlines
- When spot availability is limited or interruption risk is too high
On‑demand ensures stability even if cost is higher. ([docs.digitalocean.com](https://docs.digitalocean.com/products/droplets/concepts/spot-vs-on-demand/))
How should you manage spot GPU interruptions?
- Implement checkpointing at logical intervals (e.g. after each epoch in training).
- Use auto‑scaling groups or spot‑fleet abstraction to replenish capacity automatically. ([docs.aws.amazon.com](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-best-practices.html))
- Be flexible in instance types and Availability Zones to improve allocation success. ([docs.aws.amazon.com](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-best-practices.html))
- Multi‑region or multi‑zone fallback helps when a single spot pool drains.
- Include buffer/over‑allocation—don’t run exactly to the deadline.
Provider comparison: spot vs on‑demand GPUs at a glance
| Provider | Interruption Notice | Spot Discount | Availability Risk |
|---|---|---|---|
| AWS | ~2 minutes | Up to ~70‑90% | Moderate; spot pools may drain |
| Google Cloud | ~30 sec | 60‑91% | Good in secondary regions |
| Azure | ~30 sec | Steep on paper | Often scarce |
| Marketplaces (RunPod, Vast.ai) | Seconds or instantaneous | Deepest absolute savings | High variability |
Hyperscalers offer predictable policies with moderate volatility; marketplaces offer cheaper capacity but higher interruption and availability risk. ([computingpower.org](https://computingpower.org/providers/spot-vs-on-demand-gpu-pricing))
Common questions people search about “spot vs on‑demand gpu”
- Is spot GPU safe for production? → No, only if your code checkpoint‑resumes reliably.
- Can spot cost more than on‑demand? → Yes, short‑term supply shortages can push spot above fixed reservation prices. ([spheron.network](https://www.spheron.network/blog/gpu-cloud-pricing-comparison-2026/))
- Are savings always consistent? → No, discounts vary by region, GPU type, and time.
- How much interruption notice do providers give? → Ranges from ~30 seconds (GCP) to ~2 minutes (AWS); marketplaces may give none.
- Should I reserve or spot? → Reserve for long workloads without interruption risk; spot for burst or flexible workloads.
Frequently Asked Questions
- Can I mix spot and on‑demand GPUs in one pipeline? Yes. Hybrid strategies (e.g. train on spot, serve on‑demand) balance cost and reliability.
- Does checkpointing add cost? Slight storage and development overhead—but minimal compared to GPU runtime costs.
- What if spot fails mid‑epoch? Use fine‑grained checkpointing (e.g. every batch or validation step).
- How to estimate effective spot cost? Add reprovisioning time and lost compute to the quoted spot rate; benchmark with your workload.
- When spot pools drain, what next? Fall back to on‑demand or switch regions/zones; pre‑plan fallback policy.
Key takeaways:
- Spot GPU = deep cost savings but high termination risk.
- On‑demand = predictable pricing and continuous availability.
- Use spot for flexible, fault‑tolerant workloads; use on‑demand for production, latency‑sensitive tasks.
- Mitigate spot interruptions with checkpointing, flexible allocation, and fallback policies.
For a deeper dive into GPU cost optimization strategies like reserved commitments or auto‑scaling, see Related Cloud GPU Cost Optimization.
Need a practical guide on setting up hybrid spot/on‑demand pipelines with Kubernetes or batch schedulers? Check Build Hybrid GPU Pipelines.
About the Author:
cloudgpuhub.com – Nhon Dang is a cloud infrastructure and operations professional with over 10 years of hands‑on experience in cloud services, infrastructure, and business operations. His expertise spans the design, deployment, and operation of cloud platforms and managed services, including virtual machines (VMs), Kubernetes (K8s), object storage (S3), managed databases, Apache Kafka, and cloud GPU infrastructure. Throughout his career, Nhon has worked closely with cloud infrastructure and service operations, gaining practical experience in building reliable, scalable, and cost‑efficient cloud environments. His work combines technical expertise with business and operational insight, giving him a practical perspective on how cloud technologies perform in real‑world production environments. Nhon writes about cloud infrastructure, Kubernetes, DevOps, distributed systems, cloud computing, infrastructure operations, and cloud service management, sharing insights based on hands‑on experience rather than purely theoretical knowledge.
