Buy a Rack, Rent GPUs or Colocate? The Honest Maths of Rack-Scale AI
There are three ways to get rack-scale AI capacity. Buy the rack and host it yourself. Buy the rack and place it in someone else's data centre. Or rent GPUs by the hour and own nothing.
Each option has passionate advocates, most of whom sell one of them. The unglamorous truth is that the cheapest option is decided by a number most business cases guess at: how many hours the GPUs will be busy.
The three routes in pictures
| What | π Buy and self-host | π’ Buy and colocate | βοΈ Rent |
|---|---|---|---|
| π° Upfront cost | π΄ Rack plus facility upgrade |
π‘ Rack |
π’ None |
| π Cost per GPU-hour when busy | π’ Lowest | π’ Low | π΄ Highest |
| π΄ Cost when idle | π΄ You pay anyway | π΄ You pay anyway | π’ Zero on demand; π΄ full on a reservation |
| β±οΈ Time to first workload | π΄ Months to years | π‘ Months | π’ Hours to days |
| π Control over data and hardware | π’ Full | π’ High | π‘ Depends on provider and contract |
| π Who carries obsolescence risk | π΄ You | π΄ You | π’ The provider |
| π§° Who runs the liquid cooling | π΄ You | π‘ The operator, mostly | π’ Not your problem |
The one formula that matters
Strip away the detail and ownership economics reduce to a single line:
Cost per useful GPU-hour = total yearly cost of owning Γ· (72 GPUs Γ 8,760 hours Γ utilisation)
Total yearly cost includes the rack spread over its useful life, hosting or facility costs, power, support and staff. The number at the end, utilisation, is the share of hours the GPUs are doing paid or valuable work.
The formula's lesson is in its shape. Everything on top is fixed once you buy. Only utilisation moves, and it moves the answer a lot.
What utilisation does to the answer
| Utilisation | Owned cost per useful GPU-hour (one published US model) | Versus renting |
|---|---|---|
| π’ 80 to 90% | About US$2.56 to US$2.88 |
Well below typical rental rates |
| π‘ 60% | About US$3.84 |
Still below most rental rates |
| π΄ 30% (our extrapolation) | Roughly double the 60% figure |
Above the cheapest rental listings |
Source: DataGravity's NVL72 cost model, which uses US hardware and colocation assumptions. The 30% row is our own arithmetic from the same formula: halve the utilisation, double the cost. Singapore power and hosting costs differ, so treat these as an illustration of the shape, not a quotation.
For comparison, public listings in October 2026 showed GB300 rental starting around US$6 to US$6.50 per GPU-hour on demand, with some on-demand prices as high as US$18 and reserved capacity in between.
Sources: American Compute GB300 price comparison, GetDeploying GB300 listings
Three uncomfortable observations
1. Most organisations overestimate utilisation
Business cases are written at 80%. Real clusters often run lower. One analysis of a 64-GPU cluster at 55 to 65% utilisation put the cost of idle compute at about US$1.4 million a year. Jobs queue unevenly, teams hoard reservations, and nobody trains models at 3 a.m. on a public holiday unless a scheduler makes them.
If you cannot name the workloads that will fill the rack in month one, your utilisation assumption is a hope.
Source: Barrack AI analysis
2. A reservation is ownership without the asset
Renting sounds flexible, and on-demand renting is. But the attractive rental prices usually come with a commitment of a year or more, billed around the clock. As one provider's own guide notes, a reservation costs the same whether the GPUs are busy or idle, exactly like an owned rack. You have taken on the utilisation risk and you do not own anything at the end.
Source: GMI Cloud on reservation vs buying
3. The rack price is the smaller uncertainty
Published estimates of what a GB300 NVL72 costs vary enormously between sources, which is one reason vendors quote on request. But even a large swing in purchase price changes the cost per GPU-hour less than moving utilisation from 80% to 40%. Negotiating the rack hard and then running it half-empty is the expensive way to save money.
A decision guide
| Your situation | Lean towards |
|---|---|
| π§ͺ Exploring, workloads not yet proven | βοΈ Rent on demand |
| π Spiky demand: big training runs, quiet periods | βοΈ Rent the peaks, own a small baseline |
| π Steady, high demand you can evidence today | π’ Buy and colocate |
| π Data that must stay on hardware you control | π’ Buy and colocate, or π self-host |
| ποΈ You already run a liquid-cooled facility | π Self-host |
| πΌ You plan to sell GPU capacity to others | π’ Buy and colocate, with customers signed first |
How to test your own case in an afternoon
- List the specific workloads that will run in the first six months, with GPU-hours for each.
- Add them up and divide by the hours available. That is your evidenced utilisation.
- Get three numbers in writing: a rack quote, a hosting quote for a liquid-cooled position, and a rental quote for the same GPU-hours.
- Run the formula at your evidenced utilisation, then again at half of it.
- If ownership only wins in the optimistic case, rent first and revisit in a year.
For regulated organisations, cost is not the only factor. Data residency, export documentation and supplier checks can settle the question before the spreadsheet does. See Why Your AI Infrastructure Supplier Should Understand Export Compliance.
This article is general information to help frame a decision. It is not financial advice, and figures quoted from third parties are illustrative.
Related on SourceIT
SourceIT Approved
Researched and verified by the SourceIT team for Singapore buyers.




