@ronitrjain I don’t think there’s enough volume for such an instrument.
A good chunk of clusters are being leased through VC money. The demand isn’t organic and the output doesn’t necessarily have a quantifiable ROI.
If your neocloud is asking for 20-30% downpayment before they place a hardware PO. They’re technically adding little to no value to you.
You’re better off leasing a colo site and buying your own hardware.
@jessiedong_@CoreWeave Yes and no.
Take Aramco as an example, why don't they compete with Shell and BP? They have the oil, why don't they refine crude oil and sell direct?
Trying to do everything without partners might sound like a good idea but might end up being a waste of time and money.
This is only considering pricing from lambda labs.
What you should consider is the TCO of these GPUs.
Is the inference provider looking to lease the same nodes for 3+ years, what do they do when new models drop, buy more?
Do they have engineers that can handle the infrastructure?
These are all additional costs that were not taken into consideration.
A mere price comparison doesn’t tell the story, a year ago all GPU prices were loss leaders because demand was very low.
Thoughts from the field selling compute:
Nothing stopping in the data center
No signs that conversations are slowing just concerns on how to transact and what will happen when they go to lenders
People are concerned about getting capacity
Anyone below investment grade is gonna have some trouble buying for a little bit
This may even translate to the VC funded large inference clouds and the NVDA backed ones
Smaller inference clouds are super concerned that they won’t be able to source capacity
Everything is gonna get snapped at whatever price for the longest term possible
Hypers are gonna take back all of the power in the market