Many people who have ever had to deal with the long tail of insanities in physical hosting, like "the ceiling burst and dumped water on a rack", "the RAID controller's capacitors exploded and now you need to figure out what still works", or "for some reason three of the servers won't talk to this switch but talk to anything else using the same cables, and the switch ports work for other devices", would happily pay a premium to only deal with SaaS logistics.
(Those were all firsthand examples; I'm not saying everyone needs cloud providers, but there are reasons beyond "really good salespeople" that people opt for offloading those logistics.)
Ok but add up a trickle of those examples, with staff to handle them, and you're still comfortably in the black.
You dont even have extra organizational overhead. Every cloud first company has a head of devops sitting in the chair where head of infra would be. They somehow wind up with like half the staffing anyways compared to running bare metal.
Our colo experience was pretty smooth, nothing insane like that, and saved us a bundle. Apparently ymmv. I hate working with AWS APIs by comparison, some of the worst UX I've ever seen.
I've dealt with both. I am continually amazed that people not only pay for AWS, but that it is so complicated, and that they use it in all sorts of absurd ways, not even just hosting 'vms' but using all sorts of amazon tools to do trivial tasks. I am sorry I just dont get it. Is it like learning salesforce and once you're sucked in you're just in? I am so glad I do not work at an AWS shop anymore and am very glad to be in a position where we do not use it.
You get AWS certifications, use as many AWS services as possible to pad your CV with the service experience and certifications, get your arguments and "well architected" dogma from AWS sales. Then you can get paid more at the next company (or at least high salary offers to extract a raise).
Also building high complexity systems that require a big expensive cloud engineer staff gets you status and visibility. In a tight spot you can still blame AWS for problems.
(Reality is not quite so bleak as people are not completely cynical)
I actually do get public cloud as well as running your own racks. There is room for both. Your startup is growing 100% every quarter but you're not sure if you have real PMF yet? Yeah probably stick with cloud. Zombiecorn growing 10-20% yoy or not growing and you're paying the cloud $10M/y for 6yo CPUs? - pretty foolish to continue doing that.
This sounds like either an extremely bad experience or an account from many years ago. Things are better now.
Also like I said: physically racking only makes sense if you are huge. Managed bare metal is a whole market category and it’s spectacularly cheaper than big cloud for most work loads. For bandwidth it’s like 1000X cheaper. That is not an exaggeration.
...or "the UPS batteries caught on fire", or "the first unit failed and the contractor who was supposed to replace it replaced the second one instead, resulting in an outage".
Or something as stupid as, "vendor requires $50 failed DIMM replaced under warranty to be returned by UPS instead of chucked into the e-waste bucket, but UPS cannot pick up from DC because the driver can't be bothered to ring the bell on the DC gate".
That one alone probably resulted in our longest ever ticket.
I probably wouldn't start with buying a big server if I was just starting out. I would consider renting one from Hetzner, OVH, or many others.
There are also many other alternatives. Example: VPS providers like Vultr which have expanded to offer more traditional "cloud" features like object storage, load balancers, managed DBs, etc. Their pricing is way more competitive than AWS, especially when you consider bandwidth.
Personally, I'd rent a small server first. About 20 years ago I had a small web hosting operation with my own racked servers. It's a pain to deal with when something goes wrong. Eventually we transitioned to renting dedicated servers.
There’s a sort of arguable sweet spot where someone motivated and knowledgeable can manage their own hardware, even if it’s just rented from Hetzner etc. But that just doesn’t scale.
Once you do try to scale that - not just the hardware but the staffing that’s needed to avoid a bus factor of 1, to ensure the required uptime especially if it needs to be 24x7, to handle disaster recovery, failover, security and compliance issues etc. etc., you quickly run into all the reasons that infrastructure management services and other computing services - i.e., cloud - have won out so definitively.
If you’re small enough that none of those things matter very much to you, and knowledgeable enough to run your whole system yourself, and have the time and inclination to deal with it, go for it! But such people are outliers who are really indulging a hobby more than anything else. It generally makes extremely little business sense.
