Cloud Vs.On-Prem Guide

Cost of On-Prem vs. Cloud


Is it cheaper to host everything on-prem and pay for maintenance and workforce or is it better to host on the cloud and have all that taken care of for you?

Cloud is expensive, as one CTO remarked on the alphalist CTO slack “ Running hundreds of web servers is expensive. What they charge for network egress is outrageous, plus also managed databases, etc. are pretty much priced what they are - an expensive VM with some managed service fee on top.”

Rent vs. Own

Another big proponent for going on-prem is ‘rent-vs-own’. As DHH says of his new set up ”we're not renting these machines. We bought it. There are no payments every week or month due on the rent of the machine itself. Their aspects we ‘rent’ (e.g. pay for power, bandwidth, and some services at the data center where we rent our cabin), but we’ve bought the core hardware” 

In addition, DHH is happy that he has been able to pay off their data centers over the past few years, and they continue to service their business even though the data center is now older. As his colleague Eron Nicholson, Director of Operations of 37signals relates on the reWork podcast, a lot of the servers they use now are nearly 6 years old, but they are doing just fine and are more than paid off. Even for the newer ones, the bill is about a fifth of what they would pay every month for the cloud. 

What companies saved money by moving away from the cloud?

DHH is not the only one saving money by moving on-prem.

As discussed in the alphalist CTO newsletter - managed to reduce their annual server costs by 80% — from $1M to $200k — by moving away from Amazon Web Services (AWS). 

Furthermore, Bank Of America is reported to be saving $2 Billion a year through its own cloud infrastructure. Bank of America had to spend a lot to build its own cloud, including a $350 million charge in 2017 but this decision has led to $2 billion in annual infrastructure savings as it reduced Bank of America’s servers to 70,000 from 200,000 it’s data centers from 60 to 23.

Furthermore, Dennis Proppe reports that CrossEngange does most stuff on-prem and they would probably “have a business-model-challenging cost increase if we would just move it to the cloud. It is real-time big data processing though, so that may be a niche.”

Is it cheaper to own your hardware and host on-prem?

DHH implies that if the hardware is getting cheaper and cheaper each year - why are we letting cloud companies get such great markups if we can do it ourselves?

Hendrik Nehnes points out that that really depends on the company and stack. If the applications were built for the cloud, e.g. serverless it will be much cheaper to run the same workload and applications in the cloud as on-prem. 

Even if it does work for your tech stack, moving from the cloud to on-prem will incur 2 types of costs: Management Costs and Migration Costs.

Management Costs

“I assume the costs might just be shifted from Cloud spend towards more maintenance and tool building/configuring.” commented one CTO on the survey.

”It is impossible to find the right people and it is incredibly expensive to develop services of the same quality as we have in the cloud,” remarked another experienced CTO.

Dennis Proppe remarks “This is for sure a very valid point and something I have also seen several times - It might not be about the actual money or even about the total cost of ownership - It might boil down to the speed of execution to get to a stage where one can actually build products on top of the stack. I think the cloud has a tremendous advantage in terms of speed-to-market”

The Cost of Migrating from Cloud to On-prem

Hendrik Nehnes points out that you cannot easily migrate serverless cloud applications to on-prem without rewriting them. 

Furthermore, he adds “if you want to replicate the level of availability and stability from the cloud to on-prem you will incur a big cost e.g. at least 3 independent data centers, clustered applications, and databases... [Migration] might work out for huge companies like Bank Of America but not for mid-size players.”

What do CTOs think?
So we asked CTOs if they think that the money they pay each year for the Cloud could be better invested in owning their own cutting-edge servers.

83% of the CTOs said no. 

Case in point, of the seven CTOs who said they were paying over €1M in cloud costs a year, five disagreed that the money would be better spent on their own infrastructure.

With such a high bill, I am sure they ran their numbers. 

Avoiding Hidden Cloud Costs

David Jeffrey Ebreo said on LinkedIn.” it seems that the actual value of the cloud is not really in minimizing costs but having some control over when costs incur if at all” .

But it isn’t so simple.

Dennis Proppe reminds us “A lot of companies are very much surprised how cloud costs can go off the rails very quickly, as it is very important to set up a tight cost-controlling system (automated ideally) with the cloud, or it will be very hurtful.”

Smart CTOs know that one needs to monitor their cloud computing to make sure they are operating in a cost-effective way. This is reflected through visibility, routine monitoring, and developer empowerment. 

  • Visibility - It's great when cloud vendors give CTOs the visibility insight they need into their computing costs so they know where to save. 58% agreed or strongly agreed that the cloud company they use makes it easy for them to understand their bill. 25% disagreed or strongly disagreed. One of the CTOs that agreed stated: We use DoIT and their billing analysis. AWS is less transparent than GCP but it’s due to the fact that it was set up long ago.
  • Monitoring - When you use the cloud, you need to be on top of response time and cost. You need to check your bill frequently to make sure that some auto-scaler is not getting out of hand. 61% of CTOs said they check their computing bill every month with 25% checking their bill every week. 60% of those with a computing bill of over 1 million euros checked their computing bill every week.
  • Developer Knowledge -Of course developers need to be involved in the efficient optimization of cloud computing costs. We asked CTOs:
    1. Whether developers are responsible for efficient cloud computing (writing good code and knowing where to install what)
    2. Whether they equip their developers to perform that role.
    61% of CTOs agreed or strongly agreed that ‘Developers are responsible for optimizing the cloud compute of their code - using hardware knowledge to write effective code and know how to allocate computing resources’ however only 39% agreed or strongly agreed ‘My developers are equipped with the tools they need to understand the cost of computing and can manage it accordingly. 

CTOs Attitude to Cloud Expenses

Do they feel they are getting value for money 

When we asked CTOs “How satisfied are you with the value for money you get at your current set up”, it seemed that CTOs that spent between €25k and €1M a year were happier (NPS 7) than those that spent less (NPS 5) or more (NPS 5.6) than that amount. Dennis Proppe, CTO of CrossEngage noted on this stat that it appears the “biggest cloud customers are not as happy as the medium spenders. It makes sense that the smallest ones are not happy, because mostly they might be small, inexperienced (thus more often negatively surprised) and maybe only dabbling - But as I come from an enterprise sales company, normally the NPD should go up if you spend more: You get more focus, better support, even some tech solutions that are only made for you. I think this goes well with the citation above that even at 3m$+ of yearly cloud investment, a company is still not a large client. So there might be a clash between expectation on the client side and willingness to treat these customers special on the cloud provider side.”

How do they feel about their current cost spending?

Many CTO seems to feel that cloud costs are a fact of life. One CTO writes ’We are doing some optimization, but expect to keep about 6% - 10% of revenue for the cloud cost’

When asked in October 2022 about their current cloud spending and the answers reflected their current spending level. Of those paying over a million euros a year, 50% would like to marginally reduce it with one saying they are doing this by increasing workload and reducing spend and another saying they are trying to become more efficient and negotiate better prices. 66% of those spending between €101k - €500k annually would like to keep their current bill to stay the same and not take on any more additional spending while 71% of those paying between €25k - €100k annually would like to marginally increase their spending.


The cost is what got this discussion started in the first place. Cloud costs are high and they are at the mercy of the cloud vendor. But from speaking to CTOs, it seems like many would rather pay a cloud vendor than have the headache of managing it themselves.

I ran a lot of the platform, NETOPS and DCOPS teams at a large public cloud provider, we had millions of physical servers in 40+ global DCs, our own transoceanic fibre etc etc. The economic argument for moving away from the public cloud is 100% true, but only in VERY limited circumstances for production deployments, in my experience. Of the 100s of customers who said they fit into those circumstances, I reckon <10 actually did and eventually realised the economic gains. ....
If you have VERY large, VERY homogeneous workloads, and you are VERY good at capacity planning for growth/usage and you understand the cost structures involved then it can 100% make sense to move to your own DCs. -CTO on alphalist Slack
The cost factor plays a major role in deciding the move towards the cloud and if this benefit shrinks it’s imperative that organizations take the decision to move back. Cloud-metered services should correspond to organizations business objectives and should be well aligned with the organizations growth strategy. Non Organic growth creates a challenge for resource allocations, cost, performance, and user experience. In a long run, it's the Business that needs to survive, and having high Op ex with no fixed projection of cost is a big risk. -Sher Ali Khan, commenting on LinkedIn

Cloud vs. On-Prem

What aspect of cloud vs. on-prem would you like to explore next?

Cloud Vs. On-Prem


Cloud Vs. On-Prem


Cloud Vs. On-Prem


Cloud Vs. On-Prem

Maintenance and Management

Cloud Vs. On-Prem


Cloud Vs. On-Prem

Security and Compliance

Cloud Vs. On-Prem


Cloud Vs. On-Prem

The Hybrid Set Up