Which CTOs did we survey?

• Experience: 100% were experienced CTOs with experience leading teams of 20+ FTEs
• Type of Company: 64% of them were CTOs at Scale-Ups, 21% were CTOs at Start-Ups and 14% were CTOs at enterprise companies.
• Work Set Up: The majority of respondents had a hybrid workforce, with just 4 respondents with remote-only teams and 1 respondent with a 100% in-office workforce.
• Annual Cloud Spend: 11% spent less than €25k a year, 29% spent €25k - €100k a year, 21% spent €101k - €500k annually, 4% spend €501k - €1M annaully and 36% spends over €1M annually.
• Cloud Providers: Two-thirds of respondents utilized multiple infrastructure setups (multiple clouds etc.). Of those that used only one Cloud provider, 54% used just AWS.Overall, 35% of respondents used AWS in setup, 29% used Google Cloud in their setup and 15% included Azure in their setup.

What does moving to on-prem mean?

Before we start, I just want to clarify that migrating from the cloud means different things to different people as ‘on-prem’ has different meanings. There are different ways to on-prem. When we asked CTOs what moving to on-prem meant, we found out that 64% thought moving from the cloud to on-prem meant moving from the public cloud to rented managed dedicated servers while 25% of them thought it meant buying and racking at a co-location. None of those asked thought it meant building your custom hardware with private network infra etc.

Factors

When it comes to flexibility, the cloud is often seen as the better option compared to on-premises infrastructure. CTOs often rank flexibility as the biggest challenge they would face if they moved away from the cloud. The cloud's flexibility is particularly useful for startups starting up, when scaling up or down, and during seasonal surges in demand. On-premises infrastructure can also be flexible, but it requires a significant investment in hardware and infrastructure. Cloud infrastructure is generally more expensive than on-premises infrastructure, but the flexibility and scalability it offers may make up for the higher cost in the long run. The best solution will depend on a careful evaluation of a business's specific needs and requirements.

Read more about our analysis into the flexibility of cloud vs. on prem here.

Availability is another huge selling point of the cloud. Many companies appreciate how the multiple regions offered by public cloud prevent failures from taking the whole system down (as might be in a data center). “Cloud just makes it easier and less expensive to have redundancy to accomplish 99%+ SLAs,” claims one LinkedIn User. 

However, availability ranked as a medium size challenge when it comes to moving on Prem and it is possible to get high-availability set up using on-prem.

What is easier to maintain - on-prem or the cloud? 

Maintaining infrastructure can be challenging, with constant upkeep and management required to keep it running smoothly. In a recent survey, CTOs identified maintenance and management as the second most challenging aspect of moving to on-premises infrastructure.

There is a lot of headache that goes into maintaining infrastructure on-premises, but the cloud does not have the best reputation in terms of support. CTOs like DHH, report recieving better support from their data center than from a public cloud provider. 

But do all CTO hate the service provided by their cloud? We asked them and you can find the results here.

Another thing that comes up in the cloud vs. on-prem discussion is stress. Are CTOs who use the cloud less stressed than those who use on-prem? Although both set ups have stress, an overwhelming amount of CTOs felt less stressed using the cloud - and the sentiment increased with the annual cloud spend.

How does the workforce requirement differ between cloud vs. on-prem. When we surveyed CTOs we discovered that indeed many agree that the cloud allows them to have a smaller team. We also discovered that labour would be their third most challenging concern if they migrated to on-prem with many wondering how they would attract and retain talent in a world of resume-driven developers and with much of the younger generations without the skills required. DHH argues that most skills transfer though and he is not looking for resume-driven developers. 

There is a debate over whether it is cheaper for companies to host their own infrastructure on-premises or use the cloud. Some companies, like Bank of America, have reported saving money by hosting their own infrastructure, while others, like Prerender.io, have reported reducing costs by 80% by moving away from the cloud. However, moving from the cloud to on-premises can incur management and migration costs. It may not always be the most cost-effective option for companies, depending on their tech stack and the speed at which they need to execute.

Most CTOs surveyed (83%) do not think that the money they pay each year for cloud computing would be better invested in owning their own infrastructure. 

However, there are hidden cloud costs that can be avoided. It is important for CTOs to monitor their cloud computing costs and ensure that they are operating in a cost-effective way through visibility, routine monitoring, and empowering developers to optimize cloud computing. Only 61% of CTOs agreed that developers are responsible for optimizing cloud compute and only 39% agreed that their developers are equipped with the necessary tools to manage cloud computing costs.

The majority of CTOs surveyed believe that cloud costs are a fact of life and are satisfied with the value for money they are getting from their current set up. However, CTOs that spend between €25k and €1M a year on cloud services tend to be happier with their set up than those that spend less or more than that amount. The cost-benefit ratio is an important factor in deciding whether to move to the cloud, and if the benefit shrinks it may be necessary for organizations to move away from the cloud.

Compliance: Is it easier on-prem or the Cloud?

There is no clear-cut winner here. On the one hand, the cloud offers controls and automation that make being HIPAA-, PIC-, SOC2-, and ISO- compliant much easier. But, as the largest CTO in the community in the EU, we know that sometimes using a foreign cloud provider makes things slightly more complicated. Solutions could be found of course but GDPR does make one look at things differently. 

We asked CTOs to give a rating between 1 and 5, with 1 being compliance is easier on prem and 5 being compliance is easier with the cloud. The average result was 2.8 which means that CTOs found compliance slightly easier on-prem.

But whether you go with on-prem or the cloud, the experienced CTOs we surveyed said compliance would be the least challenging factor if you move to on-prem. 

Security: Is it safer using the cloud vs on-prem?

Put it this way, it's easier to be safer using the cloud but only if:

• you have to know how to set up the cloud correctly
• If you really know security, going on-prem would allow you to use best practices.

“It’s brave to give up the security infrastructure the clouds provide. Worth a lot, difficult to replicate,” remarked Eric Bowman, CTO of Tom Tom, on LinkedIn regarding DHH’s announcement. 

DHH claimed in a ReWork podcast episode that you get a false sense of security on the cloud where you just think, “ [security] someone else's problem”. But it's important to realize that “security is not something Amazon does for us. Absolutely not. There are some provisions for it, but you absolutely have to know what you're doing. You have to set things up in the proper way. It is extremely easy actually to expose the wrong things, even if you're using a cloud, even if you're using fully hosted services''

But what about cloud breaches? Sometimes clouds like Azure have suffered their own breaches. But Eric Bowman remarked on LinkedIn that “The breach was caused by a team at Microsoft using Azure. The team made a security mistake - not the public cloud. Security is hard, and there are very, very few (known) breaches caused by improper security by cloud providers, and thousands caused by makers who got the security part wrong.”

To this point, Daniel Vexø, a freelance security and infrastructure architect, argues with Bowman saying “ if Microsoft’s own team can't even figure out how to secure their own cloud, then how can you expect anyone else to be able to? The cloud is just a loss of control at a higher price point.

When we asked CTOs if they would find security easier using the cloud or on-prem, we got mixed responses with the cloud being perceived as only slightly more secure. (A 3.1 average with 1 being ‘security is easier on prem’ and 5 being ‘security is easier on the cloud’). 

As Chris Mavrakis put it on LinkedIn - “It’s hard enough for security teams to secure the application layer, data flows, business logic etc. Handing responsibility for the physical & general infra over to a provider who does it at scale is welcome.”

But if you were to move to on-prem, then you should know that security was rated the second least challenging part of moving to on-prem.

The cloud computing industry is dominated by a few large companies, and this dominance has raised concerns about widespread outages, policing and control, and monopolies. In a survey of CTOs, 55% agreed that they were worried that if one cloud provider went down, most of the internet would go down. 64% of CTOs agreed that it bothered them that the decentralized world we are creating will be operating on computers owned by a handful of mega-corporations. Despite these concerns, the majority of CTOs surveyed still believed that the benefits of cloud computing outweighed the risks.

One of the ideas discussed on the alphalist CTO slack is to go for a hybrid approach - put some services on the cloud and others on-prem. 

You would decide on a service-by-service basis of what to use which set up for. In the full article you will see where CTOs currently host each service and where they would ideally like to host each service.

The hybrid set up is particularly useful when it comes to database services and search & analytics engine, where the cost can be high. Some CTOs have found success with replacing managed database instances with a managed k8s cluster, while others have rented virtual machines and installed the database there. This hybrid approach allows for cost savings while still providing the necessary services.

On-Prem vs. Cloud: How to Decide

“What is the best way to support the business given the team size and set of skills?”

There is no quick answer here. It all boils down to strategy and what your company needs. You need to look at your business model and decide if the cloud is worth the risk. 

Strategy as a service is needed more than ever. More teams probably need to follow the underlying logic here, not necessarily the move away from the cloud. Understanding actual costs to key decisions based on the business context (near-term, long-term goals, outcomes, market shifts) using proper team-scrutinized metrics. These metrics and decisions should also follow some science or explanation from respected experts, not a blog article.  -Jason Hamilton commenting on LinkedIn

As you said, it's a matter of running the numbers for your situation instead of following the hype.
No matter how well it works for others, if it doesn't fit your situation, it's not for you and that does NOT mean someone is doing it wrong. -Daniel Lopez, commenting on LinkedIN

Advice from CTOs

An experienced CTO shares their experience

I ran a lot of the platform, NETOPS and DCOPS teams at a large public cloud provider, we had millions of physical servers in 40+ global DCs, our own transoceanic fibre etc etc. The economic argument for moving away from the public cloud is 100% true, but only in VERY limited circumstances for production deployments, in my experience. Of the 100s of customers who said they fit into those circumstances, I reckon <10 actually did and eventually realised the economic gains. ....
If you have VERY large, VERY homogeneous workloads, and you are VERY good at capacity planning for growth/usage and you understand the cost structures involved then it can 100% make sense to move to your own DCs.

Other factors CTOs recommend you take into account:

• “Data privacy”
• “Consider the use case (a cloud solution is often not even necessary)”
• “Environmental impact “
• “Knowing the unknowns. Every shiny technology is great to implement a Hello World on, but also every technology gets complex and hard to manage over time. It's easy to misjudge it early on.”
• “Common sense”
• “the whole DHH position is in my view idiotic and based on an initial position of being in a company with a not particularly complex technical challenge. no idea why you're giving him oxygen.”

Conclusion: We need more education

We are missing fundamental cost analysis and trade-off education in cloud/infrastructure professionals and the relationships to the business to determine what model works best in the short and long-term contexts (or if it needs to be tweaked/changed). This leaves us at the mercy of the providers to perform the analysis for us.-Joel Wilbanks commenting on LinkedIn

This article is just the start. We need more neutral parties sharing their experiences on the matter. Thank you DHH for starting the conversation and for CTOs of the alphalist CTO slack community for taking the time to answer the survey and share their insight with others. It is only with accumulated knowledge we can accelerate growth. 

We would like to acknowledge and thank the CTOs who took the time to fill in the survey as well as Dennis Proppe and Hendrik Nehnes for taking the time to review this article. We would also like to thank the people quoted in this article. We welcome your feedback.