One of the ideas discussed on the alphalist CTO slack is to go for a hybrid approach - put some services on the cloud and others on-prem. For example, the same CTO that advocated replacing a managed postgres instance with 2-3 k8s nodes, said this about going entirely on-prem: “going back to maintaining VMs / real machines, a data centre, cables, OS patches, routing, especially when you need ISO 27k1 and stuff? Never!”.

Hybrid: Deciding which Services to Put on Cloud and Which on Prem

Of course, implementing a hybrid approach is not without its challenges. Companies need to carefully evaluate their specific needs and determine which services are best suited for the cloud and which should remain on-premises. They also need to have the necessary infrastructure and expertise in place to manage and maintain both environments.

Here are some ideas we saw for what to do on-prem instead of the cloud:

Database Services + Search & Analytics Engine

The cost of database services (e.g. managed databases or RDS) and the cost of Search & Analytics Engine is quite high and on the alphalist CTO slack, some CTOs expressed how that these would be the things they would be most open to moving on-prem (while leaving other things on the cloud - thus a hybrid environment.).

One CTO relates: “ We're paying over half a million dollars per year for database (RDS) and search (ES). That can pay for a few FTEs so it might be sensible to insource Postgres and elasticsearch”.

Managed Database Services

Managed database services cost a lot. For some CTOs, the cost of a managed database is worth it but for others who want to save, there are 2 solutions we heard.

Managed Postgres Instance >> Managed k8s cluster

“Let's compare a managed service (such as Azure Postgres) and using a Postgres operator in your Kubernetes cluster. I found that many people still favor completely managed services but Microsoft and Amazon charge a high price for them. This is especially frustrating if you have idle loads.

At the same time, I see that Kubernetes operators for Postgres (see Zalando, Crunchy, Stackgres, and now also Cloudnative-PG) have a wide feature range from simple deployments to extremely complex deployments and you can run the load completely inside a Kubernetes cluster that may even exist in your organization anyway.

The load of your database setup and the load of your other applications can then be distributed in your Kubernetes node pools. Unless you only run very busy, high load services, you get quite a bit of synergy and cost savings from it. 

Till Gerken

Instead of paying 300€/month for a managed Postgres instance, you can invest the same amount of money into 2-3 k8s nodes that serve the same load by using a Postgres operator - but those nodes can then also run other loads. It's easy to configure and maintain. If you have a DB admin anyway, it doesn't matter for them if they look at the Azure dashboard or a k8s/Datadog dashboard.” suggested a CTO on the alphalist CTO slack. He continued “ We recently looked at a company that had 10k€ monthly spendings for a managed service infrastructure that we host in our (managed) k8s cluster for 600€/month.” -Till Gerken

Managed Database instance >> Rented Virtual Machine with Database installed

Instead of renting a database service in the cloud, you might find it cheaper to rent a virtual machine and install the database there. This was suggested by Antti Tolamo on LinkedIn who points out: “Many databases are not easily scalable and they have to have their own powerful instances. Plus network traffic, CPU cost, and fast disk allocation costs. And they probably have to man with some teams those servers.”

“I think this is the difference between service and infrastructure. Databases are a service. While setting up our own systems is infrastructure. Cloud is probably cheap if you set up the system as infrastructure, which means you save money compared to traditional servers.”-Antti Tolamo on LinkedIn

Other CTOs don’t find the expense of their managed database to such a concern. As a CTO shared on CTO slack

“I paid last month around 2k for managing Postgres to GCP and I'm sleeping really well at night. On the other hand, I paid 9k for BigQuery last month and I'm at least thinking about perspectively replacing it with for example Clickhouse and putting someone internally on the topic instead.” - CTO Slack

Developer Environment

A CTO explained on Slack their hybrid approach to developer environments. “ Every dev can launch cloud-based environments of our platform via Slack to develop against or test on. Those don't run on GCP but instead on Hetzner. In the end, it's all Kubernetes, so you don't really feel the difference as a dev.”

As Guenther Haslbeck points out on Linkedin: “But developing in the cloud (eg to support distributed teams and so on) does not mean “run“ in the cloud. I also use AWS/Cloud9 and so on for projects but they do run on real servers at a German provider…”

What do CTOs say about hybrid (cloud + on-prem for different services)

What do CTOs say? We didn’t see this support for hybrid in the mini-survey we did. In fact, 41% responded that ‘the ideal setup is 100% in the cloud.” Although we did see some support for moving the following on-prem - Database Services, Backups Developer Environments, and Search and Analytics Engines- it was not at the rate we expected. 

What do CTOs do On-Prem

If you have a very mature DevOps team and you can hire folks to do hardware maintenance, don’t experiment with new tech much - you can do everything on-prem. But few CTOs want that challenge and in fact, none of the CTOs we surveyed did everything on-prem. 59% were not 100% cloud-based. CTOs are doing a variety of things on-prem (hosting, database services, back ups, Search and analytics engine, developer environments etc.) as well as Open-Source-Tools as a replacement for expensive SaaS-Tools. Other things done on-prem were machine learning, virtual desktops, and infrastructure security solutions. 

What is interesting to note, for every service, not every CTO that currently did it on-prem thought it was an ideal setup. (‘Currently on Prem’ was higher than ‘Ideally on Prem’).

Conclusion

One of the ideas discussed on the alphalist CTO slack is to go for a hybrid approach - put some services on the cloud and others on-prem. This is particularly useful when it comes to database services and search & analytics engine, where the cost can be high. Some CTOs have found success with replacing managed database instances with a managed k8s cluster, while others have rented virtual machines and installed the database there. This hybrid approach allows for cost savings while still providing the necessary services.