alphalist Blog

Pentests, Partners and Policies: Tips for CTOs in 2024


What do you need to get your company cyber ready? Dr. Melanie Rieback, co-founder and CEO of Radically Open Security gave advice to CTOs on this topic in a recent alphalist CTO podcast. This is what she said.

Know your crown jewels

It is challenging to meaningfully protect everything because when you try to protect everything, you're going to wind up protecting nothing. So instead, you just need to figure out what it is that you need to protect, isolate that, and make sure that you have the strongest security just on that. You should also know what are your crown jewels. What do you think  your crown jewels are and what does the attacker think are your crown jewels (because those two things are not always the same.) 

Find a Partner in Pentesting

You don't know how to defend your company until you understand how to attack your company. 

This is where PenTesting comes in. 

Of course, you probably won't know how to attack your company because pen testing is really a skill that takes decades to learn and perfect. Within Radically Open Security, we have this peek-over-our-shoulder method, which is basically our pen testing workflow that we work in chat rooms online. We use something called RocketChat, which is an open-source, self-hosted clone of Slack. And we invite the customers into our chat rooms to actually observe and interact with our hackers while we are busy breaking your stuff. 

The whole point of it really is just to transmit the hacker mindset to everyone involved ( particularly to developers, but also security officers sysadmins, DevOps people). By being part of the process of breaking the stuff, then they understand why those problems got there in the first place.

After all, security is a long-term process and mindset, not just a set of patches that you get from a pen test report. 

So as CTO, make sure that whichever vendors that you're working with include you in the process and educate you while it's happening. Because ultimately we're going to leave, right? You're paying us by the hour after. and the real question becomes, are you still able to handle things correctly after we're gone?

This openness and transparency make things efficient for both parties. With peak-over-the-shoulder pen tests, the customers are oracles for us. If we get blocked, if we have a question if a server needs to be restarted. Just having the customer there with such short communication lines is super handy and super efficient.

 Internal Teams are great, but only if you can afford talent.

It's always a good thing to invest in your internal capacity for cybersecurity. However, that assumes you have the financial resources to do so. 

For larger organizations, it's realistic to build your own internal red team, CSIRT, security architecture, and other security departments. 

However, not every organization has that budget or capacity or is even an attractive enough place to attract quality cybersecurity professionals.  If you are a tiny company and security is not your focus, probably the best idea is to hire an external company. It isn’t just cheaper but also gives you access to better talent as smart security folks want to hang out with other smart security folks most of the time. So often it's feasible to be able to get that quality cybersecurity service by hiring an external company if you are a small company. 

 You must look for a trusted external partner to work with. Underline PARTNER as security should be a learning experience, not a transaction. 

Core and Extended Teams

Where you are big enough to have an internal team, you should always have a kind of a core team and an extended team. Even if you're a larger entity like a bank, it's still necessary to have external experts on speed dial (or rolodex or yellow pages). Ideally, these are people who you have negotiated rates with in advance as that means you get much better pricing.


Security is a mindset, not a list of tests. Ideally, one should one have an in-house cyber team - but only if they are big enough to attract top talent. If they can't get top talent, it is best to partner with an external provider. Make sure you have a partner in your external pentester - perhaps one that provides over-the-shoulder pentesting like they do at Radically Open Security. This way, your team can improve their skills as well.

Melanie Rieback

Melanie Rieback

CEO/Co-founder @ Radically Open Security

Dr. Melanie Rieback is CEO/Co-founder of Radically Open Security (the world's first not-for-profit computer security company), and "Post Growth" startup incubator Nonprofit Ventures. She is also a former Assistant Professor of Computer Science at the Free University of Amsterdam. She was named "Most Innovative IT Leader of the Netherlands" by CIO Magazine (TIM Award) in 2017, and one of the "9 Most Innovative Women in the European Union" (EU Women Innovators Prize) in 2019. She is also one of the 400 most successful women in the Netherlands by Viva Magazine (Viva400) in 2010 and 2017, and one of the fifty most inspiring women in tech (Inspiring Fifty Netherlands) in 2016, 2017, and 2019. Her company, Radically Open Security was named the 50th Most Innovative SME by the Dutch Chamber of Commerce (MKB Innovatie Top 100) in 2016.