Building a dull life

Preparing to sleep through the coming security storms

The office building was modern. Walls of glass created a bright airy atrium. We passed through security and into an elevator. When the doors opened the decor screamed American office building; the same door handles, same carpet tiles, same fire alarms I’ve seen countless times. We turned left and then left again and found ourselves in a windowless conference room. So much for natural light.

It was February 2022 and I was onsite with a customer. My goal? Convince them that if you were deploying a product into public cloud you really, really needed to consider security. But they were risk averse - they had been burnt by a disastrous upgrade in the distant past; now their culture strongly discouraged upgrading. Upgrading once a year was once too often.

But the statistics are sobering. Between 2005-2015 ~6k security vulnerabilities (CVEs) were found per year. By 2017 it reached ~15k, and topped 20k in 2021. The direction of travel was clear.

In the end we agreed a 3 monthly release cadence, plus optional emergency patch releases which could be deployed in ~4 weeks.

Was it a success? Maybe. We had moved from nearly-never to an agreed cadence. But it was still a long way from best practice; even then four weeks was an eternity in security.

Security is dull

There are many reasons companies ignore security. It doesn’t add new features. It doesn’t increase revenue. At best, it might protect you from a terrible calamity. But you will probably never know - there’s barely enough time to consider all the things that actually happen never mind the ones that didn’t. Security is insurance. It’s all too easy to view it as an unnecessary cost.

But, like it or not, security debt accumulates. And eventually it leads to massive security pushes. Take Microsoft. They had their first in 2002 when Windows XP development was halted for multiple months to focus on security. Then again in 2010. And 2023. That last time ~30,000 engineers spent 12 months focused on security. I was one of them.

Enter AI

And now we add AI into the mix. If you need any proof of how AI has advanced in the past few months, take a look at this chart.

This is Firefox, a mature, extremely well tested browser. People have been actively looking for Firefox vulnerabilities for years. Conventional wisdom would say Firefox would be well hardened by this point.

And yet.

The number of fixes has increased by 20x YoY. This is not a good kind of YoY increase.

Or take this chart from XBOW:

Mythos is clearly better than any other model. But note GPT5.5 isn’t a million miles away. GPT5.5 is publicly available. It’s tenacious. I’ve had it run for over 29 hours on a coding task. It wouldn’t be hard to get multiple GPT5.5s searching for security vulns for days.

Most code is much, much less secure than Firefox. Think about all the public internet facing legacy code written 20 years ago in memory unsafe languages. Code that people are scared to change; code where the interop relies as much on the published APIs as the unintentional bugs and subtle timing. Code people are terrified to change, never mind reimplement.

But for an attacker with GPT5.5, it’s a rich and fertile playground.

What to do?

One option is to find a bucket of sand and put your head in it. But that was never a good idea. And now it’s even worse.

In reality, there are many things you could do. I’d start by reducing the attack surface area. Products collect legacy features; code that is little used. Or not used at all. Security is the perfect excuse to jettison it.

Then there’s OSS. At a minimum you want to only be using mature, battle-tested packages written in memory safe languages. But even that is no guarantee as last week’s NPM attacks show - TanStack and node-ipc both got compromised, and the infections spread to PyPI and affected MistralAI and OpenAI. These were nasty attacks - running "npm install" at the wrong time was all it took to get infected.

There’s a tension here. Being too far behind the latest versions leaves you vulnerable to old bugs. It’s a brave person who still hosts their website on Windows XP. But last week demonstrated being too close to the bleeding edge is risky too.

There’s no right answer. No easy one size fits all. It’s nuanced. It depends. It’s horrible.

Other options

Perhaps a better approach is to adopt the hacker mindset. Go probing your code for vulns. Let’s face it, there’s nothing stopping any of us spinning up umpteen instances of GPT5.5 and leaving them probing 24x7.

But, really, I think we need to start having a serious conversation about porting legacy code to memory-safe languages. It’s increasingly clear the language moat is rapidly receding - as I’ve discovered Claude and Codex make it much easier to port code between languages. It’s not trivial - you need good oracles and oversight - but it’s doable. And it’s something Anne Currie and I discussed at length on her fantastic Asynchronous and Unreliable podcast :).

And so?

There are lots of options. I’m fortunate; I don’t have to deal with legacy code. I live in a world of Rust. I minimize the amount of open-source I use; I build simple utilities rather than go searching for open-source versions. I port key libraries I need to Rust (e.g. Opus) to reduce my exposure to FFI wrapped C or C++.

And when a language that’s better than Rust comes along, I’ll be happy to port across to it.

The kind of success I seek is the dull kind. The kind where nothing interesting happens. I’m happy not to get woken up in the middle of the night. Or be chased down the street by a horde of angry customers.

It seems pretty clear that the next security storm is building. I’m preparing as best I can. But I fear much of the rest of the world has their head in their buckets of sand. The next few months are going to be interesting.