CrowdStrike: the IT update that wrought global chaos

Somewhere, in an office of the cybersecurity giant CrowdStrike, someone must have had the worst day of their working life last week, said Matthew Field in The Daily Telegraph. 

The US tech company is not a household name, but it is a massive player in the industry: its software is embedded deep in the computers of some 23,000 corporations worldwide, including more than half of those listed on the Fortune 500.

With $3 billion (£2.33 billion) in reported revenues last year, CrowdStrike was valued at $80 billion (£62 billion) – until last Friday, when the firm that is supposed to protect IT systems caused the most significant IT outage the world has ever seen. A piece of code that it had pushed out as part of one of its routine updates to combat evolving cyber-threats contained a defect that caused Microsoft Windows to crash. 

The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

As a result, millions of people working in everything from hospitality to banking turned on their computers that morning, only to be greeted by the "blue screen of death". 

Relief and overreliance

In Britain, people woke up to find that Sky News had fallen silent, said The Times. Trains were cancelled as ticketing systems failed; Waitrose was among the retailers that could not accept contactless payments; Ladbrokes told its punters that it could not accept bets; thousands of GP surgeries were unable to access medical records or make referrals; some schools could not report pupil absences; and some hospitals were forced to cancel appointments for cancer treatments. 

Around the world, airports resorted to displaying flight information on whiteboards; security scanners failed; and passengers queued for hours to be checked in manually. Hundreds of flights to and from UK airports were cancelled on what was projected to be the busiest flying day since 2019, and the Government convened its COBR crisis response.

Among security experts in the US, the initial reaction was one of relief, that this was not a nation-state attack, said David E. Sanger in The New York Times. For two years, government cyberwarriors have been combatting Volt Typhoon, an allegedly state-supported Chinese operation to "pre-position" malware into US critical infrastructure for future sabotage operations: it is "designed to sow far greater fear and chaos" than was seen on Friday. 

But as the failure cascaded across industries, attention turned to the dangers posed by security software of this kind, said Joseph Menn in The Washington Post. To be effective, it must have privileged access to internal networks, so that it can "see everything"; but that means that if something goes wrong, the consequences, as we saw on Friday, can be catastrophic. 

And though fixing the faulty code was not complicated, it was still expensive and time-consuming, as computers had to be rebooted manually by specialists. For businesses, this should be a wake-up call, said Simon Pardo in the Daily Express. Many have been able to scale back their IT teams by buying systems from third-party providers; but that has left industries with an "overreliance on single vendor solutions" – too many eggs in one basket.

A 'perfect storm'

The CrowdStrike cascade would have been more contained, had Microsoft not got such a "stranglehold" on the corporate world, said The Observer. Most businesses use Windows. Add in the pressure on firms to boost their cybersecurity, and you have the conditions for last week’s "perfect storm". 

Of course, computing monocultures have advantages in terms of efficiency and so on, but they're not good for resilience. And hackers are aware of these vulnerabilities: indeed, it has been suggested that cybercriminals may turn out to have been behind the faulty code at CrowdStrike. There are basic lessons to be learnt from this fiasco – such as that updates should be rolled out in stages. More broadly, it highlights just how dependent we have become on "a complex web of technologies that few understand". The risk is that, because this defect was fairly quickly fixed, it will be seen as a hiccup, not as a massive red flag.

We've had plenty of those before, said Edward Lucas in the Daily Mail. In a "supply chain attack" in 2021, hackers (most likely Russian) used an update issued by a software firm called SolarWinds to gain access to the networks of thousands of organisations including the Pentagon. This year, there was a "near miss" when an engineer spotted that malware had been inserted into an update to a ubiquitous software package called XZ Utils. Were it not for his vigilance, hackers would have gained a "backdoor" to hundreds of millions of computers. 

In just a few years, we've rushed headlong into a digital future, embracing new technologies so fully that our world now barely functions offline. In the process, tech giants have become so rich and powerful that they've been able to evade accountability for any damage their products wreak. We've sacrificed security for innovation and convenience; sooner or later, we'll pay heavily for that.