The Crowdstrike Incident
Summary
In July 2024, a faulty software update from CrowdStrike, a leading cybersecurity firm, caused widespread system crashes on millions of Windows computers globally. This incident disrupted critical sectors such as transportation, healthcare, and finance, highlighting the vulnerabilities inherent in modern IT systems. This article gives us explanations about the technical backgrounds which are leading finally to the incident.
What is CrowdStrike?
CrowdStrike is the name of a publicly traded US company that provides information security and cybersecurity solutions. The company was founded in 2011 and has also assisted in investigating various cases of economic cyber espionage, including on behalf of the United States Department of Justice. Google is one of the main investors, and the company now has a market capitalization of about 65 billion US dollars. In 2013, the software product CrowdStrike Falcon was launched. CrowdStrike Falcon is a widely used Enterprise Detection and Response (EDR) protection software against cyber attacks. The product uses AI and machine learning technologies, among others, to ensure the protection of computer systems.
The Incident
On July 19, 2024, CrowdStrike released a update — Channel File 291 — for its Falcon Sensor software. This update introduced an additional input parameter (21 instead of the expected 20), which caused an out-of-bounds memory access. The error led to system crashes characterized by the “Blue Screen of Death” (BSOD) on affected Windows systems.
Technical Background
CrowdStrike Falcon
Like any software product, CrowdStrike Falcon requires maintenance. The central security scanner agent is called “CrowdStrike Falcon Sensor” by CrowdStrike. This needs constant updates to be able to respond appropriately to current threat situations and security vulnerabilities. During operation, software updates are regularly rolled out using so-called Channel files. CrowdStrike distributes dynamic updates and detection rules using the Channel files.
Chronology
On July 19, 2024, numerous system failures were detected worldwide. It was quickly determined that the affected systems were using CrowdStrike Falcon as protection software. However, this incident was not a classic cyberattack. Rather, this error occurred in connection with a regular standard procedure - the software update process.
In February 2024, a new feature was rolled out for the “Falcon Sensor”. This new function was intended to improve the detection of threats and attacks using Windows mechanisms. As part of internal stress tests, Rapid Response Updates (designation for special updates for Falcon) were tested. No abnormalities were detected here. The tests took place in March and April 2024.
On July 19, 2024, the Rapid Response Content Updates were then deployed to certain Windows hosts. Specifically, this was Channel File 291. During the root cause analysis, it was determined that the sensor expected 20 input fields, but the update provided 21 input fields. This led to a memory access outside the specified memory addresses, which ultimately resulted in a system crash - the Blue Screen of Death (BSOD).
We now want to discuss the technical background of the Rapid Response Updates. For further understanding, here is an explanation of how the Falcon Sensor software works. The CrowdStrike Falcon Sensor is an advanced security system that uses artificial intelligence and machine learning to detect and defend against malicious activities on computers. It continuously collects data from protected devices and analyzes it to identify suspicious behaviors. Both known and new attack patterns are detected. To stay up-to-date, the sensor’s protection mechanisms are regularly updated and improved through insights from the analysis of real threats.
The Faclon sensor is needed to be updated to react to the newest cyber
- Faulty Update: The update included 21 parameters while the Falcon Sensor was designed to handle only 20.
- Kernel-Level Access: The Falcon Sensor operates in “Ring Zero,” granting it deep access to Windows kernel resources. This design amplified the impact of the error.
- Race Condition: The file triggered a race condition during system startup, further contributing to crashes.
Impact
Approximately 8.5 million systems were affected globally. Key disruptions included:
- Airports: Flight delays and cancellations.
- Healthcare: Hospital systems were disrupted, delaying critical operations.
- Financial Services: ATM networks and payment systems experienced outages.
- Media: Sky News temporarily ceased broadcasting.
Linux and macOS systems were unaffected due to their different security architectures.
Response and Mitigation
- Manual Recovery: Affected systems required booting into safe mode to delete the faulty file manually.
- Patch Deployment: CrowdStrike released a corrected update within an hour of identifying the issue.
- Lessons Learned: The incident underscored the need for rigorous testing protocols and robust emergency response plans.