Reminders and Lessons from the CrowdStrike Incident

Last week’s CrowdStrike incident was a stark reminder of the importance of secure software development practices. While the specifics of the global event remain only partly known, we do know: 

• A buggy update caused widespread system crashes for CrowdStrike customers, including hospitals, banks, and airlines; 
• Some 8.5 million Windows devices were impacted;
• Software developers worldwide (with some obvious exceptions) watched with bated breath as the news broke, thankful it wasn’t their product in the spotlight; and
• These same software engineers were reminded of the critical nature of security practices and protocols in building and maintaining software.  

Among the key reminders our team took away from the CrowdStrike incident are these five:

1. Rigorous Testing is an Absolute Must

Thorough testing is the cornerstone of secure software development. This incident highlights the need to go beyond basic functionality testing and incorporate rigorous regression testing, edge case testing, and compatibility testing across diverse environments. Combining automated testing frameworks with manual testing by QA and other experts can significantly reduce the risk of bugs slipping through the cracks.

2. Secure Development Lifecycles are Essential

Implementing a Secure Development Lifecycle (SDL) ensures security is woven into every phase of the software development process. This includes secure coding practices, code reviews, vulnerability scanning, and penetration testing. An SDL fosters a culture of security within the development team, leading to more robust final products.

3. Be Prepared to Patch – Quickly

The ability to quickly deploy fixes and updates is critically important. While Crowdstrike’s prompt response in providing a patch should be applauded, the incident underscores the importance of having a well-defined patch management strategy. This includes clear communication channels with customers, efficient deployment methods, and rollback procedures.

4. Backups and Disaster Recovery Plans are Lifesavers

The ability to recover from unexpected events can make or break a company. Organizations relying on Crowdstrike’s security software were left scrambling due to the outage. This incident emphasizes the need for robust backup and disaster recovery plans. Regularly testing these plans supports a smooth recovery process in case of a software issue or cyberattack.

5. Transparency Builds Trust

While clear and timely communication is important to any software development effort, it’s especially so during a crisis. Crowdstrike’s transparency in acknowledging the issue and providing updates throughout the resolution process helped maintain customer trust. Open communication fosters a sense of partnership with customers, allowing them to make informed decisions during the disruption.

Want to chat about this topic – or learn how Revelry helps ensure secure software development? Connect with a member of our team. We love this stuff!

---

code audits code reviews communication CrowdStrike issues management patch management QA recovery security Testing transparency