How to Master Incident Response like a Pro
By James Rees, MD, Razorthorn Security
Introduction
In the world of information security, we love to believe that our countermeasures, defence in depth strategies and preventative controls will shield us from disaster. We invest in technology, develop policies, train our people and implement procedures – all in the hope that we’ll never face a serious security breach.
But as any seasoned security professional will tell you, incidents are inevitable.
At some point, you will face that moment when an incident kicks you square in the face and demands immediate attention. You can’t ignore it. You can’t run away from it. You can’t deny it’s happening (although plenty try). When that moment arrives, how you respond will define not just the outcome of the incident, but often your reputation as a security professional.
Throughout the industry, it’s widely acknowledged that whilst you can get many things wrong in information security, the one thing you absolutely must get right is incident response. When all carefully constructed defences fail – and eventually, somehow, they will – incident response becomes an organisation’s last line of defence.
I. The Psychology of Incident Response
When a security incident unfolds, the first battle isn’t with the attacker or the compromised system – it’s with psychology.
The Critical “Don’t Panic” Moment
Military training emphasises taking a “Condor moment” – that brief pause to scan your horizons, assess the situation and decide on your next move. This initial response is crucial. Without proper training, organisations often descend into chaos with people effectively running in circles in sheer panic.
You’re allowed your “Oh cr*p” moment when the cold realisation hits. What matters most is what follows.
The Schrödinger’s Cat Moment
Many incidents begin with uncertainty – what we might call the “Schrödinger’s Cat moment.” Is there truly an incident, or is this a false alarm? How serious is the situation?
Like Schrödinger’s theoretical cat, the true nature of your security incident remains unknown until investigated. This creates a psychological hurdle as some team members downplay the situation while others catastrophise it. The solution? Look inside the box and determine the facts.
Emotional Detachment and Objectivity
Maintaining emotional detachment during incident response is challenging, particularly when investigating potential insider threats involving colleagues or friends.
The investigation must be fact-based and objective: What occurred? Where did it occur? How did it occur? Why did it occur? Who was involved?
This detachment doesn’t mean being cold or uncaring. Rather, it means approaching the incident with the objectivity needed to reach the truth, even when that truth might be uncomfortable or unexpected.
II. Applying Military Principles to Incident Response
Security incidents demand decisive action under pressure – a situation where military experience provides valuable insights. The crossover between military incident handling and information security response is remarkably strong.
Drills and Repetitive Training
The military operates on a foundation of drills – performing the same actions repeatedly until they become second nature. This muscle memory proves invaluable when confronted with crisis situations.
For security teams, this translates to regular practice of incident response procedures. By running through scenarios repeatedly, teams develop natural reactions rather than panicked improvisation. When the real incident occurs, trained professionals fall back on established protocols rather than freezing in uncertainty.
Team Trust and Defined Roles
A critical element of effective incident response is trust in your team members. When an incident occurs, everyone must understand their roles and trust others to fulfil theirs. This interconnected response capability doesn’t develop spontaneously – it requires deliberate cultivation.
Security teams benefit from clearly defined responsibilities during incidents. Knowing who leads the technical investigation, who handles communications and who makes strategic decisions eliminates confusion during high pressure scenarios.
From Battlefield to Boardroom
Military personnel learn to assess situations quickly, make decisions with limited information and adapt to changing circumstances – all whilst maintaining composure. These skills translate perfectly to managing security breaches.
The ability to take that critical moment to assess before acting, to communicate clearly under pressure and to maintain focus despite chaos are invaluable traits that security professionals can cultivate through disciplined practice.
III. Building Effective Incident Response Protocols
Effective incident response doesn’t happen by accident. It requires thoughtful planning, clear categorisation and regular testing.
Incident Categorisation Systems
Every organisation should develop a system for categorising incidents based on severity and impact. A common approach uses priority levels – P1 incidents representing the most severe scenarios demanding immediate all-hands response, while lower priorities allow for more measured approaches.
A ransomware attack typically qualifies as a P1 incident for most organisations, requiring immediate attention from the entire security team and often the broader business. Other incidents might be classified at lower priority levels based on their impact, scope and urgency.
Specialised Playbooks
While having an overarching incident response plan is essential, organisations benefit from developing specialised playbooks for common scenarios. These detailed response guides provide specific steps for different incident types.
For instance, a ransomware playbook might include phases for containment, decision points about payment considerations, communication templates and recovery procedures. Similar playbooks can address data breaches, insider threats and service outages.
Each playbook should integrate with the overall incident management framework whilst providing specific guidance for the particular threat. This approach balances consistency with specialisation.
Documentation Throughout the Process
During incidents, thorough documentation often falls by the wayside as teams focus on immediate response. However, maintaining proper records proves crucial not only for post-incident analysis but also for potential regulatory reporting, legal requirements and insurance claims.
Documentation should include timeline information, actions taken, evidence preserved and decision rationale. This information becomes invaluable during the lessons-learned phase and helps demonstrate due diligence to stakeholders.
Properly documented incidents also provide valuable training material for future team members, creating an institutional knowledge base that strengthens organisational resilience over time.
IV. Communication During Incident Response
When security incidents occur, communication becomes as crucial as technical response. Poor communication magnifies damage, while effective communication minimises it.
Internal Communication Strategies
Designate a communications coordinator to shield technical responders from interruptions while keeping stakeholders informed. Establish regular status updates on a predetermined schedule to manage expectations rather than promising immediate answers.
External Communication Considerations
External communications carry significant reputational and legal implications. Avoid empty platitudes like “we take security seriously” and instead provide factual, clear information about what happened, what steps you’re taking and what actions affected parties should consider.
Templated Response Frameworks
Prepare communication templates before incidents occur for faster, more considered responses during crises. These should cover various scenarios and stakeholder groups, including customers, employees, regulators and media.
V. War-Gaming and Scenario Planning
No incident response plan survives first contact with reality unless thoroughly tested. War-gaming helps identify gaps in plans and builds team cohesion.
Approaches to Scenario Development
Effective scenario planning should include:
- Top 10 Scenarios: Most likely incidents based on your threat landscape
- Partial Availability Scenarios: Systems remain functional but compromised
- Full Failure Scenarios: Complete outages requiring rebuilding from backup
- “Purgatory” Scenarios: Ambiguous situations creating significant uncertainty
Beyond Traditional Scenarios
Unconventional scenarios often reveal surprising insights. Exercises with outlandish premises encourage creative problem solving and reveal team dynamics that standard exercises might miss.
Making Training Engaging
Escape room style exercises transform mandatory training into memorable experiences that build genuine capabilities while creating team cohesion. The most effective teams emerge from groups that have weathered significant challenges together, building trust that proves invaluable during actual incidents.
VI. Human Factors in Extended Incidents
Security incidents don’t always resolve quickly. Major breaches can require days or weeks of intensive response, creating significant human challenges alongside technical ones.
Team Rotation and Shift Management
During extended incidents, establish clear shift patterns to prevent burnout. No one can effectively manage high-stress situations for 24 hours straight. Military approaches to “stag on, stag off” (taking watch in shifts) apply perfectly to security incident management.
Create primary and secondary roles for critical functions to ensure continuous coverage without exhausting key personnel. Document handover procedures to maintain continuity between shifts.
Physical and Mental Support
Practical support matters during extended incidents. Ensure teams have access to food, drinks and comfortable rest areas. Consider arranging hotel accommodation for staff working through the night if travel home would be impractical.
Allow regular breaks for responders to step away, clear their minds and return with fresh perspective. These breaks often lead to breakthrough insights that continuous work might miss.
Recovery After Incidents
After resolving major incidents, acknowledge the toll on your team. Provide time for recovery and reflection. The traditional post-incident pub gathering serves a genuine psychological purpose – creating space to process shared experiences.
VII. Post-Incident Activities
The work doesn’t end when systems return to normal. The post-incident phase offers crucial opportunities for improvement.
The “Five Whys” Approach
Never settle for surface level explanations. The “five whys” technique involves repeatedly asking why a problem occurred to reach its root cause. For example:
- Why did customer data leak? Because a staff member emailed it externally.
- Why did they email it? Because they needed to share it with a partner.
- Why did they use email? Because the secure file sharing system was down.
- Why was it down? Because maintenance overran.
- Why wasn’t this communicated? Because the change management process failed.
This approach identifies fundamental issues rather than just addressing symptoms.
Updating Security Controls
Use incident insights to strengthen security controls. If an incident revealed a vulnerability, don’t simply patch it – consider what similar vulnerabilities might exist elsewhere.
Document all findings in your risk register, showing both the incident impact and your remediation plans. This documentation proves invaluable during security reviews and budget discussions.
Measuring Impact
Quantify the incident’s impact whenever possible. Track costs including response time, lost business, recovery expenses and reputational damage. This data helps justify security investments and prioritise future improvements.
Remember the security maxim: “Never waste a good crisis.” Major incidents often create the momentum needed for long-overdue security enhancements.
Conclusion
Security incidents are inevitable. Even with the most robust preventative measures, determined adversaries, human error or technical failures will eventually create security challenges. The differentiating factor between organisations that weather these storms and those that suffer lasting damage is effective incident response.
The military-inspired approach to incident management – built on training, preparation, clear communication and disciplined execution – provides a proven framework for handling security crises. By developing muscle memory through regular drills, security teams move from panicked reactions to methodical response.
Ultimately, good incident response reveals organisational character. It demonstrates resilience, adaptability and professionalism even under extreme pressure. Organisations that embrace this challenge emerge stronger, with improved security posture and enhanced team cohesion.
Call to Action
Review your incident response plans today. When was the last time you tested them? If the answer isn’t within the last quarter, you’re overdue.
Start small with a tabletop exercise involving key stakeholders. Progress to more complex scenarios that challenge assumptions and build response capabilities. Make these exercises engaging and relevant to your specific threats.
Document what works and what doesn’t. Update your playbooks based on these insights. Share lessons across the organisation to build a culture of security resilience.
Remember that effective incident response isn’t just about technology – it’s about people, process and preparation. Invest in all three to ensure your organisation stands ready when (not if) security incidents occur.
Join us for more cybersecurity insights on the Razorwire podcast.
Get in touch to discuss how Razorthorn can help with your cybersecurity requirements.
TALK TO US ABOUT YOUR CYBERSECURITY REQUIREMENTS
Please leave a few contact details and one of our team will get back to you.