Picture it: you’ve designed, built, and exercised your first Crisis Sim. You're pleased with the scenario and satisfied to see your team sharpen their skills, deepen their understanding, and boost their incident readiness. You can bask in the glory of this job well done for a moment, but the journey of the Crisis Sim doesn’t end here. The devil is in the details of the exercise data. 

Completing the exercise and gathering the results is only the beginning of your journey of fostering people-centric cyber resilience! 

Not sure where to start? We’ve got you covered

Remember how meticulously you mapped out those injects and options to build your scenario? The feedback options, the performance indicators, the branching paths, the exercise types? Your hard work is about to pay off. We’ve processed the exercise responses for you because you’ve earned it – and because there’s more work to be done.

Next steps for managers

Crafting outcomes from outputs

You can expand on the work you’ve already put into the exercise by leveraging both the Results and the After Action Report (AAR) for your scenario in the Immersive platform.

Follow these steps to access these items:

1. Go to Crisis Sim in the Exercise tab.
2. Locate your exercise. Hint: use the filters available on the left to show “ended” exercises.
3. Click to open your “Ended” exercise. 

From there, you’ll see how to dive into the available outputs with a few clicks!

If you need a bit more info, here are some additional guides from our Help Center:

• Where to find Crisis Sim exercise results & reports
• View Results 
• After Action Report (AAR)

Analyzing exercise results

Results

If you’re looking for granular data down to the details of each inject, you can find it here. In Results, you’ll see an overview including the summary from the exercise scenario, along with key details such as scoring and completion metrics.

Need to examine responses to specific injects? In the platform, you can quickly drill down into each inject by using the navigation on the left-hand side of the report. By selecting an inject, you can review responses and start to see patterns that emerged throughout the exercise.

If you’d prefer raw data, you can export a CSV file of your results. It's straightforward, packed with detail, and puts all the key metrics and figures within easy reach. Check out our documentation for more details on key information and metrics.

This is an invaluable resource for anyone passionate about data! It allows you to establish a foundation, set comparative standards, and ultimately gauge and improve your cyber resilience – all with concrete data to back your efforts.

If the mention of statistics and spreadsheets doesn't excite you, no worries, the Immersive platform generates an After Action Report for you 30 minutes after completion of your exercise. 

After Action Report (AAR)

Enter the After Action Report! The AAR presents an interactive visualization of your data analysis, offering valuable insights at your fingertips. And, as a bonus, you can download it as a PDF. The AAR is more than a deliverable; it’s a guide to fostering a people-centric cyber resiliency culture.

It offers an outline of the exercise and crucial data points that will help drive what you and your team do next. Overall performance, inject-by-inject analysis, and participant breakdown provide a comprehensive view of your team's current capabilities and readiness, wrapped up with relevant recommendations for you and your team.

Remember, insights are only available for data that’s collected as part of your exercise, so make sure you offer ranked inject options and enable response confidence and feedback to maximize your exercising. This is defaulted in the Immersive Crisis Sim Catalog presentation scenarios.

In the performance overview of the AAR, you'll encounter a high-level snapshot guide for your next steps. Think of this as a performance gauge (based on our experience with Immersive clients) that maps to the following:

>=75%: Excellent

>=50%: Good

>= 25%: Fair

>=0%: Needs improvement

As you dive deeper into the AAR, these broader performance indicators unfold with more granular data, and you’ll be able to understand the gaps that exist in cyber resilience for your organization. 

Mind the gap

By understanding your organization's current state, you can create targeted improvement plans, whether reinforcing strengths, addressing weaknesses, or identifying opportunities for further training and exercises. This provides a clear starting point for overall improvement and upskilling.

Inject breakdowns help pinpoint your team's strengths and weaknesses. Imagine the exercise in a real-world scenario: would there be a data breach, or would operations continue as normal? Assess your team's confidence and accuracy in their responses to identify knowledge gaps and points of failure. Use these insights not to dwell on mistakes but to improve and ensure your team is well-prepared for future challenges.

The participant breakdown takes this introspection into your team's capabilities a step further by plotting decision scores against confidence levels. This helps you understand the accuracy and confidence of your team’s responses. Are your strongest team members operating confidently? Are those with knowledge gaps posing risks by overcompensating with confidence?

Create an action plan

This data helps you prioritize your next steps. Will you address weaknesses, reinforce existing skills, or increase exercise frequency to build confidence?

There are plenty of upskilling routes to choose from. After each exercise, you'll see related Crisis Sim scenarios and lab content based on the threats and attack vectors encountered.

When creating your action plan, you should consider the following outcomes and their related recommendations:

• Weaknesses identified at the individual level ⇢ Assign recommended lab content to key users, and reinforce the importance of upskilling by communicating the purpose of the content.
  
  Hint: Don’t forget to use assignment deadlines to effectively track progress and keep the team on track.
  
  
• The participants' skills resulted in high accuracy decision-making but low confidence ⇢ Reinforce strengths with clear communication of processes and expectations.
  
  Consider reviewing your internal playbooks! Are processes clear, concise, and aligned with organizational needs and expectations? Are policies current and up to date? Are there conflicting processes or policies within your organization?
  
  
• The team performed exceptionally across the board with high confidence  ⇢ Test response readiness by exercising on a more difficult level scenario.
  
  Does the team excel in all areas, or is this an opportunity to better prepare? The landscape is constantly changing, and new threats are constantly emerging. Ensure your team has a wide breadth of knowledge and coverage by continuously proving their skills and encouraging further learning.

Three essential steps to maximize your post-simulation impact

Of course, you know your organization and teams best, so the Crisis Sim results are always best interpreted by you. Once you’ve analyzed and understood the results, prioritize these steps:

1. Review the results and gather feedback promptly to identify growth opportunities. Did outcomes align with expectations, or were there surprises?
2. Plan specific changes for future Crisis Sim exercises and build a strategic timeline. Should you adjust the difficulty or coverage areas? Is there time for additional training between exercises?
3. Create an action plan with clear objectives, owners, and deadlines to ensure individual and team development. What other organizational stakeholders should you bring in moving forward? And what will be important for them in Crisis Sim exercising?

Share your thoughts

If you’ve recently completed your first Crisis Sim exercise, what will you do next? If you’ve completed many, what tips do you have for others? Join the discussion below!