From Concept to Content: A Deep Dive into Building and Critically Analyzing Labs

Putting it all together

The main bulk of the development work is building the labs. This usually comprises two parts that require different skill sets; one is putting together the written portion of the lab (such as the briefing, tasks, and outcomes), and the other is implementing any technical needs for the practical side of the lab. While some labs may focus more on one component than the other, this general overview of lab development will demonstrate each step of the process.

Developing written content

Regardless of the lab, the written content forms the backbone of the educational material. Even with prior knowledge and planning, additional research is essential to ensure clear explanations. 

Once research is complete, an outline is drafted to focus on the flow, ensuring the information is presented logically and coherently. This step helps enhance the final product.

The final step is turning the outline into the final written content. Everyone approaches this differently, but personally, I like to note all the points I want to cover in a bullet list before expanding on each one. This method ensures all necessary information is covered, remains concise and clear, and aligns with learning outcomes and objectives.

Technical implementation

For practical labs, technical setup is key. Practical tasks should reinforce the theoretical concepts covered in the written portion, helping users understand the practical application of what they’ve learned.

Before implementing anything, the author decides what to include in the practical section. For a CTI lab on a vulnerability, the vulnerable software must be included, which involves finding and configuring it. For general topics, a custom script or program may be needed, especially for niche subjects. The key is ensuring the technical exercise is highly relevant to the subject matter.

Balancing the difficulty of practical exercises is crucial. Too easy, and users won’t engage. Too hard, and they’ll get frustrated. Tasks should challenge users to think critically and apply their knowledge without discouraging them. This requires iterative testing and feedback to fine-tune the complexity. The goal is to bridge the gap between theoretical knowledge and real-world application, making learning effective and enjoyable.

Quality assurance and finishing touches

The development process is complete, but there’s still work to do before releasing the content. We take pride in polishing our content, so the final steps are crucial.

Checking against expectations

Before the official QA process, we review the original plan to spot any discrepancies, such as unmet learning objectives or missing topics. While deviations don’t always require changes, they must be justified.

Assuring quality

A thorough QA process is vital for catching grammatical errors, technical bugs, and general improvements before release. Each lab undergoes three rounds of QA, each performed by a different person – two rounds of technical QA, and one for presentation.

Some of the steps taken during technical QA include:

• Verifying written content accuracy, flow, and completeness.
• Ensuring all learning objectives are covered.
• Identifying any critical bugs or vulnerabilities that would allow users to bypass the intended solution.
• Providing small tweaks or changes to tasks for clarity.
• Assigning relevant mappings (NICE K-numbers, MITRE tags, CWEs).

After technical QA, the lab is reviewed by our quality team to ensure it meets our presentation standards.

Once all labs in a collection pass rigorous QA, they are released for users. The final step occurs post-release on the platform.

Gathering and implementing user feedback

Users are at the heart of everything we do, and we strive to ensure our content provides real value. While our cyber experts share valuable knowledge, user feedback prevents echo chambers and highlights areas for improvement. After new releases, we conduct an evaluation stage to analyze what went well and where we can improve.

User feedback

We gather quantitative and qualitative feedback to help us identify root issues and solutions.

Quantitative feedback involves analyzing metrics like completion rates and time taken. We also examine specific changes, such as frequently missed questions or labs where users drop out. These are important things to note, but we avoid drawing conclusions solely from this data. This is where qualitative feedback comes in.

Qualitative feedback includes user opinions and experiences gathered from feedback text boxes, customer support queries, and direct conversations. These responses are stored and read by the team and provide context beyond raw numbers. Channels such as customer support queries and follow-ups with customers also help us improve our content.

Post-release reviews

We conduct post-release reviews at set intervals after content release to analyze quantitative and qualitative data. This review helps us assess the entire process and identify areas for improvement.

These reviews allow us to update content with new features, like adding auto-completable tasks for CyberPro. The reviews ensure our content remains current and enhances user experience.

Wrapping up

Hopefully, this blog post has provided insight into all the care we put into building and tailoring our content for users. This process has come a long way since we started making labs in 2017! 

Don't forget — with our new Lab Builder feature, you can now have a go at creating your own custom labs. If there's a topic that interests you and you want to share that knowledge with your team, making your own lab is a great way to do it!

If there’s any part of the process you’d like to know more about, ask in the comments. Are there any collections that made you think, “Wow, I wonder how this was made”? Let us know!