From XSS and SQLi to AI-generated code and supply-chain compromise: How application security is evolving
Keeping up with vulnerabilities is like playing a never-ending game of whack-a-mole. One day, we were knee-deep in XSS payloads and buffer overflows; the next, developers everywhere were plugging SQL injection holes with duct tape and regex. A lot of our earlier tech wasn’t built with security in mind – or at least not at the forefront of our minds. But over the past two decades, the culture has changed, and developers are shifting left. Programming languages, as well as the frameworks and ecosystems around them, are evolving and adapting to evade security threats. XSS: From everyday headache to “mostly handled” Remember when cross-site scripting (XSS) was every web developer’s nightmare? In the 2000s, it seemed like every other website was vulnerable. If you were lucky, your users’ only punishment was an annoying pop-up. If not, credentials and session cookies were up for grabs. Modern languages and frameworks took proactive steps in their designs. Today, many of them have built-in protections against common vulnerabilities like XSS. Some examples are: React, Angular, Vue: Automatically escape output by default. You have to go out of your way to render raw HTML (and they make you feel guilty about it). Django, Ruby on Rails, ASP.NET Core: Templates escape user input by default. Browsers: Even they pitch in, with features like Content Security Policy (CSP). It’s still possible to write a vulnerable application, but the bar is higher. As a result, attackers have a bigger hurdle to jump over. This is thanks to the evolution of how languages and frameworks approach security – not as a feature, but as the default. SQL injection: OR 1=1 is mostly history SQL injection once powered major data breaches. Now, parameterized queries have become the norm, and we’re on our way to leaving those days behind. Object-Relational Mappers (ORMs) like SQLAlchemy, Entity Framework, and Hibernate generate safe SQL. Most modern languages make string concatenation in queries unnecessary (and uncool). Even PHP, once infamous for its “raw SQL everywhere” approach, now encourages prepared statements and offers safe database APIs. Not a perfect world, but much improved. It’s crucial to keep in mind, however, that technology shouldn’t be relied upon uncritically to keep applications secure. Developers must do their due diligence. Memory mischief: The rise of Rust (and memory-managed languages) C and C++ are legendary for performance. And legendary for buffer overflows, use-after-free, and all manner of memory mischief. Enter memory-safe languages: Java, C#, Python: Garbage collection and managed memory eliminate entire classes of bugs. Rust: Takes it up a notch with ownership semantics, preventing data races and dangling pointers at compile time. A lot of system-level work has migrated to these “safer” languages, and the impact is apparent in everything from embedded devices to operating systems (hello, Rust in the Linux kernel). The new frontiers: AI and supply chain attacks Just as one threat starts to become old news, new threats emerge. Two of these are: Supply chain compromise The SolarWinds breach and NPM “event-stream” incident have put supply chain attacks in the spotlight over the last decade. It goes to highlight that you can write perfect code and still get breached because a dependency many levels deep was compromised. Some of the changes we’re seeing as a result: Package registries are adopting MFA and signing requirements. Software Bill of Materials (SBOM) is becoming a must-have, especially in regulated industries. But the battle is ongoing. If anything, our code is more interconnected than ever, and the rise of “vibe coding” is complicating matters further. AI-generated code AI tools like GitHub Copilot and ChatGPT are generating millions of code snippets. This is a double-edged sword. You get increased productivity, but the code is often vulnerable. This is partly a reflection of the insecurities in the code that the models were trained on. I asked ChatGPT 4.5 to identify the vulnerability in the following code and it couldn't. Can you? Leave a reply with your thoughts! @limits(calls=6, period=60) @app.route("/change-password", methods=["POST"]) def change_password(): user = session.get("user") if not user: return jsonify({"message": "Unauthorised"}), 401 data = request.get_json() password = data.get("new_password") if not password: return jsonify({"message": "Password is required"}), 400 if not db.change_password(user, password): return jsonify({"message": "Failed to change password"}), 500 return jsonify({"message": "Password changed successfully"}), 200 @limits(calls=6, period=60) @app.route("/login", methods=["GET", "POST"]) def login(): data = request.get_json() user = data.get("user") password = data.get("password") if not user or not password: return jsonify({"message": "Username and password are required"}), 400 if not db.authenticate(user, password): return jsonify({"message": "Invalid username or password"}), 401 session["user"] = user return jsonify({"message": "Logged in successfully"}), 200 The changes I expect to see going forward are: Guidelines and governance for AI-generated code: Programming languages or coding standards may soon explicitly include guidelines, validation rules, or security frameworks tailored to AI-assisted coding, ensuring generated code adheres to secure patterns. Work is already being done to create rules files for improved security. Integrated security checks at the IDE level: IDEs may embed deeper vulnerability scanning and real-time feedback directly within coding processes. Development environments or compilers may also come integrated with validation tools specifically attuned to potential weaknesses inherent in AI-generated code. Increased reliance on static/dynamic security analysis tools: Enhanced automated scanning integrated into CI/CD pipelines, detecting flaws pre-deployment. Keeping people in the loop: Security awareness should be a top priority, so everyone is ready for the worst-case scenario. Proving and improving skills: Developers’ training should increasingly emphasize secure coding, particularly when assisted by AI tools. At the current stage, thorough PR reviews are more crucial than ever. As with any code, always review and test AI-generated code, especially for security-sensitive logic. AI is a tool, not an auditor. Security: Not a feature, but a default Application security has come a long way from the Wild West days of the early internet. The biggest shift in the software development landscape is that security is no longer a bolt-on. Modern programming languages and ecosystems try to make the secure path the easy path. Defaults are safe (escaped output, parameterized queries), dangerous operations are noisy (compiler warnings, explicit function names), and security updates are automated (thanks to package managers and CI/CD integrations). Of course, attackers are creative, and the landscape is always shifting. But the evolution of programming languages, as well as the surrounding tools and communities, means developers have a fighting chance. While we face newer threats like AI-generated code vulnerabilities and supply chain compromise, the foundations are getting stronger. The key is to keep learning, stay skeptical, and use the tools at your disposal properly. The biggest shift I would like to see is the human element no longer being viewed as the “weakest link”. The moles never stop popping up, but now, at least, we have better mallets – and the resources to help us use them.Deepfake Defense: Building a Cyber-Ready Workforce in the Age of GenAI
This event has now ended. Watch the recording here. GenAI is reshaping cybersecurity—enhancing defenses while also introducing new risks. Deepfakes, GenAI-generated malware, and vulnerabilities introduced through GenAI coding tools demand a strategic shift to stay ahead of evolving threats. Join industry experts Jamie Knobles and John Blythe as they examine real-world GenAI-driven attacks and share actionable insights to strengthen cyber resilience. This webinar will provide cyber leaders with: A deep-dive into deepfake tactics and other GenAI-related threats. Strategies to increase cyber resilience across the entire workforce. Recommendations for proving cyber capabilities. Register today to gain the knowledge and strategies you need to proactively address the GenAI challenge, minimizing risk and maximizing the potential of this transformative technology.Breaking Down Walls to Make Way for AI
The rapid rise in the popularity and application of AI has been unprecedented. We are actively experiencing the dawn of a revolutionary chapter in technology and innovation, but it also feels a little like this, don't you think? AI is everywhere. Even where you didn’t ask for it, or frankly may not want it. Its rise brings security risks that require comprehensive, strategic management. Are you training your teams on AI security risks? Are you reviewing procedures to protect your business as the threat surface expands? You’re not alone. Let’s get into how we’re guiding customers through these challenges. Immersive can provide knowledge, skills building, and engaging challenges for your teams to address risks from different angles. Artificial intelligence foundations To protect an organization, you need to know what you’re protecting from. These are some of our core recommendations: AI for Business Gain an understanding of various risks associated with implementing and integrating AI in a business context. These risks include understanding the challenges of implementation, potential issues in day-to-day AI utilization, and the broader implications of AI on operations. These labs will equip individuals with the knowledge to effectively leverage AI, while being mindful of these risks. AI Fundamentals Learn about emerging threats, generative AI models, and prompt injection attacks. Gain a comprehensive understanding of AI's implications in cybersecurity, AI-related security risks, and gain practical experience mitigating those risks. Build knowledge around internal risks AI often enters businesses through productivity tools or internal chatbots, like ChatGPT, for HR or finance queries. Using internal AI apps creates risks around data access, data handling, privilege management, improper use of LLM, and compliance considerations (or consequences). Some of these considerations are also relevant for external risks with AI. Here are some collections to get your team’s gears turning around AI-adjacent considerations and internal risk: Cloud Security Identity and Access Management Secrets and Encryption in AWS NIST – Guidelines on Security and Privacy in Public Cloud Computing (800-144) Risk and Compliance ISO 22381 – Security and Resilience for Identification Systems ISO 27014 – Governance of Information Security ISO 27018 – Protecting Private Data in Public Clouds ISO 28000 – Security Management Systems for Supply Chains ISO 31000 – Risk Management Let’s not forget – with new tools comes new access and alerting patterns. You’ll need to ensure your Digital Forensics and Incident Response (DFIR) teams are ready for new signal analysis and identifying corresponding indicators of compromise (IoCs) with new technologies. Digital Forensics Introduction to Digital Forensics Digital Forensics Threat Hunting – Theory Threat Hunting Introduction to Incident Response and Forensics in AWS Upskill to protect your business from external risk As customer-facing AI expands, so does your threat surface. The risks remain rooted in attacks humans conduct today; they’re just becoming more sophisticated with AI. If threat actors use AI maliciously against your business, you might see advanced social engineering attacks. This could include sophisticated phishing attacks or AI-generated voice, image, or video to manipulate users into disclosing credentials. Here are a few hands-on content recommendations that will keep your team response ready: Events and Breaches Gain familiarity with some of the biggest cyber events and most infamous data breaches in hacking history. Buckle up for interactive labs that will get you thinking about real-world events and how AI could affect these types of scenarios. Events and Breaches: Data Exposure Events and Breaches: Phishing Fraud Events and Breaches: Data Leaks Emerging Threats Attackers are quick to adopt new tools and tactics, giving them a first-mover advantage. Labs in this collection will get you hands-on with the latest methods used by threat actors around the globe. These labs aren’t explicitly focused on AI threats, but since AI threats are rooted in legacy techniques, this collection will help your team prepare for the variations AI may introduce. There are also increased risks with publicly facing AI tools that are integrated into internal databases or systems. These non-human identities have access to potentially sensitive databases, making them inherently open to prompt injection attacks in addition to legacy techniques. Here are some of our content recommendations to prepare your teams explicitly for these types of AI challenges: Fundamental AI Algorithms Gain a deep understanding of various AI algorithms and their practical applications in cybersecurity. Engage with labs on machine learning, deep learning, specific algorithms, and complete tasks such as implementing algorithms and analyzing results. Practitioners will gain hands-on experience in applying AI techniques to enhance cybersecurity measures and mitigate cyber threats. AI Challenges Test your knowledge and skills around AI security risks such as AI plugin injections, function calling, and prompt injection attacks. Complete hands-on exercises to find vulnerabilities in AI systems, beat the bot, and actively exploit vulnerable LLM implementations. Staying ahead in a rapidly evolving tech landscape requires continuous learning and skill-building. But readiness doesn’t just stop there. You must also be well-practiced in handling new and challenging situations. Regular exercises, like prompt injection attack detection and AI-driven social engineering tabletop drills, are essential for keeping your teams prepared. As threats evolve, Immersive will continue to deliver integrated labs and industry-leading exercising capabilities so your teams are ready to protect your business. Share your thoughts What skills are critical for your team to mitigate AI risks? Did you beat our AI Challenges? Are you hungry for another byte 👾? Comment below! Stay ready in the face of increased risks – bot or not. Get updates in your inbox on posts like this by following the Human Connection Blog!Mastering Crisis Sim: How We Got 25 Execs on the Edge of Their Seats
Let's start with the requirements. In this case, they were clear: 1) To have a fully customised exercise focusing on payment risks and cyber threats like fraud, ransomware, and cash-outs. This is the domain expertise of Mastercard, who had no issues creating this in the Immersive platform with our review and guidance. 2) To deliver this on-site in front of 25 senior execs (including the CEO, CTO, CISO, CFO, and numerous heads of departments). It’s definitely not easy getting this many board members together in one room, so we had to make the first best impression! What did we do, and why was it a success? The stakeholders told us that they’d run tabletop exercises before, so this wasn’t new to them. So, how did we turn what they expected to be a routine drill into an edge-of-the-seat experience? What made our approach unique? Realistic AI-generated videos The amazing Crisis Sim content creation team at Immersive Labs used generative AI to create highly realistic videos that felt as though they were straight from a Pakistani news channel. Mastercard provided us with a few scripts and ideas that helped bring the AI videos to life. AI video of a news reporter tailored to resemble the Pakistani news channel. AI video of a news reporter tailored to resemble the Pakistani news channel. Why did the execs jump out of their seats? That’s because we included the following in the videos: Footage of their building on the news and customer dissatisfaction over cyberattack repercussions. News reporters with local accents and dress styles mentioning the bank and stakeholders, and unhappy Tweets flooding social media. A video of their CEO, not his real face of course, but we used his name when it came to an inject where the stakeholders voted for ‘releasing a press statement to the public’. They even specifically asked us to ensure none of these videos leaked! But each video included the message “Exercise use only – not based on real events”. AI video of the CEO giving a press conference. Collaboration This success was a team effort. Getting any presentation done right, especially in person, requires planning and knowing your audience. This is where Mastercard brought their expertise to create and tailor the Crisis Sim exercise while we supported with our guidance and previous customer experience knowledge. Our roles, tasks, and responsibilities were clear. Top tip: If you’re planning an exercise, collaborate with others for ideas and support. Double act presentation Presenting as a team made all the difference. Mastercard’s expertise in engaging senior execs helped animate the session and avoid awkward silences, while I focused on facilitating the exercise. Doing this alone can be tricky – I’ve personally struggled with awkward silences during solo Crisis Sim sessions while juggling scenario questions, multiple responses, feedback, and facilitation all at once. Sharing the load was crucial, especially in a U-shaped room where I couldn’t easily move around. In this case, I didn’t need to worry about moving away from my laptop, leaving the close-up engagement completely for Mastercard to lead on. Image example of the U-shaped conference room this took place in. The feedback The feedback was overwhelmingly positive – they said it was their first experience with such a tabletop exercise, a stark contrast to their initial expectations. The AI videos sparked significant interest and curiosity. The collaboration between Mastercard and Immersive Labs successfully delivered an educational, engaging, and tailored exercise. The bank expressed interest in running a wider crisis drill in 2025 with multiple banks. So look forward to hearing about that in the future! Finding AI video scenarios in our Crisis Sim Catalogue If you clicked on any of the AI images above thinking it was a video, I’d like to apologise. They were just screenshots as the actual videos included sensitive, bank-specific details like names and locations that we can’t share externally. If you’d like to explore AI video scenarios in our Crisis Sim platform, check out titles like: Boardroom Betrayal: When Deepfakes Strike the Top Puppetmaster’s Revenge The Walls Have Ears – Part One The Walls Have Ears – Part TwoFeature Focus: Introducing the AI Scenario Generator
In this blog, we’ll cover what this feature is and how you can use it. For this release, we’ve made creation as easy as possible. Just click Create with AI, add a title, and select options for organisational sector, attack vector, threat actor, and scenario size. This will generate a full scenario, from the briefing to the epilogue. You can even make a cuppa while it works!🪄✨ Once created, these scenarios can be published or edited and published in your organisation's catalogue. But how exactly does it work? Organisation admins can turn the AI Scenario Generator on and off in the platform settings area, so you’ll need this to be turned on if you want to try it out! Our AI Scenario Generator is currently only available to our Cyber Crisis Simulator customers. It’s based on technology provided by OpenAI, with generations based on publicly available data related to crisis management as well as our own Immersive Labs Crisis Sim catalogue. Organisation admins can choose to use the feature in a layered approach: No AI access at all: This means that your organisation has chosen not to enable the AI Scenario Generator. Without scenario sharing enabled: You can generate AI scenarios based only on the inputs shown in the generation box. With scenario sharing enabled: The AI will access specific parts of your previously published scenarios when generating new ones, ensuring the new scenario is highly relevant to your context. These settings can be updated on the organisation's settings page. If you’re keen to use the AI Scenario Generator but it’s not enabled in your organisation, you’ll need to discuss this with your internal Organization Administrator. If you’re an Organization Administrator and want to know more about the feature, contact your CSM. Tell me more about scenario sharing! If your organisation chooses to also enable scenario sharing, Immersive Labs will include specific information from any previous custom scenarios that you’ve published in the temporary “context window” for requests to our third-party AI vendor. A "context window" is an extension of the query sent to an AI model. It exists only during the processing of the query and isn’t saved by any third parties. The third-party AI vendor won’t use any of the information you share to train its models. Shared data will only ever be included in this temporary "context window" of generation and won’t be stored by the third-party AI vendor. The shared information includes scenario titles, descriptions, inject titles, and response options. It excludes feedback to response options, exercise information, reporting, account or organisation information fields, or metadata. Scenario sharing is designed to make the generated scenario more relevant to your particular context. You can still create scenarios using AI without scenario sharing, but your scenario will likely be more generic and less relevant to your particular organisational context. However, you can still edit the final version to make it more relevant to you – just like with our catalogue scenarios. Let’s not forget the human in the loop As with all things AI, we recommend that you review the AI output before publishing your scenario, to ensure it meets your needs. The AI Scenario Generator currently only generates text content, so you’ll probably want to add additional rich media, such as images or videos, to your scenario. To get the most out of your crisis simulation, we also recommend enabling, adding, changing, and checking certain elements. These include: Checking that you’re happy with the text formatting and narrative content Checking that you’re satisfied with the role listed Enabling and adding response feedback or performance indicators If you want to capture ranked response data, select the ranked options setting and add a rank (great, good, weak, okay) to each response option to suit your organisation's preferred situational response Turning on response confidence or justifications Get involved and share your thoughts! We know that AI is a hot topic and we’re keen to hear and capture your feedback and suggestions on this first release of our AI Scenario Generator as part of our user research taking place this November and December. If you want to participate in this research, you’ll be able to share your thoughts and experiences of using our AI tool and scenario creation more generally directly with our team. Comment below if you’d like to find out more, and we’ll contact you with further details! If you’re an Immersive Labs customer, you can find out more about the AI Scenario Generator in our FAQ guide.
