Fortian Winter Internship 2026

Damien Lowe | Winter Intern | 26 August 2026

Introduction

During my four-week winter internship with the MSS team at Fortian, I was tasked with an engineering project focused on automating validation of the detection pipeline.

The Fortian MSS team maintains a rigorous detection engineering process. Microsoft Sentinel rules deployed to customer environments are carefully written and validated against real attacker behaviour. However, manual validation takes time. An analyst must spin up a test environment, safely execute the malware or technique, and manually verify that the telemetry triggered the correct alert in Sentinel. As the ruleset grows, performing continuous regression testing on every rule becomes highly time-intensive.

My project was to take this existing validation process and fully automate it. I built a self-contained, infrastructure-as-code adversary emulation lab. The goal was to programmatically execute MITRE ATT&CK techniques and automatically verify the corresponding Sentinel alerts, allowing the team to scale its continuous testing without requiring manual analyst intervention.

Choosing the Right Architecture

Before writing any code, I evaluated the existing tools in the offensive security space to find the best fit for an automated pipeline.

Atomic Red Team is very fast to set up for a single test, but it lacks a central orchestration server and native agent management.
Mythic is an excellent command-and-control (C2) framework for red teaming, but it is not structured around repeatable, mapped test cases.
Caldera is purpose-built for automated adversary emulation. It ships with approximately 1,000 pre-built abilities in the default Stockpile plugin (rising to roughly 1,800 with the optional Atomic plugin loaded), and its default agent, Sandcat, operates across all major operating systems.

I chose MITRE Caldera. The biggest advantage of Caldera is that abilities are natively mapped to MITRE ATT&CK technique IDs. This structured mapping is exactly what you need to build a reliable automation pipeline because a specific executed technique can be linked to a specific Sentinel rule.

Building Ephemeral Infrastructure

I designed the entire environment to be ephemeral. Building persistent test labs introduces configuration drift over time and leaves a vulnerable attack surface exposed. Ephemeral infrastructure also closely mirrors standard CI/CD workflows where environments are built, tested, and destroyed on demand.

The core of the project is a Terraform deployment that provisions the entire lab in Azure. When validation testing is required, the deployment script provisions a Caldera orchestration server, spins up the target victim virtual machines across Azure, configures the network security groups, and connects the victims back to Caldera as active agents.

The sample terminal output below represents the flow when the primary setup script is executed.

$ ./setup.sh
[+] Enter Instance ID -> CALDERA-TEST-01
[+] Checking if instance ID is new... Validated.
[+] Checking if infrastructure is already spun up... Clean.
[+] How many Caldera Victims do you want executed -> 1
[+] Initialising Terraform deployment...
[+] Provisioning Caldera Orchestration Server (Ubuntu 22.04)
[+] Provisioning Target VM 01 (Windows Server 2022)
[+] Applying Network Security Group rules...
[+] Deployment complete.
[+] Caldera Server IP -> 192.168.1.50
[+] Awaiting Sandcat agent check-ins...

Solving the Automation Challenges

Making the deployment completely frictionless required solving a few mechanical issues to remove the need for human interaction.

The first major challenge was credential management. Caldera automatically generates fresh API tokens and login credentials on every new build for security purposes. If the server rebuilds with new passwords every time, an automated script cannot query the Caldera API to start an operation unless a human manually logs into the server via SSH to extract the new keys. To bypass this, I wrote a secondary bash script that is pushed to the Caldera server by Terraform during the provisioning phase. This script locates the newly generated configuration files, extracts the keys, and securely passes them back.

The second challenge was telemetry collection. For the lab to successfully validate rules, the victim machines need to begin reporting telemetry to Microsoft Defender for Endpoint immediately, from where it flows into Microsoft Sentinel via the Defender XDR connector. This is complicated by the fact that Defender onboarding packages carry an expiry date, so a pipeline cannot rely on a single package indefinitely, and offboarding is a separate process that must be handled independently of teardown. To ensure the logs were captured without manual configuration, I embedded the Defender onboarding script into the initial boot sequence of the virtual machines. By forcing the onboarding script to execute on first boot and triggering a restart, the victim machines came online fully monitored. This meant the lab had active telemetry flowing shortly after provisioning completed, without any manual configuration step.

The Proof of Concept: T1497.001

With the infrastructure pipeline working, I ran a proof of concept targeting MITRE technique T1497.001, Virtualization/Sandbox Evasion: System Checks. Since the targets in my lab were virtual machines, it was a well-suited environment for testing evasion techniques that actively look for virtualisation. Malware routinely runs system checks to see if it is inside a sandbox before executing a payload, checking hardware indicators like CPU core counts or MAC address vendor prefixes. I built custom adversary profiles in Caldera running 20 distinct abilities designed to detect virtualised environments. I executed the emulation through Caldera, and the telemetry successfully reached Sentinel.

Automating the Correlation

At this stage, the attacks were executing and Sentinel was generating alerts. To make this a true automation pipeline, I needed to remove the final manual step of an analyst checking the alert queue. I wrote a Python script to query the Caldera API and retrieve the exact timestamp at which a specific attack was executed. The script then queries the Sentinel API to determine whether an alert was generated within that time window. If the script found a match, it validated the rule automatically. The correlation script automates the existing manual validation process, executing it in seconds.

Next Steps and Conclusion

This proof of concept demonstrated that Fortian's rule validation process can be automated end to end for a representative technique, using ephemeral environments. Because the infrastructure is destroyed immediately after the automated tests are run, the entire architecture is highly cost-efficient. There are no idle servers left running to generate unnecessary cloud spend.

The next step for this system is scaling the custom adversary profiles across the remaining ruleset and integrating the validation pipeline into a continuous deployment workflow. This ensures that every time a detection rule is updated, it is automatically regression-tested against live emulation before being promoted to production.

The Fortian MSS team operates at a very high technical standard. Having the autonomy to design this infrastructure end to end was a valuable experience, and it was rewarding to build automation that directly supports the team's existing security operations.