Comparison of Open Source Adversary Emulation Tools

If you liked my older post titled “List of Adversary Emulation Tools“, I am sure you must want to know what is the difference or comparison between the different tools. This post is an attempt to do just that -to list down the comparison of open source adversary emulation tools. I have compared their capabilities against the 11 tactics mentioned in the MITRE ATT&CK framework.

Comparison of Open Source Adversary Emulation Tools:

As mentioned earlier, the 11 tactics that Mitre CALDERA, Uber Metta, APTSimulator, Endgame Red Team Automation & Guardicore Infection Monkey are being compared against are:

1. Initial Access: The initial access tactic represents the vectors adversaries use to gain an initial foothold within a network.
2. Execution: This tactic represents techniques that result in execution of adversary-controlled code on a local or remote system.
3. Persistence: Persistence is any access, action, or configuration change to a system that gives an adversary a persistent presence on that system.
4. Privilege Escalation: Privilege escalation is the result of actions that allows an adversary to obtain a higher level of permissions on a system or network
5. Defense Evasion: This tactic consists of techniques an adversary may use to evade detection or avoid other defenses.
6. Credential Access: Credential access represents techniques resulting in access to or control over system, domain, or service credentials that are used within an enterprise environment.
7. Discovery: Discovery consists of techniques that allow the adversary to gain knowledge about the system and internal network.
8. Lateral Movement: Lateral movement consists of techniques that enable an adversary to access and control remote systems on a network and could, but does not necessarily, include execution of tools on remote systems.
9. Collection: Collection consists of techniques used to identify and gather information, such as sensitive files, from a target network prior to exfiltration.
10. Exfiltration: Exfiltration refers to techniques and attributes that result or aid in the adversary removing files and information from a target network.
11. Command & Control: The C&C tactic represents how adversaries communicate with systems under their control within a target network.

Now that these are listed, this is the actual comparison:

Please note that this is a very broad comparison based on the ATT&CK tactics and does not represent the full coverage by any tool. For example, Uber Metta seems to fare better than all the tools. However, it limited coverage for all the techniques and as of now, there are a good 219 techniques!