AMTSO’s June 2021 Testing Town Hall event was held online on Wednesday, June 9th. The event provided attendees with a rare opportunity to hear first-hand about the latest developments in the testing world, and to ask questions of the expert presenters.
The challenges of IoT security testing
After the usual summary of recent and upcoming tests results, the event’s first speaker, Vladislav Iliushin of Avast’s IoT Lab, offered some insights into the work he and his team have been doing to benchmark IoT security solutions. The talk started with in introduction to AMTSO’s IoT Working Group, working to develop guidance on testing solutions intended to protect IoT devices from danger, as well as summarizing a test of secure routers recently run by AV-Comparatives. This led in to some background on why Avast established its IoT Lab, to help address the shortage of information on how well such solutions are performing, the same issue driving the efforts of AMTSO’s IoT group.
Vlad flagged up the issues of visibility facing developers of IoT security solutions, leading to bias in their evaluations of their own output thanks to greater familiarity with the specific issues they have already addressed and potential lack of knowledge of other areas. He stressed the importance of focussing on protection features, rather than areas such as quality of life and the security of solutions themselves, which may be of some interest but are not considered key to this type of benchmarking. Vlad’s key argument was that such benchmarking is vital to the success of IoT security solutions, demonstrating where individual developers are doing well and highlighting areas where improvement is required.
Taking questions from the attendees, Vlad noted that the IoT space is extremely diverse and many devices have broad flexibility, making it difficult to focus on specific protocols or data sources and requiring security providers to monitor and track a wide range of potential threat vectors.
Discussion also covered the risks presented by traffic between IoT devices within a network as well as communications extending beyond the network edge, and the requirements of an advanced IoT testing facility, including Faraday cages to minimize the impact of wifi networks.
The session ended with a look ahead to the possible future of IoT security, and AMTSO noted that members will continue to work with Vlad and his team, as well as other experts and interested parties, in the ongoing development of IoT security testing guidance.
Testing how security products fight targeted attacks
The next speaker was Stefan Dumitrascu, CTO at AMTSO tester member SE Labs. Presenting standing up to reproduce the feeling of a physical conference, Stefan introduced his topic of “Transparency in APT-style testing”. After a brief summary of SE Labs’ various testing efforts, Stefan reviewed the complex world of advanced threat testing, evaluating how well security solutions protect against targeted and tailored attacks.
Highlighting the importance of the information shared with participating vendors on how such testing is carried out, both before and after testing, for full transparency, Stefan looked at the sources of information used in designing complex APT-style test cases, ranging from the MITRE ATT&CK framework to data from public and private research carried out by third parties as well as analysis of APT-related malware samples gathered from the real world. Such diverse information sources, varying both from source to source and over time in a single evolving source, make testing protection against advanced threats a complex and resource-intensive effort.
Having reviewed these issues, Stefan introduced attendees to a new system in development at SE Labs, aiming to collate and centralize information on how threats work and how best to test protections against them, as well as how security products work and how best they can be configured and operated, aimed at informing vendors, testers and others, and encouraging and facilitating more advanced testing. Stefan noted that the initial iteration of the project is expected to go live in October.
Simulating ransomware attacks
The final speaker of the day was Alexander Adamov of NioGuard Security Lab, also an AMTSO tester member. Following up on a presentation given at a previous AMTSO Testing Town Hall, Alexander presented on his company’s simulation-based approach to testing anti-ransomware protections.
These simulations are developed in an iterative process using machine learning to find the methods and techniques most likely to bypass detection of unwanted activity, focussing on the encryption of files used in almost all variants of ransomware attacks.
Having demonstrated how this process works to refine techniques and increase ability to bypass detection, Alexander moved on to provide a series of demonstrations of ransomware simulations, designed to mimic major ransomware threats, targeting popular cloud storage and webmail services such as Google Drive, Gmail, and Microsoft’s Outlook.com and OneDrive. The demos showed successful encryption of files and emails via various different methods with a very small amount of effort from the attacker, and highlighted the vulnerability of such services to attack.
Next AMTSO Testing Town Hall
The next AMTSO Testing Town Hall will be on September 8th, 2021, and is open to all AMTSO members and the public. To find out more about becoming an AMTSO member, see our joining page. To register your interest in attending our next Testing Town Hall event, please contact us at [email protected] or complete the event registration form.