AMTSO News 2023-06-15

Welcome to our regular roundup of events and activities in and around AMTSO. 

XDR Testing Criteria

A launch meeting was held this week for the newly-formed XDR testing working group, working on a set of criteria to define high-quality tests of XDR-type solutions. The project will be driven by AMTSO tester members, with additional feedback to be sought from users of test data in the CSO/CISO community, as well as the wider AMTSO membership; representatives of multiple AMTSO member testing labs attended the call to provide their input and ideas. We expect to see some initial outcomes in place in time for the next full member meeting later this year, alongside several further initiatives which we plan to launch in the coming weeks.

CSO/CISO conference plans

A meeting was also held this week of the “New Frontiers” group within AMTSO, working to advance our plans to host a conference bringing together testers with the CSO/CISO community to share ideas and help fill knowledge gaps. A broad-ranging discussion was held on the potential shape of the event, its focus, and potential attendees. Numerous next steps have been put in place to be fulfilled ahead of the next meeting of the group in two weeks’ time. We currently anticipate holding the first such event next to the next member meeting.

Meetings and Events

Work is under way to finalise plans for the next member meeting, currently expected to take place in London in early October as a one-day event ahead of the planned CSO/CISO event noted above. We expect to be able to provide full details of the plans to members ahead of the next issue of this newsletter.

For members only, recordings of most sessions at our recent Budapest member meeting have now been posted to the member site, along with slide decks from most sessions; more decks will be added as they are shared by the various speakers. For the wider public, we have also posted edited recordings from our most recent Testing Town Hall event, including a summary of the update provided by MITRE on their plans for upcoming ATT&CK evaluations. More details on MITRE’s plans can be found in their Medium post here.

Our next Testing Town Hall will be held on July 5th, we expect to feature a talk from SecureIQLab on their upcoming XDR test (more details below), alongside other presentations from AMTSO members and guests. Non-members can sign up to attend our Testing Town Hall events via the events page.

Board Elections

Voting in our annual Board elections closed on June 5th. David Ellis of SecureIQLab and Jeffrey Wu of Testing Ground Labs were elected to represent Tester members, while Vladislav Iliushin of Ellio Technology and Luis Corrons of Avast were elected by non-tester members. Our congratulations to those elected, and our thanks to everyone who stood in the elections; the new Board will be formed on July 1st at the start of our new company year.

RTTL and ThreatList

We held our regular RTTL update meeting this week, reviewing progress and tracking submissions to the system. A major topic was our ThreatList project, for which an oversight board has now been selected featuring several members of the former WildList board; they held an initial call last week and are moving ahead with final review of the proposed procedures and protocols of the new system, which we hope to have fully operational very soon. Other work in this area includes the creation of web pages to display details of the new system and provide a vector for recruiting new contributors.

AMTSO Standard and Test Calendar

Our test calendar shows 10 tests published in the last 30 days, including several running under the AMTSO Standard. There were also a number of new test notifications issued, as detailed below. Tests expected to be released in the next few weeks include the first under our Standard from our newest Tester member AVLab Cybersecurity Foundation, as well as an email security test from SELabs.

  • AMTSO Compliance Confirmation Reports have been issued to Testing Ground Labs (TGL) for their April 2023 Android Consumer Malware Detection Test Report (AMTSO Test ID: AMTSO-LS1-TP074) and Android Enterprise Malware Detection Test Report (AMTSO Test ID: AMTSO-LS1-TP075) on Monday, June 5th, 2023.
  • MRG-Effitas received AMTSO Compliance Confirmation Reports for their Q1’2023 Android Test (AMTSO-LS1-TP071) on Monday, June 5th, 2023 and Q1’2023 360 Degree Assessment Endpoint Test  (AMTSO Test ID: AMTSO-LS1-TP072) on Monday, June 12th, 2023.
  • Public Test Notifications were issued for four tests during the reporting period:
  • TGL (Testing Ground Labs) June 2023 Consumer Android Malware Detection Test (AMTSO Test ID: AMTSO-LS1-TP078) 
  • SecureIQLab Q3’2023 Extended Detection and Response XDR CyberRisk Test (AMTSO Test ID: AMTSO-LS1-TP079) 
  • MRG-Effitas Q2’2023 360 Degree Android Assessment and Certification (AMTSO Test ID: AMTSO-LS1-TP080) 
  • MRG-Effitas Q2’2023 360 Degree Assessment and Certification (Endpoint) (AMTSO Test ID: AMTSO-LS1-TP081)

Information on all published and upcoming tests being tracked by AMTSO can be found in our test calendar.


We send this newsletter to all AMTSO member representatives, as well as non-members who have engaged with AMTSO recently and have an interest in what’s going on in the AMTSO community. If you have any friends or colleagues who would like to be kept informed of developments in the testing world, they can sign up here. If you’d like to know more about joining the AMTSO community, there’s information and an application form here.