AMTSO News 2023-06-29

Welcome to our regular roundup of events and activities in and around AMTSO. 

Meetings and Events

Our next member meeting will be taking place on October 2nd, in London. Details have been circulated to members this week, and registration has opened, with five member companies already registered to attend. Further information will be added to the meeting page on the AMTSO member website as it becomes available.

On October 3rd we plan to hold our first event bringing together testers and CISOs/CSOs to share ideas, pinpoint where the information gaps are, and work out how best to fill them. The gathering will also provide an opportunity to share pain points related to security products with the wide range of vendors in the AMTSO community. A call was held this week with the team running this project, we are building an initial pitch sheet and will be asking AMTSO members to reach out to their contacts in the CISO/CSO community to help us find the best people to participate in panels and other discussions.

Both events will be held ahead of, and in the same venue as, the VB2023 conference taking place later the same week.

Our next Testing Town Hall will be held next week, starting at 8am Pacific time on July 5th. Agenda items so far include a talk from SecureIQLab on their XDR testing processes, and Massive Computing on the data they hope to share with test labs; there will also be the usual round-up of testing news and related topics from AMTSO. Full recordings will be made available to AMTSO members shortly after the call, edited highlights may be posted to the event page on the public website for wider consumption. Non-members can sign up to attend our Testing Town Hall events via the events page.

RTTL and ThreatList

The RTTL team held their regular call this week, with the biggest topic for discussion being the implementation of a series of major changes to the sample-sharing system, including the addition of a sandbox to analyse submitted files and a raft of improvements to the underlying structures and statistics display system. The ThreatList project continues to move forward, with a meeting of the new oversight board set for next week to finalize review of the proposed structure and processes of the new system; with that in place, we expect to open it up to contributions in the next few weeks.

AMTSO Standard and Test Calendar

June has been a busy month for testing, with 14 new tests added to our test calendar. These include the first test to be confirmed compliant with the AMTSO Testing Protocol Standard from AMTSO’s newest test-lab member, AVLab Cybersecurity Foundation, as well as Android and ransomware tests from Testing Ground Labs; VBSpam and VB100 data from Virus Bulletin; more Android results from AV-Test; annual Mac, anti-phishing and parental control reports from AV-Comparatives; and an email security report from SE Labs released today. Coming up in July we expect to see Q2 results from MRG Effitas, Advanced Cloud Firewall data from SecureIQLab, and more regular reports from AppEsteem, Testing Ground Labs, and SE Labs.

Here’s a summary of activities from our Standard compliance team:

  • Phase 1 Commentary Collection covering the TGL (Testing Ground Labs) June 2023 Consumer Android Malware Detection Test (AMTSO Test ID: AMTSO-LS1-TP078) closed on Thursday, June 22nd, 2023.
  • Phase 1 Commentary Collections covering the MRG-Effitas Q2’2023 360 Degree Android Assessment and Certification (AMTSO Test ID: AMTSO-LS1-TP080) and MRG-Effitas Q2’2023 360 Degree Assessment and Certification (Endpoint) (AMTSO Test ID: AMTSO-LS1-TP081) both closed on Friday, June 23rd, 2023.  One issue was raised by a non-member test subject vendor regarding the Endpoint Testing Methodology involving Phishing Tests, discussion of which with the test lab has resulted in improvements to the testing methodology for future tests.
  • Phase 2 Commentary Collection covering the Testing Ground Labs (TGL) May 2023 Consumer Ransomware (Windows) Test (AMTSO Test ID: AMTSO-LS1-TP077) closed on Tuesday, June 27th, 2023.  The associated AMTSO Compliance Confirmation Report was issued later the same day.
  • Phase 2 Commentary Collection covering the AVLab Cybersecurity Foundation May 2023 Advanced In-the-Wild Malware Test Report (AMTSO Test ID: AMTSO-LS1-TP073) closes on Thursday, June 29th, 2023.
  • Phase 2 Commentary Collection covering the SE Labs Email Security Services Test Q1 2023 (AMTSO Test ID: AMTSO-LS1-TP068) will commence next week.

Information on all published and upcoming tests being tracked by AMTSO can be found in our test calendar.

Promotion and Outreach

Our marketing team has been working on the next batch of entries in our series of interviews, with the next entries expected soon. We have also been posting photos of our recent Budapest meeting to our LinkedIn and Twitter, along with reminders about some of the past interviews with members. We plan to schedule a meeting of our “Visibility” group in the next few weeks to start planning for the London meetings and the next round of Testing Town Hall events.

Testing Criteria, Guidelines and Best Practices

We expect to hold another call next week for our group working on criteria and resources for XDR testing. Also coming up in July are plans to launch projects addressing anti-phishing testing and firewalls. We hope to see initial outcomes from all of these projects ready to open out to the wider AMTSO membership at our upcoming London meeting. More details to come in future newsletters.


We send this newsletter to all AMTSO member representatives, as well as non-members who have engaged with AMTSO recently and have an interest in what’s going on in the AMTSO community. If you have any friends or colleagues who would like to be kept informed of developments in the testing world, they can sign up here. If you’d like to know more about joining the AMTSO community, there’s information and an application form here.