2021: AMTSO’s year in review

We can certainly conclude 2021 sighing “what a year” for the second time in a row. With the pandemic still holding us behind our video cameras instead of allowing big trips and in-person meetings, this is the second year with no in-person meetings at AMTSO. We had to postpone our Venice meeting, which was originally planned for April 2020, first to 2021 and then to 2022 – and honestly when we did so we didn’t think its occurrence would be in question again twelve months later. Still, given the circumstances this has been a year full of activities, discussions, progress, and achievements.

Looking back at what has happened throughout the year, let’s start with some numbers:

  • 96 tests covered and tracked since the start of 2021 on the AMTSO Testing Calendar. We update our Test Calendar at least weekly with all major security group tests from a wide range of test labs, all tests can be filtered by lab, platform, and all vendors covered to easily find the tests most relevant to your needs.
  • 15 tests run under the AMTSO Standard V1.3 during 2021.
  • 12 videos available covering testing advances in 2021 on the AMTSO YouTube Channel. These videos result from our Testing Town Halls which we’re now running openly, also for non-AMTSO members to attend, to give our community visibility, and room to grow.
  • Over 420 followers for AMTSO’s Twitter feed looking at presentations, tests, white papers, and more. This is an increase by 8% vs. last year. We also grew our LinkedIn group by 20% more members.
  • 22 Unwanted Software Working Group team meetings. The definition of potentially unwanted software isn’t easy, it’s in the grey zone. Therefore, there is a lot of room for debate. As an outcome of our Unwanted Software Working Group meetings, we are progressing towards a shareable prototype.
  • 23 Real Time Threat List (RTTL) Working Group team meetings. RTTL provides a source of high-quality, prevalent, real-world samples for testing purposes. Last year, we opened up RTTL for CERTs, and non-member security providers and researchers to contribute and access the list. This year, we added more CERTs to our RTTL program, and started making samples available more widely. We added granular controls to let contributors manage who has access to their samples in April, worked on various fixes and expansions throughout year, and did a major upgrade to the system’s hardware in October/November.
  • Countless other working group meetings held:
    • IoT Working Group: IoT testing is still in its infancy, and we issued an initial position paper of the IoT Working Group in March, and a first draft of our guidance November.
    • Test Evaluations Investment Group: This group is monitoring Standard and compliance activities, and developing a Basic Criteria concept. The group issued the first draft for comment with other members at the October member meeting.
    • Knowledge Sharing Investment Group: This group oversees and supports the RTTL, USC, and IoT working groups, and is working on a refresh of the guidance library.
    • Test Visibility Investment Group: This group supports the planning of our Testing Town Hall events. This year, the group also developed and published major upgrades of the AMTSO Testing Calendar in April.

Meetings and Testing Town Halls

Besides the working group meetings, we held member meetings, and member update meetings, in February, May, and October. At our member update meeting in February, we welcomed a special guest from Microsoft, who spoke about new testing features in MS365.

We also held regular Testing Town Halls in January, April, June and November, with tester and vendor members speaking about the latest developments in testing, ranging from AV-Test’s approach to APT testing to Sophos’ call for more transparency in machine learning used in security, to Avast’s ideas around IoT security testing, to SE Labs’ simulation of APT attacks in testing. We also proudly presented speakers from NetSecOPEN, Cybereason, the Cyber Threat Alliance, SecureIQLab, NioGuard Security Lab, and MRG Effitas at our Testing Town Halls this year.


Our marketing team this year worked on AMTSO’s visibility, by feeding our Twitter and LinkedIn channels with frequent updates on tests, Testing Town Halls, and news around AMTSO. To increase awareness of AMTSO, we kicked off a new interview series that introduces tester and vendor members, discussing why fair cybersecurity testing is important to them, and AMTSO’s role in supporting this. The marketing team also has started working with an external design agency on a new corporate design, with changes to be introduced in the new year.

New partnership

This year, we were also happy to announce that NetSecOPEN, an open standards, membership-driven network security industry group, has joined AMTSO as a member. At the same time, AMTSO has become a NetSecOPEN member, facilitating knowledge exchange between the two cybersecurity testing standard organizations.

Election of new board

AMTSO couldn’t exist as an organization without all our voluntary contributors, and in August this year, we announced the election of a new board, appointing Alexander Ivanyuk, Senior Director of Product and Technology, at global technology company Acronis, and David Ellis, Vice President of Sales and Corporate Relations at cloud security validation provider SecureIQLab, as new board members. Simon Edwards, CEO at SE Labs, a private and independently-owned security testing company, and longtime AMTSO board member, stepped up as a new co-chairman of the board. He shares the position with Luis Corrons, Security Evangelist at Avast, a global leader in digital security and privacy, who was also re-elected to the board this year. Additional board members are Maik Morgenstern, CEO and Technical Director of AV-Test, Jesse Song, CEO of SKD Labs, Alexander Vukcevic, Director of Avira Protection Labs at NortonLifeLock Inc., and Glaucia Young, Engineering Director at Microsoft, and we’re thankful for all their contributions.

Truly, what a year. It’s great to see what we can achieve as a community even in a purely remote working situation. We’re looking forward to the next year, with many interesting discussions, developments with AMTSO, and our members. Enjoy the Holidays, and have a very Happy New Year!