AMTSO – the First Ten Years

AMTSO was formed in February 2008. Now ten years down the line, the organization continues to work to improve testing.

At our foundation meeting in Bilbao in 2008, AMTSO gathered expertise from 25 companies, representing the bulk of the security industry at the time, as well as most serious testing labs. By the end of that first year we had already published our first guideline documents, including our “Fundamental Principles of Testing”.

Over the years many of the original names ceased to exist, mainly thanks to mergers and acquisitions, but the majority are still around and still working with AMTSO to define and promote better tests. Many more guidance papers have been developed and published, and AMTSO continues to work to address new issues in testing, often emerging from changes in the approaches taken to security, which in turn react to new trends in malware.

From our initial focus on “dynamic” and “whole product” testing, approaches which have become the accepted norm of good testing, AMTSO has broadened its horizons to cover network-based security products, mobile solution testing, and most recently “targeted attacks”, providing advice on how best to measure protection against the APTs and other advanced, sophisticated threats facing governments and major corporations.

Along the way we have developed our popular “Security Features Check” (SFC), which gives end-users the tools they need to check the setup and operation of the products they are running, and depending on to keep them safe.

We also built our Real-Time Threat List system (RTTL), facilitating sharing of samples gathered by our members and considered of value in testing. The system allows a wide range of additional metadata to be collected and stored alongside samples and URLs, enabling testers to refine their sample sets to reflect specific time periods, prevalence levels, user types and even regional variations.

In 2016 AMTSO stepped up a gear, starting work on our first true Standards. Work has continued on our Testing Protocol Standard since then, going through various phases of development, review and trials. In December 2017, during our meeting in Beijing, China, the AMTSO membership approved moving forward to a public pilot, the first results of which should be posted to our revamped site in the next few weeks.

In 2018, AMTSO has over 55 members, covering a security market which has grown and diversified to keep pace with the growing menace of malware and cybercrime. Testing remains crucial to the industry, and to the people relying on security solutions to protect their systems and their data.

With our focus clearly on the Standards as they emerge from development into full public trials, we have kept working on our other projects, a thorough review and relaunch of the RTTL system expected to complete this year. Our growing Contact List system is a spin-off from the Standards, meeting the need for open and verifiable notification of upcoming tests.

We have also started on the next stage of the Standards project, looking into ways to provide independent and rigorous analysis of the relevance and usefulness of specific tests, and we hope to start another round of expansions and improvements to the SFC system this year.

We continue to hold regular member meetings, in varied locations around the world to cater to our global membership. Our next member meeting is due to take place in late May in Portland, Oregon, alongside the 2018 CARO Workshop. The CARO Workshop series developed from the same testing meeting at which the idea of AMTSO was first proposed, and is now a well-established annual event; this year, AMTSO is helping to organize and run the workshop.

The launch of our revamped website brings with it opportunities for greater outreach and promotion of our activities, something which is vital to our Standards project. The Standards require visibility, to become something readers of test data look out for and trust. This blog is a first step along the road to greater recognition of AMTSO and what it stands for.

We do welcome input from anyone with an interest in the testing area, and ask that any questions, ideas or comments be addressed to [email protected].