AMTSO Publishes New Guidelines for the Anti-Malware Testing Industry
Thursday 10th June, London, UK
The Anti-Malware Testing Standards Organization (AMTSO), an organization formed to encourage improved methodologies for testing security programs, has added major resources for anti-malware software testers and other interested parties to its guideline document and resources repositories.
AMTSO members unanimously adopted two testing guidelines papers on 25th May at its members’ meeting in Helsinki, Finland. The adoption of these papers, (both are immediately available from here), helps to further AMTSO’s mission to provide standards and guidelines promoting and facilitating accurate and reliable anti-malware testing.
Most tests focus only on simplistic detection rates, in some cases actually disabling part of the product’s functionality in order to isolate a single protective layer. “Whole Product Protection Testing Guidelines” advocates a more balanced look at the effectiveness of products, taking into account multiple layers of detection and protection. “This Guidelines document marks an important step in developing tests which accurately measure how an entire product actually functions when exposed to threats,” said Dr. Igor Muttik, representing McAfee. “Too many current tests focus on individual technologies, such as ‘On Demand Scans.’ Only by testing all of a product’s protection capabilities in a comprehensive test, can one provide a more realistic view of the security offered to computer users by contemporary security suites. ”
“Performance Testing Guidelines” illustrates that it is as easy to over-emphasize irrelevant metrics by implementing poorly-conceived benchmarking methodologies, as it is to bias a detection test. “The Performance Testing Guidelines examines the myriad – and often subtle – complexities in conducting speed tests,” said Mikko Hypponen, Chief Research Officer of F-Secure. “It is very tempting to take a simplistic approach to measuring speed and footprint of an anti-virus program. However, there is as much art as there is science in understanding the various elements which can skew the results for the unwary tester. This document will help testers understand these issues and allow them to take the necessary steps to minimize them and take them into account.”
Further to AMTSO’s mission to raise awareness of the topical issues in testing by providing access to other best-of-breed information resources, four major research papers have been added to the testing-related resources page. All four papers, presented at recent high-profile security conferences, discuss testing methodologies that reflect the real world better, and are donated by AMTSO members ESET, McAfee, Trend Micro and West Coast Labs .
AMTSO members are currently working on several more guidance documents, and extending its co-operative links with other organizations.
AMTSO is comprised of 37 members, representing testers, vendors, academics and publishers involved in anti-malware research. Founded in 2008, AMTSO members have cooperatively developed and adopted standards, guidelines, educational materials, and a review analysis process aimed at improving the efficacy and reliability of anti-malware testing.
