Test Lab: MRG Effitas
Test Title: MRG Effitas Q1 2018 360 Degree Assessment and Certification
AMTSO Test ID: AMTSO-PP1-TP202
-
Statement from Test Lab:
March 2018 – 360 Degree Assessment and Certification
A first-of-its-kind test that covers all angles, the MRG Effitas 360 Degree Protection Test targets the key threats faced by internet users. In each test case we employ the full spectrum of Early Life Malware. We use a Time-To-Detect metric to measure how long it takes each application to detect and neutralize missed threats.
Participants: 10/19
Compliance Status: Partial Compliance
Note: Some issues were noted in the compliance check; as a complete list of test subjects was not available at the start of the test, proper notification, access to Participant status and configuration audit, and Commentary rights were not provided to all test subjects. Operationally, AMTSO will prevent this situation in future tests and ensure all test subjects receive full notification and per-product requests for Commentary.Publication Date: May 17, 2018
Participation information
Test Subject Vendor Status Phase 1 Feedback Attestations Comments (Phase 1) Phase 2 feedback Comments (Phase 2) Avast PARTICIPANT None (Waived) None None None AVG PARTICIPANT None (Waived) None None None Avira PARTICIPANT Completed Completed None None None BitDefender PARTICIPANT Completed Completed None NO ISSUES None ESET PARTICIPANT Completed Completed None NO ISSUES None F-Secure PARTICIPANT Completed Completed SUBMITTED SUBMITTED SUBMITTED GData INCLUDED None None None None None Kaspersky PARTICIPANT Completed Completed None None None Malwarebytes INCLUDED None None None None None McAfee INCLUDED None None None None None Microsoft PARTICIPANT None (Waived) None None None Panda INCLUDED None None None None None SurfRight INCLUDED None None None None None Symantec INCLUDED None None None None None ThreatTrack INCLUDED None None None None None Trend Micro INCLUDED None None None None None Watchdog INCLUDED None None None SUBMITTED None Webroot PARTICIPANT None (Waived) None None None Zemana PARTICIPANT None (Waived) None NO ISSUES SUBMITTED Notable Commentary:
F-SECURE (Phase 1):
“The initial invitation to the test and possible “Voluntary Participant” status we as a company was unknown to me at least until this questionnaire was presented to us. This might be the case with other organizations as well; I propose that whenever such a test is conducted the participant status is clearly outlined when the materials regarding the test itself are being delivered.(This would serve two purposes — all the relevant people from the tests perspective would get the information, and it would still allow correction to be made if not everybody agrees with the status.)
I left the “I confirm that the Tester established a process allowing me to adopt Voluntary Participant status in this Test” checked out as the participant status question wasn’t addressed to me personally at the time it was asked. After discussing with it, it became clear that the decision was presented to us (read: us as in F-Secure Oyj.)”
ZEMANA (Phase 2):
“Keep up the good work guys!”F-SECURE (Phase 2):
“I didn’t receive any invitations for a configurations check.“ -
Notification date: January 24, 2018
Test plan shared on notificationCommencement date: Feb 12-Mar 30, 2018
Phase 1 Commentary Completion Date: February 26, 2018
Completion date (expected): May 7, 2018
Phase 2 Commentary Completion Date: May 24, 2018
Publication date: May 17, 2018
Compliance report date: June 7, 2018 -
avast! Internet Security 18.3.2333
AVG Internet Security 18.3.3051
Avira Internet Security 15.0.34.27
BitDefender Internet Security 2018 22.0.20.260
ESET Internet Security 11.0.159.9
F-Secure Business Computer Protection 18.4
GData Internet Security 25.4.0.2
Kaspersky Internet Security 2018 18.0.0.405 (g)
Malwarebytes AntiMalware 3.4.4.2398
McAfee Total Protection 16.0 R10
Microsoft Windows Defender with SmartScreen 4.12.16299.15
Panda Internet Security 17.0.1
SurfRight HitmanPro 3.8.0 Build 292
Symantec Norton Security 22.14.0.54
ThreatTrack Vipre Advanced Security 10.1.4.33
Trend Micro Maximum Security 12.0.1226
Watchdog Anti-Malware 2.74.186.426
Webroot SecureAnywhere 9.0.19.43
Zemana AntiMalware 2.74.2.150 -
Phase 1 Commentary Completion Date: January 31, 2018
Phase 2 Commentary Completion Date: May 24, 2018
-
Avast
Vendor Status: PARTICIPANT
Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Tester waived attestation requirements)
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
AVG
Vendor Status: PARTICIPANT
Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Tester waived attestation requirements)
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Avira
Vendor Status: PARTICIPANT
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Bitdefender
Vendor Status: PARTICIPANT
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
ESET
Vendor Status: PARTICIPANT
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
F-Secure
Vendor Status: PARTICIPANT
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Vendor Confirmation Status: SOME ITEMS MARKED AS LESS THAN SATISFACTORY
Item “the Tester established a process allowing me to adopt Voluntary Participant status in this Test” – marked as NOT CONFIRMED.
Commentary (Phase 1): COMMENTARY SUBMITTED as follows:
“The initial invitation to the test and possible “Voluntary Participant” status we as a company was unknown to me at least until this questionnaire was presented to us. This might be the case with other organizations as well; I propose that whenever such a test is conducted the participant status is clearly outlined when the materials regarding the test itself are being delivered.
(This would serve two purposes — all the relevant people from the tests perspective would get the information, and it would still allow correction to be made if not everybody agrees with the status.)
I left the “I confirm that the Tester established a process allowing me to adopt Voluntary Participant status in this Test” checked out as the participant status question wasn’t addressed to me personally at the time it was asked. After discussing with it, it became clear that the decision was presented to us (read: us as in F-Secure Oyj.)”
Vendor Confirmation Status: SOME ITEMS MARKED AS LESS THAN SATISFACTORY
Item: “Configuration checks” – marked as “Configuration audit not offered”
Commentary (Phase 2): COMMENTARY SUBMITTED as follows:
“I didn’t receive any invitations for a configurations check.“
-
GData
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Kaspersky Labs
Vendor Status: PARTICIPANT
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Malwarebytes
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
McAfee
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Microsoft
Vendor Status: PARTICIPANT
Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Tester waived attestation requirements)
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Panda
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Surfright
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Symantec
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
ThreatTrack
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Trend Micro
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Watchdog
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: SOME ITEMS MARKED AS LESS THAN SATISFACTORY
Configuration checks: I was not offered an opportunity to audit the configuration of my product
Adherence to the test plan: I confirm that, to the best of my knowledge, the tester followed the published test plan throughout
Communications: I confirm that all communications between the tester and my company were handled in a timely, relevant and fair manner
Feedback and disputes: I was not given an opportunity to review and dispute the test dataCommentary (Phase 2): NO COMMENTARY SUBMITTED
-
Webroot
Vendor Status: PARTICIPANT
Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Tester waived attestation requirements)
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: PENDINGCommentary (Phase 2): PENDING
-
Zemana
Vendor Status: PARTICIPANT
Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Tester waived attestation requirements)
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY
Commentary (Phase 2): COMMENTARY SUBMITTED as follows:
Other comments on the execution of this test: “Keep up the good work guys!“
-
Avast
-
Compliance Status: Partial Compliance with Draft Standard v6.1 – 2018-06-05
Note: Some issues were noted in the compliance check; as a complete list of test subjects was not available at the start of the test, proper notification, access to Participant status and configuration audit, and Commentary rights were not provided to all test subjects. Operationally, AMTSO will prevent this situation in future tests and ensure all test subjects receive full notification and per-product requests for Commentary.