MRG Effitas Q4 2018 360 Degree Assessment and Certification

Test Lab

MRG Effitas

Test Title

MRG Effitas Q4 2018 360 Degree Assessment and Certification






AVAST, Avira, Bitdefender, ESET, F-Secure, Kaspersky, McAfee, Microsoft, Symantec

Publication date


Statement from Test Lab

“A first-of-its-kind test that covers all angles, our pioneering 360 Degree Protection Test targets the key threats faced by internet users. In each test case we employ the full spectrum of Early Life Malware. We use a Time-To-Detect metric to measure how long it takes each application to detect and neutralize missed threats.”

Tested products

VendorProductVendor status
AVASTBusiness Antivirusparticipant
AviraAntivirus Pro - Business editionparticipant
BitdefenderGravityzone Advanced Business Security - cloudparticipant
ESETEndpoint Securityparticipant
F-SecureProtection Service for Businessparticipant
KasperskySmall Office Securityparticipant
MicrosoftWindows Defenderincluded
McAfeeEndpoint Securityincluded
SymantecEndpoint Protectionparticipant

AMTSO Standard compliance info

Notification issued


Notification method

Publicly posted test plan, Contact list notification

Test plan

Commencement date



These Vendors chose to adopt Participant status under the AMTSO Standard, gaining certain guaranteed rights in return for attestations.

“Included” Vendors


These Vendors did not chose to adopt Participant status under the AMTSO Standard, but may have engaged with the test lab in other ways.

Commentary dates
CommentaryStart dateEnd date
Phase 1 Commentary2018-11-162018-11-27
Phase 2 Commentary2019-02-112019-02-18

Commentary received

VendorCommentary phaseComment
SymantecPhase 1

3.6.a: Microsoft Edge is not the most widely used browser in the windows OS. Moreover, it is also the most restrictive browser, resulting in a number of Symantec’s protection technologies being unavailable. Based on market share, Chrome would be the most appropriate choice.

Response from Tester: We got the same feedback from multiple vendors. We can’t change this in Q4, but we will experiment with this in Q1.

3.7: The tester should ensure that malicious actions occurred on the system and refrain from simply checking if the application “ran” as an indicator of protection. 3.8: Same complaint at 3.7. The tester is advised to verify that malicious actions occurred.

Response from Tester: We are checking for malicious actions. Methodology/test plan will be updated in Q1.

4.0.7: The tester should clarify if the “Original site” is the one hosting the malware. If so, how would it be ensured that the site is serving a deterministic malicious program and there is no change in content over the period of testing?

Response from Tester: We already use a replay proxy to prevent such issues. Once a content is cached by our replay proxy, the same content is given back during the test. Methodology/test plan will be updated in Q1 to reflect this.

5.6: Use of internal servers for FP testing is not a real-world scenario. There is also no mention of the scale/impact of a FP in the test plan.

Response from Tester: We will discuss internal server issue internally. There is no scale/impact of the FP test, as there is no “one, final ranking” where ITW, performance and FP results are combined together.

6: In the performance section – the tester should clarify number of runs used and the methodology used to address variations between those runs.

Response from Tester: Agree. Methodology/test plan will be updated in Q1

AMTSO Standard compliance status

Confirmed compliant with AMTSO Standard v1.1Compliance report