Test Lab: MRG Effitas
Test Title: MRG Effitas Q3 2020 360 Degree Assessment and Certification
AMTSO Test ID: AMTSO-LS1-TP028
-
Statement from Test Lab:
“A first-of-its-kind test that covers all angles, our pioneering 360 Degree Protection Test targets the key threats faced by internet users. In each test case we employ the full spectrum of Early Life Malware. We use a Time-To-Detect metric to measure how long it takes each application to detect and neutralize missed threats.”
Participants: 4
These Vendors chose to adopt Participant status under the AMTSO Standard, gaining certain guaranteed rights in return for attestations“Included” Vendors: 4
These Vendors did not chose to adopt Participant status under the AMTSO Standard, but may have engaged with the test lab in other waysCompliance Status: Confirmed compliant with AMTSO Standard v1.3
Publication Date: 2020-12-16
Participation information
Vendor Status Phase 1 Feedback Attestations Comments (Phase 1) Phase 2 feedback Comments (Phase 2) Avast PARTICIPANT Completed Completed None Completed None Avira PARTICIPANT Completed Completed None Completed None BitDefender INCLUDED None None None None None Broadcom/Symantec PARTICIPANT Completed Completed Submitted None None ESET INCLUDED None None None None None F-Secure PARTICIPANT Completed Completed None Completed Submitted Microsoft INCLUDED None None None None None Trend Micro INCLUDED None None None None None Notable Commentary
Broadcom/Symantec (Phase 1):
“Financial malware simulator test methodology – A test should use real malware from the industry instead of generated samples.”F-Secure (Phase 2):
“This comment is regarding the test that was conducted in MRG Effitas 360 degree assessment 2020 Q2 and 360 degree assessment 2020 Q3 Tests.
In our opinion the sample used for clean files test were not simulating the real world scenario as experienced every day by users.
If we want to simulate a real-world scenario for normal users, the clean set for false positive testing should be composed of common/popular applications from popular software/websites. In the case that handcrafted or newly created applications are to be used, they need to be tested with proper paramaters such as being found in locations related to code development.“ -
Notification date: 2020-07-17
Notification method: Publicly posted test plan, Contact List noticeCommencement date: 2020-07-24
Phase 1 Commentary dates: 2020-08-10 – 2020-08-19
Phase 2 Commentary dates: 2020-12-16 – 2020-12-22Publication date: 2020-12-16
-
MRG Effitas “Q3 2020 360 Degree Assessment and Certification” (ID AMTSO-LS1-TP028) covered the following products:
Avast Business Antivirus
Avira Antivirus Pro Business
BitDefender GravityZone Advanced Business Security Cloud
Broadcom/Symantec Endpoint Protection
ESET Endpoint Security
F-Secure Protection Services for Buiness
Microsoft Windows Defender
Trend Micro Worry-Free Services with XGEN -
Phase 1 Commentary dates: 2020-08-10 – 2020-08-19
Phase 2 Commentary dates: 2020-12-16 – 2020-12-22Commentary received:
-
Avast
Vendor Status: PARTICIPANT
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Avira
Vendor Status: PARTICIPANT
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Bitdefender
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Broadcom/Symantec
Vendor Status: INCLUDED
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Commentary (Phase 1): COMMENTARY SUBMITTED as follows:
“Financial malware simulator test methodology – A test should use real malware from the industry instead of generated samples.”
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Eset
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
F-Secure
Vendor Status: PARTICIPANT
Vendor Attestation Status: ALL ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: SOME ITEMS MARKED AS LESS THAN SATISFACTORY
Commentary (Phase 2): COMMENTARY SUBMITTED as follows:
On section “Feedback and Disputes”:
“This comment is regarding the test that was conducted in MRG Effitas 360 degree assessment 2020 Q2 and 360 degree assessment 2020 Q3 Tests.
In our opinion the sample used for clean files test were not simulating the real world scenario as experienced every day by users.
If we want to simulate a real-world scenario for normal users, the clean set for false positive testing should be composed of common/popular applications from popular software/websites. In the case that handcrafted or newly created applications are to be used, they need to be tested with proper paramaters such as being found in locations related to code development.“ -
Microsoft
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Trend Micro
Vendor Status: INCLUDED
Vendor Attestation Status: NO ATTESTATIONS PROVIDED
Commentary (Phase 1): NO COMMENTARY SUBMITTED
Vendor Confirmation Status: NO RESPONSE
Commentary (Phase 2): NO COMMENTARY SUBMITTED
-
Avast
-
Compliance Status: Confirmed compliant with AMTSO Standard v1.3