Test Lab: SE Labs Logo
Test Title: “Q3 2018 Endpoint Protection: Enterprise, Small Business, and Consumer”
AMTSO Test ID: AMTSO-LS1-TP002

  • Statement from Test Lab:

    “September 2018 – Endpoint Protection
    Enterprise, Small Business, and Consumer

    SE Labs tests a variety of endpoint security products from a range of well-known vendors in an effort to judge which were the most effective. Each enterprise, small business, or consumer class product are exposed to the same threats, which are a mixture of targeted attacks using well-established techniques, public email, and web-based threats that are known or found to be live on the internet at the time of the test. The Test Reports indicate how effectively the products were at detecting and/or protecting against those threats in real time.”

    Participants: 27/33

    Compliance Status: Confirmed Compliant with AMTSO Standard v1.0

    Publication Date: 2018-10-25

    Published Test Report (Enterprise – PDF)

    Published Test Report (Small Business – PDF)

    Published Test Report (Consumer – PDF)

    Participation information
    Consumer Status Phase 1 Feedback Attestations Comments (Phase 1) Phase 2 Feedback Comments (Phase 2)
    AVAST PARTICIPANT Completed Completed None Completed Submitted
    AVG PARTICIPANT Completed Completed None Completed Submitted
    Avira INCLUDED Completed None Submitted Completed Submitted
    ESET PARTICIPANT Completed Completed None Completed None
    F-Secure PARTICIPANT Completed Completed None None None
    G-Data INCLUDED None None None None None
    Kaspersky Lab PARTICIPANT Completed Completed None Completed None
    McAfee PARTICIPANT None (Waived) None None None
    Microsoft PARTICIPANT Completed Completed None None None
    Symantec PARTICIPANT Completed Completed None Completed Submitted
    Trend Micro INCLUDED None None None None None
    Webroot PARTICIPANT Completed Completed None Completed Submitted
    Small Business
    BitDefender PARTICIPANT Completed Completed None None None
    ESET PARTICIPANT Completed Completed None Completed None
    Kaspersky Lab PARTICIPANT Completed Completed None Completed None
    McAfee PARTICIPANT None (Waived) None None None
    Microsoft PARTICIPANT Completed Completed None None None
    Sophos PARTICIPANT Completed Completed None Completed None
    Symantec PARTICIPANT Completed Completed None Completed Submitted
    Trend Micro INCLUDED None None None None None
    Enterprise
    BitDefender PARTICIPANT Completed Completed None Completed None
    Crowdstrike PARTICIPANT Completed Completed None Completed Submitted
    ESET PARTICIPANT Completed Completed None Completed None
    Kaspersky Lab PARTICIPANT Completed Completed None Completed None
    MalwareBytes INCLUDED None None None None None
    McAfee PARTICIPANT None (Waived) None None None
    Microsoft PARTICIPANT Completed Completed None None None
    Sophos PARTICIPANT Completed Completed None Completed None
    Symantec PARTICIPANT Completed Completed None Completed Submitted
    Trend Micro INCLUDED None None None None None
    Webroot PARTICIPANT Completed Completed None Completed Submitted
    Notable Commentary:

    AVIRA (Phase 1):
    My company has not voluntarily decided to participate in this test but agree that our product will be tested.

  • Notification date: 2018-06-19
    Notification method: Publicly posted Test Plan, Contact List announcement

    Test Plan

    Commencement date: 2018-06-25

    Phase 1 Commentary completion date: 2018-07-05
    Phase 2 Commentary completion date: 2018-10-08

    Publication Date: 2018-10-25

    Published Test Report (Enterprise – PDF)

    Published Test Report (Small Business – PDF)

    Published Test Report (Consumer – PDF)

  • SE Labs “Q3 2018 Endpoint Protection: Enterprise, Small Business, and Consumer” covers the following products:

    Test component: “Home Anti-Malware Protection Jul-Sep 2018“:

    AVAST Avast Free Antivirus
    AVG AVG Antivirus Free Edition
    Avira Avira Free Security Suite
    ESET ESET Smart Security
    F-Secure SAFE
    G-Data G-Data Internet Security
    Kaspersky Lab Kaspersky Internet Security
    McAfee McAfee Internet Security
    Microsoft Windows Defender
    Symantec Norton Security
    Trend Micro Trend Micro Internet Security
    Webroot Webroot Antivirus

    Test component: “Small Business Endpoint Protection“:

    BitDefender GravityZone
    ESET ESET Endpoint Security
    Kaspersky Lab Kaspersky Small Office Security
    McAfee Small Business
    Microsoft Microsoft System Center Endpoint Protection
    Sophos Sophos Central Endpoint
    Symantec Symantec Endpoint Protection Cloud
    Trend Micro Trend Micro Worry Free Security Services

    Test component: “Enterprise Endpoint Protection“:

    BitDefender GravityZone
    ESET ESET Endpoint Security
    Crowdstrike Crowdstrike Falcon
    Kaspersky Lab Kaspersky Endpoint Security
    MalwareBytes MalwareBytes Endpoint Security
    McAfee McAfee Endpoint Security
    Microsoft Microsoft System Center Endpoint Protection
    Sophos Sophos Central Endpoint
    Symantec Symantec Endpoint Security Enterprise Edition
    Trend Micro Trend Micro OfficeScan, Intrusion Defense Firewall
    Webroot SecureAnywhere Endpoint

  • Phase 1 Commentary Completion Date: July 5, 2018

    Phase 2 Commentary Due Date: TBC

    Commentary received:

    Avast

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: SOME ITEMS MARKED AS LESS THAN SATISFACTORY

    Commentary (Phase 2): COMMENTARY SUBMITTED as follows:

    Item “Communications” – marked as “I have some issues with the way communications between the tester and my company were handled”

    Regarding item “Communications”: “I haven’t seen any communication from the tester regarding this test.

    AVG

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: SOME ITEMS MARKED AS LESS THAN SATISFACTORY

    Commentary (Phase 2): COMMENTARY SUBMITTED as follows:

    Item “Communications” – marked as “I have some issues with the way communications between the tester and my company were handled”

    Regarding item “Communications”: “I haven’t seen any communication from the tester regarding this test.

    Avira

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): COMMENTARY SUBMITTED as follows:

    My company has not voluntarily decided to participate in this test but agree that our product will be tested.

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): COMMENTARY SUBMITTED as follows:

    As Avira participate with the product involuntarily, Avira has no opportunity to review or dispute test data.

    BitDefender

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    CrowdStrike

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: SOME ITEMS MARKED AS LESS THAN SATISFACTORY

    Commentary (Phase 2): COMMENTARY SUBMITTED as follows:

    Item “Communications” – marked as “I have some issues with the way communications between the tester and my company were handled”

    Regarding item “Communications”: “The dispute phase results of the test did not match what we expected after consulting with the tester.

    Item “Feedback and Disputes” – marked as “I was given an opportunity to review and dispute the test data, but there were issues with the process”

    Regarding item “Feedback and Disputes”: “Disputes are still not resolved, and had long communication delays, as well as contradictory information from the tester. Current events affected the dispute process.

    General feedback:

    The sample selection was much less diverse than expected for this test, consisting of large numbers of very similar attacks. This feedback was given to the tester as soon as possible.

    ESET

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    F-Secure

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    G Data

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Kaspersky Lab

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    MalwareBytes

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    McAfee

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: (Attestation requirements waived by test lab)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Microsoft

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Notes: Product 1 of 3 (Windows Defender) – marked “Variances exist between our commercially-available product and the tested configuration, and these variances have been duly disclosed to the Tester.

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Sophos

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Symantec

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): COMMENTARY SUBMITTED as follows:

    Symantec products are used in Default configuration mode in SELabs Public test.

    Trend Micro

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Webroot

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): COMMENTARY SUBMITTED as follows:

    Dispute process still underway. Simon has been good about communicating the progress.

  • Compliance Status: Confirmed Compliant with AMTSO Standard v1.0

    Compliance report