Test Lab: SE Labs Logo
Test Title: SE Labs Q1 2019 Endpoint Protection: Enterprise, Small Business, and Consumer
AMTSO Test ID: AMTSO-LS1-TP008

  • Statement from Test Lab:

    “January-March 2019 – Endpoint Protection Enterprise, Small Business, and Consumer

    SE Labs tests a variety of endpoint security products from a range of well-known vendors in an effort to judge which were the most effective. Each enterprise, small business, or consumer class product are exposed to the same threats, which are a mixture of targeted attacks using well-established techniques, public email, and web-based threats that are known or found to be live on the internet at the time of the test. The Test Reports indicate how effectively the products were at detecting and/or protecting against those threats in real time.”

    Test Plan

    Participants: 26
    These Vendors chose to adopt Participant status under the AMTSO Standard, gaining certain guaranteed rights in return for attestations

    “Included” Vendors: 6
    These Vendors did not chose to adopt Participant status under the AMTSO Standard, but may have engaged with the test lab in other ways

    Compliance Status: Confirmed compliant with AMTSO Standard v1.1

    Published test Report (Consumer)

    Published test Report (SMB)

    Published test Report (Enterprise)

    Publication Date: 2019-06-05

    Participation information
    Consumer Product title Status Phase 1 Feedback Attestations Comments (Phase 1) Phase 2 feedback Comments (Phase 2)
    AVAST Free Antivirus PARTICIPANT Completed Completed None Completed None
    AVG Antivirus Free Edition PARTICIPANT Completed Completed None Completed None
    Avira Free Security Suite PARTICIPANT None (Waived) None Completed None
    BullGuard Internet Security INCLUDED None None None None None
    eScan Internet Security Suite INCLUDED None None None None None
    ESET Internet Security PARTICIPANT Completed Completed None None None
    F-Secure Safe INCLUDED None None None None None
    G-Data Internet Security INCLUDED None None None None None
    K7 Computing Antivirus Premium INCLUDED Completed Completed Submitted Completed Submitted
    Kaspersky Internet Security PARTICIPANT Completed Completed None None None
    McAfee Internet Security PARTICIPANT None (Waived) None None None
    Microsoft Windows Defender PARTICIPANT None (Waived) None None None
    Symantec Norton Security PARTICIPANT Completed Completed None Completed None
    Trend Micro Internet Security PARTICIPANT Completed Completed None None None
    SMB
    BitDefender Gravity Zone Endpoint Security PARTICIPANT Completed Completed None None None
    ESET Endpoint Security PARTICIPANT Completed Completed None None None
    Kaspersky Small Office Security PARTICIPANT Completed Completed None None None
    McAfee Endpoint Security PARTICIPANT None (Waived) None None None
    Microsoft Windows Defender ATP’s Antivirus PARTICIPANT None (Waived) None None None
    Sophos Intercept X PARTICIPANT Completed Completed None Completed None
    Symantec Endpoint Protection Cloud PARTICIPANT Completed Completed None Completed None
    Trend Micro Worry Free Security Services PARTICIPANT Completed Completed None None None
    Enterprise
    BitDefender Gravity Zone Endpoint Security PARTICIPANT Completed Completed None None None
    Crowdstrike Falcon PARTICIPANT None (Waived) None None None
    ESET Endpoint Security PARTICIPANT Completed Completed None None None
    Kaspersky Endpoint Security PARTICIPANT Completed Completed None None None
    McAfee Endpoint Security PARTICIPANT None (Waived) None None None
    Microsoft Windows Defender ATP’s Antivirus PARTICIPANT None (Waived) None None None
    Sophos Intercept X PARTICIPANT Completed Completed None Completed None
    Symantec Endpoint Security Enterprise Edition PARTICIPANT Completed Completed None Completed None
    Trend Micro OfficeScan, Intrusion Defense Firewall PARTICIPANT Completed Completed None None None
    TrustPort Antivirus for Business INCLUDED None None None None None
    Notable Commentary:

    K7 Computing (Phase 1):

    “K7 was never notified about being included in the test. Therefore K7 was not given the option of providing a suitable product to include in the test. The tester has itself chosen the most basic K7 consumer security product, without K7’s knowledge, which is entirely unsuitable given the premise of the Test Plan. Due diligence on suitability to match the nature of the test has NOT been taken into account at all.”

    K7 Computing (Phase 2):

    “As mentioned earlier, for reasons best known to SELabs, K7’s flagship K7 Total Security product which supports robust web security was NOT chosen for this test series despite its features being best suited to the test plan executed. Instead, K7’s most basic product has been tested without providing K7 recourse to choosing to be a Participant. K7 has not even been given the preliminary test results, and hence has not had any opportunity to dispute the test results.”

    Response from Test Lab:

    “We generally respond to requests for results from vendors rather than proactively sending them out. We do not currently have a formal relationship with K7 and so there is no existing process whereby we send data without a request. Had K7 requested this data we would have provided it as soon as possible.

    The product selection was made and tested before K7 communicated that it preferred its more fully-featured product to be included in such a challenging test. It is possible that the delay between our initial discussions about this and the results being published has caused confusion at K7’s end, causing the team to think that we selected Antivirus Premium after these discussions, when in fact these results are from earlier in the year. As agreed with K7 after the first stage of commentary, SE Labs has included a comment in the report that registers K7’s belief that K7 Total Security would have been a more appropriate product to include.

    SE Labs notes that the test includes a selection of products, many of which are free and/or considered not to be premium offerings. Therefore, while we understand K7’s preference for having a more fully-featured product undergo testing, the test is a fair comparison of a number of anti-malware products available to the public within a range of difference prices.“

  • Notification date: 2019-01-04
    Notification method: Publicly posted Test Plan, Contact List announcement

    Test Plan

    Commencement date: 2019-01-09

    Phase 1 Commentary dates: 2019-01-23 – 2018-01-30
    Phase 2 Commentary dates: 2019-06-03 – 2019-06-10

    Publication date: 2019-06-05

    Published test Report (Consumer)

    Published test Report (SMB)

    Published test Report (Enterprise)

  • SE Labs “Q1 2019 Endpoint Protection: Enterprise, Small Business, and Consumer” covers the following products:

    Test component: “Home Anti-Malware Protection Q1 2019“:

    AVAST Free Antivirus
    AVG Antivirus Free Edition
    Avira Free Security Suite
    BullGuard Internet Security
    eScan Internet Security Suite
    ESET Internet Security
    F-Secure Safe
    G-Data Internet Security
    K7 Antivirus Premium
    Kaspersky Internet Security
    McAfee Internet Security
    Microsoft Windows Defender
    Symantec Norton Security
    Trend Micro Internet Security

    Test component: “Small Business Endpoint Protection“:

    BitDefender Gravity Zone Endpoint Security
    ESET Endpoint Security
    Kaspersky Small Office Security
    McAfee Endpoint Security
    Microsoft Windows Defender ATP’s Antivirus
    Sophos Intercept X
    Symantec Endpoint Protection Cloud
    Trend Micro Worry Free Security Services

    Test component: “Enterprise Endpoint Protection“:

    BitDefender Gravity Zone Endpoint Security
    Crowdstrike Falcon
    ESET Endpoint Security
    Kaspersky Endpoint Security
    McAfee Endpoint Security
    Microsoft Windows Defender ATP’s Antivirus
    Sophos Intercept X
    Symantec Endpoint Security Enterprise Edition
    Trend Micro OfficeScan, Intrusion Defense Firewall
    TrustPort Antivirus for Business

  • Phase 1 Commentary dates: 2019-01-23 – 2018-01-30
    Phase 2 Commentary dates: 2019-06-03 – 2019-06-10

    Commentary received:

    Avast

    Test Components covered: Home

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    AVG

    Test Components covered: Home

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Avira

    Test Components covered: Home, SMB, Enterprise

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED (WAIVED)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    BitDefender

    Test Components covered: SMB, Enterprise

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    BullGuard

    Test Components covered: Home

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    CrowdStrike

    Test Components covered: Enterprise

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    eScan

    Test Components covered: Home

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    ESET

    Test Components covered: Home, SMB, Enterprise

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    F-Secure

    Test Components covered: Home

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    G Data

    Test Components covered: Home

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    K7 Computing

    Test Components covered: Home

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): COMMENTARY SUBMITTED as follows:

    “K7 was never notified about being included in the test. Therefore K7 was not given the option of providing a suitable product to include in the test. The tester has itself chosen the most basic K7 consumer security product, without K7’s knowledge, which is entirely unsuitable given the premise of the Test Plan. Due diligence on suitability to match the nature of the test has NOT been taken into account at all.”

    Vendor Confirmation Status: COMPLETED

    COMMENTARY SUBMITTED as follows:

    “As mentioned earlier, for reasons best known to SELabs, K7’s flagship K7 Total Security product which supports robust web security was NOT chosen for this test series despite its features being best suited to the test plan executed. Instead, K7’s most basic product has been tested without providing K7 recourse to choosing to be a Participant. K7 has not even been given the preliminary test results, and hence has not had any opportunity to dispute the test results.”

    Response from Test Lab:

    “We generally respond to requests for results from vendors rather than proactively sending them out. We do not currently have a formal relationship with K7 and so there is no existing process whereby we send data without a request. Had K7 requested this data we would have provided it as soon as possible.

    The product selection was made and tested before K7 communicated that it preferred its more fully-featured product to be included in such a challenging test. It is possible that the delay between our initial discussions about this and the results being published has caused confusion at K7’s end, causing the team to think that we selected Antivirus Premium after these discussions, when in fact these results are from earlier in the year. As agreed with K7 after the first stage of commentary, SE Labs has included a comment in the report that registers K7’s belief that K7 Total Security would have been a more appropriate product to include.

    SE Labs notes that the test includes a selection of products, many of which are free and/or considered not to be premium offerings. Therefore, while we understand K7’s preference for having a more fully-featured product undergo testing, the test is a fair comparison of a number of anti-malware products available to the public within a range of difference prices.”

    Kaspersky Lab

    Test Components covered: Home, SMB, Enterprise

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    McAfee

    Test Components covered: Home, SMB, Enterprise

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Microsoft

    Test Components covered: Home, SMB, Enterprise

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Sophos

    Test Components covered: SMB, Enterprise

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Symantec

    Test Components covered: Home, SMB, Enterprise

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Waived)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Trend Micro

    Test Components covered: Home, SMB, Enterprise

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Waived)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Trustport

    Test Components covered: SMB, Enterprise

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

  • Compliance Status: Confirmed compliant with AMTSO Standard v1.1

    Compliance report