Test Lab: NioGuard Security Lab Logo
Test Title: NioGuard Security Lab 2019 Anti-Cryptojacking Test
AMTSO Test ID: AMTSO-LS1-TP014

  • Statement from Test Lab:

    “NioGuard Security Lab (Tester) tests a variety of corporate endpoint security products from the industry leaders in an effort to judge which were the most effective in detecting unauthorized cryptomining called cryptojacking. Each product is exposed to the same threats that represent cryptominers in different forms. The Test Report indicates how effectively the products were at detecting those threats in real time.”

    Participants: 1
    These Vendors chose to adopt Participant status under the AMTSO Standard, gaining certain guaranteed rights in return for attestations

    “Included” Vendors: 0
    These Vendors did not chose to adopt Participant status under the AMTSO Standard, but may have engaged with the test lab in other ways

    Compliance Status: CONFIRMED COMPLIANT with AMTSO Standard v1.1

    Published test Report

    Publication Date: 2019-08-05

    Participation information
    Vendor Status Phase 1 Feedback Attestations Comments (Phase 1) Phase 2 feedback Comments (Phase 2)
    Acronis PARTICIPANT Completed Completed None Completed None
    OTHER VENDORS All other vendor data ANONYMIZED

    NOTE: Multiple additional vendors were listed as within scope in the Test Plan; no information has been published as to which of these were ultimately covered in the final report, where data for all but one participant has been anonymized. All potential Test Subject Vendors were contacted for feedback.

    Notable Commentary:

    Kaspersky (Phase 1):
    “After scrutinizing the Test Plan document, we‘ve made a decision not to participate in this test.
    There is no regulation about legitimacy of mining tools and their definition at the moment, so different points of view, which are similar to a PUA topic, could happen.    ”

    K7 Computing (Phase 1):
    “Feedback on improvement of the test has been provided separately to the Tester.”

    “We believe improvements are warranted.”

    Sophos (Phase 1):
    “Sophos is always open to tests that comply with AMTSO standards. However, we are respectively declining the invitation to participate in the Nioguard Anti-Cryptojacking Test. While we are confident in our ability to protect against malicious cryptomining, we do not feel that a comparison with the Acronis Backup product is a like-for-like comparison and are concerned a comparative test would lead to unnecessary market confusion.“

  • Test Plan

    Notification date: 2019-06-17
    Notification method: Publicly posted Test Plan, Contact List announcement

    Commencement date: 2019-07-01

    Phase 1 Commentary dates: 2019-06-21 – 2019-07-01
    Phase 2 Commentary dates: 2019-08-08 – 2019-08-15

    Publication date: 2019-08-05

    Published test Report

  • NioGuard Security Labs “Anti-Cryptojacking Test 2019” (ID AMTSO-LS1-TP014) covered the following products:

    Acronis “Acronis Backup”

    NOTE: Multiple additional products tested, but not identified in final test report

  • Phase 1 Commentary dates: 2019-06-21 – 2019-07-01
    Phase 2 Commentary dates: 2019-08-08 – 2019-08-15

    Commentary received:

    Acronis

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Avast

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Avira

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Bitdefender

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    CrowdStrike

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    ESET

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    F-Secure

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    FireEye

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    K7 Computing

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): COMMENTARY SUBMITTED as follows:

    “Feedback on improvement of the test has been provided separately to the Tester.”

    “We believe improvements are warranted.”

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Kaspersky

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): COMMENTARY SUBMITTED as follows:

    “After scrutinizing the Test Plan document, we‘ve made a decision not to participate in this test.
    There is no regulation about legitimacy of mining tools and their definition at the moment, so different points of view, which are similar to a PUA topic, could happen.    ”

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    MalwareBytes

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    McAfee

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Microsoft

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Sophos

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): COMMENTARY SUBMITTED as follows:

    “Sophos is always open to tests that comply with AMTSO standards. However, we are respectively declining the invitation to participate in the Nioguard Anti-Cryptojacking Test. While we are confident in our ability to protect against malicious cryptomining, we do not feel that a comparison with the Acronis Backup product is a like-for-like comparison and are concerned a comparative test would lead to unnecessary market confusion.”

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Symantec

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Trend Micro

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    VIPRE

    Vendor Status: UNKOWN (results anonymized)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

  • Compliance Status: CONFIRMED COMPLIANT with AMTSO Standard v1.1

    Compliance report