Test Lab: NioGuard Security Lab Logo
Test Title: NioGuard Security Lab 2019 Anti-Cryptojacking Test
AMTSO Test ID: AMTSO-LS1-TP014

  • Statement from Test Lab:

    “NioGuard Security Lab (Tester) tests a variety of corporate endpoint security products from the industry leaders in an effort to judge which were the most effective in detecting unauthorized cryptomining called cryptojacking. Each product is exposed to the same threats that represent cryptominers in different forms. The Test Report indicates how effectively the products were at detecting those threats in real time.”

    Participants: 1
    These Vendors chose to adopt Participant status under the AMTSO Standard, gaining certain guaranteed rights in return for attestations

    “Included” Vendors: 0
    These Vendors did not chose to adopt Participant status under the AMTSO Standard, but may have engaged with the test lab in other ways

    Compliance Status: CONFIRMED COMPLIANT with AMTSO Standard v1.1

    Published test Report

    Publication Date: 2019-08-05

    Participation information
    Vendor Status Phase 1 Feedback Attestations Comments (Phase 1) Phase 2 feedback Comments (Phase 2)
    Acronis PARTICIPANT Completed Completed None Completed None
    OTHER VENDORS All other vendor data ANONYMIZED

    NOTE: Multiple additional vendors were listed as within scope in the Test Plan; no information has been published as to which of these were ultimately covered in the final report, where data for all but one participant has been anonymized. All potential Test Subject Vendors were contacted for feedback.

    Notable Commentary:

    Kaspersky (Phase 1):
    “After scrutinizing the Test Plan document, we‘ve made a decision not to participate in this test.
    There is no regulation about legitimacy of mining tools and their definition at the moment, so different points of view, which are similar to a PUA topic, could happen.    ”

    K7 Computing (Phase 1):
    “Feedback on improvement of the test has been provided separately to the Tester.”

    “We believe improvements are warranted.”

    Sophos (Phase 1):
    “Sophos is always open to tests that comply with AMTSO standards. However, we are respectively declining the invitation to participate in the Nioguard Anti-Cryptojacking Test. While we are confident in our ability to protect against malicious cryptomining, we do not feel that a comparison with the Acronis Backup product is a like-for-like comparison and are concerned a comparative test would lead to unnecessary market confusion.“

  • Test Plan

    Notification date: 2019-06-17
    Notification method: Publicly posted Test Plan, Contact List announcement

    Commencement date: 2019-07-01

    Phase 1 Commentary dates: 2019-06-21 – 2019-07-01
    Phase 2 Commentary dates: 2019-08-08 – 2019-08-15

    Publication date: 2019-08-05

    Published test Report

  • NioGuard Security Labs “Anti-Cryptojacking Test 2019” (ID AMTSO-LS1-TP014) covered the following products:

    Acronis “Acronis Backup”

    NOTE: Multiple additional products tested, but not identified in final test report

  • Phase 1 Commentary dates: 2019-06-21 – 2019-07-01
    Phase 2 Commentary dates: 2019-08-08 – 2019-08-15

    Commentary received:

    • Acronis

      Vendor Status: PARTICIPANT

      Vendor Attestation Status: ALL ATTESTATIONS PROVIDED

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Avast

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Avira

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Bitdefender

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • CrowdStrike

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • ESET

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • F-Secure

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • FireEye

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • K7 Computing

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): COMMENTARY SUBMITTED as follows:

      “Feedback on improvement of the test has been provided separately to the Tester.”

      “We believe improvements are warranted.”

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Kaspersky

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): COMMENTARY SUBMITTED as follows:

      “After scrutinizing the Test Plan document, we‘ve made a decision not to participate in this test.
      There is no regulation about legitimacy of mining tools and their definition at the moment, so different points of view, which are similar to a PUA topic, could happen.      ”

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • MalwareBytes

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • McAfee

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Microsoft

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Sophos

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): COMMENTARY SUBMITTED as follows:

      “Sophos is always open to tests that comply with AMTSO standards. However, we are respectively declining the invitation to participate in the Nioguard Anti-Cryptojacking Test. While we are confident in our ability to protect against malicious cryptomining, we do not feel that a comparison with the Acronis Backup product is a like-for-like comparison and are concerned a comparative test would lead to unnecessary market confusion.”

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Symantec

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Trend Micro

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • VIPRE

      Vendor Status: UNKOWN (results anonymized)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

  • Compliance Status: CONFIRMED COMPLIANT with AMTSO Standard v1.1

    Compliance report