Test Lab: SE Labs Logo
Test Title: Q1 2020 Email Security Service Protection
AMTSO Test ID: AMTSO-LS1-TP022

  • Statement from Test Lab:

    “Q1 2020 Email Security Service Protection

    SE Labs tests a variety of email filtering services to judge which are the most effective. Each service is exposed to the same threats at the same time. The threats are an equal mixture of commodity, social engineering, phishing, malware and legitimate samples. The malicious samples contained within the test are all generated from commodity samples discovered independently by SE Labs, thereby ensuring that they reflect reality to the highest degree. The legitimate samples are all generated in-house and are designed to be demonstrably benign. The resulting test reports show how effective each Test Subject Vendor’s products are at detecting malicious samples while still allowing through legitimate ones”

    Test Plan

    Participants: 1
    These Vendors chose to adopt Participant status under the AMTSO Standard, gaining certain guaranteed rights in return for attestations

    “Included” Vendors: 5
    These Vendors did not chose to adopt Participant status under the AMTSO Standard, but may have engaged with the test lab in other ways

    Compliance Status: Confirmed compliant with AMTSO Standard v1.3

    Published test Report

    Publication Date: 2020-04-03

    Participation information
    Consumer Product title Status Phase 1 Feedback Attestations Comments (Phase 1) Phase 2 feedback Comments (Phase 2)
    Fortinet FortiMail Cloud – Gateway Premium INCLUDED None None None None None
    Google GSuite Business/GSuite Enterprise INCLUDED None None None None None
    Kaspersky Security for Office 365 PARTICIPANT None (Waived) None Completed None
    Microsoft Office 365/Office 365 with Advanced Threat Protection INCLUDED None None None None None
    Mimecast Secure Email Gateway INCLUDED None None None None None
    Perception Point Advanced Email Security INCLUDED None None None None None
    Notable Commentary:

    No Commentary received.

  • Notification date: 2020-02-03
    Notification method: Publicly posted Test Plan, Contact List announcement

    Test Plan

    Commencement date: 2020-02-03

    Phase 1 Commentary dates: 2020-02-17 – 2020-02-27
    Phase 2 Commentary dates: 2020-04-02 – 2020-04-10

    Publication Date: 2020-04-03

    Published test Report

  • SE Labs “Q1 2020 Email Security Service Protection” covers the following products:

    Fortinet FortiMail Cloud – Gateway Premium
    Google GSuite Business
    Google GSuite Enterprise
    Kaspersky Security for Office 365
    Microsoft Office 365
    Microsoft Office 365 with Advanced Threat Protection
    Mimecast Secure Email Gateway
    Perception Point Advanced Email Security

  • Phase 1 Commentary dates: 2020-02-17 – 2020-02-27
    Phase 2 Commentary dates: 2020-04-02 – 2020-04-10

    Commentary received:

    • Fortinet

      Vendor Status: INCLUDED

      Vendor Attestation Status: NO ATTESTATIONS PROVIDED

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Vendor Confirmation Status: NO RESPONSE

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Google

      Vendor Status: INCLUDED

      Vendor Attestation Status: NO ATTESTATIONS PROVIDED

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Vendor Confirmation Status: NO RESPONSE

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Kaspersky

      Vendor Status: PARTICIPANT

      Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Waived)

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Microsoft

      Vendor Status: INCLUDED

      Vendor Attestation Status: NO ATTESTATIONS PROVIDED

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Vendor Confirmation Status: NO RESPONSE

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Mimecast

      Vendor Status: INCLUDED

      Vendor Attestation Status: NO ATTESTATIONS PROVIDED

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Vendor Confirmation Status: NO RESPONSE

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

    • Perception Point

      Vendor Status: INCLUDED

      Vendor Attestation Status: NO ATTESTATIONS PROVIDED

      Commentary (Phase 1): NO COMMENTARY SUBMITTED

      Vendor Confirmation Status: NO RESPONSE

      Commentary (Phase 2): NO COMMENTARY SUBMITTED

  • Compliance Status: Confirmed compliant with AMTSO Standard v1.3

    Compliance report