Test Lab: SE Labs Logo
Test Title: Q1 2020 Email Security Service Protection
AMTSO Test ID: AMTSO-LS1-TP022

  • Statement from Test Lab:

    “Q1 2020 Email Security Service Protection

    SE Labs tests a variety of email filtering services to judge which are the most effective. Each service is exposed to the same threats at the same time. The threats are an equal mixture of commodity, social engineering, phishing, malware and legitimate samples. The malicious samples contained within the test are all generated from commodity samples discovered independently by SE Labs, thereby ensuring that they reflect reality to the highest degree. The legitimate samples are all generated in-house and are designed to be demonstrably benign. The resulting test reports show how effective each Test Subject Vendor’s products are at detecting malicious samples while still allowing through legitimate ones”

    Test Plan

    Participants: 1
    These Vendors chose to adopt Participant status under the AMTSO Standard, gaining certain guaranteed rights in return for attestations

    “Included” Vendors: 5
    These Vendors did not chose to adopt Participant status under the AMTSO Standard, but may have engaged with the test lab in other ways

    Compliance Status: Confirmed compliant with AMTSO Standard v1.3

    Published test Report

    Publication Date: 2020-04-03

    Participation information
    Consumer Product title Status Phase 1 Feedback Attestations Comments (Phase 1) Phase 2 feedback Comments (Phase 2)
    Fortinet FortiMail Cloud – Gateway Premium INCLUDED None None None None None
    Google GSuite Business/GSuite Enterprise INCLUDED None None None None None
    Kaspersky Security for Office 365 PARTICIPANT None (Waived) None Completed None
    Microsoft Office 365/Office 365 with Advanced Threat Protection INCLUDED None None None None None
    Mimecast Secure Email Gateway INCLUDED None None None None None
    Perception Point Advanced Email Security INCLUDED None None None None None
    Notable Commentary:

    No Commentary received.

  • Notification date: 2020-02-03
    Notification method: Publicly posted Test Plan, Contact List announcement

    Test Plan

    Commencement date: 2020-02-03

    Phase 1 Commentary dates: 2020-02-17 – 2020-02-27
    Phase 2 Commentary dates: 2020-04-02 – 2020-04-10

    Publication Date: 2020-04-03

    Published test Report

  • SE Labs “Q1 2020 Email Security Service Protection” covers the following products:

    Fortinet FortiMail Cloud – Gateway Premium
    Google GSuite Business
    Google GSuite Enterprise
    Kaspersky Security for Office 365
    Microsoft Office 365
    Microsoft Office 365 with Advanced Threat Protection
    Mimecast Secure Email Gateway
    Perception Point Advanced Email Security

  • Phase 1 Commentary dates: 2020-02-17 – 2020-02-27
    Phase 2 Commentary dates: 2020-04-02 – 2020-04-10

    Commentary received:

    Fortinet

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Google

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Kaspersky

    Vendor Status: PARTICIPANT

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED (Waived)

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: ALL ITEMS MARKED AS SATISFACTORY

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Microsoft

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Mimecast

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

    Perception Point

    Vendor Status: INCLUDED

    Vendor Attestation Status: NO ATTESTATIONS PROVIDED

    Commentary (Phase 1): NO COMMENTARY SUBMITTED

    Vendor Confirmation Status: NO RESPONSE

    Commentary (Phase 2): NO COMMENTARY SUBMITTED

  • Compliance Status: Confirmed compliant with AMTSO Standard v1.3

    Compliance report