Interview with David Ellis at SecureIQLab: “Using Obscure Test Cases Will Make Testing Meaningless”

Today, we have David Ellis as a guest in our AMTSO member interview series. Many of us AMTSO members met David first when he presented about the “New Era of Testing Cloud Security” at our Testing Town Hall in April 2021, and he was elected to join our board in July. He brings extensive experience in developing testing and validation metrics, documenting testing methodologies from his current role at cloud security validation provider, SecureIQLab. He also has business development and sales experience from his previous positions at Forcepoint, NSS Labs, and HighSide.

David, in your own words, what’s your AMTSO story, why and when did you and SecureIQLab decide to join?

Our aim at SecureIQLab is to help improve the cybersecurity posture fundamentally with cloud security based solutions and other emerging technologies through accountability and transparency between testers and the vendor community at large. We recognize that the cybersecurity problem is bigger than we are and that our collaborative approach aligns well with AMTSO and its mission.

AMTSO develops and evolves their Testing Protocol Standard for use in testing involving the development, use, and validation of anti-malware solutions. This standard provides the framework needed for fair, open, and trustworthy testing of security solutions and is the main reason we joined AMTSO. We are currently using the AMTSO Testing Protocol Standard in our testing methodologies.

We at SecureIQLab would like to help AMTSO in charting a new course of validating solutions in the era of constantly evolving Cloud security-based technologies. SecureIQLab strives to elevate testing standards overall and help add value to the community at large. We want AMTSO to be the testing standard for testers, security vendors, organizations and the industry overall and not just another type of compliance, certification or validation -based organization that is fundamentally driven by the security vendors.

We were excited to officially join AMTSO at the start of 2021.

What benefits do you get from AMTSO? How does AMTSO help your company?

The first benefit from joining AMTSO is to partner with a community that is passionate about testing. A quick example of this was when we announced to AMTSO our plans for testing cloud WAF solutions, we were immediately and proactively assisted with refining our test methodology to the AMTSO Test Protocol Standard.

The second benefit is instantly connecting us with some of the larger security vendors that have an interest in testing. AMTSO’s leadership has deep roots in the cybersecurity industry and have connections in the cybersecurity vendor space that go well beyond the AMTSO membership list which we hope will bring in emerging technologies and solutions especially around cloud security.

Cybersecurity testing has historically been a touchy subject. There is a lot at stake in cybersecurity testing and there have previously been difficulties between testers and vendors. This is not to say that contentions won’t ever arise again, but AMTSO has weathered a few storms and has improved their testing standards through these challenges. Additionally, AMTSO provides a forum in which testers and vendors can air differences and find resolutions. Our expertise and AMTSO with their history have the ability to grow together.

This is certainly true – testing isn’t an easy subject, and there are many different views on this. What does fair or ethical testing mean to you?

Ethical testing starts with a good test methodology that resonates with the industry at large and aims to solve for relevant real-world use cases and test scenarios. The AMTSO Test Protocol Standard is a good framework to build an ethical test on.

Additionally, ethical testing must revolve around testing against a broad selection of industry-relevant attacks. Using only some obscure test cases using the traditional closed centric method of testing will make testing meaningless.

Finally, fair and ethical testing means transparency: Transparency in how the testing is scored, and as much transparency as possible in the testing process without collaterally creating additional security risks. This allows others to evaluate the test.

These sound like great values in terms of testing. Let’s speak about cloud security, the solutions you are testing at SecureIQLab. What problem does cloud security solve within the cyber industry?

Cloud security solves two major problems within the cyber industry. First, cloud security helps organizations deal with a distributed workforce. Second, cloud security allows for organizations to take advantage of global talent more readily in cybersecurity, in essence, the cloud has made cybersecurity more widely available and adoptable. SaaS model apps and convenient work remote would not be possible without the development of cloud-based security.

Have your clients seen a higher demand for cloud security during the pandemic – and is this something that’s also reflecting positively on your business?

Yes, there certainly has been a higher interest and demand for cloud security since the pandemic forced a remote work environment. While many of our clients were already deep into the process of moving securely to the cloud, the increased need for cloud security has had a positive impact on our business.

And what challenges are you faced with in your particular field?

The biggest challenge we face is assumptions. Assumptions range from thinking what used to work is still good enough to assuming that the security tools we are familiar with are still the right solutions. For example, take cloud WAF technology. We had assumed that cloud WAF technology was mature enough to not require tuning. This assumption turned out to only being partially correct. From our testing, it was determined that many of the products tested still require some degree of tuning to provide effective security.

Speaking of tests, there are also plenty of assumptions that surround testing. Many of these deal with transparency, process, and vendor bias. Developing a methodology that aligns with the AMTSO Testing Standard goes a long way towards resolving assumptions in these areas.

Thanks for these insights. One final question – what do you enjoy most about working in the cybersecurity industry?

What I enjoy most about working in the cybersecurity industry is solving problems and making a difference. Cybersecurity involves working with organizations that are comprised of a lot of stakeholders and intricate relations between business units. Helping organizations solve security really boils down to working with others to solve human security challenges. Solving these challenges help harden enterprises and governments and assist them in being more stable. This in turn benefits the workers and citizens of those organizations and countries.

One recent project that I enjoyed working on involved the security of an application that was distributed across multiple regions. In addition to regional variations in regulations and compliance, the security posture and integrity of the data at rest and in motion across the cloud infrastructure were a chief concern. Ensuring the necessary security measures are propagated and are in place across the infrastructure provided our team with unique challenges to solve. There is never a dull moment in cybersecurity.

Statements made by individuals may not reflect the views or opinions of their employers, of other AMTSO members, or of the organization as a whole.