Paul Walker has worked at McAfee, then known as Network Associates, since 2002, starting as a program manager. Since then, he has undertaken several projects and program management roles, with the majority of his time spent focusing on the work of McAfee Labs. In 2019 he became McAfee’s External Test Manager where he took on the role as the company’s representative to AMTSO. In addition to these responsibilities, Paul leads program management in the Office of the Chief Technology Officer. At AMTSO, Paul reviews test notifications and coordinates responses with a range of teams in McAfee.
What benefits do you get from AMTSO? How does AMTSO help McAfee?
It takes a while to understand the role of AMTSO and why it is important for McAfee and the cybersecurity industry as a whole. When I look back to the start of my McAfee career, external testers were often just seen as a nuisance and a distraction. Over the years I have seen that relationship change, and there is now much more of a partnership between testers and vendors. A big part of that change has been the raising of standards, and while there are many tests that are still not run through AMTSO, the establishment of the standard has raised the bar on fairness, visibility, and relevance for everyone.
From your vendor perspective, what are some of the biggest challenges in anti-malware testing?
There are several challenges I see. One is ensuring that product testing is aligned with how a consumer would experience our product in the real world. Testers need to find the right level between automation, which can create environmental differences to how a customer might experience a threat, while still ensuring that all products are tested against an attack at the same time. This is because environmental differences can change the result of dynamic malware detection technologies, and differences in timing can result in the reputations of URLs and files changing, creating advantages for those products tested later in the cycle.
Another challenge, and potentially a whole new area for testers and AMTSO, is around testing the whole product offering from vendors in a measurable, objective way. The testers, in the main, do a great job of doing focused testing (malware protection, false propensity, performance, phishing, etc.), but testing the whole product is still left to industry magazines. This means holistic reviews become more subjective and less repeatable. If the results from testers are to help consumers choose the right product for them, then focused tests alone do not tell the whole story. Consumers need to also look at other security features, like parental controls, VPN (both performance and security) and other identity protection features available in or with the product. While some testers do offer parental control, VPN, and phishing tests, it’s only the magazines that are pulling everything together for the consumer.
What does fair or ethical testing mean to you?
This is an interesting question. For me it is not just about the test being transparent, equal and unbiased to all vendors and open for feedback, as a fair test also means pairing relevant threats with relevant defences. It is about ensuring that the test report is telling the reader how the product will perform in their home today, for their grandparents or for their children. Too often we have seen test results that show a product doing well in a test that would not fare so well in the real world. Similarly, we have experienced test results that were less than perfect due to something that a consumer would never experience or would be of no consequence to them. For example, does a consumer really care if an application launch time goes up by 100% if that is only from 0.1s to 0.2s? Completely unnoticeable to a consumer. As vendors, we want to focus 100% of our effort on protecting our customers and providing an excellent user experience, and testing should be completely aligned with that. If achieving perfect test outcomes deviates from that, it can encourage vendors to commit resources to just do well in a test environment when those same resources should be spent on protecting customers. These trade-offs mean a poor test can actually have a negative impact on customer outcomes, whereas a good, fair and ethical test that closely resembles a “real world” environment will drive improvements to customer cybersecurity outcomes.
What challenges do you currently see in today’s threat landscape?
A big area of concern in the consumer threat landscape is social engineering attacks, particularly those associated with the abuse of trusted services. We see a lot of attacks that leverage legitimate and reputable platforms. While the route of the attack is legitimate, the content delivered is not, leading to potential identity theft or fraudulent transactions.
How does McAfee address these social engineering attacks?
There are a number of routes that McAfee is following, each of which layer up to help protect our customers. These include customer education, policy-based coverage, as well as more contextual and granular protection. For example, we are educating our customers that they will never receive an email from McAfee telling them how many viruses their system is infected with and how they can use our products to check their system health. Policy and contextual-based coverage shut down vectors of attack, such as blocking executable and script email attachments, preventing Microsoft Word from running Powershell commands, and detecting other similar behaviours that legitimate processes do not do in a consumer environment.
Thank you very much for these very interesting insights, Paul!
Statements made by individuals may not reflect the views or opinions of their employers, of other AMTSO members, or of the organization as a whole.