AMTSO’s AI Security Working Group has completed development of its first set of guidelines, which are now being made available for review by the wider AMTSO membership and the public. Developed at speed over the last few months to address the fast-changing world of agentic AI, the “Guidelines for Testing of Agentic Security Products” provide detailed guidance and advice for testers working on the evaluation of tools claiming to protect and secure users and data in an increasingly agentic world.
Alongside this release, our Sandbox Working Group has built a new section for our Sandbox Evaluation Framework covering LLM handling within sandbox tools. The additions expand the framework and provide new KPIs to add to the battery of existing criteria for evaluating sandboxes and similar malware analysis systems. The proposed additions were previously shared with the AI Working Group due to the overlap in subject matter, and are now also being opened up for wider comment. A redline version showing the changes from v1.0 is available to AMTSO members on request.
Both papers will be open for review until Monday, July 20th. Comments, criticism, and suggestions are welcome – please send all feedback to info@amtso.org. On completion of the review period, all feedback will be reviewed by the working groups and actioned as necessary; depending on the complexity of any further adjustments required, the papers will either return for a further round of review or proceed to the final stages of member approval for official adoption as AMTSO papers.