Dear members,
Below you can find the minutes from today's call of the 'Sevilla Crew' sub-group (AI Tech & Risk).
Online paper the 'Sevilla Crew' is working on: https://docs.google.com/document/d/1y6TS0pCuybYDAsIwR7Vf8T2II6KQVzTpTfU13OIqr-8/edit#heading=h.jila3fjjswb1
Discussion on LLM-enabled Off-Sec Process
- Kirti presented a visual representation of the LLM-enabled Off-Sec process, emphasizing the balance between AI benefits and risks, and the differences from conventional processes.
Document Collaboration and Feedback Incorporation
- The document's length limit of 10 to 15 pages.
Scope of Document and Offensive Security
- The discussion highlights concerns about the scope of the document being limited to offensive security, excluding the aspect of fixing vulnerabilities.
- A decision is made to focus on defining guardrails for offensive security to set boundaries on what actions can be taken to prevent potential damage.
Hallucinations and Agentic Design Pattern
- Concerns raised about the potential for hallucinations in AI systems when lacking updated knowledge, leading to inaccurate responses.
- Discussion on the importance of a knowledge base for AI systems to prevent hallucinations and ensure accurate execution.
Validation of Generated Content
- Mj Schwenger highlighted the need for validation of generated content to ensure coherence and relevance from a human perspective.
AI Model Governance and Threat Detection
- Chopra expressed concerns about the potential risks associated with not protecting AI models properly, highlighting the need for governance and balance between benefits and risks.
- Chopra emphasized the importance of balancing benefits with practical considerations for the audience and encouraged further writing and visual aids to convey the message effectively.
- Adam emphasized the importance of continuous improvement in writing and content creation, acknowledging the need for restructuring after graphics are added.
Next call for the 'Sevilla Crew':
Wednesday, April 17th, at 09:00 a.m. PDT / 12:00 p.m. EDT / 16:00 GMT
url: https://cloudsecurityalliance.zoom.us/j/81167498763?pwd=suTY3T6xYOhpmhMx7QMUUPGzdN9h3O.1
Kind regards,
Marina
------------------------------
Marina Bregkou,
Senior Research Analyst,
CSA
------------------------------