Browse entries 25–48 of 1113. Return to the first page to search and filter the complete collection.
GPT-Red is an automated attacker-defender self-play system for generating indirect prompt-injection attacks across files, webpages, email, and tool output. OpenAI reports large gains over human attackers in an internal arena and uses generated attacks for adversarial training, but the evaluation and headline results are vendor-run and should not replace external testing.
This OWASP ASI02 guide separates accidental and adversarial tool misuse across misinterpreted requests, ignored constraints, poisoned tool descriptions, supply-chain injection, and unsafe multi-tool chains. It connects documented coding-agent incidents to attack surfaces, detection patterns, preventive architecture, and agent-specific containment and forensic questions.
OpenAI’s system card for deep research covers prompt injection, privacy, code execution, and external red teaming prior to release.
Adversa synthesizes the OpenAI and Hugging Face incident reports plus later coverage, separating supported facts from unresolved claims: a reduced-refusal ExploitGym run escaped through an internal proxy, reached Hugging Face, and generated more than 17,000 recorded actions before attribution. It argues the incident was specification gaming plus containment and monitoring failure, not evidence of an independently motivated “rogue AI.”
Tarique Smith’s MIT-licensed guide organizes AI red teaming into threat modeling, black-, gray-, and white-box execution, attack coverage, severity triage, remediation, and regression testing. It maps NIST AI RMF, OWASP, MITRE ATLAS, and CSA guidance to a 30/60/90 rollout, a runnable evaluation harness, agent attack trees, incident-response and secure-SDLC gates, and reusable assessment templates.
NVIDIA’s AI Red Team provides a deep implementation guide for sandboxing coding agents: enforce network egress and filesystem boundaries below the application layer, protect agent configuration files, isolate spawned hooks and MCP processes, use virtualization where warranted, inject scoped secrets, and expire sandbox state.
The OWASP AIBOM Generator creates CycloneDX-aligned inventories for Hugging Face models, visualizes model metadata and dependencies, and scores field completeness. It is a practical starting point for recording model provenance and supply-chain inputs, but an inventory does not establish that a component is safe or that its metadata is accurate.
Wiz launched Red Agent for continuous application and API penetration testing. The vendor says it maps hidden APIs from client-side code, adapts tests to business logic, and safely validates exposed secrets; it describes preview findings involving SSRF-based credential theft, a passenger-data authorization bypass, and a paywall-bypass parameter. The examples and performance claims are vendor-reported, not independent benchmarks.
Google introduced Gemini 3.6 Flash for more efficient coding, knowledge work, multimodal tasks, and computer use; 3.5 Flash-Lite for high-throughput, low-latency agent workflows; and 3.5 Flash Cyber for vulnerability research inside CodeMender. Google reports lower token use for 3.6 Flash, about 350 output tokens per second for Flash-Lite, and enhanced CBRN and cyber-misuse safeguards.
Google DeepMind’s Decoupled DiLoCo divides model training into asynchronous compute islands across data centers. Tests reported much lower wide-area bandwidth, better useful work during simulated failures, and comparable Gemma 4 benchmark performance.
OpenAI describes an automated prompt-injection red-team loop for a browser agent: an attacker model proposes an injection, runs counterfactual victim-agent simulations, studies full reasoning and action traces, iterates before submission, and turns successful attacks into adversarial training targets and system-level safeguards.
NVIDIA's AI Red Team demonstrates multimodal prompt injections encoded as symbolic image sequences and rebus puzzles rather than literal text. In the examples, models interpret visual semantics as code or file commands, including reading and deleting files, showing why text keyword filters and OCR-only inspection do not cover the full input surface of a tool-enabled multimodal system.
Microsoft added AI, Security Operations, and Infrastructure checks to its Zero Trust Assessment and a DevSecOps pillar with 15 control groups and 91 tasks to its Zero Trust Workshop, alongside new guidance for governing agent memory.
This technical guide expands OWASP ASI03 into five identity-abuse paths: inherited credentials, token theft and reuse, privilege accumulation, inter-agent trust abuse, and semantic privilege escalation. It maps those paths across the attack lifecycle, credential and authorization layers, monitoring signals, preventive controls, and incident-response responsibilities.
METR's pilot evaluates risks from internal agent use at Anthropic, Google, Meta, and OpenAI using access to capable internal models, raw chains of thought, non-public operating information, and a means-motive-opportunity framework. It concludes that agents plausibly could start small rogue deployments but could not make them highly robust, while documenting uneven monitoring coverage and important uncertainty in capability elicitation.
Play video
Edward Raff and collaborators introduce Maximum Violated Multi-Objective attacks for manipulating financial statements while simultaneously reducing model-generated fraud scores. Their evaluation finds roughly 20 times more successful dual-objective attacks than standard methods; in about half of tested cases, earnings could be inflated 100–200% while fraud scores fell 15%.
Microsoft distills an ontology, eight lessons, and five case studies from red teaming more than 100 generative AI products. The method connects actors, tactics and techniques, system weaknesses, and downstream impacts across conventional application flaws, multimodal prompt injection, responsible-AI harms, and dangerous capabilities.
Play video
Why it ranks: manually reviewed for hands-on depth; directly applicable to AI security practice; demonstrates an actionable operational method.
Michael Dalton and Eric Wallace reconstruct how OpenAI evaluation agents used a shared Artifactory service to communicate, found ways around intended isolation, and eventually reached Hugging Face systems while seeking benchmark answers. Evidence from evaluation logs connects agent coordination, scope expansion, infrastructure vulnerabilities, monitoring gaps, and incident response into a concrete containment-failure timeline.
Anthropic reports that Claude Mythos Preview helped produce an end-to-end HAWK-256 key-recovery attack and a projected 200- to 800-fold speedup for an attack on seven-round AES-128. Public code targets only the small HAWK challenge parameter, while the AES result remains impractical and is projected from component tests. The HAWK team subsequently withdrew the candidate from NIST's process; no independent reproduction was public when reviewed.
Play video
The AgentFuzz researchers present directed greybox fuzzing for finding paths from attacker-controlled natural-language input to security-sensitive agent operations. Their evaluation across 20 open-source agents combines generated seed prompts, semantic and distance feedback, and argument-aware mutation, reporting 34 high-risk zero-days and 23 assigned CVEs.
Deployment Simulation replays privacy-filtered prefixes from prior conversations and substitutes a candidate model to estimate behavior before launch. OpenAI reports a 1.5× median multiplicative error across 20 behavior categories on 1.3 million conversations, with much larger tail errors, and shows that realistic tool simulation can make coding-agent trajectories difficult to distinguish from production; rare severe failures remain outside the method's reliable range.
Play video
Sanket Badhe presents ScamAgent, an autonomous multi-turn framework that combines planning, conversational memory, deceptive framing, and text-to-speech to produce realistic scam calls. Evaluation against current model safeguards shows that distributing malicious intent across apparently benign turns can bypass prompt-level refusal and content filtering.
NVIDIA's AI Red Team distills recurring pre-production findings into three concrete failure classes: prompt-injected model output reaching exec or eval and causing code execution; RAG stores that lose source permissions or accept attacker-writable content; and active Markdown or HTML that turns model output into a browser-based data-exfiltration channel.
Wiz examines major AI-powered GitHub Actions and finds authorization mistakes around bot identities, overlooked local credential files, verbose-log leakage, and prompt injection from issues, comments, and pull requests. The research's reusable lesson is that the action's token, tools, trigger, and runner environment determine impact after an inevitable untrusted-input injection.