M29.9 CONNECT THE MECHANISM
Misuse in practice: jailbreaks, deepfakes, and provenance
A finance worker joined a video call with his CFO and colleagues, then sent $25 million. None of them were real. See how AI misuse works, and how to check what you see and hear.
LESSON OVERVIEW14 min lesson
Lesson overview
A finance worker joined a video call with his CFO and colleagues, then sent $25 million. None of them were real. See how AI misuse works, and how to check what you see and hear.
What you’ll explore
- Misuse of generative AI works through jailbreaks, impersonation, and synthetic media. Detectors, content credentials, and watermarks help but can be evaded or stripped, so disclosure rules and verification through independent channels carry much of the protection.
GO TO THE SOURCE
Original explanations, connected to the research.
Jailbroken: How Does LLM Safety Training Fail? (Wei, Haghtalab & Steinhardt, 2023)Many-shot jailbreaking (Anil et al., Anthropic, 2024)Low-Resource Languages Jailbreak GPT-4 (Yong, Menghini & Bach, 2023)Universal and Transferable Adversarial Attacks on Aligned Language Models (Zou et al., 2023)Arup lost $25mn in Hong Kong deepfake video conference scam (CNN, May 2024)Scammers use AI to enhance their family emergency schemes (US Federal Trade Commission, March 2023)FCC makes AI-generated voices in robocalls illegal (February 2024)Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers (VALL-E; Wang et al., 2023)The DeepFake Detection Challenge (DFDC) Dataset (Dolhansky et al., 2020)New AI classifier for indicating AI-written text (OpenAI, 2023; withdrawn July 2023)Coalition for Content Provenance and Authenticity (C2PA)A Watermark for Large Language Models (Kirchenbauer et al., 2023)Scalable watermarking for identifying large language model outputs (Dathathri et al., Nature, 2024)SynthID (Google DeepMind)TAKE IT DOWN Act (S.146, 119th Congress, 2025)Suggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.