Experts say combining deepfake audio and video, virtual avatars, and large language models like GPT could turbocharge the work of scammers and propagandists
Alex Pasternack / Fast Company : Thanks: @pasternack
Context & Ripple Effects
The deepfake story has come full circle since warnings that AI-generated audio and video could undermine elections. In between, vendors like Synthesia made synthetic video a legitimate business with multilingual, personalized training videos, while researchers pushed realism further — Microsoft's VASA-1 can now produce a realistic talking face from a single portrait photo plus an audio file.
Fast Company's report names the inflection point: each component was already deployed separately, but combining deepfake audio and video, virtual avatars, and LLMs like GPT turns them into a composable pipeline. Criminals are not waiting — the Financial Times documented how deepfakes and voice cloning already power financial scams, forcing banks and fintechs into defensive AI investment.
First-order effects
- Scam operations gain an end-to-end stack: LLM-written scripts, cloned voices, and photorealistic talking avatars remove the manual skill that previously limited who could run convincing impersonation fraud.
- Banks and fintechs already investing in AI to counter voice-clone fraud must broaden defenses from audio-only detection to full multimodal verification.
Second-order effects
- Detection vendors and platforms inherit an arms race they cannot fully win, shifting spending toward provenance, liveness checks, and out-of-band verification rather than content-level fakes.
- Legitimate synthetic-media businesses like Synthesia face a trust penalty: every high-profile abuse of the same techniques raises the cost of selling corporate deepfakes.
Third-order effects
- If composed pipelines become standard for both propaganda and fraud, authenticity shifts from inspecting content to verifying identity through separate channels — making verification infrastructure a default layer of digital commerce.
- The dual-use pattern hardens: the exact toolset marketed for training and communication is the toolset of the propagandist, pushing regulators toward rules on synthetic media that apply to vendors regardless of intent.
The trend: Synthetic media is consolidating from isolated novelties into a composable fraud-and-propaganda toolkit, with detection and verification racing to keep up.