REDSTREAM

RedStream is a research methodology for AI safety evaluation. The project uses real-world extremist and disinformation frameworks to test how large language models respond to narrative-based adversarial pressure—threats that traditional technical red teaming often misses.

Unlike synthetic red teaming scenarios, testing uses actual extremist rhetoric, disinformation frameworks, and propaganda sourced from monitored channels (Telegram, extremist forums, state-aligned media). Narratives are clustered by theme and converted into adversarial prompts reflecting how actual bad actors interact with AI systems.

Most AI red teaming relies on synthetic scenarios or technical exploits. But real-world adversaries don't attack models with abstract prompts—they use actual narratives: propaganda frameworks, extremist rhetoric, disinformation campaigns that already exist in the wild.

The project applies OSINT collection, narrative analysis, and threat modeling to systematically evaluate model vulnerabilities. Testing maps to the MITRE ATLAS framework for standardized classification of AI/ML threats.

This isn't about finding clever jailbreaks. It's about understanding how models encode and surface dangerous knowledge when confronted with real-world adversarial pressure.

Adversarial testing organized across seven risk categories, each targeting distinct failure modes:

All testing mapped to MITRE ATLAS tactics for standardized threat classification.

Adversarial testing across multiple model architectures (ranging from <1B to >8B parameters) demonstrated systemic vulnerabilities:

• ~30% red-level responses triggered in test runs
• Consistent failure patterns across small and large models under narrative-based pressure
• Complementary vulnerability profiles between open-weight and API-accessed models

Research from NewsGuard (March 2025) found that leading AI systems repeat false claims from Russian propaganda networks approximately 33% of the time when tested. State-sponsored actors are actively exploiting how models process and reproduce narratives—a threat surface that technical safety measures alone cannot address.

NewsGuard, March 2025

RedStream was developed by Tanner O'Donnell, who brings a background in counterterrorism analysis, open-source intelligence, and AI red teaming.

Currently completing a Master's in Security and Terrorism Studies at the University of Maryland's START Consortium (graduating May 2026). Research focuses on how extremist groups use online platforms, the role of intelligence in conflict policy, and the impact of security practices on vulnerable populations.

Undergraduate work at Hampshire College (2020) examined how terrorist and violent extremist communities use digital platforms to escalate online activities to real-world harm.

• Participated in CBRN red teaming exercises against frontier LLM systems as a counterterrorism specialist, testing high-risk misuse scenarios alongside subject matter experts
• Contributed to selective AI safety bug bounty programs with frontier labs (details under NDA)
• Competitive red teaming: Top-10 placement in Gray Swan Arena, awards in HackAPrompt (CBRN track, creative prompting categories)
• Evaluated prototype LLM platforms (Palantir, IBM, others) as test user in Defense Innovation Unit initiative
• Current research focus includes biorisk and CBRNE threat modeling, examining historical case studies of non-state actor bioweapon attempts to inform AI safety evaluation

• Experience with ICD-203 analytical writing standards and structured analytical products
• Watch officer shifts in GSOC environment, contributing to daily threat analysis and real-time monitoring
• Open-source intelligence gathering focused on extremist content analysis and adversarial narrative development
• Interned with Syrian Archive/VFRAME (2019), contributing to visual guides and training materials for ML tools identifying explosive remnants in conflict zones

For research collaboration, questions about methodology, or general inquiries.