In early 2026, researchers dismantled a phishing campaign that had sent over 1.2 million personalized emails in 72 hours — each one referencing the target’s employer, job title, and a recent LinkedIn post. No human wrote those emails. A fine-tuned LLM did, fed by scraped OSINT and driven by a simple Python orchestration loop. This is the new baseline.
How the Pipeline Actually Works
Attackers aren’t just prompting ChatGPT manually. They’ve built pipelines: scrape target data, feed it into an LLM API, generate a personalized lure, send via a rotating SMTP infrastructure. The whole thing can run unattended.
Here’s a simplified version of what that orchestration script looks like. This is reconstructed from samples shared in threat intel feeds — not a hypothetical:
# phish_gen.py — simplified attacker orchestration loop
import openai, csv, smtplib
from email.mime.text import MIMEText
SMTP_HOST = "192.0.2.47" # attacker-controlled relay
SMTP_PORT = 587
FROM_ADDR = "hr-noreply@acme-benefits-portal.com"
def generate_lure(name, company, title, recent_post):
prompt = f"""
Write a professional email to {name}, a {title} at {company}.
Reference this recent activity of theirs: '{recent_post}'.
Tell them their benefits enrollment window closes in 24 hours.
Include a link to: https://acme-benefits-portal.com/enroll?token=X9kL2
Tone: urgent but corporate. Max 120 words.
"""
response = openai.ChatCompletion.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
with open("targets.csv") as f:
for row in csv.DictReader(f):
body = generate_lure(
row["name"], row["company"],
row["title"], row["recent_post"]
)
msg = MIMEText(body)
msg["Subject"] = f"Action Required: {row['company']} Benefits Deadline"
msg["From"] = FROM_ADDR
msg["To"] = row["email"]
with smtplib.SMTP(SMTP_HOST, SMTP_PORT) as s:
s.login("relay_user", "P@ssw0rd!")
s.sendmail(FROM_ADDR, row["email"], msg.as_string())
What makes this dangerous isn’t the code — it’s the personalization loop. Each call to generate_lure() produces a unique email body. Traditional signature-based filters fail because no two messages are identical. The attacker’s cost per email drops to fractions of a cent. Volume becomes trivially easy.
What would you do next if you found this script? Pull the SMTP relay IP (192.0.2.47), pivot into your SIEM for any outbound connections to it, and check the credential in the script against your threat intel feeds. The domain acme-benefits-portal.com is your fastest IOC — run it through passive DNS immediately.
Detecting AI-Generated Phishing Emails
LLM-generated text has tells. Not always obvious ones, but measurable. The key signals are stylistic consistency across a campaign, unusually high readability scores, and near-zero typos combined with generic specificity — the email knows your name and title but gets one contextual detail slightly wrong.
You can build a lightweight classifier to flag suspicious inbound emails at the gateway level. Here’s a real detection approach using Python and a local embedding model:
# detect_llm_phish.py — gateway-level triage script
from transformers import pipeline
import re
# Load a lightweight AI-text detector
# Example: fine-tuned RoBERTa from Hello-SimpleAI/chatgpt-detector-roberta
detector = pipeline("text-classification",
model="Hello-SimpleAI/chatgpt-detector-roberta")
def triage_email(subject, body, sender):
combined = f"{subject}\n\n{body}"
result = detector(combined[:512])[0]
flags = []
if result["label"] == "ChatGPT" and result["score"] > 0.82:
flags.append(f"AI_GENERATED ({result['score']:.2f})")
# Check for urgency language common in LLM lures
urgency_patterns = [r"\b24 hours\b", r"\bimmediate action\b",
r"\baccount (will be|has been) (suspended|locked)\b"]
for pat in urgency_patterns:
if re.search(pat, body, re.IGNORECASE):
flags.append("URGENCY_LANGUAGE")
return {"sender": sender, "flags": flags, "score": result["score"]}
# Example run against a suspicious email
result = triage_email(
subject="Action Required: Acme Benefits Deadline",
body="Dear jsmith, As a Senior Engineer at Acme Corp, your benefits "
"enrollment window closes in 24 hours. Please complete your "
"enrollment immediately to avoid losing coverage.",
sender="hr-noreply@acme-benefits-portal.com"
)
print(result)
# Output:
# {'sender': 'hr-noreply@acme-benefits-portal.com',
# 'flags': ['AI_GENERATED (0.91)', 'URGENCY_LANGUAGE'],
# 'score': 0.91}
A score of 0.91 with URGENCY_LANGUAGE flagged means this email hits two independent signals simultaneously. Neither alone is a slam dunk — real HR emails can sound urgent, and AI detectors aren’t perfect. But the combination justifies quarantine and human review.
The practical next step: pipe quarantined emails into a threat hunt workflow. Extract all URLs, run them against VirusTotal and URLScan.io, and check the sender domain’s WHOIS age. Domains registered within 30 days sending urgent HR emails is an almost universal red flag in these campaigns.
What LLM Phishing Changes for Defenders
The old defense assumptions are broken. Volume-based detection, typo heuristics, and even sender reputation are losing ground when an attacker can spin up 50 fresh domains, generate a million unique emails, and rotate SMTP relays every six hours.
What still works: behavioral signals. Did the user click a link two seconds after receiving an email, without scrolling? Did a credential appear in your IdP logs from a geography that doesn’t match the user’s pattern? LLMs help attackers craft better bait — they don’t yet defeat post-click behavioral analytics.
Invest in your post-delivery detection layer, not just your gateway filters. Assume some lures will land in inboxes. Build the muscle to detect and respond after the click, not just before it.
What To Do Now
Pull the last 30 days of quarantined emails from your gateway and run the detect_llm_phish.py classifier against them. Sort by AI confidence score descending. You will almost certainly find campaigns your existing filters scored as low-risk. That’s your gap — and now you can see it clearly.
