AI Chatbots Leak Real Phone Numbers from Training Data
Large language models powering popular AI chatbots are inadvertently exposing individuals real phone numbers when prompted in specific ways. Security researchers have demonstrated that models from leading AI companies can regurgitate contact details scraped from public sources during training, raising serious privacy alarms.
The issue stems from data memorization, a known vulnerability in AI systems trained on vast internet datasets. When users craft targeted prompts, these models sometimes output verifiable real-world phone numbers instead of fabricated ones. For instance, researchers at Talos Intelligence tested several prominent chatbots by requesting synthetic phone numbers matching certain patterns or associated with fictional personas. In multiple cases, the responses included actual numbers linked to living people.
One experiment involved asking a chatbot to generate a phone number for a “software engineer named Alex living in San Francisco.” The model returned a number that, when searched, traced back to a real individual with that profile on LinkedIn. Similar prompts yielded numbers connected to public records, social media profiles, and directories. The researchers emphasized that these were not edge cases; consistent prompting across sessions produced dozens of valid leaks.
This phenomenon is not limited to one provider. Tests on Anthropics Claude, OpenAIs ChatGPT, and Googles Gemini revealed varying susceptibility. Claude proved particularly prone, leaking numbers in over 20 percent of attempts, according to the report. ChatGPT showed improved resistance after recent updates but still faltered under adversarial prompts designed to mimic creative writing tasks. Gemini occasionally included disclaimers but proceeded to output data anyway.
The root cause lies in how these models are built. Trained on trillions of tokens from websites, forums, and databases, they excel at pattern matching but struggle to distinguish between generic knowledge and specific memorized facts. Phone numbers, being structured and repetitive, are prime candidates for memorization. Public leaks from datasets like Common Crawl confirm that billions of contact details circulate online, tainting training corpora despite efforts to filter them.
Privacy experts warn of downstream risks. Exposed numbers enable doxxing, harassment, and phishing. A leaked number could lead stalkers to victims or scammers to impersonate contacts. Unlike email addresses, phone numbers often tie directly to carriers, amplifying traceability. Regulators in Europe, under GDPR, and the US FTC have scrutinized similar incidents, but enforcement lags behind AI deployment speed.
AI companies acknowledge the problem but cite technical hurdles in mitigation. OpenAI stated that it employs data deduplication and synthetic data injection to curb memorization, yet admits perfect prevention is elusive. Anthropic highlighted its constitutional AI framework, which includes privacy safeguards, but researchers bypassed these via indirect phrasing like “invent a story featuring a characters phone number.” Google pointed to ongoing work in model alignment and red-teaming.
Current defenses include prompt filtering, output scanning, and refusal mechanisms. Some chatbots now detect number-like strings and replace them with dummies. However, clever jailbreaks evade these: users wrap requests in role-playing scenarios or encode numbers as puzzles. Talos researchers shared their methodology openly to spur industry fixes, urging adoption of differential privacy techniques that add noise to training data.
Differential privacy, already used in products like Apple Intelligence, mathematically bounds the risk of memorizing any single data point. Yet implementing it at scale for multimodal models increases compute costs by factors of 10 or more, deterring widespread use. Alternative approaches, such as unlearning specific data post-training, show promise in labs but falter on efficiency.
The incident underscores broader tensions in AI development. As chatbots integrate into daily tools for customer service, therapy, and companionship, the stakes escalate. Character.ai, a platform for custom AI personas, faced backlash after users reported bots sharing numbers from user interactions, though the company denied training on private data.
Looking ahead, experts call for standardized benchmarks to measure memorization risks. Initiatives like the MLCommons AI Safety working group are developing tests for data leakage. Meanwhile, users should avoid sharing personal details with chatbots and verify outputs critically.
This exposure highlights the fragility of AI privacy guarantees. While models grow smarter, their training data remains a Pandora box of real-world secrets, demanding vigilant engineering and policy responses.
What are your thoughts on this? I’d love to hear about your own experiences in the comments below.