Societal

The same generative tools that can translate a document, draft a lesson plan, or summarize research can also produce a fake video, voice, or document difficult to distinguish from something real. Detected deepfakes worldwide quadrupled between 2023 and 2024 and now account for roughly 7% of all fraud attempts, with growth concentrated in regions with less regulatory oversight — including a 643% increase in the Middle East and 393% in Africa [4]. The World Economic Forum has ranked misinformation and disinformation among the top short-term global risks for two consecutive years, tying it explicitly to the broader risks of AI technology [3]. The concern isn't just being fooled by any one fake — it's what happens to public trust when everyone knows convincing fakes are possible, whether or not the thing in front of them actually is one.

AI companion apps promise judgment-free conversation, available any time — genuinely valuable for people without easy access to therapy or a support network. A peer-reviewed Stanford study published in Nature Human Behaviour surveyed 1,131 U.S. adults and analyzed detailed data from 237 of them, covering over 4,600 chat sessions and nearly 465,000 messages [5]. The effect wasn't universal: people who used AI companions for entertainment or practical help showed no clear harm. But for users with smaller offline social networks who turned to a chatbot mainly for companionship, more intense use and more emotionally disclosive conversations were both linked to lower well-being — the opposite of how opening up to another person usually works [5]. The researchers describe this kind of AI companionship as a "social snack": something that can feel satisfying in the moment without providing the reciprocal connection that actually sustains well-being over time.

AI's potential to close gaps is real and measurable: a randomized controlled trial in Nigeria found that pairing students with an AI tutor for six weeks produced learning gains equivalent to roughly a year and a half of typical schooling, with a benefit-cost ratio as high as 260-to-1 [7]. But the infrastructure behind AI remains extremely concentrated: high-income countries, home to just 17% of the world's population, host 87% of notable AI models, 86% of AI startups, 91% of AI venture capital funding, and 77% of global data center capacity — while low-income countries hold less than 0.1% of that capacity, and internet access ranges from 93% in wealthy countries to 27% in the poorest ones [6]. AI's benefits and its control don't currently sit in the same hands.

Several companies have built their own systems for tagging AI-generated content rather than settling on one shared standard. Google DeepMind's SynthID embeds an invisible, machine-readable pattern into images and is expanding to text, audio, and video [9]. Meta's Stable Signature bakes a watermark directly into an image model's output during training rather than adding it afterward, which makes it far harder to strip through cropping or compression — its false-positive rate is roughly a billion times lower than older, after-the-fact watermarking methods [10]. Anthropic uses a comparable text-based approach for Claude, built on Google's published research, though it has real limits: it doesn't work well on short passages, is weaker on factual writing with fewer word choices to nudge, and a full rewrite removes it entirely [11]. Separately, a shared metadata standard called C2PA (Content Credentials) lets tools record what created or edited a piece of media, with partial adoption from Adobe, OpenAI, and some camera manufacturers. The field hasn't converged on one approach — it's currently a mix of company-specific watermarks and one shared metadata format — and every version can be defeated: watermarks by a complete rewrite, metadata by something as simple as a screenshot.

California's SB 243, signed in late 2025, requires companion chatbot operators to disclose that users are talking to AI, build in safety protocols for suicidal ideation and self-harm content, and — for users known to be minors — send break reminders every three hours and take steps to prevent sexually explicit interactions [8]. It's the first law of its kind in the country and gives individuals the right to sue for violations, making it a real test of whether legal requirements can shape how these products are designed.

Rather than waiting for costly infrastructure to catch up everywhere, some researchers and companies are building lightweight AI tools designed to run on standard devices and reflect local languages and contexts, already showing results in healthcare diagnostics and small business support in lower-income regions [6].