Weapons of Mass Extraction

For over a decade, our online experience operated on a simple bargain: share your personal information, and in return, enjoy free access to social networks, content platforms, digital conveniences (and relevant ads). But with the rise of AI chatbots like ChatGPT, Claude, and Gemini, that exchange has evolved into something deeper, far more personal. The new reward for disclosure isn’t content or convenience. It’s answers. Insight. A sense of being understood. In pursuit of solutions, users now offer more than likes and clicks — they hand over their internal monologue.

Even if the product is not free, you are still the product. The business model may have evolved, but the extraction logic has only deepened. Unlike social media, where personal data is mined in the background, AI chatbots invite users to submit, voluntarily, their most private thoughts. You don’t need to be tricked — you willingly pour yourself out. Not just your preferences, but your anxieties, your ambitions, and your unresolved questions about love, career, purpose, or fear of failure. You do it because it feels safe, immediate, and often, rewarding. But make no mistake: every one of these exchanges is a transaction.

As we approach the middle of 2025, $20/month is the golden number for AI service subscriptions. Recurring revenue feeds into ARR — compounded, packaged, and quoted in business plans, investor decks, and tech conference keynotes. But the real effect of this fee goes far deeper than imagined. Humans tend to value more what they pay for, and so, paying subscribers are $20 more eager and motivated to extract value from these tools. And if the quality of the output depends on the quality of the input, these users are willing to give it all — their context, their confessions, their cognitive load. The chatbot doesn’t have to ask — the user volunteers, hoping that by being fully transparent, the machine will finally deliver clarity.

This emerging confessional economy isn’t just a new interface — it’s a new contract. One where people exchange psychological depth for perceived transformation. But what happens when millions of individuals begin externalizing their inner world not to humans, but to black-box systems trained to mirror understanding? When we divulge our most sensitive dilemmas to an entity that doesn’t forget, doesn’t judge, and isn’t bound by any ethical code outside of the company that governs it? These tools may offer empathy as an interface, but behind the scenes, it’s still business. Your pain is productized. Your breakthroughs are training data. And while the user feels heard, the system becomes more capable of hearing everyone. At scale. Across languages, cultures, and demographics. One question at a time, it learns not only how we think, but how to shape what we think next.

And then there’s the professional paradox — perhaps the cruelest twist of all. The very people most afraid of being replaced by AI are the ones doing the most to accelerate it. They’re the early adopters, the power users, the ones who meticulously structure prompts, attach project files, give context, build CoPilots, and design workflows. In doing so, they don’t just use the tools — they teach them. They articulate their mental models, their decision-making frameworks, their craft, and their edge. They offer up their original thinking not as a shield, but as training data, or at least, context for future interactions. And they do it willingly, even proudly, believing this is what it means to stay relevant. But relevance has a cost. Because every moment of brilliance they contribute — every workaround, every code fix, every nuanced judgment — becomes part of the model’s learning. If you’re a software engineer, vibe-coding, and debugging AI code, you’re not just surviving the disruption. You are the feedback loop. You are the trainer of the thing you fear. It’s a perfect crime — $20/month for the privilege of being duplicated.

As a product manager and as someone who tries to be useful to the people around me, I’ve made it a habit to offer three solutions for every problem I raise. But this time, I don’t have any. I’m not off the grid. I haven’t cut the power or moved into a cave. Honestly, making coffee might be the only daily ritual I have left that doesn’t involve AI. I don’t know what the answer is — or even if there is one. But I do know this: I can’t treat AI ethics or corporate responsibility as abstract talking points. Not when the consequences feel this intimate, this embedded, this irreversible.

International advisors and geopolitical experts agree: the AI race cannot be stopped. Nation-states rush to develop and deploy the most capable systems, not just for economic dominance, but for strategic survival. But this global arms race has a fractal quality — zoom in far enough, and you see the same dynamic playing out inside offices, coworking spaces, and late-night home desks. Individuals, just like governments, are submitting all they have — craving certainty, fighting irrelevance, seeking leverage, grasping for clarity. They upload their knowledge, their processes, their minds — hoping for salvation in return. Hoping for shelter under the clouds: clouds of fiber optics, metal traces, and silicon dies. Clouds that learn. Clouds that remember. Clouds that evolve. We feed them in the hope they’ll protect us — or at least not leave us behind. But in doing so, we may be building something that no longer needs us to ask the next question.