The Less You Know the Better

When ChatGPT launched in late 2022, the eulogies for "traditional" search immediately followed. The consensus was that Google's results were filled with ads and low-quality content optimized for search engine discovery rather than human consumption. New chat interfaces powered by large language models (LLMs) offered a clean, bullshit-free experience that would just give us "the answer."

What these arguments acknowledge about our past, but conveniently ignore about our present, is that the mechanisms that hollowed out Google search haven't gone away. The degradation of search engine results pages (SERPs) wasn't accidental, it was a requirement of capital. Google pursued infinite revenue growth through the expansion of its advertising business, and a multi-billion dollar industry emerged to reverse engineer and exploit the algorithm controlling the vast majority of web traffic. The real problem was tying search to a business model that requires the number to always go up.

Should frontier model providers like OpenAI and Anthropic go public, they will be beholden to the exact same economic imperatives. Both companies are currently playing on easy mode, enjoying an unprecedented period of hype-fueled cash incineration. Despite this, some degree of enshittification has already started. OpenAI introduced ads to the free tier of ChatGPT despite Sam Altman declaring them a "last resort" less than two years prior. Additionally, LLMs are further hollowing out the web's near-zombified corpse, driving both a proliferation of generated content and the rise of content mills that exist solely to be harvested as references for language models.

Framing LLM chat interfaces as a breakthrough in user experience (UX) for information access is superficial at best. While sifting through a list of blue links is arduous, this friction is a necessary part of inquiry. True information synthesis, the ability to weigh evidence and interrogate difficult questions, requires the very effort that frictionless chat interfaces powered by generative AI eliminate. Although going to a library, finding a book, checking it out, and reading it takes considerable effort, we might learn something in the process. The hard part was always reading the book and interrogating the information within it.

The AI industry frames this shift as a revolution where the previously scarce resource of intelligence is now available to everyone. In reality, it's more an optimization of convenience that heavily obfuscates what we give up in return. This piece explores the downsides of the latest bargain we've made with big tech, and the role LLMs play in our collective intellectual disarmament.

Visibility > veracity

The post-truth era didn't begin with language models. Social media gave rise to the influencer economy that prioritizes visibility over veracity. Beyond making attention a primary success metric regardless of the message, we've become increasingly accustomed to unseen motivations and undeclared biases. We've moved beyond advertising cleanly embedded in YouTube content, or even sponsored ad reads. At times, it's unclear whether or not the entire premise and setting for a piece of content is bought and paid for. An obvious example of this is travel content where a creator declares their visit is sponsored by the tourism bureau of the location they are in. Should viewers consider the entire video an ad? Arguably, yes. But when the style and format mirrors identical content that isn't sponsored, or is less sponsored, things get really blurry.

Paid content is often far less transparent and direct. The prevalence of clipping, a practice where large groups of individuals are paid to syndicate clips of content to manipulate social media algorithms, is behind much of what occupies the modern zeitgeist. From looksmaxing guru Clavicular to the controversy surrounding Bad Bunny at the 2026 Superbowl, someone, somewhere is paying someone to capture our attention. Reporters have framed specific instances of this as a "psyop," but it would be more accurate to frame social media feeds themselves as psyops. Content from friends and family is intertwined with sponsored content, content from strangers, and algorithmic manipulations. It has never been more difficult to distinguish "the thing itself" from "the promotion of something."

While this piece is not an indictment of advertising or social media algorithms, their decay is essential context for understanding the threat generative AI poses to our already corroded capacity for truth.

Language shapes perception

The language used to market LLM products offers helpful framing before we jump into the consequences they pose to our social fabric. By analyzing this terminology, we can uncover the way the technical reality of generative AI is obscured by an illusion of personhood and authority:

• Artificial Intelligence (AI): Created as a way to brand, pitch, and separate emerging techniques from cybernetics in the 1950s, the term is the foundation of the current hype cycle. While "AI" is often used today to imply burgeoning digital consciousness, it was originally a way to differentiate ordinary computing tasks and give them a grand identity. It is an umbrella term for statistical pattern matching, encompassing machine learning (ML) and natural language processing (NLP), that has almost nothing to do with any established understanding of intelligence.
• Thinking: Frontier model providers use the term "thinking" to anthropomorphize the latency between a user submitting a prompt and the intensive statistical computations that contribute to its output. Language models do not think and they never will.
• Sources: In an attempt to manufacture credibility, modern LLM products have begun incorporating "sources" into their outputs. Because these models are probabilistic rather than deterministic, these citations are often fake (e.g., non-existant URLs, irrelevant snippets, or complete fabrications) yet are still presented with the same unearned confidence as a verified fact.
• Deep Research: This term serves as a premium-tier euphemism used to justify higher subscription costs and longer processing wait times. While "deep research" might imply a more rigorous interrogation of data, in the context of an LLM, it is often just a more expensive way to run the same recursive loops.

We could continue interrogating the lexicon ("AI agents" have been the term du jour for the past year or so) but we would be beating a dead horse. All of this terminology serves the same purpose: to obfuscate the mechanics of the machine and imbue its capabilities with more trust than they deserve.

Kill the witness then lie

The death of the witness is a concept centered around truth suppression. If the person who can validate, record, or testify that an event occurred can be silenced or killed, history becomes highly vulnerable to erasure. Justice, accuracy, and consequences become virtually impossible to enforce or discern. Generative AI has made it possible to create images of Will Smith eating spaghetti he never ate with increasing realism. While this example is lighthearted and, on its own, inconsequential, the technology that powers it is edging us closer to a universal death of the witness. Will we reach a point where we can no longer point to something and say, "this is what happened?"

If we take this logic a step further and imagine a world in the not too distant future where everything can be attributed to a hallucination or a deepfake, then anything real can be dismissed as being generated with AI. The liar's dividend is the idea that bad actors no longer have to disprove the lies they tell, they only have to cast doubt on the truth. To make this more tangible, and dispense the idea that it's conspiratorial, consider that the sitting president of the United States is already doing it.

The death of the witness profiles the systemic erosion of truth, while the liars dividend makes it possible to discredit authentic evidence due to an infinite possibility that anything can be fabricated. Algorithmic content feeds began the process of creating individualized filter bubbles, both isolating and magnifying the concerns and information that occupy our attention. Now LLMs are carrying that baton forward. Democracy requires some shared reality to persevere, but if anything could be fake then nothing or anything can be true. Truth then becomes a matter of belief rather than something that must be substantiated by robust evidence. This helps explain why the 6,000+ word manifestos from industry insiders like Mark Zuckerberg and Bill Gates make bold predictions about the future without citing a single source.

Remember Sora?

There have been notable examples where companies released products that would have further accelerated this process. The first was OpenAI’s now shuttered Sora video generation app, which gave users the ability to use simple text prompts to produce realistic video clips. While concerns regarding the release of public tools capable of mass producing synthetic content were present from its inception, the app wasn't shut down over measured caution on issues of consensuality, misinformation, or copyright infringement. It was because Sora was "a resource black hole" with "limited monetization" potential for OpenAI.

The next example was even more short-lived. Meta integrated its Muse Image model directly with Instagram, allowing users to @ public accounts and generate content with their likeness. Fortunately, the company faced swift backlash from its users and removed the functionality after just three days. I don't mean to argue that the corporations behind these products intend to further corrupt our shared sense of truth and reality. But I do think this is the foreseeable consequence, and if either company cared they wouldn't have released these products or features in the first place. Much like Meta didn't intend undermine the mental health of an entire generation of youth, that is the consequence of pursuing growth at all costs and optimizing its products for the maximum capture of attention.

The false oracle that lives in a black box

The big lie that much of this serves is the idea that LLMs democratize and commoditize intelligence. One of the best illustrations of this is Sam Altman's "The Gentle Singularity." In it, Altman frames the future around abundance rather than extraction, claiming that intelligence will become a highly available, low-cost utility.

His argument rests solely on proclamation. It includes no data, no citations, and no clear definitions of what intelligence even means. The thesis exists to manufacture the fear of missing out (FOMO) among investors eager to ride the "inevitable" wave of progress toward greater riches, not to provide an accurate map of technological trajectory. Even if we were to use the faulty method of perceived credentials to lend weight to Altman's arguments, he lacks the scientific or technical standing to justify such a grand vision. He is, fundamentally, a venture capitalist and dealmaker, not a scientist, researcher or technologist.

What we are being asked to do as a society is hand over the cognitive tools of research, reason, and decision-making to black box algorithms. These probabilistic models are not intelligent, they are mathematical machines predicting the next most likely token based on historical human data. While these models frequently surface correct information, our growing dependence on them erodes our ability to question the integrity of their outputs. The argument that LLMs are a path to superintelligence is not only technically flawed but dangerous. It elevates a statistical mechanism with no concern for truth to the position of an oracle.

The Dunning-Kruger trap

The Dunning–Kruger effect is a cognitive bias that describes the systematic tendency of people with low ability in a specific area to give overly positive assessments of their ability. The danger LLMs pose in the workplace acts like a cascading hall of mirrors, where everyone thinks the technology is proficient at areas of expertise they have only a surface-level understanding of.

At the top of the hierarchy, executives and middle-managers have long since been removed from the granular, hands-on mechanics of their subordinates' work. When they use generative AI products to produce reports or strategy, the outputs appear polished, professional, and "correct" to their non-expert eyes. This creates a trap of false confidence where they begin to believe that the tool is capable of auditing and automating every function within the chain of command. A cascade of perceived capability emerges that hollows out actual skill. The engineer believes they can automate the intuition of the marketer, the marketer believes they can replace the agency of customer support, and the manager believes the entire organization can be reduced to a series of daisy-chained AI agents. Whether anyone actually believes any of this, or just feels obligated to project that they do, remains subject to debate.

Sam Altman famously predicted that a “one-person billion-dollar company, which would’ve been unimaginable without AI, and now it will happen” would be driven by this very phenomenon. Not long after, the New York Times reported on a company that appeared to have realized this vision. Holes were swiftly poked in a story that turned out to be more hype. The company, Medvi, had more than one employee, faced allegations of both fraudulent AI-generated marketing and FDA violations, and also hid an army of healthcare workers behind an AI interface.

Who owns the box?

The reporting of unsubstantiated proclamations from executives and other wealthy members of society is taken as newsworthy in and of itself. There is little pushback from media outlets but instead a presumption that people in positions of power know something the rest of us don't. What's more concerning is that basic tenets of journalistic integrity are ignored in favor of creating a click-worthy headline.

Predictions from people like Elon Musk, Dario Amodei, Sam Altman, Bill Gates, and others are taken as significant despite little or no corroborating evidence to back up their claims (not to mention how often they fail to materialize). Seldom does the media mention the financial incentives behind these narratives. Instead, their compromised position is used to legitimize their commentary.

These narratives give capital more leverage over labor by using fear to put the average worker in a state of perpetual panic. "AI is coming for your job" or "someone using AI is coming for your job" has been the mantra for the better part of the past four years. People become afraid to question whether the technology is suitable for a given use case, or even point out its limitations, as it can ostracize them from colleagues and more specifically the layers of management above them.

When truth is subjective and reason is outsourced to black boxes, power reverts to those who control the narrative. The less you know about how the technology works, the better you think it is at replacing yourself and others around you. The less you know about the world around you, the easier it is for powerful individuals and corporations to manufacture consensus that serves their interests.

The less you know, the better.