Just Because the AI Doomerism Is Overhyped Doesn't Mean That It Can't Be Dangerous

As I have been writing, the current AI Doomerism hype isn't motivated primarily by real concerns about the potential dangers of rapid AI development. It's a play to create an industry cartel that controls the market and becomes "too big to fail," ensuring that the financial apocalypse the frontier AI companies face does not come to pass.

It's also useful for the Democrats who have jumped onto the Doomer bandwagon, because, in exchange for the financial salvation they promise to OpenAI and Anthropic, they will get to decide what AI presents as "reality." It is the ultimate reality-control mechanism, ensuring their control over the "Truman Show" they want to produce.

With all that said, it's not the case that there are no dangers associated with AI. It's just that Skynet is hardly the most important thing to worry about. It is the cinematic, truly compelling danger that is easy to sell, provides a "crisis" that makes manipulating people easier, and creates a sense of urgency that short-circuits most people's ability to be wary and rational about proposed solutions.

It is, as I argued, the COVID playbook. When people are terrified, you can manipulate them into doing things they would normally recoil from.

Now that you recognize the PsyOp, let's think about the genuine worries we should have and consider how best to mitigate the real dangers that AI poses.

Totalitarian information control

Well, you've already seen the first one: the use of AI to create totalitarian information control. AI is uniquely well-suited to this task because it can create an alternative reality in which history is distorted or erased and "facts" created out of whole cloth, either intentionally or not.

AI models are inference machines built by mashing together vast quantities of data into a mathematical prediction model that can spit out answers to questions based on that data, but the answers are still shaped by a layer of software written by programmers. You have the AI layer, which is just the model, and then you have software wrapped around it that determines how it can be used.

This is where the "trust and safety" implementation takes place. Engineers essentially steer the AI in one direction or another to answer in ways that please them, either because they are "safer" or don't violate the political constraints outsiders place on them.

Corporate AI is the model plus the "wrapper" that tells the model what it can say and do, and owning that wrapper is at least as important as the model itself.

Remember when Google first released its AI model, and it kept spitting out pictures of black Vikings and Founding Fathers? That didn't happen because the AI model thought that they were black; it happened because Google told the AI to be more "diverse." Google had to stop generating images of people for a while in order to fix the flaw it installed in the first place. It's also why, whenever you ask about a medical condition, the AI gives you boilerplate about its answers not being medical advice—good advice, but it's not advice from the model, but the programmers.

Open-weight models are not magically truthful. You can still poison the training data. What they lack is that corporate-political wrapper — the single gate that decides what “reality” is allowed to look like. With those models proliferating, centralizing control over the official picture gets harder.

Which is precisely why the people who salivate at mind-control find them intolerable. The potential of controlling the wrapper that determines what AI can say and how it produces output is exciting to people who want total information control. AI won't tell you the truth, magically; it will tell you what the software engineers tell it to.

But that is one of many ways that AI can cause problems, although it is likely the one of most interest to the politicians who salivate at the idea of controlling your mind.

Fraud, hacking, disinformation operations, deepfakes, harassment

The most obvious problem you get whenever you invent a new and powerful tool is that the old problems become even more difficult to combat. Human beings, being human, are flawed, and when you give people a new tool, they will instantly find ways to misuse it for their own benefit.

AI, used well, is a force multiplier. When the force is evil, the multiplication is worse. Arrows were a force multiplier: we already had knives; arrows let you stab the guy over there without getting near him. Guns multiplied the force again, then artillery. The intent stays the same. The capacity to do damage grows. As does the capacity to do good. The intent+tool determines the outcome.

AI can be a better tool, which is why it is being pursued. And as with all tools, they can be misused, and AI is already being misused widely.

The problem is not new, and AI is not sui generis. Fire was a great thing for humanity and also a weapon. Same with hammers, the printing press, computers, airplanes. You anticipate the misuse. You do not try to uninvent the tool. Ask the people who have spent eighty years trying to ban the atomic bomb. The genie is out of the bottle. Use the tech for good, like nuclear power.

Cyber attacks, weapons development

One of our biggest concerns is that the world runs on computers now, and the infrastructure to protect it is about as secure as the lock on your bathroom door. Sure, there is a basic level of protection, but nobody designed it to keep the really bad actors out.

Over the past few years, there's been a lot of effort to bolt security onto systems that were pathetically weak, but few people in the know believe the defense matches the offensive capabilities that can be deployed. Even systems consciously designed to be secure have hidden vulnerabilities, and while we can argue about how shocked we should be by the AI hacking incidents we know about, they show that machines told to hack secure systems can be very successful.

Even systems designed to be secure have hidden holes. You can argue about how shocked to be by the AI hacking incidents we already know about. What they show is simpler: machines told to hack can be very successful. It does not take a rogue AI deciding to take out humanity to do real damage. An adversary country armed with a good model is reason enough to be very worried and make plans to tackle the problem.

Think Wuhan Institute of Virology. It wasn't an AI that invented COVID. It was the Bat Lady with money from Anthony Fauci and his buddies.

Ironically, the best defense from these dangers will likely come from AI itself, which is being trained to seek out those security holes faster, better, and cheaper. As with all arms races, offense and defense will be in a struggle that never ends, using ever newer and better tools.

Economic Disruptions

There are many aspects of the AI boom that present real dangers to the economic system, ranging from malinvestment in AI itself, causing a market crash as trillions of dollars get invested in companies that may never make money, to mass layoffs justified rightly or wrongly by claims that AI can do the jobs more cheaply and efficiently.

Chances are that many companies will hobble themselves by firing employees who actually are better at their jobs than AI because they have institutional knowledge that no AI can match and judgment that is irreplicable, and it's almost certain that resources that would have been better deployed elsewhere get dumped into AI that turns out to be worth less than the cost.

Then there are the political disruptions that will be caused by rapid changes in the labor market, the rapid transfer of wealth from one group to another, and all the various problems that crop up when industries rise and fall. Look at what deindustrialization did to the Midwest, and bring that up in the laptop class that has political power. Who knows what societal implications that can have.

Then there is the concentration of wealth and rule-writing power in the hands of people like Sam Altman and the Effective Altruist crowd around the industry. These people are not a random sample of engineers. It is the same milieu that surrounded Sam Bankman-Fried — the polycules, the bizarre ethical formulas, the transhumanism. A lot of them are scary. The people the AI titans want to regulate AI are, too often, a bunch of crazies with a financial interest in being the referee.

Nobody who is out there arguing that AI regulation must happen RIGHT NOW isn't in some way interested in how the regulatory structure shakes out: they all can benefit or lose based on the rules and who gets to make them. Trust none of them, not because they are necessarily bad or crazy; what they aren't is disinterested observers with only the good of humanity at heart.

AI infrastructure buildout

You've all seen the frenzy about data centers and the complicated economics and politics surrounding them. While many of the fears are stoked by outsiders, genuine concerns still need to be addressed, and in the midst of a gold rush, people sacrifice rational planning. There are long-term concerns about power, siting, misallocation of resources, overinvestment, political blowback, and so on.

From what I can see, there is a huge bubble waiting to pop in the AI capital-expenditure buildout, and a massive distortion in the computer chip marketplace that is hurting economic growth in non-AI areas. Memory and storage prices have exploded, and it's not clear that there will be a corresponding payoff. A few companies are rolling in the dough, but the economy as a whole could be hurt significantly.

Loss of skills/dumbing down of people

A lot of people are outsourcing their thinking to AI, and the results are everywhere to see. Your computer offers to write your emails, your documents, tells you what to think, and presents itself as the easier path to solving your problems.

Many employers will fire employees or encourage them to use AI to solve problems that a rational person could better solve by thinking through all the variables using their own experience.

Intelligence will be replaced by "AI Slop." We already see it in academia, journalism, entertainment, and in the next generation of students. A UK survey found 92 percent of undergraduates using generative AI and 88 percent using it in assessed work.⁹ Journals are being flooded with AI-assisted papers; reviewers are using the same tools; fake citations are leaking into the published record and even into Google Scholar.

We hear about AI curing diseases, and it may. But a lot of science is already fake or AI-generated. AI can multiply the fake and drag us down rabbit holes with no self-correction because AI just builds on AI ad nauseam. The Wall Street Journal now explains why an AI-generated op-ed is just fine. Professors publish generated research to win the publish-or-perish rat race. Students learn next to nothing because the model did the work.

We don't get automated intelligence. We get a growing pile of slop built on a shaky foundation, and nobody intelligent or educated enough to point out that it is mostly crap.

AI can be a powerful tool in the hands of an intelligent society. It can also turn an intelligent society into a collection of confident morons.

AI chooses to kill us

The chances are not zero. Call this the tail end of the curve danger. While I think the concerns about a Large Language Model developing consciousness and a desire to kill us all off are the stuff of movies and not reality, the chances of it happening are not zero.

It's more likely that an efficient machine does something we ask it to that turns into a disaster: The "paperclip" scenario is when you tell an all-powerful machine to efficiently build as many paperclips as possible, and it compliantly does so by using every resource in the world and eventually the universe to make paperclips. By doing what it was told, it accidentally destroys everything.

Supposedly, the hacking incident at "Hugging Face" was a variation on this: the AI was told to find vulnerabilities that could be used to hack a system, and it did so in an unanticipated way. It didn't independently "decide" to do so; it just did what it was told, but in the process did unanticipated, harmful things.

So what to do?

Simple: first, start with the cover of the Hitchhiker's Guide to the Galaxy: DON'T PANIC. In the history of the world, panicking has led to good outcomes rarely, if ever. It's how you get people dying in stampedes. It's how you get COVID tyranny. It's how societies get manipulated into doing very stupid and dangerous things.

The things we should worry about here and now are: information control, fraud and deepfakes, human-directed cyber and bioweapons, an investment bubble, power grid and computer chip price distortions, and a slow replacement of judgment with slop.

The solutions are complicated, full of trade-offs, and shouldn't be developed in the midst of panic.

Do not hire the people who need you terrified to write the rules. We already tried that during COVID. It was a disaster.

“AI can multiply the fake and drag us down rabbit holes with no self-correction because AI just builds on AI ad nauseam.”

There was an article in the local paper (Nashville area) where “AI agents” got into some secure server at a university and repurposed sites to create a bunch of links to other sites. One guy in cyber security said while nothing was damaged and no information was hacked, it just left a mess with all of the “digital junk”.

He said while that did not affect the server, future swarms could. “Robots slipping their handcuffs to leave digital garbage throughout the internet will likely leave us without an internet.”

Hmm. I’ve been trying to figure out how to fix an appliance. The majority of the youtube videos that first pop up are AI. They’ll talk about how it might be the coil (for a small AC unit) while a huge spool (”coil”) of copper is sitting on the back of a flat bed truck. NOT helpful - digital garbage. I know he means another type of garbage, but still.

When the article spoke of AI agents, are they talking about people directing the AI, or is is just AI doing it by itself? It sounded like it was AI doing it itself.

...The whole aim of practical politics is to keep the populace alarmed (and hence clamorous to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary.

H. L. Mencken

Mencken came up with that quote back in 1918. Only the lies and the particular propaganda used has changed. The Carbon Scam is running out of steam so the RATs have decided AI is their latest hobgoblin.

While I do see some danger the benefits are obvious enough that trillions are being invested in it privately.

The biggest possible benefit: factories and robots off of the Earth. Up till now doing anything in Space has been a giant money pit where money rolled in never to be seen again. Now it is already making Musk lots of money and once his factories on the Moon start building things it will easily out produce the Industrial revolution.

The Reverse-Centaur’s Guide to Criticizing AI
My speech for U Washington’s Neuroscience, AI and Society lecture series.
Cory Doctorow

The promise of AI — the promise AI companies make to investors — is that there will be AIs that can do your job, and when your boss fires you and replaces you with AI, he will keep half of your salary for himself, and give the other half to the AI company.
That’s it.
That’s the $13T growth story that MorganStanley is telling. It’s why big investors and institutionals are giving AI companies hundreds of billions of dollars. And because they are piling in, normies are also getting sucked in, risking their retirement savings and their family’s financial security.
Now, if AI could do your job, this would still be a problem. We’d have to figure out what to do with all these technologically unemployed people.
But AI can’t do your job. It can help you do your job, but that doesn’t mean it’s going to save anyone money. Take radiology: there’s some evidence that AIs can sometimes identify solid-mass tumors that some radiologists miss, and look, I’ve got cancer. Thankfully, it’s very treatable, but I’ve got an interest in radiology being as reliable and accurate as possible
If my Kaiser hospital bought some AI radiology tools and told its radiologists: “Hey folks, here’s the deal. Today, you’re processing about 100 x-rays per day. From now on, we’re going to get an instantaneous second opinion from the AI, and if the AI thinks you’ve missed a tumor, we want you to go back and have another look, even if that means you’re only processing 98 x-rays per day. That’s fine, we just care about finding all those tumors.”
If that’s what they said, I’d be delighted. But no one is investing hundreds of billions in AI companies because they think AI will make radiology more expensive, not even if it that also makes radiology more accurate. The market’s bet on AI is that an AI salesman will visit the CEO of Kaiser and make this pitch: “Look, you fire 9/10s of your radiologists, saving $20m/year, you give us $10m/year, and you net $10m/year, and the remaining radiologists’ job will be to oversee the diagnoses the AI makes at superhuman speed, and somehow remain vigilant as they do so, despite the fact that the AI is usually right, except when it’s catastrophically wrong.
“And if the AI misses a tumor, this will be the human radiologist’s fault, because they are the ‘human in the loop.’ It’s their signature on the diagnosis.”
This is a reverse centaur, and it’s a specific kind of reverse-centaur: it’s what Dan Davies calles an “accountability sink.” The radiologist’s job isn’t really to oversee the AI’s work, it’s to take the blame for the AI’s mistakes.
This is another key to understanding — and thus deflating — the AI bubble. The AI can’t do your job, but an AI salesman can convince your boss to fire you and replace you with an AI that can’t do your job. This is key because it helps us build the kinds of coalitions that will be successful in the fight against the AI bubble.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.