The biggest AI flops of 2024

The past 12 months have been undeniably busy for those working in AI. There have been more successful product launches than we can count, and even Nobel Prizes. But it hasn’t always been smooth sailing.

AI is an unpredictable technology, and the increasing availability of generative models has led people to test their limits in new, weird, and sometimes harmful ways. These were some of 2024’s biggest AI misfires. 

AI slop infiltrated almost every corner of the internet

Generative AI makes creating reams of text, images, videos, and other types of material a breeze. Because it takes just a few seconds between entering a prompt for your model of choice to spit out the result, these models have become a quick, easy way to produce content on a massive scale. And 2024 was the year we started calling this (generally poor quality) media what it is—AI slop.  

This low-stakes way of creating AI slop means it can now be found in pretty much every corner of the internet: from the newsletters in your inbox and books sold on Amazon, to ads and articles across the web and shonky pictures on your social media feeds. The more emotionally evocative these pictures are (wounded veterans, crying children, a signal of support in the Israel-Palestine conflict) the more likely they are to be shared, resulting in higher engagement and ad revenue for their savvy creators.

AI slop isn’t just annoying—its rise poses a genuine problem for the future of the very models that helped to produce it. Because those models are trained on data scraped from the internet, the increasing number of junky websites containing AI garbage means there’s a very real danger models’ output and performance will get steadily worse. 

AI art is warping our expectations of real events

2024 was also the year that the effects of surreal AI images started seeping into our real lives. Willy’s Chocolate Experience, a wildly unofficial immersive event inspired by Roald Dahl’s Charlie and the Chocolate Factory, made headlines across the world in February after its fantastical AI-generated marketing materials gave visitors the impression it would be much grander than the sparsely-decorated warehouse its producers created.

Similarly, hundreds of people lined the streets of Dublin for a Halloween parade that didn’t exist. A Pakistan-based website used AI to create a list of events in the city, which was shared widely across social media ahead of October 31. Although the SEO-baiting site (myspirithalloween.com) has since been taken down, both events illustrate how misplaced public trust in AI-generated material online can come back to haunt us.

Grok allows users to create images of pretty much any scenario

The vast majority of major AI image generators have guardrails—rules that dictate what AI models can and can’t do—to prevent users from creating violent, explicit, illegal, and other types of harmful content. Sometimes these guardrails are just meant to make sure that no one makes blatant use of others’ intellectual property. But Grok, an assistant made by Elon Musk’s AI company, called xAI, ignores almost all of these principles in line with Musk’s rejection of what he calls “woke AI.”

Whereas other image models will generally refuse to create images of celebrities, copyrighted material, violence, or terrorism—unless they’re tricked into ignoring these rules—Grok will happily generate images of Donald Trump firing a bazooka, or Mickey Mouse holding a bomb. While it draws the line at generating nude images, its refusal to play by the rules undermines other companies’ efforts to steer clear of creating problematic material.

Sexually explicit deepfakes of Taylor Swift circulated online

In January, non-consensual deepfake nudes of singer Taylor Swift started circulating on social media, including X and Facebook. A Telegram community tricked Microsoft’s AI image generator Designer into making the explicit images, demonstrating how guardrails can be circumvented even when they are in place. 

While Microsoft quickly closed the system’s loopholes, the incident shone a light on the platforms’ poor content-moderation policies, after posts containing the images circulated widely and remained live for days. But the most chilling takeaway is how powerless we still are to fight non-consensual deepfake porn. While watermarking and data-poisoning tools can help, they’ll need to be adopted much more widely to make a difference.

Business chatbots went haywire

As AI becomes more widespread, businesses are racing to adopt generative tools to save time and money, and to maximize efficiency. The problem is—chatbots make stuff up and can’t be relied upon to always provide you with accurate information.

Air Canada found this out the hard way after its chatbot advised a customer to follow a bereavement refund policy that didn’t exist. In February, a Canadian small-claims tribunal upheld the customer’s legal complaint, despite the airline’s assertion that the chatbot was a “separate legal entity that is responsible for its own actions.”

In other high-profile examples of how chatbots can do more harm than good, delivery firm DPD’s bot cheerfully swore and called itself useless with little prompting, while a different bot set up to provide New Yorkers with accurate information about their city’s government ended up dispensing guidance on how to break the law.

AI gadgets aren’t exactly setting the market alight

Hardware assistants are something the AI industry tried, and failed, to crack in 2024. Humane attempted to sell customers on the promise of the Ai Pin, a wearable lapel computer, but even slashing its price failed to boost weak sales. The Rabbit R1, a ChatGPT-based personal assistant device, suffered a similar fate, following a rash of critical reviews and reports that it was slow and buggy. Both products seemed to be trying to solve a problem that did not actually exist. 

AI search summaries went awry

Have you ever added glue to a pizza, or eaten a small rock? These are just some of the outlandish suggestions that Google’s AI Overviews feature gave web users in May after the search giant added generated responses to the top of search results. Because AI systems can’t tell the difference between a factually correct news story and a joke post on Reddit, users raced to find the strangest responses AI Overviews could generate.

But AI summaries can also have serious consequences. A new iPhone feature that groups app notifications together and creates summaries of their contents, recently generated a false BBC News headline. The summary falsely stated that Luigi Mangione, who has been charged with the murder of healthcare insurance CEO Brian Thompson, had shot himself. The same feature had previously created a headline claiming that Israeli prime minister Benjamin Netanyahu had been arrested, which was also incorrect. These kinds of errors can inadvertently spread misinformation and undermine trust in news organizations.

The humans behind the robots

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Here’s a question. Imagine that, for $15,000, you could purchase a robot to pitch in with all the mundane tasks in your household. The catch (aside from the price tag) is that for 80% of those tasks, the robot’s AI training isn’t good enough for it to act on its own. Instead, it’s aided by a remote assistant working from the Philippines to help it navigate your home and clear your table or put away groceries. Would you want one?

That’s the question at the center of my story for our magazine, published online today, on whether we will trust humanoid robots enough to welcome them into our most private spaces, particularly if they’re part of an asymmetric labor arrangement in which workers in low-wage countries perform physical tasks for us in our homes through robot interfaces. In the piece, I wrote about one robotics company called Prosper and its massive effort—bringing in former Pixar designers and professional butlers—to design a trustworthy household robot named Alfie. It’s quite a ride. Read the story here.

There’s one larger question that the story raises, though, about just how profound a shift in labor dynamics robotics could bring in the coming years. 

For decades, robots have found success on assembly lines and in other somewhat predictable environments. Then, in the last couple of years, robots started being able to learn tasks more quickly thanks to AI, and that has broadened their applications to tasks in more chaotic settings, like picking orders in warehouses. But a growing number of well-funded companies are pushing for an even more monumental shift. 

Prosper and others are betting that they don’t have to build a perfect robot that can do everything on its own. Instead, they can build one that’s pretty good, but receives help from remote operators anywhere in the world. If that works well enough, they’re hoping to bring robots into jobs that most of us would have guessed couldn’t be automated: the work of hotel housekeepers, care providers in hospitals, or domestic help. “Almost any indoor physical labor” is on the table, Prosper’s founder and CEO, Shariq Hashme, told me. 

Until now, we’ve mostly thought about automation and outsourcing as two separate forces that can affect the labor market. Jobs might be outsourced overseas or lost to automation, but not both. A job that couldn’t be sent offshore and could not yet be fully automated by machines, like cleaning a hotel room, wasn’t going anywhere. Now, advancements in robotics are promising that employers can outsource such a job to low-wage countries without needing the technology to fully automate it. 

It’s a tall order, to be clear. Robots, as advanced as they’ve gotten, may find it difficult to move around complex environments like hotels and hospitals, even with assistance. That will take years to change. However, robots will only get more nimble, as will the systems that enable them to be controlled from halfway around the world. Eventually, the bets made by these companies may pay off.

What would that mean? One, the labor movement’s battle with AI—which this year has focused its attention on automation at ports and generative AI’s theft of artists’ work—will have a whole new battle to fight. It won’t just be dock workers, delivery drivers, and actors seeking contracts to protect their jobs from automation—it will be hospitality and domestic workers too, along with many others. 

Second, our expectations of privacy would radically shift. People buying those hypothetical household robots would have to be comfortable with the idea that someone that they have never met is seeing their dirty laundry—literally and figuratively. 

Some of those changes might happen sooner rather than later. For robots to learn how to navigate places effectively, they need training data, and this year has already seen a race to collect new data sets to help them learn. To achieve their ambitions for teleoperated robots, companies will expand their search for training data to hospitals, workplaces, hotels, and more. 

Now read the rest of The Algorithm

Deeper Learning

This is where the data to build AI comes from

AI developers often don’t really know or share much about the sources of the data they are using, and the Data Provenance Initiative, a group of over 50 researchers from both academia and industry, wanted to fix that. They dug into 4,000 public data sets spanning over 600 languages, 67 countries, and three decades to understand what’s feeding today’s top AI models, and how that will affect the rest of us. 

Why it matters: AI is being incorporated into everything, and what goes into the AI models determines what comes out. However, the team found that AI’s data practices risk concentrating power overwhelmingly in the hands of a few dominant technology companies, a shift from how AI models were being trained just a decade ago. Over 90% of the data sets that the researchers analyzed came from Europe and North America, and over 70% of data for both speech and image data sets comes from YouTube. This concentration means that AI models are unlikely to “capture all the nuances of humanity and all the ways that we exist,” says Sara Hooker, a researcher involved in the project. Read more from Melissa Heikkilä.

Bits and Bytes

In the shadows of Arizona’s data center boom, thousands live without power

As new research shows that AI’s emissions have soared, Arizona is expanding plans for AI data centers while rejecting plans to finally provide electricity to parts of the Navajo Nation’s land. (Washington Post)

AI is changing how we study bird migration

After decades of frustration, machine-learning tools are unlocking a treasure trove of acoustic data for ecologists. (MIT Technology Review)

OpenAI unveils a more advanced reasoning model in race with Google

The new o3 model, unveiled during a livestreamed event on Friday, spends more time computing an answer before responding to user queries, with the goal of solving more complex multi-step problems. (Bloomberg)

How your car might be making roads safer

Researchers say data from long-haul trucks and General Motors cars is critical for addressing traffic congestion and road safety. Data privacy experts have concerns. (New York Times)

This is where the data to build AI comes from

AI is all about data. Reams and reams of data are needed to train algorithms to do what we want, and what goes into the AI models determines what comes out. But here’s the problem: AI developers and researchers don’t really know much about the sources of the data they are using. AI’s data collection practices are immature compared with the sophistication of AI model development. Massive data sets often lack clear information about what is in them and where it came from. 

The Data Provenance Initiative, a group of over 50 researchers from both academia and industry, wanted to fix that. They wanted to know, very simply: Where does the data to build AI come from? They audited nearly 4,000 public data sets spanning over 600 languages, 67 countries, and three decades. The data came from 800 unique sources and nearly 700 organizations. 

Their findings, shared exclusively with MIT Technology Review, show a worrying trend: AI’s data practices risk concentrating power overwhelmingly in the hands of a few dominant technology companies. 

In the early 2010s, data sets came from a variety of sources, says Shayne Longpre, a researcher at MIT who is part of the project. 

It came not just from encyclopedias and the web, but also from sources such as parliamentary transcripts, earning calls, and weather reports. Back then, AI data sets were specifically curated and collected from different sources to suit individual tasks, Longpre says.

Then transformers, the architecture underpinning language models, were invented in 2017, and the AI sector started seeing performance get better the bigger the models and data sets were. Today, most AI data sets are built by indiscriminately hoovering material from the internet. Since 2018, the web has been the dominant source for data sets used in all media, such as audio, images, and video, and a gap between scraped data and more curated data sets has emerged and widened.

“In foundation model development, nothing seems to matter more for the capabilities than the scale and heterogeneity of the data and the web,” says Longpre. The need for scale has also boosted the use of synthetic data massively.

The past few years have also seen the rise of multimodal generative AI models, which can generate videos and images. Like large language models, they need as much data as possible, and the best source for that has become YouTube. 

For video models, as you can see in this chart, over 70% of data for both speech and image data sets comes from one source.

This could be a boon for Alphabet, Google’s parent company, which owns YouTube. Whereas text is distributed across the web and controlled by many different websites and platforms, video data is extremely concentrated in one platform.

“It gives a huge concentration of power over a lot of the most important data on the web to one company,” says Longpre. 

And because Google is also developing its own AI models, its massive advantage also raises questions about how the company will make this data available for competitors, says Sarah Myers West, the co–executive director at the AI Now Institute.

“It’s important to think about data not as though it’s sort of this naturally occurring resource, but it’s something that is created through particular processes,” says Myers West.

“If the data sets on which most of the AI that we’re interacting with reflect the intentions and the design of big, profit-motivated corporations—that’s reshaping the infrastructures of our world in ways that reflect the interests of those big corporations,” she says.

This monoculture also raises questions about how accurately the human experience is portrayed in the data set and what kinds of models we are building, says Sara Hooker, the vice president of research at the technology company Cohere, who is also part of the Data Provenance Initiative.

People upload videos to YouTube with a particular audience in mind, and the way people act in those videos is often intended for very specific effect. “Does [the data] capture all the nuances of humanity and all the ways that we exist?” says Hooker. 

Hidden restrictions

AI companies don’t usually share what data they used to train their models. One reason is that they want to protect their competitive edge. The other is that because of the complicated and opaque way data sets are bundled, packaged, and distributed, they likely don’t even know where all the data came from.

They also probably don’t have complete information about any constraints on how that data is supposed to be used or shared. The researchers at the Data Provenance Initiative found that data sets often have restrictive licenses or terms attached to them, which should limit their use for commercial purposes, for example.

“This lack of consistency across the data lineage makes it very hard for developers to make the right choice about what data to use,” says Hooker.

It also makes it almost impossible to be completely certain you haven’t trained your model on copyrighted data, adds Longpre.

More recently, companies such as OpenAI and Google have struck exclusive data-sharing deals with publishers, major forums such as Reddit, and social media platforms on the web. But this becomes another way for them to concentrate their power.

“These exclusive contracts can partition the internet into various zones of who can get access to it and who can’t,” says Longpre.

The trend benefits the biggest AI players, who can afford such deals, at the expense of researchers, nonprofits, and smaller companies, who will struggle to get access. The largest companies also have the best resources for crawling data sets.

“This is a new wave of asymmetric access that we haven’t seen to this extent on the open web,” Longpre says.

The West vs. the rest

The data that is used to train AI models is also heavily skewed to the Western world. Over 90% of the data sets that the researchers analyzed came from Europe and North America, and fewer than 4% came from Africa. 

“These data sets are reflecting one part of our world and our culture, but completely omitting others,” says Hooker.

The dominance of the English language in training data is partly explained by the fact that the internet is still over 90% in English, and there are still a lot of places on Earth where there’s really poor internet connection or none at all, says Giada Pistilli, principal ethicist at Hugging Face, who was not part of the research team. But another reason is convenience, she adds: Putting together data sets in other languages and taking other cultures into account requires conscious intention and a lot of work. 

The Western focus of these data sets becomes particularly clear with multimodal models. When an AI model is prompted for the sights and sounds of a wedding, for example, it might only be able to represent Western weddings, because that’s all that it has been trained on, Hooker says. 

This reinforces biases and could lead to AI models that push a certain US-centric worldview, erasing other languages and cultures.

“We are using these models all over the world, and there’s a massive discrepancy between the world we’re seeing and what’s invisible to these models,” Hooker says. 

AI’s search for more energy is growing more urgent

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

If you drove by one of the 2,990 data centers in the United States, you’d probably think little more than “Huh, that’s a boring-looking building.” You might not even notice it at all. However, these facilities underpin our entire digital world, and they are responsible for tons of greenhouse-gas emissions. New research shows just how much those emissions have skyrocketed during the AI boom. 

Since 2018, carbon emissions from data centers in the US have tripled, according to new research led by a team at the Harvard T.H. Chan School of Public Health. That puts data centers slightly below domestic commercial airlines as a source of this pollution.

That leaves a big problem for the world’s leading AI companies, which are caught between pressure to meet their own sustainability goals and the relentless competition in AI that’s leading them to build bigger models requiring tons of energy. The trend toward ever more energy-intensive new AI models, including video generators like OpenAI’s Sora, will only send those numbers higher. 

A growing coalition of companies is looking toward nuclear energy as a way to power artificial intelligence. Meta announced on December 3 it was looking for nuclear partners, and Microsoft is working to restart the Three Mile Island nuclear plant by 2028. Amazon signed nuclear agreements in October. 

However, nuclear plants take ages to come online. And though public support has increased in recent years, and president-elect Donald Trump has signaled support, only a slight majority of Americans say they favor more nuclear plants to generate electricity. 

Though OpenAI CEO Sam Altman pitched the White House in September on an unprecedented effort to build more data centers, the AI industry is looking far beyond the United States. Countries in Southeast Asia, like Malaysia, Indonesia, Thailand, and Vietnam, are all courting AI companies, hoping to be their new data center hubs. 

In the meantime, AI companies will continue to use up power from their current sources, which are far from renewable. Since so many data centers are located in coal-producing regions, like Virginia, the “carbon intensity” of the energy they use is 48% higher than the national average. The researchers found that 95% of data centers in the US are built in places with sources of electricity that are dirtier than the national average. Read more about the new research here.

Deeper Learning

We saw a demo of the new AI system powering Anduril’s vision for war

We’re living through the first drone wars, but AI is poised to change the future of warfare even more drastically. I saw that firsthand during a visit to a test site in Southern California run by Anduril, the maker of AI-powered drones, autonomous submarines, and missiles. Anduril has built a way for the military to command much of its hardware—from drones to radars to unmanned fighter jets—from a single computer screen. 

Why it matters: Anduril, other companies in defense tech, and growing numbers of people within the Pentagon itself are increasingly adopting a new worldview: A future “great power” conflict—military jargon for a global war involving multiple countries—will not be won by the entity with the most advanced drones or firepower, or even the cheapest firepower. It will be won by whoever can sort through and share information the fastest. The Pentagon is betting lots of energy and money that AI—despite its flaws and risks—will be what puts the US and its allies ahead in that fight. Read more here.

Bits and Bytes

Bluesky has an impersonator problem 

The platform’s rise has brought with it a surge of crypto scammers, as my colleague Melissa Heikkilä experienced firsthand. (MIT Technology Review)

Tech’s elite make large donations to Trump ahead of his inauguration 

Leaders in Big Tech, who have been lambasted by Donald Trump, have made sizable donations to his ​​inauguration committee. (The Washington Post)

Inside the premiere of the first commercially streaming AI-generated movies

The films, according to writer Jason Koebler, showed the telltale flaws of AI-generated video: dead eyes, vacant expressions, unnatural movements, and a reliance on voice-overs, since dialogue doesn’t work well. The company behind the films is confident viewers will stomach them anyway. (404 Media)

Meta asked California’s attorney general to stop OpenAI from becoming for-profit

Meta now joins Elon Musk in alleging that OpenAI has improperly enjoyed the benefits of nonprofit status while developing its technology. (Wall Street Journal)

How Silicon Valley is disrupting democracy

Two books explore the price we’ve paid for handing over unprecedented power to Big Tech—and explain why it’s imperative we start taking it back. (MIT Technology Review)