Bans on deepfakes take us only so far—here’s what we really need

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

There has been some really encouraging news in the fight against deepfakes. A couple of weeks ago the US Federal Trade Commission announced it is finalizing rules banning the use of deepfakes that impersonate people. Leading AI startups and big tech companies also unveiled their voluntary commitments to combatting the deceptive use of AI in 2024 elections. And last Friday, a group of civil society groups, including the Future of Life Institute, SAG-AFTRA, and Encode Justice came out with a new campaign calling for a ban on deepfakes.

These initiatives are a great start and raise public awareness—but the devil will be in the details. Existing rules in the UK and some US states already ban the creation and/or dissemination of deepfakes. The FTC would make it illegal for AI platforms to create content that impersonates people and would allow the agency to force scammers to return the money they made from such scams. 

But there is a big elephant in the room: outright bans might not even be technically feasible. There is no button someone can flick on and off, says Daniel Leufer, a senior policy analyst at the digital rights organization Access Now. 

That is because the genie is out of the bottle. 

Big Tech gets a lot of heat for the harm deepfakes cause, but to their credit, these companies do try to use their content moderation systems to detect and block attempts to generate, say, deepfake porn. (That’s not to say they are perfect. The deepfake porn targeting Taylor Swift reportedly came from a Microsoft system.) 

The bigger problem is that many of the harmful deepfakes come from open-source systems or systems built by state actors, and they are disseminated on end-to-end-encrypted platforms such as Telegram, where they cannot be traced.

Regulation really needs to tackle every actor in the deepfake pipeline, says Leufer. That may mean holding companies big and small accountable for allowing not just the creation of deepfakes but also their spread. So “model marketplaces,” such as Hugging Face or GitHub, may need to be included in talks about regulation to slow the spread of deepfakes. 

These model marketplaces make it easy to access open-source models such as Stable Diffusion, which people can use to build their own deepfake apps. These platforms are already taking action. Hugging Face and GitHub have put into place measures that add friction to the processes people use to access tools and make harmful content. Hugging Face is also a vocal proponent of OpenRAIL licenses, which make users commit to using the models in a certain way. The company also allows people to automatically integrate provenance data that meets high technical standards into their workflow. 

Other popular solutions include better watermarking and content provenance techniques, which would help with detection. But these detection tools are no silver bullet. 

Rules that require all AI-generated content to be watermarked are impossible to enforce, and it’s also highly possible that watermarks could end up doing the opposite of what they’re supposed to do, Leufer says. For one thing, in open-source systems, watermarking and provenance techniques can be removed by bad actors. This is because everyone has access to the model’s source code, so specific users can simply remove any techniques they don’t want.

If only the biggest companies or most popular proprietary platforms offer watermarks on their AI-generated content, then the absence of a watermark could come to signify that content is not AI generated, says Leufer. 

“Enforcing watermarking on all the content that you can enforce it on would actually lend credibility to the most harmful stuff that’s coming from the systems that we can’t intervene in,” he says. 

I asked Leufer if there are any promising approaches he sees out there that give him hope. He paused to think and finally suggested looking at the bigger picture. Deepfakes are just another symptom of the problems we have had with information and disinformation on social media, he said: “This could be the thing that tips the scales to really do something about regulating these platforms and drives a push to really allow for public understanding and transparency.” 

Now read the rest of The Algorithm

Deeper Learning

Watch this robot as it learns to stitch up wounds

An AI-trained surgical robot that can make a few stitches on its own is a small step toward systems that can aid surgeons with such repetitive tasks. A video taken by researchers at the University of California, Berkeley, shows the two-armed robot completing six stitches in a row on a simple wound in imitation skin, passing the needle through the tissue and from one robotic arm to the other while maintaining tension on the thread. 

A helping hand: Though many doctors today get help from robots for procedures ranging from hernia repairs to coronary bypasses, those are used to assist surgeons, not replace them. This new research marks progress toward robots that can operate more autonomously on very intricate, complicated tasks like suturing. The lessons learned in its development could also be useful in other fields of robotics. Read more from James O’Donnell here. 

Bits and Bytes

Wikimedia’s CTO: In the age of AI, human contributors still matter
Selena Deckelmann argues that in this era of machine-generated content, Wikipedia becomes even more valuable. (MIT Technology Review) 

Air Canada has to honor a refund policy its chatbot made up
The airline was forced to offer a customer a partial refund after its customer service chatbot inaccurately explained the company’s bereavement travel policy. Expect more cases like this as long as the tech sector sells chatbots that still make things up and have security flaws. (Wired)

Reddit has a new AI training deal to sell user content
The company has struck a $60 million deal to give an unnamed AI company access to the user-created content on its platform. OpenAI and Apple have reportedly been knocking on publishers’ doors trying to strike similar deals. Reddit’s human-written content is a gold mine for AI companies looking for high-quality training data for their language models. (Bloomberg) 

Google pauses Gemini’s ability to generate AI images of people after diversity errors
It’s no surprise that AI models are biased. I’ve written about how they are outright racist. But  Google’s effort to make its model more inclusive backfired after the model flat-out refused to generate images of white people. (The Verge) 

ChatGPT goes temporarily “insane” with unexpected outputs, spooking users
Last week, a bug made the popular chatbot produce bizarre and random responses to user queries. (Ars Technica) 

I went for a walk with Gary Marcus, AI’s loudest critic

Gary Marcus meets me outside the post office of Vancouver’s Granville Island wearing neon-coral sneakers and a blue Arc’teryx jacket. I’m in town for a family thing, and Marcus has lived in the city since 2018, after 20 years in New York City. “I just find it to be paradise,” he tells me, as I join him on his daily walk around Granville Island and nearby Kitsilano Beach. We’ve agreed to walk and talk about—what else—the current state of AI. 

“I’m depressed about it,” he tells me. “When I went into this field, it was not so that we could have a massive turnover of wealth from artists to big corporations.” I take a big sip of my black dark-roast coffee. Off we go. 

Marcus, a professor emeritus at NYU, is a prominent AI researcher and cognitive scientist who has positioned himself as a vocal critic of deep learning and AI. He is a divisive figure. You might recognize him from the spicy feuds on X with AI heavyweights such as Yann LeCun and Geoffrey Hinton. (“All attempts to socialize me have failed,” he jokes.) It is on walks like this that Marcus often does most of his tweeting.

This week has been a big news week in AI. Google DeepMind launched the next generation of its powerful artificial-intelligence model Gemini, which has an enhanced ability to work with large amounts of video, text, and images. And OpenAI has built a striking new generative video model called Sora that can take a short text description and turn it into a detailed, high-definition film clip up to a minute long. AI video generation has been around for a while, but Sora seems to have upped the ante. My X timeline has been flooded with stunning clips people have generated using the software. OpenAI claims that its results suggest that scaling video generation models like Sora “is a promising path towards building general purpose simulators of the physical world.” You can read more about Sora from Will Douglas Heaven here. 

But—surprise—Marcus is not impressed. “If you look at [the videos] for a second, you’re like, ‘Wow, that’s amazing.’ But if you look at them carefully, [the AI system] still doesn’t really understand common sense,” he says. In some videos, the physics are clearly off, and animals and people spontaneously appear and disappear, or things fly backwards, for example. 

For Marcus, generative video is yet another example of the exploitative business model of tech companies. Many artists and writers and even the New York Times have sued AI companies, claiming that their practice of indiscriminately scraping the internet for data to train their models violates their intellectual property. Copyright issues are top of Marcus’s mind. He managed to get popular AI image generators to generate scenes from Marvel movies or famous characters such as the Minions, Sonic the Hedgehog, and Darth Vader. He has started lobbying for clearer rules on what goes into AI models.  

“Video generation should not be done with copyrighted materials taken without consent, in systems that are opaque, where we can’t understand what’s going on,” he says. “It shouldn’t be a legal thing. It’s certainly not an ethical thing.” 

We stop at a scenic spot. It’s a beautiful route, with views of the city, the mountains, and the beach. A speckle of sun hits the peak of a mountain just across the bay. We could not be further away from Silicon Valley, the epicenter of today’s AI boom. “​​I’m not a religious person, but these kinds of tableaux … just continue to blow my mind,” Marcus says. 

But despite the tranquility of the surroundings, it is on walks like this that Marcus often uses X to rail against the power structures of Silicon Valley. Right now, he says, he identifies as an activist. 

When I ask him what motivates him, he replies without missing a beat: “The people who are running AI don’t really care that much about what you might call responsible AI, and that the consequences for society may be severe.” 

Late last year he wrote a book, called Taming Silicon Valley, which is coming out this fall. It is his manifesto on how AI should be regulated, but also a call to action. “We need to get the public involved in the struggle to try to get the AI companies to behave responsibly,” he says. 

There are a bunch of different things people can do, ranging from boycotting some of the software until people clean up their act to choosing electoral candidates around their tech policies, he says. 

Action and AI policy are needed urgently, he argues, because we are in a very narrow window during which we can fix things in AI. The risk is that we make the same mistakes regulators made with social media companies. 

“What we saw with social media is just going to be like an appetizer compared to what’s going to happen,” he says. 

Around 12 000 steps later, we’re back at Granville Island’s Public Market. I’m starving, so Marcus shows me a spot that serves good bagels. We both get the lox with cream cheese and eat it outside in the sun before parting ways.  

Later that day, Marcus would send out a flurry of tweets about Sora, having seen enough evidence to call it: “Sora is fantastic, but it is akin to morphing and splicing, rather than a path to the physical reasoning we would need for AGI,” he wrote. “We will see more systemic glitches as more people have access. Many will be hard to remedy.” 

Don’t say he didn’t warn you. 

_______________________________________

DEEPER LEARNING

A new satellite will use Google’s AI to map methane leaks from space

A methane-measuring satellite will launch next month that aims to use Google’s AI to quantify, map, and reduce leaks. The mission is part of a collaboration with the nonprofit Environmental Defense Fund, and the result, they say, will be the most detailed portrait yet of methane emissions. It should help to identify where the worst spots are and who is responsible.

Putting methane on the map: With methane responsible for roughly a third of the warming caused by greenhouse gases, regulators in the US and elsewhere are pushing for stronger rules to curb the leaks that spring from oil and gas plants. MethaneSAT will measure the plumes of methane that billow invisibly from oil and gas operations around the globe, and Google and EDF will then map those leaks for use by researchers, regulators, and the public. Read more from our new AI reporter James O’Donnell. James will cover the intersection between AI and hardware, such as robotics and chips. 

_____________________________________________________________________

BITS AND BYTES

Is AI going to change how we define videos? 

Systems like OpenAI’s Sora don’t make recordings. They render ideas. Does it matter that they’re not real? (New Yorker)

Early adopters of Microsoft’s AI bot are wondering if it’s worth the money

Testers have had it in their hands for six months now, and the results are mixed, to say the least. (WSJ)

The White House will spend $1.5 billion on a new chip factory

The massive grant, part of the CHIPS and Science Act, will help the US establish a homegrown supply for some of the most critical components of modern life. (WP)

AI hype has echoes of the telecom boom and bust

When a chief executive asks for trillions, not billions, when raising funds you know a sector might be getting a bit too hot. (FT)

Why Big Tech’s watermarking plans are some welcome good news

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

This week I am happy to bring you some encouraging news from the world of AI. Following the depressing Taylor Swift deepfake porn scandal and the proliferation of political deepfakes, such as AI-generated robocalls of President Biden asking voters to stay home, tech companies are stepping up and putting into place measures to better detect AI-generated content. 

On February 6, Meta said it was going to label AI-generated images on Facebook, Instagram, and Threads. When someone uses Meta’s AI tools to create images, the company will add visible markers to the image, as well as invisible watermarks and metadata in the image file. The company says its standards are in line with best practices laid out by the Partnership on AI, an AI research nonprofit.

Big Tech is also throwing its weight behind a promising technical standard that could add a “nutrition label” to images, video, and audio. Called C2PA, it’s an open-source internet protocol that relies on cryptography to encode details about the origins of a piece of content, or what technologists refer to as “provenance” information. The developers of C2PA often compare the protocol to a nutrition label, but one that says where content came from and who—or what—created it. Read more about it here. 

On February 8, Google announced it is joining other tech giants such as Microsoft and Adobe in the steering committee of C2PA and will include its watermark SynthID in all AI-generated images in its new Gemini tools. Meta says it is also participating in C2PA. Having an industry-wide standard makes it easier for companies to detect AI-generated content, no matter which system it was created with.

OpenAI too announced new content provenance measures last week. It says it will add watermarks to the metadata of images generated with ChatGPT and DALL-E 3, its image-making AI. OpenAI says it will now include a visible label in images to signal they have been created with AI. 

These methods are a promising start, but they’re not foolproof. Watermarks in metadata are easy to circumvent by taking a screenshot of images and just using that, while visual labels can be cropped or edited out. There is perhaps more hope for invisible watermarks like Google’s SynthID, which subtly changes the pixels of an image so that computer programs can detect the watermark but the human eye cannot. These are harder to tamper with. What’s more, there aren’t reliable ways to label and detect AI-generated video, audio, or even text. 

But there is still value in creating these provenance tools. As Henry Ajder, a generative-AI expert, told me a couple of weeks ago when I interviewed him about how to prevent deepfake porn, the point is to create a “perverse customer journey.” In other words, add barriers and friction to the deepfake pipeline in order to slow down the creation and sharing of harmful content as much as possible. A determined person will likely still be able to override these protections, but every little bit helps. 

There are also many nontechnical fixes tech companies could introduce to prevent problems such as deepfake porn. Major cloud service providers and app stores, such as Google, Amazon, Microsoft, and Apple could move to ban services that can be used to create nonconsensual deepfake nudes. And watermarks should be included in all AI-generated content across the board, even by smaller startups developing the technology.

What gives me hope is that alongside these voluntary measures we’re starting to see binding regulations, such as the EU’s AI Act and the Digital Services Act, which require tech companies to disclose AI-generated content and take down harmful content faster. There’s also renewed interest among US lawmakers in passing some binding rules on deepfakes. And following AI-generated robocalls of President Biden telling voters not to vote, the US Federal Communications Commission announced last week that it was banning the use of AI in these calls. 

In general I’m pretty skeptical about voluntary guidelines and rules, because there’s no real accountability mechanism and companies can choose to change these rules whenever they want. The tech sector has a really bad track record for regulating itself. In the cutthroat, growth-driven tech world, things like responsible AI are often the first to face cuts. 

But despite that, these announcements are extremely welcome. They’re also much better than the status quo, which is next to nothing. 

Deeper Learning

Google’s Gemini is now in everything. Here’s how you can try it out.

In the biggest mass-market AI launch yet, Google is rolling out Gemini, its family of large language models, across almost all its products, from Android to the iOS Google app to Gmail to Docs and more. You can now get your hands on Gemini Ultra, the most powerful version of the model, for the first time. 

Bard is dead; long live Gemini: Google is also sunsetting Bard, its ChatGPT rival. Bard, which has been powered by a version of Gemini since December, will now be known as Gemini too. By baking Gemini into its ubiquitous tools, Google is hoping to make up lost ground and even overtake its rival OpenAI. Read more from Will Douglas Heaven. 

Bits and Bytes

A chatbot helped more people access mental-health services
An AI chatbot from a startup called Limbic helped increase the number of patients referred for mental-health services through England’s National Health Service (particularly among members of underrepresented groups, who are less likely to seek help), new research has found. (MIT Technology Review) 

This robot can tidy a room without any help
A new system called OK-Robot could train robots to pick up and move objects in settings they haven’t encountered before. It’s an approach that might be able to plug the gap between rapidly improving AI models and actual robot capabilities, because it doesn’t require any additional costly, complex training. (MIT Technology Review) 

Inside OpenAI’s plan to make AI more “democratic”
This feature looks at how computer scientists at OpenAI are trying to address the technical problem of how to align their AIs to human values. But a bigger question remains unanswered: Exactly whose values should AI reflect? And who should get to decide? 
(Time) 

OpenAI’s Sam Altman wants trillions to build chips for AI
The CEO has often complained that the company does not have enough computing power to train and run its powerful AI models. Altman is reportedly talking with investors in the United Arab Emirates government to raise up to $7 trillion to boost the world’s chip-building capacity. (The Wall Street Journal) 

A new app to “dignify” women
Ugh. In contrast to apps that sexualize images of women, some 4Chan users are using generative AI to add clothes, erase their tattoos and piercings, and make them look more modest. How about … we just leave women alone. (404 Media) 

Dear Taylor Swift, we’re sorry about those explicit deepfakes

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Hi, Taylor.

I can only imagine how you must be feeling after sexually explicit deepfake videos of you went viral on X. Disgusted. Distressed, perhaps. Humiliated, even. 

I’m really sorry this is happening to you. Nobody deserves to have their image exploited like that. But if you aren’t already, I’m asking you to be furious. 

Furious that this is happening to you and so many other women and marginalized people around the world. Furious that our current laws are woefully inept at protecting us from violations like this. Furious that men (because let’s face it, it’s mostly men doing this) can violate us in such an intimate way and walk away unscathed and unidentified. Furious that the companies that enable this material to be created and shared widely face no consequences either, and can profit off such a horrendous use of their technology. 

Deepfake porn has been around for years, but its latest incarnation is its worst one yet. Generative AI has made it ridiculously easy and cheap to create realistic deepfakes. And nearly all deepfakes are made for porn. Only one image plucked off social media is enough to generate something passable. Anyone who has ever posted or had a photo published of them online is a sitting duck. 

First, the bad news. At the moment, we have no good ways to fight this. I just published a story looking at three ways we can combat nonconsensual deepfake porn, which include watermarks and data-poisoning tools. But the reality is that there is no neat technical fix for this problem. The fixes we do have are still experimental and haven’t been adopted widely by the tech sector, which limits their power. 

The tech sector has thus far been unwilling or unmotivated to make changes that would prevent such material from being created with their tools or shared on their platforms. That is why we need regulation. 

People with power, like yourself, can fight with money and lawyers. But low-income women, women of color, women fleeing abusive partners, women journalists, and even children are all seeing their likeness stolen and pornified, with no way to seek justice or support. Any one of your fans could be hurt by this development. 

The good news is that the fact that this happened to you means politicians in the US are listening. You have a rare opportunity, and momentum, to push through real, actionable change. 

I know you fight for what is right and aren’t afraid to speak up when you see injustice. There will be intense lobbying against any rules that would affect tech companies. But you have a platform and the power to convince lawmakers across the board that rules to combat these sorts of deepfakes are a necessity. Tech companies and politicians need to know that the days of dithering are over. The people creating these deepfakes need to be held accountable. 

You once caused an actual earthquake. Winning the fight against nonconsensual deepfakes would have an even more earth-shaking impact.

What to expect from the coming year in AI

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Happy new year! I hope you had a relaxing break. I spent it up in the Arctic Circle skiing, going to the sauna, and playing card games with my family by the fire. 10/10 would recommend. 

I also had plenty of time to reflect on the past year. There are so many more of you reading The Algorithm than when we first started this newsletter, and for that I am eternally grateful. Thank you for joining me on this wild AI ride. Here’s a cheerleading pug as a little present! 

So what can we expect in 2024? All signs point to there being immense pressure on AI companies to show that generative AI can make money and that Silicon Valley can produce the “killer app” for AI. Big Tech, generative AI’s biggest cheerleaders, is betting big on customized chatbots, which will allow anyone to become a generative-AI app engineer, with no coding skills needed. Things are already moving fast: OpenAI is reportedly set to launch its GPT app store as early as this week. We’ll also see cool new developments in AI-generated video, a whole lot more AI-powered election misinformation, and robots that multitask. My colleague Will Douglas Heaven and I shared our four predictions for AI in 2024 last week—read the full story here. 

This year will also be another huge year for AI regulation around the world. In 2023 the first sweeping AI law was agreed upon in the European Union, Senate hearings and executive orders unfolded in the US, and China introduced specific rules for things like recommender algorithms. If last year lawmakers agreed on a vision, 2024 will be the year policies start to morph into concrete action. Together with my colleagues Tate Ryan-Mosley and Zeyi Yang, I’ve written a piece that walks you through what to expect in AI regulation in the coming year. Read it here. 

But even as the generative-AI revolution unfolds at a breakneck pace, there are still some big unresolved questions that urgently need answering, writes Will. He highlights problems around bias, copyright, and the high cost of building AI, among other issues. Read more here. 

My addition to the list would be generative models’ huge security vulnerabilities. Large language models, the AI tech that powers applications such as ChatGPT, are really easy to hack. For example, AI assistants or chatbots that can browse the internet are very susceptible to an attack called indirect prompt injection, which allows outsiders to control the bot by sneaking in invisible prompts that make the bots behave in the way the attacker wants them to. This could make them powerful tools for phishing and scamming, as I wrote back in April. Researchers have also successfully managed to poison AI data sets with corrupt data, which can break AI models for good. (Of course, it’s not always a malicious actor trying to do this. Using a new tool called Nightshade, artists can add invisible changes to the pixels in their art before they upload it online so that if it’s scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways.) 

Despite these vulnerabilities, tech companies are in a race to roll out AI-powered products, such as assistants or chatbots that can browse the web. It’s fairly easy for hackers to manipulate AI systems by poisoning them with dodgy data, so it’s only a matter of time until we see an AI system being hacked in this way. That’s why I was pleased to see NIST, the US technology standards agency, raise awareness about these problems and offer mitigation techniques in a new guidance published at the end of last week. Unfortunately, there is currently no reliable fix for these security problems, and much more research is needed to understand them better.

AI’s role in our societies and lives will only grow bigger as tech companies integrate it into the software we all depend on daily, despite these flaws. As regulation catches up, keeping an open, critical mind when it comes to AI is more important than ever.

Deeper Learning

How machine learning might unlock earthquake prediction

Our current earthquake early warning systems give people crucial moments to prepare for the worst, but they have their limitations. There are false positives and false negatives. What’s more, they react only to an earthquake that has already begun—we can’t predict an earthquake the way we can forecast the weather. If we could, it would  let us do a lot more to manage risk, from shutting down the power grid to evacuating residents.

Enter AI: Some scientists are hoping to tease out hints of earthquakes from data—signals in seismic noise, animal behavior, and electromagnetism—with the ultimate goal of issuing warnings before the shaking begins. Artificial intelligence and other techniques are giving scientists hope in the quest to forecast quakes in time to help people find safety. Read more from Allie Hutchison. 

Bits and Bytes

AI for everything is one of MIT Technology Review’s 10 breakthrough technologies
We couldn’t put together a list of the tech that’s most likely to have an impact on the world without mentioning AI. Last year tools like ChatGPT reached mass adoption in record time, and reset the course of an entire industry. We haven’t even begun to make sense of it all, let alone reckon with its impact. (MIT Technology Review) 

Isomorphic Labs has announced it’s working with two pharma companies
Google DeepMind’s drug discovery spinoff has two new “strategic collaborations” with major pharma companies Eli Lilly and Novartis. The deals are worth nearly $3 billion to Isomorphic Labs and offer the company funding to help discover potential new treatments using AI, the company said. 

We learned more about OpenAI’s board saga
Helen Toner, an AI researcher at Georgetown’s Center for Security and Emerging Technology and a former member of OpenAI’s board, talks to the Wall Street Journal about why she agreed to fire CEO Sam Altman. Without going into details, she underscores that it wasn’t safety that led to the fallout, but a lack of trust. Meanwhile, Microsoft executive Dee Templeton has joined OpenAI’s board as a nonvoting observer. 

A new kind of AI copy can fully replicate famous people. The law is powerless.
Famous people are finding convincing AI replicas in their likeness. A new draft bill in the US called the No Fakes Act would require the creators of these AI replicas to license their use from the original human. But this bill would not apply in cases where the replicated human or the AI system is outside the US. It’s another example of just how incredibly difficult AI regulation is. (Politico)

The largest AI image data set was taken offline after researchers found it is full of child sexual abuse material
Stanford researchers made the explosive discovery about the open-source LAION data set, which powers models such as Stable Diffusion. We knew indiscriminate scraping of the internet meant AI data sets contain tons of biased and harmful content, but this revelation is shocking. We desperately need better data practices in AI! (404 Media) 

Four trends that changed AI in 2023

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

This has been one of the craziest years in AI in a long time: endless product launches, boardroom coups, intense policy debates about AI doom, and a race to find the next big thing. But we’ve also seen concrete tools and policies aimed at getting the AI sector to behave more responsibly and hold powerful players accountable. That gives me a lot of hope for the future of AI. 

Here’s what 2023 taught me: 

1. Generative AI left the lab with a vengeance, but it’s not clear where it will go next

The year started with Big Tech going all in on generative AI. The runaway success of OpenAI’s ChatGPT prompted every major tech company to release its own version. This year might go down in history as the year we saw the most AI launches: Meta’s LLaMA 2, Google’s Bard chatbot and Gemini, Baidu’s Ernie Bot, OpenAI’s GPT-4, and a handful of other models, including one from a French open-source challenger, Mistral. 

But despite the initial hype, we haven’t seen any AI applications become an overnight success. Microsoft and Google pitched powerful AI-powered search, but it turned out to be more of a dud than a killer app. The fundamental flaws in language models, such as the fact that they frequently make stuff up, led to some embarrassing (and, let’s be honest, hilarious) gaffes. Microsoft’s Bing would frequently reply to people’s questions with conspiracy theories, and suggested that a New York Times reporter leave his wife. Google’s Bard generated factually incorrect answers for its marketing campaign, which wiped $100 billion off the company’s share price.

There is now a frenetic hunt for a popular AI product that everyone will want to adopt. Both OpenAI and Google are experimenting with allowing companies and developers to create customized AI chatbots and letting people build their own applications using AI—no coding skills needed. Perhaps generative AI will end up embedded in boring but useful tools to help us boost our productivity at work. It might take the form of AI assistants—maybe with voice capabilities—and coding support. Next year will be crucial in determining the real value of generative AI.

2. We learned a lot about how language models actually work, but we still know very little

Even though tech companies are rolling out large language models into products at a frenetic pace, there is still a lot we don’t know about how they work. They make stuff up and have severe gender and ethnic biases. This year we also found out that different language models generate texts with different political biases, and that they make great tools for hacking people’s private information. Text-to-image models can be prompted to spit out copyrighted images and pictures of real people, and they can easily be tricked into generating disturbing images. It’s been great to see so much research into the flaws of these models, because this could take us a step closer to understanding why they behave the way they do, and ultimately fix them.

Generative models can be very unpredictable, and this year there were lots of attempts to try to make them behave as their creators want them to. OpenAI shared that it uses a technique called reinforcement learning from human feedback, which uses feedback from users to help guide ChatGPT to more desirable answers. A study from the AI lab Anthropic showed how simple natural-language instructions can steer large language models to make their results less toxic. But sadly, a lot of these attempts end up being quick fixes rather than permanent ones. Then there are misguided approaches like banning seemingly innocuous words such as “placenta” from image-generating AI systems to avoid producing gore. Tech companies come up with workarounds like these because they don’t know why models generate the content they do. 

We also got a better sense of AI’s true carbon footprint. Generating an image using a powerful AI model takes as much energy as fully charging your smartphone, researchers at the AI startup Hugging Face and Carnegie Mellon University found. Until now, the exact amount of energy generative AI uses has been a missing piece of the puzzle. More research into this could help us shift the way we use AI to be more sustainable. 

3. AI doomerism went mainstream

Chatter about the possibility that AI poses an existential risk to humans became familiar this year. Hundreds of scientists, business leaders, and policymakers have spoken up, from deep-learning pioneers Geoffrey Hinton and Yoshua Bengio to the CEOs of top AI firms, such as Sam Altman and Demis Hassabis, to the California congressman Ted Lieu and the former president of Estonia Kersti Kaljulaid.

Existential risk has become one of the biggest memes in AI. The hypothesis is that one day we will build an AI that is far smarter than humans, and this could lead to grave consequences. It’s an ideology championed by many in Silicon Valley, including Ilya Sutskever, OpenAI’s chief scientist, who played a pivotal role in ousting OpenAI CEO Sam Altman (and then reinstating him a few days later). 

But not everyone agrees with this idea. Meta’s AI leaders Yann LeCun and Joelle Pineau have said that these fears are “ridiculous” and the conversation about AI risks has become “unhinged.” Many other power players in AI, such as researcher Joy Buolamwini, say that focusing on hypothetical risks distracts from the very real harms AI is causing today. 

Nevertheless, the increased attention on the technology’s potential to cause extreme harm has prompted many important conversations about AI policy and animated lawmakers all over the world to take action. 

4. The days of the AI Wild West are over

Thanks to ChatGPT, everyone from the US Senate to the G7 was talking about AI policy and regulation this year. In early December, European lawmakers wrapped up a busy policy year when they agreed on the AI Act, which will introduce binding rules and standards on how to develop the riskiest AI more responsibly. It will also ban certain “unacceptable” applications of AI, such as police use of facial recognition in public places. 

The White House, meanwhile, introduced an executive order on AI, plus voluntary commitments from leading AI companies. Its efforts aimed to bring more transparency and standards for AI and gave a lot of freedom to agencies to adapt AI rules to fit their sectors. 

One concrete policy proposal that got a lot of attention was watermarks—invisible signals in text and images that can be detected by computers, in order to flag AI-generated content. These could be used to track plagiarism or help fight disinformation, and this year we saw research that succeeded in applying them to AI-generated text and images.

It wasn’t just lawmakers that were busy, but lawyers too. We saw a record number of  lawsuits, as artists and writers argued that AI companies had scraped their intellectual property without their consent and with no compensation. In an exciting counter-offensive, researchers at the University of Chicago developed Nightshade, a new data-poisoning tool that lets artists fight back against generative AI by messing up training data in ways that could cause serious damage to image-generating AI models. There is a resistance brewing, and I expect more grassroots efforts to shift tech’s power balance next year. 

Deeper Learning

Now we know what OpenAI’s superalignment team has been up to

OpenAI has announced the first results from its superalignment team, its in-house initiative dedicated to preventing a superintelligence—a hypothetical future AI that can outsmart humans—from going rogue. The team is led by chief scientist Ilya Sutskever, who was part of the group that just last month fired OpenAI’s CEO, Sam Altman, only to reinstate him a few days later.

Business as usual: Unlike many of the company’s announcements, this heralds no big breakthrough. In a low-key research paper, the team describes a technique that lets a less powerful large language model supervise a more powerful one—and suggests that this might be a small step toward figuring out how humans might supervise superhuman machines. Read more from Will Douglas Heaven. 

Bits and Bytes

Google DeepMind used a large language model to solve an unsolvable math problem
In a paper published in Nature, the company says it is the first time a large language model has been used to discover a solution to a long-standing scientific puzzle—producing verifiable and valuable new information that did not previously exist. (MIT Technology Review)

This new system can teach a robot a simple household task within 20 minutes
A new open-source system, called Dobb-E, was trained using data collected from real homes. It can help to teach a robot how to open an air fryer, close a door, or straighten a cushion, among other tasks. It could also help the field of robotics overcome one of its biggest challenges: a lack of training data.  (MIT Technology Review)

ChatGPT is turning the internet into plumbing
German media giant Axel Springer, which owns Politico and Business Insider, announced a partnership with OpenAI, in which the tech company will be able to use its news articles as training data and the news organizations will be able to use ChatGPT to do summaries of news. This column has a clever point: tech companies are increasingly becoming gatekeepers for online content, and journalism is just “plumbing for a digital faucet.” (The Atlantic)

Meet the former French official pushing for looser AI rules after joining startup Mistral
A profile of Mistral AI cofounder Cédric O, who used to be France’s digital minister. Before joining France’s AI unicorn, he was a vocal proponent of strict laws for tech, but he lobbied hard against rules in the AI Act that would have restricted Mistral’s models. He was successful: the company’s models don’t meet the computing threshold set by the law, and its open-source models are also exempt from transparency obligations. (Bloomberg) 

Five things you need to know about the EU’s new AI Act

This story is from The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

It’s done. It’s over. Two and a half years after it was first introduced—after months of lobbying and political arm-wrestling, plus grueling final negotiations that took nearly 40 hours—EU lawmakers have reached a deal over the AI Act. It will be the world’s first sweeping AI law. 

The AI Act was conceived as a landmark bill that would mitigate harm in areas where using AI poses the biggest risk to fundamental rights, such as health care, education, border surveillance, and public services, as well as banning uses that pose an “unacceptable risk.” 

“High risk” AI systems will have to adhere to strict rules that require risk-mitigation systems, high-quality data sets, better documentation, and human oversight, for example. The vast majority of AI uses, such as recommender systems and spam filters, will get a free pass. 

The AI Act is a major deal in that it will introduce important rules and enforcement mechanisms to a hugely influential sector that is currently a Wild West. 

Here are MIT Technology Review’s key takeaways: 

1. The AI Act ushers in important, binding rules on transparency and ethics

Tech companies love to talk about how committed they are to AI ethics. But when it comes to concrete measures, the conversation dries up. And anyway, actions speak louder than words. Responsible AI teams are often the first to see cuts during layoffs, and in truth, tech companies can decide to change their AI ethics policies at any time. OpenAI, for example, started off as an “open” AI research lab before closing up public access to its research to protect its competitive advantage, just like every other AI startup. 

The AI Act will change that. The regulation imposes legally binding rules requiring tech companies to notify people when they are interacting with a chatbot or with biometric categorization or emotion recognition systems. It’ll also require them to label deepfakes and AI-generated content, and design systems in such a way that AI-generated media can be detected. This is a step beyond the voluntary commitments that leading AI companies made to the White House to simply develop AI provenance tools, such as watermarking. 

The bill will also require all organizations that offer essential services, such as insurance and banking, to conduct an impact assessment on how using AI systems will affect people’s fundamental rights. 

2. AI companies still have a lot of wiggle room

When the AI Act was first introduced, in 2021, people were still talking about the metaverse. (Can you imagine!) 

Fast-forward to now, and in a post-ChatGPT world, lawmakers felt they had to take so-called foundation models—powerful AI models that can be used for many different purposes—into account in the regulation. This sparked intense debate over what sorts of models should be regulated, and whether regulation would kill innovation. 

The AI Act will require foundation models and AI systems built on top of them to draw up better documentation, comply with EU copyright law, and share more information about what data the model was trained on. For the most powerful models, there are extra requirements. Tech companies will have to share how secure and energy efficient their AI models are, for example. 

But here’s the catch: The compromise lawmakers found was to apply a stricter set of rules only the most powerful AI models, as categorized by the computing power needed to train them. And it will be up to companies to assess whether they fall under stricter rules. 

A European Commission official would not confirm whether the current cutoff would capture powerful models such as OpenAI’s GPT-4 or Google’s Gemini, because only the companies themselves know how much computing power was used to train their models. The official did say that as the technology develops, the EU could change the way it measures how powerful AI models are. 

3. The EU will become the world’s premier AI police

The AI Act will set up a new European AI Office to coordinate compliance, implementation, and enforcement. It will be the first body globally to enforce binding rules on AI, and the EU hopes this will help it become the world’s go-to tech regulator. The AI Act’s governance mechanism also includes a scientific panel of independent experts to offer guidance on the systemic risks AI poses, and how to classify and test models. 

The fines for noncompliance are steep: from 1.5% to 7% of a firm’s global sales turnover, depending on the severity of the offense and size of the company. 

Europe will also become the one of the first places in the world where citizens will be able to launch complaints about AI systems and receive explanations about how AI systems came to the conclusions that affect them. 

By becoming the first to formalize rules around AI, the EU retains its first-mover advantage. Much like the GDPR, the AI Act could become a global standard. Companies elsewhere that want to do business in the world’s second-largest economy will have to comply with the law. The EU’s rules also go a step further than ones introduced by the US, such as the White House executive order, because they are binding. 

4. National security always wins

Some AI uses are now completely banned in the EU: biometric categorization systems that use sensitive characteristics; untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases like Clearview AI; emotion recognition at work or in schools; social scoring; AI systems that manipulate human behavior; and AI that is used to exploit people’s vulnerabilities. 

Predictive policing is also banned, unless it is used with “clear human assessment and objective facts, which basically do not simply leave the decision of going after a certain individual in a criminal investigation only because an algorithm says so,” according to an EU Commission official.

However, the AI Act does not apply to AI systems that have been developed exclusively for military and defense uses. 

One of the bloodiest fights over the AI Act has always been how to regulate police use of biometric systems in public places, which many fear could lead to mass surveillance. While the European Parliament pushed for a near-total ban on the technology, some EU countries, such as France, have resisted this fiercely. They want to use it to fight crime and terrorism. 

European police forces will only be able to use biometric identification systems in public places if they get court approval first, and only for 16 different specific crimes, such as terrorism, human trafficking, sexual exploitation of children, and drug trafficking. Law enforcement authorities may also use high-risk AI systems that don’t pass European standards in “exceptional circumstances relating to public security.” 

5. What next? 

It might take weeks or even months before we see the final wording of the bill. The text still needs to go through technical tinkering, and has to be approved by European countries and the EU Parliament before it officially enters into law. 

Once it is in force, tech companies have two years to implement the rules. The bans on AI uses will apply after six months, and companies developing foundation models will have to comply with the law within one year.