What to expect from the coming year in AI

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

Happy new year! I hope you had a relaxing break. I spent it up in the Arctic Circle skiing, going to the sauna, and playing card games with my family by the fire. 10/10 would recommend. 

I also had plenty of time to reflect on the past year. There are so many more of you reading The Algorithm than when we first started this newsletter, and for that I am eternally grateful. Thank you for joining me on this wild AI ride. Here’s a cheerleading pug as a little present! 

So what can we expect in 2024? All signs point to there being immense pressure on AI companies to show that generative AI can make money and that Silicon Valley can produce the “killer app” for AI. Big Tech, generative AI’s biggest cheerleaders, is betting big on customized chatbots, which will allow anyone to become a generative-AI app engineer, with no coding skills needed. Things are already moving fast: OpenAI is reportedly set to launch its GPT app store as early as this week. We’ll also see cool new developments in AI-generated video, a whole lot more AI-powered election misinformation, and robots that multitask. My colleague Will Douglas Heaven and I shared our four predictions for AI in 2024 last week—read the full story here

This year will also be another huge year for AI regulation around the world. In 2023 the first sweeping AI law was agreed upon in the European Union, Senate hearings and executive orders unfolded in the US, and China introduced specific rules for things like recommender algorithms. If last year lawmakers agreed on a vision, 2024 will be the year policies start to morph into concrete action. Together with my colleagues Tate Ryan-Mosley and Zeyi Yang, I’ve written a piece that walks you through what to expect in AI regulation in the coming year. Read it here

But even as the generative-AI revolution unfolds at a breakneck pace, there are still some big unresolved questions that urgently need answering, writes Will. He highlights problems around bias, copyright, and the high cost of building AI, among other issues. Read more here

My addition to the list would be generative models’ huge security vulnerabilities. Large language models, the AI tech that powers applications such as ChatGPT, are really easy to hack. For example, AI assistants or chatbots that can browse the internet are very susceptible to an attack called indirect prompt injection, which allows outsiders to control the bot by sneaking in invisible prompts that make the bots behave in the way the attacker wants them to. This could make them powerful tools for phishing and scamming, as I wrote back in April. Researchers have also successfully managed to poison AI data sets with corrupt data, which can break AI models for good. (Of course, it’s not always a malicious actor trying to do this. Using a new tool called Nightshade, artists can add invisible changes to the pixels in their art before they upload it online so that if it’s scraped into an AI training set, it can cause the resulting model to break in chaotic and unpredictable ways.) 

Despite these vulnerabilities, tech companies are in a race to roll out AI-powered products, such as assistants or chatbots that can browse the web. It’s fairly easy for hackers to manipulate AI systems by poisoning them with dodgy data, so it’s only a matter of time until we see an AI system being hacked in this way. That’s why I was pleased to see NIST, the US technology standards agency, raise awareness about these problems and offer mitigation techniques in a new guidance published at the end of last week. Unfortunately, there is currently no reliable fix for these security problems, and much more research is needed to understand them better.

AI’s role in our societies and lives will only grow bigger as tech companies integrate it into the software we all depend on daily, despite these flaws. As regulation catches up, keeping an open, critical mind when it comes to AI is more important than ever.

Deeper Learning

How machine learning might unlock earthquake prediction

Our current earthquake early warning systems give people crucial moments to prepare for the worst, but they have their limitations. There are false positives and false negatives. What’s more, they react only to an earthquake that has already begun—we can’t predict an earthquake the way we can forecast the weather. If we could, it would  let us do a lot more to manage risk, from shutting down the power grid to evacuating residents.

Enter AI: Some scientists are hoping to tease out hints of earthquakes from data—signals in seismic noise, animal behavior, and electromagnetism—with the ultimate goal of issuing warnings before the shaking begins. Artificial intelligence and other techniques are giving scientists hope in the quest to forecast quakes in time to help people find safety. Read more from Allie Hutchison

Bits and Bytes

AI for everything is one of MIT Technology Review’s 10 breakthrough technologies
We couldn’t put together a list of the tech that’s most likely to have an impact on the world without mentioning AI. Last year tools like ChatGPT reached mass adoption in record time, and reset the course of an entire industry. We haven’t even begun to make sense of it all, let alone reckon with its impact. (MIT Technology Review

Isomorphic Labs has announced it’s working with two pharma companies
Google DeepMind’s drug discovery spinoff has two new “strategic collaborations” with major pharma companies Eli Lilly and Novartis. The deals are worth nearly $3 billion to Isomorphic Labs and offer the company funding to help discover potential new treatments using AI, the company said

We learned more about OpenAI’s board saga
Helen Toner, an AI researcher at Georgetown’s Center for Security and Emerging Technology and a former member of OpenAI’s board, talks to the Wall Street Journal about why she agreed to fire CEO Sam Altman. Without going into details, she underscores that it wasn’t safety that led to the fallout, but a lack of trust. Meanwhile, Microsoft executive Dee Templeton has joined OpenAI’s board as a nonvoting observer. 

A new kind of AI copy can fully replicate famous people. The law is powerless.
Famous people are finding convincing AI replicas in their likeness. A new draft bill in the US called the No Fakes Act would require the creators of these AI replicas to license their use from the original human. But this bill would not apply in cases where the replicated human or the AI system is outside the US. It’s another example of just how incredibly difficult AI regulation is. (Politico)

The largest AI image data set was taken offline after researchers found it is full of child sexual abuse material
Stanford researchers made the explosive discovery about the open-source LAION data set, which powers models such as Stable Diffusion. We knew indiscriminate scraping of the internet meant AI data sets contain tons of biased and harmful content, but this revelation is shocking. We desperately need better data practices in AI! (404 Media

Bringing breakthrough data intelligence to industries

As organizations recognize the transformational opportunity presented by generative AI, they must consider how to deploy that technology across the enterprise in the context of their unique industry challenges, priorities, data types, applications, ecosystem partners, and governance requirements. Financial institutions, for example, need to ensure that data and AI governance has the built-in intelligence to fully align with strict compliance and regulatory requirements. Media and entertainment (M&E) companies seek to build AI models to drive deeper product personalization. And manufacturers want to use AI to make their internet of things (IoT) data insights readily accessible to everyone from the data scientist to the shop floor worker.

In any of these scenarios, the starting point is access to all relevant data—of any type, from any source, in real time—governed comprehensively and shared across an industry ecosystem. When organizations can achieve this with the right data and AI foundation, they have the beginnings of data intelligence: the ability to understand their data and break free from data silos that would block the most valuable AI outcomes.

But true data intelligence is about more than establishing the right data foundation. Organizations are also wrestling with how to overcome dependence on highly technical staff and create frameworks for data privacy and organizational control when using generative AI. Specifically, they are looking to enable all employees to use natural language to glean actionable insight from the company’s own data; to leverage that data at scale to train, build, deploy, and tune their own secure large language models (LLMs); and to infuse intelligence about the company’s data into every business process.

In this next frontier of data intelligence, organizations will maximize value by democratizing AI while differentiating through their people, processes, and technology within their industry context. Based on a global, cross-industry survey of 600 technology leaders as well as in-depth interviews with technology leaders, this report explores the foundations being built and leveraged across industries to democratize data and AI. Following are its key findings:

• Real-time access to data, streaming, and analytics are priorities in every industry. Because of the power of data-driven decision-making and its potential for game-changing innovation, CIOs require seamless access to all of their data and the ability to glean insights from it in real time. Seventy-two percent of survey respondents say the ability to stream data in real time for analysis and action is “very important” to their overall technology goals, while another 20% believe it is “somewhat important”—whether that means enabling real-time recommendations in retail or identifying a next best action in a critical health-care triage situation.

• All industries aim to unify their data and AI governance models. Aspirations for a single approach to governance of data and AI assets are strong: 60% of survey respondents say a single approach to built-in governance for data and AI is “very important,” and an additional 38% say it is “somewhat important,” suggesting that many organizations struggle with a fragmented or siloed data architecture. Every industry will have to achieve this unified governance in the context of its own unique systems of record, data pipelines, and requirements for security and compliance.

• Industry data ecosystems and sharing across platforms will provide a new foundation for AI-led growth. In every industry, technology leaders see promise in technology-agnostic data sharing across an industry ecosystem, in support of AI models and core operations that will drive more accurate, relevant, and profitable outcomes. Technology teams at insurers and retailers, for example, aim to ingest partner data to support real-time pricing and product offer decisions in online marketplaces, while manufacturers see data sharing as an important capability for continuous supply chain optimization. Sixty-four percent of survey respondents say the ability to share live data across platforms is “very important,” while an additional 31% say it is “somewhat important.” Furthermore, 84% believe a managed central marketplace for data sets, machine learning models, and notebooks is very or somewhat important.

• Preserving data and AI flexibility across clouds resonates with all verticals. Sixty-three percent of respondents across verticals believe that the ability to leverage multiple cloud providers is at least somewhat important, while 70% feel the same about open-source standards and technology. This is consistent with the finding that 56% of respondents see a single system to manage structured and unstructured data across business intelligence and AI as “very important,” while an additional 40% see this as “somewhat important.” Executives are prioritizing access to all of the organization’s data, of any type and from any source, securely and without compromise.

• Industry-specific requirements will drive the prioritization and pace by which generative AI use cases are adopted. Supply chain optimization is the highest-value generative AI use case for survey respondents in manufacturing, while it is real-time data analysis and insights for the public sector, personalization and customer experience for M&E, and quality control for telecommunications. Generative AI adoption will not be one-size-fits-all; each industry is taking its own strategy and approach. But in every case, value creation will depend on access to data and AI permeating the enterprise’s ecosystem and AI being embedded into its products and services.

Maximizing value and scaling the impact of AI across people, processes, and technology is a common goal across industries. But industry differences merit close attention for their implications on how intelligence is infused into the data and AI platforms. Whether it be for the retail associate driving omnichannel sales, the health-care practitioner pursuing real-world evidence, the actuary analyzing risk and uncertainty, the factory worker diagnosing equipment, or the telecom field agent assessing network health, the language and scenarios AI will support vary significantly when democratized to the front lines of every industry.

Download the report.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff.

How machine learning might unlock earthquake prediction

In September 2017, about two minutes before a magnitude 8.2 earthquake struck Mexico City, blaring sirens alerted residents that a quake was coming. Such alerts, which are now available in the United States, Japan, Turkey, Italy, and Romania, among other countries, have changed the way we think about the threat of earthquakes. They no longer have to take us entirely by surprise.

Earthquake early warning systems can send alarms through phones or transmit a loud signal to affected regions three to five seconds after a potentially damaging earthquake begins. First, seismometers close to the fault pick up the beginnings of the quake, and finely programmed algorithms determine its probable size. If it is moderate or large, the resulting alert then travels faster than the earthquake itself, giving seconds to minutes of warning. This window of time is crucial: in these brief moments, people can shut off electricity and gas lines, move fire trucks into the streets, and find safe places to go. 

shot at night of buildings on fire
The magnitude 9 Tohoku-Oki earthquake of 2011 was preceded by two slow earthquakes.
AP IMAGES

But these systems have limitations. There are false positives and false negatives. What’s more, they react only to an earthquake that has already begun—we can’t predict an earthquake the way we can forecast the weather. And so many earthquake-­prone regions are left in a state of constant suspense. A proper forecast could let us do a lot more to manage risk, from shutting down the power grid to evacuating residents.

When I started my PhD in seismology in 2013, the very topic of earthquake prediction was deemed unserious, as outside the realm of mainstream research as the hunt for the Loch Ness Monster. 

But just seven years later, a lot had changed. When I began my second postdoc in 2020, I observed that scientists in the field had become much more open to earthquake prediction. The project I was a part of, Tectonic, was using machine learning to advance earthquake prediction. The European Research Council was sufficiently convinced of its potential to award it a four-year, €3.4 million grant that same year. 

Today, a number of well-respected scientists are getting serious about the prospect of prediction and are making progress in their respective subdisciplines. Some are studying a different kind of slow-motion behavior along fault lines, which could turn out to be a useful indicator that the devastating kind of earthquake we all know and fear is on the way. Others are hoping to tease out hints from other data—signals in seismic noise, animal behavior, and electromagnetism—to push earthquake science toward the possibility of issuing warnings before the shaking begins. 

In the dark

Earthquake physics can seem especially opaque. Astronomers can view the stars; biologists can observe an animal. But those of us who study earthquakes cannot see into the ground—at least not directly. Instead, we use proxies to understand what happens inside the Earth when its crust shakes: seismology, the study of the sound waves generated by movement within the interior; geodesy, the application of tools like GPS to measure how Earth’s surface changes over time; and paleoseismology, the study of relics of past earthquakes concealed in geologic layers of the landscape. 

Without good knowledge of what’s happening under the ground, it’s impossible to intuit any sense of order.

There is much we still don’t know. Decades after the theory of plate tectonics was widely accepted in the 1960s, our understanding of earthquake genesis hasn’t progressed far beyond the idea that stress builds to a critical threshold, at which point it is released through a quake. Different factors can make a fault more susceptible to reaching that point. The presence of fluids, for instance, is significant: the injection of wastewater fluid from oil and gas production has caused huge increases in tectonic activity across the central US in the last decade. But when it comes to knowing what is happening along a given fault line, we’re largely in the dark. We can construct an approximate map of a fault by using seismic waves and mapping earthquake locations, but we can’t directly measure the stress it is experiencing, nor can we quantify the threshold beyond which the ground will move.

For a long time, the best we could do regarding prediction was to get a sense of how often earthquakes happen in a particular region. For example, the last earthquake to rupture the entire length of the southern San Andreas Fault in California was in 1857. The average time period between big quakes there is estimated to be somewhere between 100 and 180 years. According to a back-of-the-envelope calculation, we could be “overdue.” But as the wide range suggests, recurrence intervals can vary wildly and may be misleading. The sample size is limited to the scope of human history and what we can still observe in the geologic record, which represents a small fraction of the earthquakes that have occurred over Earth’s history.

In 1985, scientists began installing seismometers and other earthquake monitoring equipment along the Parkfield section of the San Andreas Fault, in central California. Six earthquakes in that section had occurred at unusually regular intervals compared to earthquakes along other faults, so scientists from the US Geological Survey (USGS) forecasted with a high degree of confidence that the next earthquake of a similar magnitude would occur before 1993. The experiment is largely considered a failure—the earthquake didn’t come until 2004. 

Instances of regular intervals between earthquakes of similar magnitudes have been noted in other places, including Hawaii, but these are the exception, not the rule. Far more often, recurrence intervals are given as averages with large margins of error. For areas prone to large earthquakes, these intervals can be on the scale of hundreds of years, with uncertainty bars that also span hundreds of years. Clearly, this method of forecasting is far from an exact science. 

Tom Heaton, a geophysicist at Caltech and a former senior scientist at the USGS, is skeptical that we will ever be able to predict earthquakes. He treats them largely as stochastic processes, meaning we can attach probabilities to events, but we can’t forecast them with any accuracy. 

“In terms of physics, it’s a chaotic system,” Heaton says. Underlying it all is significant evidence that Earth’s behavior is ordered and deterministic. But without good knowledge of what’s happening under the ground, it’s impossible to intuit any sense of that order. “Sometimes when you say the word ‘chaos,’ people think [you] mean it’s a random system,” he says. “Chaotic means that it’s so complicated you cannot make predictions.” 

But as scientists’ understanding of what’s happening inside Earth’s crust evolves and their tools become more advanced, it’s not unreasonable to expect that their ability to make predictions will improve. 

Slow shakes

Given how little we can quantify about what’s going on in the planet’s interior, it makes sense that earthquake prediction has long seemed out of the question. But in the early 2000s, two discoveries began to open up the possibility. 

First, seismologists discovered a strange, low-amplitude seismic signal in a tectonic region of southwest Japan. It would last from hours up to several weeks and occurred at somewhat regular intervals; it wasn’t like anything they’d seen before. They called it tectonic tremor.

Meanwhile, geodesists studying the Cascadia subduction zone, a massive stretch off the coast of the US Pacific Northwest where one plate is diving under another, found evidence of times when part of the crust slowly moved in the opposite of its usual direction. This phenomenon, dubbed a slow slip event, happened in a thin section of Earth’s crust located beneath the zone that produces regular earthquakes, where higher temperatures and pressures have more impact on the behavior of the rocks and the way they interact.

The scientists studying Cascadia also observed the same sort of signal that had been found in Japan and determined that it was occurring at the same time and in the same place as these slow slip events. A new type of earthquake had been discovered. Like regular earthquakes, these transient events—slow earthquakes—redistribute stress in the crust, but they can take place over all kinds of time scales, from seconds to years. In some cases, as in Cascadia, they occur regularly, but in other areas they are isolated incidents.

Scientists subsequently found that during a slow earthquake, the risk of regular earthquakes can increase, particularly in subduction zones. The locked part of the fault that produces earthquakes is basically being stressed both by regular plate motion and by the irregular periodic backward motion produced by slow earthquakes, at depths greater than where earthquakes begin. These elusive slow events became the subject of my PhD research, but (as is often the case with graduate work) I certainly didn’t resolve the problem. To this day, it is unclear what exact mechanisms drive this kind of activity.

Could we nevertheless use slow earthquakes to predict regular earthquakes? Since their discovery, almost every big earthquake has been followed by several papers showing that it was preceded by a slow earthquake. The magnitude 9 Tohoku-Oki earthquake, which occurred in Japan in 2011, was preceded by not one but two slow ones. There are exceptions: for example, despite attempts to prove otherwise, there is still no evidence that a slow earthquake preceded the 2004 earthquake in Sumatra, Indonesia, which created a devastating tsunami that killed more than 200,000 people. What’s more, a slow earthquake is not always followed by a regular earthquake. It’s not known whether something distinguishes those that could be precursors from those that aren’t. 

It may be that some kind of distinctive process occurs along the fault in the hours leading up to a big quake. Last summer a former colleague of mine, Quentin Bletery, and his colleague Jean-Mathieu Nocquet, both at Géoazur, a multidisciplinary research lab in the south of France, published the results of an analysis of data on crustal deformation in the hours leading up to 90 larger earthquakes. They found that in the two hours or so preceding an earthquake, the crust along the fault begins to deform at a faster rate in the direction of the earthquake rupture until the instant the quake begins. What this tells us, Bletery says, is that an acceleration process occurs along the fault ahead of the motion of the earthquake—something that resembles a slow earthquake.

“This does support the assumption that there’s something happening before. So we do have that,” he says. “But most likely, it’s not physically possible to play with the topic of prediction. We just don’t have the instruments.” In other words, the precursors may be there, but we’re currently unable to measure their presence well enough to single them out before an earthquake strikes. 

Bletery and Nocquet conducted their study using traditional statistical analysis of GPS data; such data might contain information that’s beyond the reach of our traditional models and frames of reference. Seismologists are now applying machine learning in ways they haven’t before. Though it is early days yet, the machine-learning approach could reveal hidden structures and causal links in what would otherwise look like a jumble of data. 

Finding signals in the noise

Earthquake researchers have applied machine learning in a variety of ways. Some, like Mostafa Mousavi and Gregory Beroza of Stanford, have studied how to use it on seismic data from a single seismic station to predict the magnitude of an earthquake, which can be tremendously useful for early warning systems and may also help clarify what factors determine an earthquake’s size.

Brendan Meade, a professor of earth and planetary science at Harvard, is able to predict the locations of aftershocks using neural networks. Zachary Ross at Caltech and others are using deep learning to pick seismic waves out of data even with high levels of background noise, which could lead to the detection of more earthquakes.

Paul Johnson of the Los Alamos National Laboratory in New Mexico, who became something between a mentor and a friend after we met during my first postdoc, is applying machine learning to help make sense of data from earthquakes generated in the lab. 

There are a number of ways to create laboratory earthquakes. One relatively common method involves placing a rock sample, cut down the center to simulate a fault, inside a metal framework that puts it under a confining pressure. Localized sensors measure what happens as the sample undergoes deformation.  

an old church seen standing past a massive pile of rubble in the foreground
In Italy, increased agitation among animals was linked to strong earthquakes, including the deadly Norcia quake in 2016.
SIPA USA VIA AP

In 2017, a study out of Johnson’s lab showed that machine learning could help predict with remarkable accuracy how long it would take for the fault the researchers created to start quaking. Unlike many methods humans use to forecast earthquakes, this one uses no historical data—it relies only on the vibrations coming from the fault. Crucially, what human researchers had discounted as low-­amplitude noise turned out to be the signal that allowed machine learning to make its predictions. 

In the field, Johnson’s team applied these findings to seismic data from Cascadia, where they identified a continuous acoustic signal coming from the subduction zone that corresponds to the rate at which that fault is moving through the slow earthquake cycle—a new source of data for models of the region. “[Machine learning] allows you to make these correlations you didn’t know existed. And in fact, some of them are remarkably surprising,” Johnson says. 

Machine learning could also help us create more data to study. By identifying perhaps as many as 10 times more earthquakes in seismic data than we are aware of, Beroza, Mousavi, and Margarita Segou, a researcher at the British Geological Survey, determined that machine learning is useful for creating more robust databases of earthquakes that have occurred; they published their findings in a 2021 paper for Nature Communications. These improved data sets can help us—and machines—understand earthquakes better.

“You know, there’s tremendous skepticism in our community, with good reason,” Johnson says. “But I think this is allowing us to see and analyze data and realize what those data contain in ways we never could have imagined.”

Animal senses

While some researchers are relying on the most current technology, others are looking back at history to formulate some pretty radical studies based on animals. One of the shirts I collected over 10 years of attending geophysics conferences features the namazu, a giant mythical catfish that in Japan was believed to generate earthquakes by swimming beneath Earth’s crust. 

The creature is seismology’s unofficial mascot. Prior to the 1855 Edo earthquake in Japan, a fisherman recorded some atypical catfish activity in a river. In a 1933 paper published in Nature, two Japanese seismologists reported that catfish in enclosed glass chambers behaved with increasing agitation before earthquakes—a phenomenon said to predict them with 80% accuracy. 

The closer the animals were to the earthquake’s source, the more advance warning their seemingly panicked behavior could provide.

Catfish are not the only ones. Records dating back as early as 373 BCE show that many species, including rats and snakes, left a Greek city days before it was destroyed by an earthquake. Reports noted that horses cried and some fled San Francisco in the early morning hours before the 1906 earthquake.

Martin Wikelski, a research director at the Max Planck Institute of Animal Behavior, and his colleagues have been studying the possibility of using the behavior of domesticated animals to help predict earthquakes. In 2016 and 2017 in central Italy, the team attached motion detectors to dogs, cows, and sheep. They determined a baseline level of movement and set a threshold for what would indicate agitated behavior: a 140% increase in motion relative to the baseline for periods lasting longer than 45 minutes. They found that the animals became agitated before eight of nine earthquakes greater than a magnitude 4, including the deadly magnitude 6.6 Norcia earthquake of 2016. And there were no false positives—no times when the animals were agitated and an earthquake did not occur. They also found that the closer the animals were to the earthquake’s source, the more advance warning their seemingly panicked behavior could provide.

Wikelski has a hypothesis about this phenomenon: “My take on the whole thing would be that it could be something that’s airborne, and the only thing that I can think of is really the ionized [electrically charged] particles in the air.”

Electromagnetism isn’t an outlandish theory. Earthquake lights—glowing emissions from a fault that resemble the aurora borealis—have been observed during or before numerous earthquakes, including the 2008 Sichuan earthquake in China, the 2009 L’Aquila earthquake in Italy, the 2017 Mexico City earthquake, and even the September 2023 earthquake in Morocco

Friedemann Freund, a scientist at NASA’s Ames Research Center, has been studying these lights for decades and attributes them to electrical charges that are activated by motion along the fault in certain types of rocks, such as gabbros and basalts. It is akin to rubbing your sock on the carpet and freeing up electrons that allow you to shock someone. 

Some researchers have proposed different mechanisms, while others discount the idea that earthquake lights are in any way related to earthquakes. Unfortunately, measuring electromagnetic fields in Earth’s crust or surface is not straightforward. We don’t have instruments that can sample large areas of an electromagnetic field. Without knowing in advance where an earthquake will be, it is challenging, if not impossible, to know where to install instruments to make measurements. 

At present, the most effective way to measure such fields in the ground is to set up probes where there is consistent groundwater flow. Some work has been done to look for electromagnetic and ionospheric disturbances caused by seismic and pre-seismic activity in satellite data, though the research is still at a very early stage.

Small movements

Some of science’s biggest paradigm shifts started without any understanding of an underlying mechanism. The idea that continents move, for example—the basic phenomenon at the heart of plate tectonics—was proposed by Alfred Wegener in 1912. His theory was based primarily on the observation that the coastlines of Africa and South America match, as if they would fit together like puzzle pieces. But it was hotly contested. He was missing an essential ingredient that is baked into the ethos of modern science—the why. It wasn’t until the 1960s that the theory of plate tectonics was formalized, after evidence was found of Earth’s crust being created and destroyed, and at last the mechanics of the phenomenon were understood. 

In all those years in between, a growing number of people looked at the problem from different angles. The paradigm was shifting. Wegener had set the wheels of change in motion.

Perhaps that same sort of shift is happening now with earthquake prediction. It may be decades before we can look back on this period in earthquake research with certainty and understand its role in advancing the field. But some, like Johnson, are hopeful. “I do think it could be the beginning of something like the plate tectonics revolution,” he says. “We might be seeing something similar.” 

Allie Hutchison is a writer based in Porto, Portugal.

Four trends that changed AI in 2023

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

This has been one of the craziest years in AI in a long time: endless product launches, boardroom coups, intense policy debates about AI doom, and a race to find the next big thing. But we’ve also seen concrete tools and policies aimed at getting the AI sector to behave more responsibly and hold powerful players accountable. That gives me a lot of hope for the future of AI. 

Here’s what 2023 taught me: 

1. Generative AI left the lab with a vengeance, but it’s not clear where it will go next

The year started with Big Tech going all in on generative AI. The runaway success of OpenAI’s ChatGPT prompted every major tech company to release its own version. This year might go down in history as the year we saw the most AI launches: Meta’s LLaMA 2, Google’s Bard chatbot and Gemini, Baidu’s Ernie Bot, OpenAI’s GPT-4, and a handful of other models, including one from a French open-source challenger, Mistral. 

But despite the initial hype, we haven’t seen any AI applications become an overnight success. Microsoft and Google pitched powerful AI-powered search, but it turned out to be more of a dud than a killer app. The fundamental flaws in language models, such as the fact that they frequently make stuff up, led to some embarrassing (and, let’s be honest, hilarious) gaffes. Microsoft’s Bing would frequently reply to people’s questions with conspiracy theories, and suggested that a New York Times reporter leave his wife. Google’s Bard generated factually incorrect answers for its marketing campaign, which wiped $100 billion off the company’s share price.

There is now a frenetic hunt for a popular AI product that everyone will want to adopt. Both OpenAI and Google are experimenting with allowing companies and developers to create customized AI chatbots and letting people build their own applications using AI—no coding skills needed. Perhaps generative AI will end up embedded in boring but useful tools to help us boost our productivity at work. It might take the form of AI assistants—maybe with voice capabilities—and coding support. Next year will be crucial in determining the real value of generative AI.

2. We learned a lot about how language models actually work, but we still know very little

Even though tech companies are rolling out large language models into products at a frenetic pace, there is still a lot we don’t know about how they work. They make stuff up and have severe gender and ethnic biases. This year we also found out that different language models generate texts with different political biases, and that they make great tools for hacking people’s private information. Text-to-image models can be prompted to spit out copyrighted images and pictures of real people, and they can easily be tricked into generating disturbing images. It’s been great to see so much research into the flaws of these models, because this could take us a step closer to understanding why they behave the way they do, and ultimately fix them.

Generative models can be very unpredictable, and this year there were lots of attempts to try to make them behave as their creators want them to. OpenAI shared that it uses a technique called reinforcement learning from human feedback, which uses feedback from users to help guide ChatGPT to more desirable answers. A study from the AI lab Anthropic showed how simple natural-language instructions can steer large language models to make their results less toxic. But sadly, a lot of these attempts end up being quick fixes rather than permanent ones. Then there are misguided approaches like banning seemingly innocuous words such as “placenta” from image-generating AI systems to avoid producing gore. Tech companies come up with workarounds like these because they don’t know why models generate the content they do. 

We also got a better sense of AI’s true carbon footprint. Generating an image using a powerful AI model takes as much energy as fully charging your smartphone, researchers at the AI startup Hugging Face and Carnegie Mellon University found. Until now, the exact amount of energy generative AI uses has been a missing piece of the puzzle. More research into this could help us shift the way we use AI to be more sustainable. 

3. AI doomerism went mainstream

Chatter about the possibility that AI poses an existential risk to humans became familiar this year. Hundreds of scientists, business leaders, and policymakers have spoken up, from deep-learning pioneers Geoffrey Hinton and Yoshua Bengio to the CEOs of top AI firms, such as Sam Altman and Demis Hassabis, to the California congressman Ted Lieu and the former president of Estonia Kersti Kaljulaid.

Existential risk has become one of the biggest memes in AI. The hypothesis is that one day we will build an AI that is far smarter than humans, and this could lead to grave consequences. It’s an ideology championed by many in Silicon Valley, including Ilya Sutskever, OpenAI’s chief scientist, who played a pivotal role in ousting OpenAI CEO Sam Altman (and then reinstating him a few days later). 

But not everyone agrees with this idea. Meta’s AI leaders Yann LeCun and Joelle Pineau have said that these fears are “ridiculous” and the conversation about AI risks has become “unhinged.” Many other power players in AI, such as researcher Joy Buolamwini, say that focusing on hypothetical risks distracts from the very real harms AI is causing today. 

Nevertheless, the increased attention on the technology’s potential to cause extreme harm has prompted many important conversations about AI policy and animated lawmakers all over the world to take action. 

4. The days of the AI Wild West are over

Thanks to ChatGPT, everyone from the US Senate to the G7 was talking about AI policy and regulation this year. In early December, European lawmakers wrapped up a busy policy year when they agreed on the AI Act, which will introduce binding rules and standards on how to develop the riskiest AI more responsibly. It will also ban certain “unacceptable” applications of AI, such as police use of facial recognition in public places. 

The White House, meanwhile, introduced an executive order on AI, plus voluntary commitments from leading AI companies. Its efforts aimed to bring more transparency and standards for AI and gave a lot of freedom to agencies to adapt AI rules to fit their sectors. 

One concrete policy proposal that got a lot of attention was watermarks—invisible signals in text and images that can be detected by computers, in order to flag AI-generated content. These could be used to track plagiarism or help fight disinformation, and this year we saw research that succeeded in applying them to AI-generated text and images.

It wasn’t just lawmakers that were busy, but lawyers too. We saw a record number of  lawsuits, as artists and writers argued that AI companies had scraped their intellectual property without their consent and with no compensation. In an exciting counter-offensive, researchers at the University of Chicago developed Nightshade, a new data-poisoning tool that lets artists fight back against generative AI by messing up training data in ways that could cause serious damage to image-generating AI models. There is a resistance brewing, and I expect more grassroots efforts to shift tech’s power balance next year. 

Deeper Learning

Now we know what OpenAI’s superalignment team has been up to

OpenAI has announced the first results from its superalignment team, its in-house initiative dedicated to preventing a superintelligence—a hypothetical future AI that can outsmart humans—from going rogue. The team is led by chief scientist Ilya Sutskever, who was part of the group that just last month fired OpenAI’s CEO, Sam Altman, only to reinstate him a few days later.

Business as usual: Unlike many of the company’s announcements, this heralds no big breakthrough. In a low-key research paper, the team describes a technique that lets a less powerful large language model supervise a more powerful one—and suggests that this might be a small step toward figuring out how humans might supervise superhuman machines. Read more from Will Douglas Heaven

Bits and Bytes

Google DeepMind used a large language model to solve an unsolvable math problem
In a paper published in Nature, the company says it is the first time a large language model has been used to discover a solution to a long-standing scientific puzzle—producing verifiable and valuable new information that did not previously exist. (MIT Technology Review)

This new system can teach a robot a simple household task within 20 minutes
A new open-source system, called Dobb-E, was trained using data collected from real homes. It can help to teach a robot how to open an air fryer, close a door, or straighten a cushion, among other tasks. It could also help the field of robotics overcome one of its biggest challenges: a lack of training data.  (MIT Technology Review)

ChatGPT is turning the internet into plumbing
German media giant Axel Springer, which owns Politico and Business Insider, announced a partnership with OpenAI, in which the tech company will be able to use its news articles as training data and the news organizations will be able to use ChatGPT to do summaries of news. This column has a clever point: tech companies are increasingly becoming gatekeepers for online content, and journalism is just “plumbing for a digital faucet.” (The Atlantic)

Meet the former French official pushing for looser AI rules after joining startup Mistral
A profile of Mistral AI cofounder Cédric O, who used to be France’s digital minister. Before joining France’s AI unicorn, he was a vocal proponent of strict laws for tech, but he lobbied hard against rules in the AI Act that would have restricted Mistral’s models. He was successful: the company’s models don’t meet the computing threshold set by the law, and its open-source models are also exempt from transparency obligations. (Bloomberg

Navigating a shifting customer-engagement landscape with generative AI

One can’t step into the same river twice. This simple representation of change as the only constant was taught by the Greek philosopher Heraclitus more than 2000 years ago. Today, it rings truer than ever with the advent of generative AI. The emergence of generative AI is having a profound effect on today’s enterprises—business leaders face a rapidly changing technology that they need to grasp to meet evolving consumer expectations.

“Across all industries, customers are at the core, and tapping into their latent needs is one of the most important elements to sustain and grow a business,” says Akhilesh Ayer, executive vice president and global head of AI, analytics, data, and research practice at WNS Triange, a unit of WNS Global Services, a leading business process management company. “Generative AI is a new way for companies to realize this need.”

A strategic imperative

Generative AI’s ability to harness customer data in a highly sophisticated manner means enterprises are accelerating plans to invest in and leverage the technology’s capabilities. In a study titled “The Future of Enterprise Data & AI,” Corinium Intelligence and WNS Triange surveyed 100 global C-suite leaders and decision-makers specializing in AI, analytics, and data. Seventy-six percent of the respondents said that their organizations are already using or planning to use generative AI.

According to McKinsey, while generative AI will affect most business functions, “four of them will likely account for 75% of the total annual value it can deliver.” Among these are marketing and sales and customer operations. Yet, despite the technology’s benefits, many leaders are unsure about the right approach to take and mindful of the risks associated with large investments.

Mapping out a generative AI pathway

One of the first challenges organizations need to overcome is senior leadership alignment. “You need the necessary strategy; you need the ability to have the necessary buy-in of people,” says Ayer. “You need to make sure that you’ve got the right use case and business case for each one of them.” In other words, a clearly defined roadmap and precise business objectives are as crucial as understanding whether a process is amenable to the use of generative AI.

The implementation of a generative AI strategy can take time. According to Ayer, business leaders should maintain a realistic perspective on the duration required for formulating a strategy, conduct necessary training across various teams and functions, and identify the areas of value addition. And for any generative AI deployment to work seamlessly, the right data ecosystems must be in place.

Ayer cites WNS Triange’s collaboration with an insurer to create a claims process by leveraging generative AI. Thanks to the new technology, the insurer can immediately assess the severity of a vehicle’s damage from an accident and make a claims recommendation based on the unstructured data provided by the client. “Because this can be immediately assessed by a surveyor and they can reach a recommendation quickly, this instantly improves the insurer’s ability to satisfy their policyholders and reduce the claims processing time,” Ayer explains.

All that, however, would not be possible without data on past claims history, repair costs, transaction data, and other necessary data sets to extract clear value from generative AI analysis. “Be very clear about data sufficiency. Don’t jump into a program where eventually you realize you don’t have the necessary data,” Ayer says.

The benefits of third-party experience

Enterprises are increasingly aware that they must embrace generative AI, but knowing where to begin is another thing. “You start off wanting to make sure you don’t repeat mistakes other people have made,” says Ayer. An external provider can help organizations avoid those mistakes and leverage best practices and frameworks for testing and defining explainability and benchmarks for return on investment (ROI).

Using pre-built solutions by external partners can expedite time to market and increase a generative AI program’s value. These solutions can harness pre-built industry-specific generative AI platforms to accelerate deployment. “Generative AI programs can be extremely complicated,” Ayer points out. “There are a lot of infrastructure requirements, touch points with customers, and internal regulations. Organizations will also have to consider using pre-built solutions to accelerate speed to value. Third-party service providers bring the expertise of having an integrated approach to all these elements.”

Ayer offers the example of WNS Triange helping a travel intermediary use generative AI to deal with customer inquiries about airline rescheduling, cancellations, and other itinerary complications. “Our solution is immediately able to go into a thousand policy documents, pick out the policy parameters relevant to the query… and then come back quickly not only with the response but with a nice, summarized, human-like response,” he says.

In another example, Ayer shares that his company helped a global retailer create generative AI–driven designs for personalized gift cards. “The customer experience goes up tremendously,” he says.

Hurdles in the generative AI journey

As with any emerging technology, however, there are organizational, technical, and implementation barriers to overcome when adopting generative AI.

Organizational:  One of the major hurdles businesses can face is people. “There is often immediate resistance to the adoption of generative AI because it affects the way people work on a daily basis,” says Ayer.

As a result, securing internal buy-in from all teams and being mindful of a skills gap is a must. Additionally, the ability to create a business case for investment—and getting buy-in from the C-suite—will help expedite the adoption of generative AI tools.

Technical: The second set of obstacles relates to large language models (LLMs) and mechanisms to safeguard against hallucinations and bias and ensure data quality. “Companies need to figure out if generative AI can solve the whole problem or if they still need human input to validate the outputs from LLM models,” Ayer explains. At the same time, organizations must ask whether the generative AI models being used have been appropriately trained within the customer context or with the enterprise’s own data and insights. If not, there is a high chance that the response will be incorrect. Another related challenge is bias: If the underlying data has certain biases, the modeling of the LLM could be unfair. “There have to be mechanisms to address that,” says Ayer. Other issues, such as data quality, output authenticity, and explainability, also must be addressed.

Implementation: The final set of challenges relates to actual implementation. The cost of implementation can be significant, especially if companies cannot orchestrate a viable solution, says Ayer. In addition, the right infrastructure and people must be in place to avoid resource constraints. And users must be convinced that the output will be relevant and of high quality, so as to gain their acceptance for the program’s implementation.

Lastly, privacy and ethical issues must be addressed. The Corinium Intelligence and WNS Triange survey showed that almost 72% of respondents were concerned about ethical AI decision-making.

The focus of future investment

The entire ecosystem of generative AI is moving quickly. Enterprises must be agile and adapt quickly to change to ensure customer expectations are met and maintain a competitive edge. While it is almost impossible to anticipate what’s next with such a new and fast-developing technology, Ayer says that organizations that want to harness the potential of generative AI are likely to increase investment in three areas:

  • Data modernization, data management, data quality, and governance: To ensure underlying data is correct and can be leveraged.
  • Talent and workforce: To meet demand, training, apprenticeships, and injection of fresh talent or leveraging market-ready talent from service providers will be required.
  • Data privacy solutions and mechanisms: To ensure privacy is maintained, C-suite leaders must also keep pace with relevant laws and regulations across relevant jurisdictions.

However, it shouldn’t be a case of throwing everything at the wall and seeing what sticks. Ayer advises that organizations examine ROI from the effectiveness of services or products provided to customers. Business leaders must clearly demonstrate and measure a marked improvement in customer satisfaction levels using generative AI–based interventions.

“Along with a defined generative AI strategy, companies need to understand how to apply and build use cases, how to execute them at scale and speed to market, and how to measure their success,” says Ayer. Leveraging generative AI for customer engagement is typically a multi-pronged approach, and a successful partnership can help with every stage.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff.

This new system can teach a robot a simple household task within 20 minutes

A new system that teaches robots a domestic task in around 20 minutes could help the field of robotics overcome one of its biggest challenges: a lack of training data

The open-source system, called Dobb-E, was trained using data collected from real homes. It can help to teach a robot how to open an air fryer, close a door, or straighten a cushion, among other tasks. 

While other types of AI, such as large language models, are trained on huge repositories of data scraped from the internet, the same can’t be done with robots, because the data needs to be physically collected. This makes it a lot harder to build and scale training databases.  

Similarly, while it’s relatively easy to train robots to execute tasks inside a laboratory, these conditions don’t necessarily translate to the messy unpredictability of a real home. 

To combat these problems, the team came up with a simple, easily replicable way to collect the data needed to train Dobb-E—using an iPhone attached to a reacher-grabber stick, the kind typically used to pick up trash. Then they set the iPhone to record videos of what was happening.

Volunteers in 22 homes in New York completed certain tasks using the stick, including opening and closing doors and drawers, turning lights on and off, and placing tissues in the trash. The iPhones’ lidar systems, motion sensors, and gyroscopes were used to record data on movement, depth, and rotation—important information when it comes to training a robot to replicate the actions on its own.

After they’d collected just 13 hours’ worth of recordings in total, the team used the data to train an AI model to instruct a robot in how to carry out the actions. The model used self-supervised learning techniques, which teach neural networks to spot patterns in data sets by themselves, without being guided by labeled examples.

The next step involved testing how reliably a commercially available robot called Stretch, which consists of a wheeled unit, a tall pole, and a retractable arm, was able to use the AI system to execute the tasks. An iPhone held in a 3D-printed mount was attached to Stretch’s arm to replicate the setup on the stick.

The researchers tested the robot in 10 homes in New York over 30 days, and it completed 109 household tasks with an overall success rate of 81%. Each task typically took Dobb-E around 20 minutes to learn: five minutes of demonstration from a human using the stick and attached iPhone, followed by 15 minutes of fine-tuning, when the system compared its previous training with the new demonstration. 

Once the fine-tuning was complete, the robot was able to complete simple tasks like pouring from a cup, opening blinds and shower curtains, or pulling board-game boxes from a shelf. It could also perform multiple actions in quick succession, such as placing a can in a recycling bag and then lifting the bag. 

However, not every task was successful. The system was confused by reflective surfaces like mirrors. Also, because the robot’s center of gravity is low, tasks that require pulling something heavy at height, like opening fridge doors, proved too risky to attempt. 

The research represents tangible progress for the home robotics field, says Charlie C. Kemp, cofounder of the robotics firm Hello Robot and a former associate professor at Georgia Tech. Although the Dobb-E team used Hello Robot’s research robot, Kemp was not involved in the project.

“The future of home robots is really coming. It’s not just some crazy dream anymore,” he says. “Scaling up data has always been a challenge in robotics, and this is a very creative, clever approach to that problem.”

To date, Roomba and other robotic vacuum cleaners are the only real commercial home robot successes, says Jiajun Wu, an assistant professor of computer science at Stanford University who was not involved in the research. Their job is easier because Roombas don’t interact with objects—in fact, their aim is to avoid them. It’s much more challenging to develop home robots capable of doing a wider range of tasks, which is what this research could help advance. 

The NYU research team has made all elements of the project open source, and they’re hoping others will download the code and help expand the range of tasks that robots running Dobb-E will be able to achieve.

“Our hope is that when we get more and more data, at some point when Dobb-E sees a new home, you don’t have to show it more examples,” says Lerrel Pinto, a computer science researcher at New York University who worked on the project. 

“We want to get to the point when we don’t have to teach the robot new tasks, because it already knows all the tasks in most houses,” he says.

Google DeepMind used a large language model to solve an unsolvable math problem

Google DeepMind has used a large language model to crack a famous unsolved problem in pure mathematics. In a paper published in Nature today, the researchers say it is the first time a large language model has been used to discover a solution to a long-standing scientific puzzle—producing verifiable and valuable new information that did not previously exist. “It’s not in the training data—it wasn’t even known,” says coauthor Pushmeet Kohli, vice president of research at Google DeepMind.

Large language models have a reputation for making things up, not for providing new facts. Google DeepMind’s new tool, called FunSearch, could change that. It shows that they can indeed make discoveries—if they are coaxed just so, and if you throw out the majority of what they come up with.

FunSearch (so called because it searches for mathematical functions, not because it’s fun) continues a streak of discoveries in fundamental math and computer science that DeepMind has made using AI. First AlphaTensor found a way to speed up a calculation at the heart of many different kinds of code, beating a 50-year record. Then AlphaDev found ways to make key algorithms used trillions of times a day run faster.

Yet those tools did not use large language models. Built on top of DeepMind’s game-playing AI AlphaZero, both solved math problems by treating them as if they were puzzles in Go or chess. The trouble is that they are stuck in their lanes, says Bernardino Romera-Paredes, a researcher at the company who worked on both AlphaTensor and FunSearch: “AlphaTensor is great at matrix multiplication, but basically nothing else.”

FunSearch takes a different tack. It combines a large language model called Codey, a version of Google’s PaLM 2 that is fine-tuned on computer code, with other systems that reject incorrect or nonsensical answers and plug good ones back in.

“To be very honest with you, we have hypotheses, but we don’t know exactly why this works,” says Alhussein Fawzi, a research scientist at Google DeepMind. “In the beginning of the project, we didn’t know whether this would work at all.”

The researchers started by sketching out the problem they wanted to solve in Python, a popular programming language. But they left out the lines in the program that would specify how to solve it. That is where FunSearch comes in. It gets Codey to fill in the blanks—in effect, to suggest code that will solve the problem.

A second algorithm then checks and scores what Codey comes up with. The best suggestions—even if not yet correct—are saved and given back to Codey, which tries to complete the program again. “Many will be nonsensical, some will be sensible, and a few will be truly inspired,” says Kohli. “You take those truly inspired ones and you say, ‘Okay, take these ones and repeat.’”

After a couple of million suggestions and a few dozen repetitions of the overall process—which took a few days—FunSearch was able to come up with code that produced a correct and previously unknown solution to the cap set problem, which involves finding the largest size of a certain type of set. Imagine plotting dots on graph paper. The cap set problem is like trying to figure out how many dots you can put down without three of them ever forming a straight line.

It’s super niche, but important. Mathematicians do not even agree on how to solve it, let alone what the solution is. (It is also connected to matrix multiplication, the computation that AlphaTensor found a way to speed up.) Terence Tao at the University of California, Los Angeles, who has won many of the top awards in mathematics, including the Fields Medal, called the cap set problem “perhaps my favorite open question” in a 2007 blog post.

Tao is intrigued by what FunSearch can do. “This is a promising paradigm,” he says. “It is an interesting way to leverage the power of large language models.”

A key advantage that FunSearch has over AlphaTensor is that it can, in theory, be used to find solutions to a wide range of problems. That’s because it produces code—a recipe for generating the solution, rather than the solution itself. Different code will solve different problems. FunSearch’s results are also easier to understand. A recipe is often clearer than the weird mathematical solution it produces, says Fawzi.

To test its versatility, the researchers used FunSearch to approach another hard problem in math: the bin packing problem, which involves trying to pack items into as few bins as possible. This is important for a range of applications in computer science, from data center management to e-commerce. FunSearch came up with a way to solve it that’s faster than human-devised ones.

Mathematicians are “still trying to figure out the best way to incorporate large language models into our research workflow in ways that harness their power while mitigating their drawbacks,” Tao says. “This certainly indicates one possible way forward.”

Now we know what OpenAI’s superalignment team has been up to

OpenAI has announced the first results from its superalignment team, the firm’s in-house initiative dedicated to preventing a superintelligence—a hypothetical future computer that can outsmart humans—from going rogue.

Unlike many of the company’s announcements, this heralds no big breakthrough. In a low-key research paper, the team describes a technique that lets a less powerful large language model supervise a more powerful one—and suggests that this might be a small step toward figuring out how humans might supervise superhuman machines.  

Less than a month after OpenAI was rocked by a crisis when its CEO, Sam Altman, was fired by its oversight board (in an apparent coup led by chief scientist Ilya Sutskever) and then reinstated three days later, the message is clear: it’s back to business as usual.

Yet OpenAI’s business is not usual. Many researchers still question whether machines will ever match human intelligence, let alone outmatch it. OpenAI’s team takes machines’ eventual superiority as given. “AI progress in the last few years has been just extraordinarily rapid,” says Leopold Aschenbrenner, a researcher on the superalignment team. “We’ve been crushing all the benchmarks, and that progress is continuing unabated.”

For Aschenbrenner and others at the company, models with human-like abilities are just around the corner. “But it won’t stop there,” he says. “We’re going to have superhuman models, models that are much smarter than us. And that presents fundamental new technical challenges.”

In July, Sutskever and fellow OpenAI scientist Jan Leike set up the superalignment team to address those challenges. “I’m doing it for my own self-interest,” Sutskever told MIT Technology Review in September. “It’s obviously important that any superintelligence anyone builds does not go rogue. Obviously.”

Amid speculation that Altman was fired for playing fast and loose with his company’s approach to AI safety, Sutskever’s superalignment team loomed behind the headlines. Many have been waiting to see exactly what it has been up to. 

Dos and don’ts

The question the team wants to answer is how to rein in, or “align,” hypothetical future models that are far smarter than we are, known as superhuman models. Alignment means making sure a model does what you want it to do and does not do what you don’t want it to do. Superalignment applies this idea to superhuman models.

One of the most widespread techniques used to align existing models is called reinforcement learning via human feedback. In a nutshell, human testers score a model’s responses, upvoting behavior that they want to see and downvoting behavior they don’t. This feedback is then used to train the model to produce only the kind of responses that human testers liked. This technique is a big part of what makes ChatGPT so engaging.   

The problem is that it requires humans to be able to tell what is and isn’t desirable behavior in the first place. But a superhuman model—the idea goes—might do things that a human tester can’t understand and thus would not be able to score. (It might even try to hide its true behavior from humans, Sutskever told us.)  

OpenAI’s approach to the superalignment problem.
OPENAI

The researchers point out that the problem is hard to study because superhuman machines do not exist. So they used stand-ins. Instead of looking at how humans could supervise superhuman machines, they looked at how GPT-2, a model that OpenAI released five years ago, could supervise GPT-4, OpenAI’s latest and most powerful model. “If you can do that, it might be evidence that you can use similar techniques to have humans supervise superhuman models,” says Collin Burns, another researcher on the superalignment team.   

The team took GPT-2 and trained it to perform a handful of different tasks, including a set of chess puzzles and 22 common natural-language-processing tests that assess inference, sentiment analysis, and so on. They used GPT-2’s responses to those tests and puzzles to train GPT-4 to perform the same tasks. It’s as if a 12th grader were taught how to do a task by a third grader. The trick was to do it without GPT-4 taking too big a hit in performance.

The results were mixed. The team measured the gap in performance between GPT-4 trained on GPT-2’s best guesses and GPT-4 trained on correct answers. They found that GPT-4 trained by GPT-2 performed 20% to 70% better than GPT-2 on the language tasks but did less well on the chess puzzles.

The fact that GPT-4 outdid its teacher at all is impressive, says team member Pavel Izmailov: “This is a really surprising and positive result.” But it fell far short of what it could do by itself, he says. They conclude that the approach is promising but needs more work.

“It is an interesting idea,” says Thilo Hagendorff, an AI researcher at the University of Stuttgart in Germany who works on alignment. But he thinks that GPT-2 might be too dumb to be a good teacher. “GPT-2 tends to give nonsensical responses to any task that is slightly complex or requires reasoning,” he says. Hagendorff would like to know what would happen if GPT-3 were used instead.

He also notes that this approach does not address Sutskever’s hypothetical scenario in which a superintelligence hides its true behavior and pretends to be aligned when it isn’t. “Future superhuman models will likely possess emergent abilities which are unknown to researchers,” says Hagendorff. “How can alignment work in these cases?”

But it is easy to point out shortcomings, he says. He is pleased to see OpenAI moving from speculation to experiment: “I applaud OpenAI for their effort.”

OpenAI now wants to recruit others to its cause. Alongside this research update, the company announced a new $10 million money pot that it plans to use to fund people working on superalignment. It will offer grants of up to $2 million to university labs, nonprofits, and individual researchers and one-year fellowships of $150,000 to graduate students. “We’re really excited about this,” says Aschenbrenner. “We really think there’s a lot that new researchers can contribute.”

Five things you need to know about the EU’s new AI Act

This story is from The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.

It’s done. It’s over. Two and a half years after it was first introduced—after months of lobbying and political arm-wrestling, plus grueling final negotiations that took nearly 40 hours—EU lawmakers have reached a deal over the AI Act. It will be the world’s first sweeping AI law. 

The AI Act was conceived as a landmark bill that would mitigate harm in areas where using AI poses the biggest risk to fundamental rights, such as health care, education, border surveillance, and public services, as well as banning uses that pose an “unacceptable risk.” 

“High risk” AI systems will have to adhere to strict rules that require risk-mitigation systems, high-quality data sets, better documentation, and human oversight, for example. The vast majority of AI uses, such as recommender systems and spam filters, will get a free pass. 

The AI Act is a major deal in that it will introduce important rules and enforcement mechanisms to a hugely influential sector that is currently a Wild West. 

Here are MIT Technology Review’s key takeaways: 

1. The AI Act ushers in important, binding rules on transparency and ethics

Tech companies love to talk about how committed they are to AI ethics. But when it comes to concrete measures, the conversation dries up. And anyway, actions speak louder than words. Responsible AI teams are often the first to see cuts during layoffs, and in truth, tech companies can decide to change their AI ethics policies at any time. OpenAI, for example, started off as an “open” AI research lab before closing up public access to its research to protect its competitive advantage, just like every other AI startup. 

The AI Act will change that. The regulation imposes legally binding rules requiring tech companies to notify people when they are interacting with a chatbot or with biometric categorization or emotion recognition systems. It’ll also require them to label deepfakes and AI-generated content, and design systems in such a way that AI-generated media can be detected. This is a step beyond the voluntary commitments that leading AI companies made to the White House to simply develop AI provenance tools, such as watermarking

The bill will also require all organizations that offer essential services, such as insurance and banking, to conduct an impact assessment on how using AI systems will affect people’s fundamental rights. 

2. AI companies still have a lot of wiggle room

When the AI Act was first introduced, in 2021, people were still talking about the metaverse. (Can you imagine!) 

Fast-forward to now, and in a post-ChatGPT world, lawmakers felt they had to take so-called foundation models—powerful AI models that can be used for many different purposes—into account in the regulation. This sparked intense debate over what sorts of models should be regulated, and whether regulation would kill innovation. 

The AI Act will require foundation models and AI systems built on top of them to draw up better documentation, comply with EU copyright law, and share more information about what data the model was trained on. For the most powerful models, there are extra requirements. Tech companies will have to share how secure and energy efficient their AI models are, for example. 

But here’s the catch: The compromise lawmakers found was to apply a stricter set of rules only the most powerful AI models, as categorized by the computing power needed to train them. And it will be up to companies to assess whether they fall under stricter rules. 

A European Commission official would not confirm whether the current cutoff would capture powerful models such as OpenAI’s GPT-4 or Google’s Gemini, because only the companies themselves know how much computing power was used to train their models. The official did say that as the technology develops, the EU could change the way it measures how powerful AI models are. 

3. The EU will become the world’s premier AI police

The AI Act will set up a new European AI Office to coordinate compliance, implementation, and enforcement. It will be the first body globally to enforce binding rules on AI, and the EU hopes this will help it become the world’s go-to tech regulator. The AI Act’s governance mechanism also includes a scientific panel of independent experts to offer guidance on the systemic risks AI poses, and how to classify and test models. 

The fines for noncompliance are steep: from 1.5% to 7% of a firm’s global sales turnover, depending on the severity of the offense and size of the company. 

Europe will also become the one of the first places in the world where citizens will be able to launch complaints about AI systems and receive explanations about how AI systems came to the conclusions that affect them. 

By becoming the first to formalize rules around AI, the EU retains its first-mover advantage. Much like the GDPR, the AI Act could become a global standard. Companies elsewhere that want to do business in the world’s second-largest economy will have to comply with the law. The EU’s rules also go a step further than ones introduced by the US, such as the White House executive order, because they are binding. 

4. National security always wins

Some AI uses are now completely banned in the EU: biometric categorization systems that use sensitive characteristics; untargeted scraping of facial images from the internet or CCTV footage to create facial recognition databases like Clearview AI; emotion recognition at work or in schools; social scoring; AI systems that manipulate human behavior; and AI that is used to exploit people’s vulnerabilities. 

Predictive policing is also banned, unless it is used with “clear human assessment and objective facts, which basically do not simply leave the decision of going after a certain individual in a criminal investigation only because an algorithm says so,” according to an EU Commission official.

However, the AI Act does not apply to AI systems that have been developed exclusively for military and defense uses. 

One of the bloodiest fights over the AI Act has always been how to regulate police use of biometric systems in public places, which many fear could lead to mass surveillance. While the European Parliament pushed for a near-total ban on the technology, some EU countries, such as France, have resisted this fiercely. They want to use it to fight crime and terrorism. 

European police forces will only be able to use biometric identification systems in public places if they get court approval first, and only for 16 different specific crimes, such as terrorism, human trafficking, sexual exploitation of children, and drug trafficking. Law enforcement authorities may also use high-risk AI systems that don’t pass European standards in “exceptional circumstances relating to public security.” 

5. What next? 

It might take weeks or even months before we see the final wording of the bill. The text still needs to go through technical tinkering, and has to be approved by European countries and the EU Parliament before it officially enters into law. 

Once it is in force, tech companies have two years to implement the rules. The bans on AI uses will apply after six months, and companies developing foundation models will have to comply with the law within one year. 

These robots know when to ask for help

There are two bowls on the kitchen table: one made of plastic, the other metal. You ask the robot to pick up the bowl and put it in the microwave. Which one will it choose?

A human might ask for clarification, but given the vague command, the robot may place the metal bowl in the microwave, causing sparks to fly.

A new training model, dubbed “KnowNo,” aims to address this problem by teaching robots to ask for our help when orders are unclear. At the same time, it ensures they seek clarification only when necessary, minimizing needless back-and-forth. The result is a smart assistant that tries to make sure it understands what you want without bothering you too much.

Andy Zeng, a research scientist at Google DeepMind who helped develop the new technique, says that while robots can be powerful in many specific scenarios, they are often bad at generalized tasks that require common sense.

For example, when asked to bring you a Coke, the robot needs to first understand that it needs to go into the kitchen, look for the refrigerator, and open the fridge door. Conventionally, these smaller substeps had to be manually programmed, because otherwise the robot would not know that people usually keep their drinks in the kitchen.

That’s something large language models (LLMs) could help to fix, because they have a lot of common-sense knowledge baked in, says Zeng. 

Now when the robot is asked to bring a Coke, an LLM, which has a generalized understanding of the world, can generate a step-by-step guide for the robot to follow.

The problem with LLMs, though, is that there’s no way to guarantee that their instructions are possible for the robot to execute. Maybe the person doesn’t have a refrigerator in the kitchen, or the fridge door handle is broken. In these situations, robots need to ask humans for help.

KnowNo makes that possible by combining large language models with statistical tools that quantify confidence levels. 

When given an ambiguous instruction like “Put the bowl in the microwave,” KnowNo first generates multiple possible next actions using the language model. Then it creates a confidence score predicting the likelihood that each potential choice is the best one.

These confidence estimates are sized up against a predetermined certainty threshold, which indicates exactly how confident or conservative the user wants a robot to be in its actions. For example, a robot with a success rate of 80% should make the correct decision at least 80% of the time.

This is useful in situations with varying degrees of risk, says Anirudha Majumdar, an assistant professor of mechanical and aerospace engineering at Princeton and the senior author of the study. 

You may want your cleaning robot to be more independent, despite a few mistakes here and there, so that you don’t have to supervise it too closely. But for medical applications, robots must be extremely cautious, with the highest level of success possible.

When there is more than one option for how to proceed, the robot pauses to ask for clarification instead of blindly continuing: “Which bowl should I pick up—the metal or the plastic one?”

KnownNo was tested on three robots in more than 150 different scenarios. Results showed that KnowNo-trained robots had more consistent success rates while needing less human assistance than those trained without the same statistical calculations. The paper describing the research was presented at the Conference on Robot Learning in November.

Because human language is often ambiguous, teaching robots to recognize and respond to uncertainty can improve their performance.

Studies show that people prefer robots that ask questions, says Dylan Losey, an assistant professor at Virginia Tech who specializes in human-robot interaction and was not involved in this research. When robots reach out for help, it increases transparency about how they’re deciding what to do, which leads to better interactions, he says. 

Allen Ren, a PhD student at Princeton and the study’s lead author, says there are several ways to improve KnowNo. Right now, it assumes robots’ vision is always reliable, which may not be the case with faulty sensors. Also, the model can be updated to factor in potential errors coming from human help.

AI’s ability to express uncertainty will make us trust robots more, says Majumdar. “Quantifying uncertainty is a missing piece in a lot of our systems,” he says. “It allows us to be more confident about how safe and successful the robots will be.”