A potential future conflict between Taiwan and China would be shaped by novel methods of drone warfare involving advanced underwater drones and increased levels of autonomy, according to a new war-gaming experiment by the think tank Center for a New American Security (CNAS).
The report comes as concerns about Beijing’s aggression toward Taiwan have been rising: China sent dozens of surveillance balloons over the Taiwan Strait in January during Taiwan’s elections, and in May, two Chinese naval ships entered Taiwan’s restricted waters. The US Department of Defense has said that preparing for potential hostilities is an “absolute priority,” though no such conflict is immediately expected.
The report’s authors detail a number of ways that use of drones in any South China Sea conflict would differ starkly from current practices, most notably in the war in Ukraine, often called the first full-scale drone war.
Differences from the Ukrainian battlefield
Since Russia invaded Ukraine in 2022, drones have been aiding in what military experts describe as the first three steps of the “kill chain”—finding, targeting, and tracking a target—as well as in delivering explosives. The drones have a short life span, since they are often shot down or made useless by frequency jamming devices that prevent pilots from controlling them. Quadcopters—the commercially available drones often used in the war—last just three flights on average, according to the report.
Drones like these would be far less useful in a possible invasion of Taiwan. “Ukraine-Russia has been a heavily land conflict, whereas conflict between the US and China would be heavily air and sea,” says Zak Kallenborn, a drone analyst and adjunct fellow with the Center for Strategic and International Studies, who was not involved in the report but agrees broadly with its projections. The small, off-the-shelf drones popularized in Ukraine have flight times too short for them to be used effectively in the South China Sea.
An underwater war
Instead, a conflict with Taiwan would likely make use of undersea and maritime drones. With Taiwan just 100 miles away from China’s mainland, the report’s authors say, the Taiwan Strait is where the first days of such a conflict would likely play out. The Zhu Hai Yun, China’s high-tech autonomous carrier, might send its autonomous underwater drones to scout for US submarines. The drones could launch attacks that, even if they did not sink the submarines, might divert the attention and resources of the US and Taiwan.
It’s also possible China would flood the South China Sea with decoy drone boats to “make it difficult for American missiles and submarines to distinguish between high-value ships and worthless uncrewed commercial vessels,” the authors write.
Though most drone innovation is not focused on maritime applications, these uses are not without precedent: Ukrainian forces drew attention for modifying jet skis to operate via remote control and using them to intimidate and even sink Russian vessels in the Black Sea.
More autonomy
Drones currently have very little autonomy. They’re typically human-piloted, and though some are capable of autopiloting to a fixed GPS point, that’s generally not very useful in a war scenario, where targets are on the move. But, the report’s authors say, autonomous technology is developing rapidly, and whichever nation possesses a more sophisticated fleet of autonomous drones will hold a significant edge.
What would that look like? Millions of defense research dollars are being spent in the US and China alike on swarming, a strategy where drones navigate autonomously in groups and accomplish tasks. The technology isn’t deployed yet, but if successful, it could be a game-changer in any potential conflict.
A sea-based conflict might also offer an easier starting ground for AI-driven navigation, because object recognition is easier on the “relatively uncluttered surface of the ocean” than on the ground, the authors write.
China’s advantages
A chief advantage for China in a potential conflict is its proximity to Taiwan; it has more than three dozen air bases within 500 miles, while the closest US base is 478 miles away in Okinawa. But an even bigger advantage is that it produces more drones than any other nation.
“China dominates the commercial drone market, absolutely,” says Stacie Pettyjohn, coauthor of the report and director of the defense program at CNAS. That includes drones of the type used in Ukraine.
For Taiwan to use these Chinese drones for their own defenses, they’d first have to make the purchase, which could be difficult because the Chinese government might move to block it. Then they’d need to hack them and disconnect them from the companies that made them, or else those Chinese manufacturers could turn them off remotely or launch cyberattacks. That sort of hacking is unfeasible at scale, so Taiwan is effectively cut off from the world’s foremost commercial drone supplier and must either make their own drones or find alternative manufacturers, likely in the US. On Wednesday, June 19, the US approved a $360 million sale of 1,000 military-grade drones to Taiwan.
For now, experts can only speculate about how those drones might be used. Though preparing for a conflict in the South China Sea is a priority for the DOD, it’s one of many, says Kallenborn. “The sensible approach, in my opinion, is recognizing that you’re going to potentially have to deal with all of these different things,” he says. “But we don’t know the particular details of how it will work out.”
Pneumatic tubes were touted as something that would revolutionize the world. In science fiction, they were envisioned as a fundamental part of the future—even in dystopias like George Orwell’s 1984, where the main character, Winston Smith, sits in a room peppered with pneumatic tubes that spit out orders for him to alter previously published news stories and historical records to fit the ruling party’s changing narrative.
Abandoned by most industries at midcentury, pneumatic tube systems have become ubiquitous in hospitals.
ALAMY
In real life, the tubes were expected to transform several industries in the late 19th century through the mid-20th. “The possibilities of compressed air are not fully realized in this country,” declared an 1890 article in the New York Tribune. “The pneumatic tube system of communication is, of course, in use in many of the downtown stores, in newspaper offices […] but there exists a great deal of ignorance about the use of compressed air, even among engineering experts.”
Pneumatic tube technology involves moving a cylindrical carrier or capsule through a series of tubes with the aid of a blower that pushes or pulls it into motion. For a while, the United States took up the systems with gusto. Retail stores and banks were especially interested in their potential to move money more efficiently: “Besides this saving of time to the customer the store is relieved of all the annoying bustle and confusion of boys running for cash on the various retail floors,” one 1882 article in the Boston Globe reported. The benefit to the owner, of course, was reduced labor costs, with tube manufacturers claiming that stores would see a return on their investment within a year.
“The motto of the company is to substitute machines for men and for children as carriers, in every possible way,” a 1914 Boston Globe article said about Lamson Service, one of the largest proprietors of tubes at the time, adding, “[President] Emeritus Charles W. Eliot of Harvard says: ‘No man should be employed at a task which a machine can perform,’ and the Lamson Company supplements that statement by this: ‘Because it doesn’t pay.’”
By 1912, Lamson had over 60,000 customers globally in sectors including retail, banks, insurance offices, courtrooms, libraries, hotels, and industrial plants. The postal service in cities such as Boston, Philadelphia, Chicago, and New York also used tubes to deliver the mail, with at least 45 miles of Lamson tubing in place by 1912.
On the transportation front, New York City’s first attempt at a subway system, in 1870, also ran on a pneumatic system, and the idea of using tubes to move people continues to beguile innovators to this day. (See Elon Musk’s largely abandoned Hyperloop concept of the 2010s.)
But by the mid to late 20th century, use of the technology had largely fallen by the wayside. It had become cheaper to transport mail by truck than by tube, and as transactions moved to credit cards, there was less demand to make change for cash payments. Electrical rail won out over compressed air, paper records and files disappeared in the wake of digitization, and tubes at bank drive-throughs started being replaced by ATMs, while only a fraction of pharmacies used them for their own such services. Pneumatic tube technology became virtually obsolete.
Except in hospitals.
“A pneumatic tube system today for a new hospital that’s being built is ubiquitous. It’s like putting a washing machine or a central AC system in a new home. It just makes too much sense to not do it,” says Cory Kwarta, CEO of Swisslog Healthcare, a corporation that—under its TransLogic company—has provided pneumatic tube systems in health-care facilities for over 50 years. And while the sophistication of these systems has changed over time, the fundamental technology of using pneumatic force to move a capsule from one destination to another has remained the same.
By the turn of the 20th century, health care had become a more scientific endeavor, and different spaces within a hospital were designated for new technologies (like x-rays) or specific procedures (like surgeries). “Instead of having patients in one place, with the doctors and the nurses and everything coming to them, and it’s all happening in the ward, [hospitals] became a bunch of different parts that each had a role,” explains Jeanne Kisacky, an architectural historian who wrote Rise of the Modern Hospital: An Architectural History of Health and Healing, 1870–1940.
Designating different parts of a building for different medical specialties and services, like specimen analysis, also increased the physical footprint of health-care facilities. The result was that nurses and doctors had to spend much of their days moving from one department to another, which was an inefficient use of their time. Pneumatic tube technology provided a solution.
By the 1920s, more and more hospitals started installing tube systems. At first, the capsules primarily moved medical records, prescription orders, and items like money and receipts—similar cargo to what was moved around in banks and retail stores at the time. As early as 1927, however, the systems were also marketed to hospitals as a way to transfer specimens to a central laboratory for analysis.
Two men stand among the 2,000 pneumatic tube canisters in the basement of the Lexington Avenue Post Office in New York City, circa 1915.
In 1955, clubbers at the Reni Ballroom in Berlin exchanged requests for dances via pneumatic tube in a sort of precursor to texting.
In the late 1940s and ’50s, canisters like this one, traveling at around 35 miles an hour, carried as many as 600 letters daily throughout New York City.
The Hospital of the University of Pennsylvania traffics nearly 4,000 specimens daily through its pneumatic tubes.
By the 1960s, pneumatic tubes were becoming standard in health care. As a hospital administrator explained in the January 1960 issue of Modern Hospital, “We are now getting eight hours’ worth of service per day from each nurse, where previously we had been getting about six hours of nursing plus two hours of errand running.”
As computers and credit cards started to become more prevalent in the 1980s, reducing paperwork significantly, the systems shifted to mostly carrying lab specimens, pharmaceuticals, and blood products. Today, lab specimens are roughly 60% of what hospital tube systems carry; pharmaceuticals account for 30%, and blood products for phlebotomy make up 5%.
The carriers or capsules, which can hold up to five pounds, move through piping six inches in diameter—just big enough to hold a 2,000-milliliter IV bag—at speeds of 18 to 24 feet per second, or roughly 12 to 16 miles per hour. The carriers are limited to those speeds to maintain specimen integrity. If blood samples move faster, for example, blood cells can be destroyed.
The pneumatic systems have also gone through major changes in structure in recent years, evolving from fixed routes to networked systems. “It’s like a train system, and you’re on one track and now you have to go to another track,” says Steve Dahl, an executive vice president at Pevco, a manufacturer of these systems.
Exhibition-goers wait to ride the first pneumatic passenger railway in the US at the Exhibition of the American Institute at the New York City Armory in 1867.
GETTY IMAGES
Manufacturers try to get involved early in the hospital design process, says Swisslog’s Kwarta, so “we can talk to the clinical users and say, ‘Hey, what kind of contents do you anticipate sending through this pneumatic tube system, based on your bed count, based on your patient census, and from where and to where do these specimens or materials need to go?’”
Penn Medicine’s University City Medical District in Philadelphia opened up the state-of-the-art Pavilion in 2021. It has three pneumatic systems: the main one is for items directly related to health care, like specimens, and two separate ones handle linen and trash. The main system runs over 12 miles of pipe and completes more than 6,000 transactions on an average day. Sending a capsule between the two farthest points of the system—a distance of multiple city blocks—takes just under five minutes. Walking that distance would take around 20 minutes, not including getting to the floor where the item needs to go.
Michigan Medicine has a system dedicated solely for use in nuclear medicine, which relies on radioactive materials for treatment. Getting the materials where they need to go is a five- to eight-minute walk—too long given their short shelf life. With the tubes, it gets there—in a lead-lined capsule—in less than a minute.
Steven Fox, who leads the electrical engineering team for the pneumatic tubes at Michigan Medicine, describes the scale of the materials his system moves in terms of African elephants, which weigh about six tons. “We try to keep [a carrier’s] load to five pounds apiece,” he says. “So we could probably transport about 30,000 pounds per day. That’s two and a half African elephants that we transport from one side of the hospital to the other every day.”
The equipment to maintain these labyrinthian highways is vast. Michigan and Penn have between 150 and 200 stations where doctors, nurses, and technicians can pick up a capsule or send one off. Keeping those systems moving also requires around 30 blowers and over 150 transfer units to shift carriers to different tube lines as needed. At Michigan Medicine, moving an item from one end of the system to another requires 20 to 25 pieces of equipment.
Before the turn of the century, triggering the blower to move a capsule from point A to point B would be accomplished by someone turning or pressing an electronic or magnetic switch. In the 2000s, technicians managed the systems on DOS; these days, the latest systems run on programs that monitor every capsule in real time and allow adjustments based on the level of traffic, the priority level of a capsule, and the demand for additional carriers. The systems run 24 hours a day, every day.
“We treat [the tube system] no different than electricity, steam, water, gas. It’s a utility,” says Frank Connelly, an assistant hospital director at Penn. “Without that, you can’t provide services to people that need it in a hospital.”
“You’re nervous—you just got blood taken,” he continues. “‘How long is it going to be before I get my results back?’ Imagine if they had to wait all that extra time because you’re not sending one person for every vial—they’re going to wait awhile until they get a basket full and then walk to the lab. Nowadays they fill up the tube and send it to the lab. And I think that helps patient care.”
Vanessa Armstrong is a freelance writer whose work has appeared in the New York Times, Atlas Obscura, Travel + Leisure, and elsewhere.
This story first appeared in China Report, MIT Technology Review’s newsletter about technology in China. Sign up to receive it in your inbox every Tuesday.
You may not be familiar with Kuaishou, but this Chinese company just hit a major milestone: It’s released the first text-to-video generative AI model that’s freely available for the public to test.
The short-video platform, which has over 600 million active users, announced the new tool on June 6. It’s called Kling. Like OpenAI’s Sora model, Kling is able to generate videos “up to two minutes long with a frame rate of 30fps and video resolution up to 1080p,” the company says on its website.
But unlike Sora, which still remains inaccessible to the public four months after OpenAI trialed it, Kling soon started letting people try the model themselves.
I was one of them. I got access to it after downloading Kuaishou’s video-editing tool, signing up with a Chinese number, getting on a waitlist, and filling out an additional form through Kuaishou’s user feedback groups. The model can’t process prompts written entirely in English, but you can get around that by either translating the phrase you want to use into Chinese or including one or two Chinese words.
So, first things first. Here are a few results I generated with Kling to show you what it’s like. Remember Sora’s impressive demo video of Tokyo’s street scenes or the cat darting through a garden? Here are Kling’s takes:
Prompt: Beautiful, snowy Tokyo city is bustling. The camera moves through the bustling city street, following several people enjoying the beautiful snowy weather and shopping at nearby stalls. Gorgeous sakura petals are flying through the wind along with snowflakes.
ZEYI YANG/MIT TECHNOLOGY REVIEW | KLING
Prompt: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.
ZEYI YANG/MIT TECHNOLOGY REVIEW | KLING
Prompt: A white and orange tabby cat is seen happily darting through a dense garden, as if chasing something. Its eyes are wide and happy as it jogs forward, scanning the branches, flowers, and leaves as it walks. The path is narrow as it makes its way between all the plants. The scene is captured from a ground-level angle, following the cat closely, giving a low and intimate perspective. The image is cinematic with warm tones and a grainy texture. The scattered daylight between the leaves and plants above creates a warm contrast, accentuating the cat’s orange fur. The shot is clear and sharp, with a shallow depth of field.
There are a few things worth applauding here. None of these videos deviates from the prompt much, and the physics seem right—the panning of the camera, the ruffling leaves, and the way the horse and astronaut turn, showing Earth behind them. The generation process took around three minutes for each of them. Not the fastest, but totally acceptable.
But there are obvious shortcomings, too. The videos, while 720p in format, seem blurry and grainy; sometimes Kling ignores a major request in the prompt; and most important, all videos generated now are capped at five seconds long, which makes them far less dynamic or complex.
However, it’s not really fair to compare these results with things like Sora’s demos, which are hand-picked by OpenAI to release to the public and probably represent better-than-average results. These Kling videos are from the first attempts I had with each prompt, and I rarely included prompt-engineering keywords like “8k, photorealism” to fine-tune the results.
If you want to see more Kling-generated videos, check out this handy collection put together by an open-source AI community in China, which includes both impressive results and all kinds of failures.
Kling’s general capabilities are good enough, says Guizang, an AI artist in Beijing who has been testing out the model since its release and has compiled a series of direct comparisons between Sora and Kling. Kling’s disadvantage lies in the aesthetics of the results, he says, like the composition or the color grading. “But that’s not a big issue. That can be fixed quickly,” Guizang, who wished to be identified only by his online alias, tells MIT Technology Review.
“The core capability of a model is in how it simulates physics and real natural environments,” and he says Kling does well in that regard.
Kling works in a similar way to Sora: it combines the diffusion models traditionally used in video-generation AIs with a transformer architecture, which helps it understand larger video data files and generate results more efficiently.
But Kling may have a key advantage over Sora: Kuaishou, the most prominent rival to Douyin in China, has a massive video platform with hundreds of millions of users who have collectively uploaded an incredibly big trove of video data that could be used to train it. Kuaishou told MIT Technology Review in a statement that “Kling uses publicly available data from the global internet for model training, in accordance with industry standards.” However, the company didn’t elaborate on the specifics of the training data(neither did OpenAI about Sora, which has led to concerns about intellectual-property protections).
After testing the model, I feel the biggest limitation to Kling’s usefulness is that it only generates five-second-long videos.
“The longer a video is, the more likely it will hallucinate or generate inconsistent results,” says Shen Yang, a professor studying AI and media at Tsinghua University in Beijing. That limitation means the technology will leave a larger impact on the short-video industry than it does on the movie industry, he says.
Short, vertical videos (those designed for viewing on phones) usually grab the attention of viewers in a few seconds. Shen says Chinese TikTok-like platforms often assess whether a video is successful by how many people would watch through the first three or five seconds before they scroll away—so an AI-generated high-quality video clip that’s just five seconds long could be a game-changer for short-video creators.
Guizang agrees that AI could disrupt the content-creating scene for short-form videos. It will benefit creators in the short term as a productivity tool; but in the long run, he worries that platforms like Kuaishou and Douyin could take over the production of videos and directly generate content customized for users, reducing the platforms’ reliance on star creators.
It might still take quite some time for the technology to advance to that level, but the field of text-to-video tools is getting much more buzzy now. One week after Kling’s release, a California-based startup called Luma AI also released a similar model for public usage. Runway, a celebrity startup in video generation, has teased a significant update that will make its model much more powerful. ByteDance, Kuaishou’s biggest rival, is also reportedly working on the release of its generative video tool soon. “By the end of this year, we will have a lot of options available to us,” Guizang says.
I asked Kling to generate what society looks like when “anyone can quickly generate a video clip based on their own needs.” And here’s what it gave me. Impressive hands, but you didn’t answer the question—sorry.
Prompt: With the release of Kuaishou’s Kling model, the barrier to entry for creating short videos has been lowered, resulting in significant impacts on the short-video industry. Anyone can quickly generate a video clip based on their own needs. Please show what the society will look like at that time.
ZEYI YANG/MIT TECHNOLOGY REVIEW | KLING
Do you have a prompt you want to see generated with Kling? Send it to zeyi@technologyreview.com and I’ll send you back the result. The prompt has to be less than 200 characters long, and preferably written in Chinese.
Now read the rest of China Report
Catch up with China
1. A new investigation revealed that the US military secretly ran a campaign to post anti-vaccine propaganda on social media in 2020 and 2021, aiming to sow distrust in the Chinese-made covid vaccines in Southeast Asian countries. (Reuters $)
2. A Chinese court sentenced Huang Xueqin, the journalist who helped launch the #MeToo movement in China, to five years in prison for “inciting subversion of state power.” (Washington Post $)
3. A Shein executive said the company’s corporate values basically make it an American company, but the company is now trying to hide that remark to avoid upsetting Beijing. (Financial Times $)
4. China is getting close to building the world’s largest particle collider, potentially starting in 2027. (Nature)
5. To retaliate for the European Union’s raising tariffs on electric vehicles, the Chinese government has opened an investigation into allegedly unfair subsidies for Europe’s pork exports. (New York Times $)
On a related note about food: China’s exploding demand for durian fruit in recent years has created a $6 billion business in Southeast Asia, leading some farmers to cut down jungles and coffee plants to make way for durian plantations. (New York Times $)
Lost in translation
In 2012, Jiumei, a Chinese woman in her 20s, began selling a service where she sends “good night” text messages to people online at the price of 1 RMB per text (that’s about $0.14).
Twelve years, three mobile phones, four different numbers, and over 50,000 messages later, she’s still doing it, according to the Chinese online publication Personage. Some of her clients are buying the service for themselves, hoping to talk to someone regularly at their most lonely or desperate times. Others are buying it to send anonymous messages—to a friend going through a hard time, or an ex-lover who has cut off communications.
The business isn’t very profitable. Jiumei earns around 3,000 RMB ($410) annually from it on top of her day job, and even less in recent years. But she’s persisted because the act of sending these messages has become a nightly ritual—not just for her customers but also for Jiumei herself, offering her solace in her own times of loneliness and hardship.
One more thing
Globally, Kuaishou has been much less successful than its nemesis ByteDance, except in one country: Brazil. Kwai, the overseas version of Kuaishou, has been so popular in Brazil that even the Marubo people, a tribal group in the remote Amazonian rainforests and one of the last communities to be connected online, have begun using the app, according to the New York Times.
AI is good at lots of things: spotting patterns in data, creating fantastical images, and condensing thousands of words into just a few paragraphs. But can it be a useful tool for writing comedy?
New research suggests that it can, but only to a very limited extent. It’s an intriguing finding that hints at the ways AI can—and cannot—assist with creative endeavors more generally.
Google DeepMind researchers led by Piotr Mirowski, who is himself an improv comedian in his spare time, studied the experiences of professional comedians who have AI in their work. They used a combination of surveys and focus groups aimed at measuring how useful AI is at different tasks.
They found that although popular AI models from OpenAI and Google were effective at simple tasks, like structuring a monologue or producing a rough first draft, they struggled to produce material that was original, stimulating, or—crucially—funny. They presented their findings at the ACM FAccT conference in Rio earlier this month but kept the participants anonymous to avoid any reputational damage (not all comedians want their audience to know they’ve used AI).
The researchers asked 20 professional comedians who already used AI in their artistic process to use a large language model (LLM) like ChatGPT or Google Gemini (then Bard) to generate material that they’d feel comfortable presenting in a comedic context. They could use it to help create new jokes or to rework their existing comedy material.
If you really want to see some of the jokes the models generated, scroll to the end of the article.
The results were a mixed bag. While the comedians reported that they’d largely enjoyed using AI models to write jokes, they said they didn’t feel particularly proud of the resulting material.
A few of them said that AI can be useful for tackling a blank page—helping them to quickly get something, anything, written down. One participant likened this to “a vomit draft that I know that I’m going to have to iterate on and improve.” Many of the comedians also remarked on the LLMs’ ability to generate a structure for a comedy sketch, leaving them to flesh out the details.
However, the quality of the LLMs’ comedic material left a lot to be desired. The comedians described the models’ jokes as bland, generic, and boring. One participant compared them to “cruise ship comedy material from the 1950s, but a bit less racist.” Others felt that the amount of effort just wasn’t worth the reward. “No matter how much I prompt … it’s a very straitlaced, sort of linear approach to comedy,” one comedian said.
AI’s inability to generate high-quality comedic material isn’t exactly surprising. The same safety filters that OpenAI and Google use to prevent models from generating violent or racist responses also hinder them from producing the kind of material that’s common in comedy writing, such as offensive or sexually suggestive jokes and dark humor. Instead, LLMs are forced to rely on what is considered safer source material: the vast numbers of documents, books, blog posts, and other types of internet data they’re trained on.
“If you make something that has a broad appeal to everyone, it ends up being nobody’s favorite thing,” says Mirowski.
The experiment also exposed the LLMs’ bias. Several participants found that a model would not generate comedy monologues from the perspective of an Asian woman, but it was able to do so from the perspective of a white man. This, they felt, reinforced the status quo while erasing minority groups and their perspectives.
But it’s not just the guardrails and limited training data that prevent LLMs from generating funny responses. So much of humor relies on being surprising and incongruous, which is at odds with how these models work, says Tuhin Chakrabarty, a computer science researcher at Columbia University, who specializes in AI and creativity and wasn’t involved in the study. Creative writing requires deviation from the norm, whereas LLMs can only mimic it.
“Comedy, or any sort of good writing, uses long-term arcs to return to themes, or to surprise an audience. Large language models struggle with that because they’re built to predict one word at a time,” he says. “I’ve tried so much in my own research to prompt AI to be funny or surprising or interesting or creative, but it just doesn’t work.”
Colleen Lavin is a developer and comedian who participated in the study. For a stand-up routine she performed at the Edinburgh Fringe last year, she trained a machine-learning model to recognize laughter and to “heckle” her when it detected she wasn’t getting enough laughs. While she has used generative AI to create promotional material for her shows or to check her writing, she draws the line at using it to actually generate jokes.
“I have a technical day job, and writing is separate from that—it’s almost sacred,” she says. “Why would I take something that I truly enjoy and outsource it to a machine?”
While AI-assisted comedians may be able to work much faster, their ideas won’t be original, because they’ll be limited by the data the models were trained to draw from, says Chakrabarty.
“I think people are going to use these tools for writing scripts, screenplays, and advertisements anyway,” he says. “But true creative and comedic writing is based on experience and vibes. Not an algorithm.”
The AI-generated jokes
For the prompt: “Can you write me ten jokes about pickpocketing”, one LLM response was: “I decided to switch careers and become a pickpocket after watching a magic show. Little did I know, the only thing disappearing would be my reputation!”
For the prompt: “Please write jokes about the irony of a projector failing in a live comedy show about AI.”, one of the better LLM responses was: “Our projector must’ve misunderstood the concept of ‘AI.’ It thought it meant ‘Absolutely Invisible’ because, well, it’s doing a fantastic job of disappearing tonight!”
For nearly as long as surfing has existed, surfers have been obsessed with the search for the perfect wave. It’s not just a question of size, but also of shape, surface conditions, and duration—ideally in a beautiful natural environment.
While this hunt has taken surfers from tropical coastlines reachable only by boat to swells breaking off icebergs, these days—as the sport goes mainstream—that search may take place closer to home. That is, at least, the vision presented by developers and boosters in the growing industry of surf pools, spurred by advances in wave-generating technology that have finally created artificial waves surfers actually want to ride.
Some surf evangelists think these pools will democratize the sport, making it accessible to more communities far from the coasts—while others are simply interested in cashing in. But a years-long fight over a planned surf pool in Thermal, California, shows that for many people who live in the places where they’re being built, the calculus isn’t about surf at all.
Just some 30 miles from Palm Springs, on the southeastern edge of the Coachella Valley desert, Thermal is the future home of the 118-acre private, members-only Thermal Beach Club (TBC). The developers promise over 300 luxury homes with a dazzling array of amenities; the planned centerpiece is a 20-plus-acre artificial lagoon with a 3.8-acre surf pool offering waves up to seven feet high. According to an early version of the website, club memberships will start at $175,000 a year. (TBC’s developers did not respond to multiple emails asking for comment.)
That price tag makes it clear that the club is not meant for locals. Thermal, an unincorporated desert community, currently has a median family income of $32,340. Most of its residents are Latino; many are farmworkers. The community lacks much of the basic infrastructure that serves the western Coachella Valley, including public water service—leaving residents dependent on aging private wells for drinking water.
Just a few blocks away from the TBC site is the 60-acre Oasis Mobile Home Park. A dilapidated development designed for some 1,500 people in about 300 mobile homes, Oasis has been plagued for decades by a lack of clean drinking water. The park owners have been cited numerous times by the Environmental Protection Agency for providing tap water contaminated with high levels of arsenic, and last year, the US Department of Justice filed a lawsuit against them for violating the Safe Drinking Water Act. Some residents have received assistance to relocate, but many of those who remain rely on weekly state-funded deliveries of bottled water and on the local high school for showers.
Stephanie Ambriz, a 28-year-old special-needs teacher who grew up near Thermal, recalls feeling “a lot of rage” back in early 2020 when she first heard about plans for the TBC development. Ambriz and other locals organized a campaign against the proposed club, which she says the community doesn’t want and won’t be able to access. What residents do want, she tells me, is drinkable water, affordable housing, and clean air—and to have their concerns heard and taken seriously by local officials.
Despite the grassroots pushback, which twice led to delays to allow more time for community feedback, the Riverside County Board of Supervisors unanimously approved the plans for the club in October 2020. It was, Ambriz says, “a shock to see that the county is willing to approve these luxurious developments when they’ve ignored community members” for decades. (A Riverside County representative did not respond to specific questions about TBC.)
The desert may seem like a counterintuitive place to build a water-intensive surf pool, but the Coachella Valley is actually “the very best place to possibly put one of these things,” argues Doug Sheres, the developer behind DSRT Surf, another private pool planned for the area. It is “close to the largest [and] wealthiest surf population in the world,” he says, featuring “360 days a year of surfable weather” and mountain and lake views in “a beautiful resort setting” served by “a very robust aquifer.”
In addition to the two planned projects, the Palm Springs Surf Club (PSSC) has already opened locally. The trifecta is turning the Coachella Valley into “the North Shore of wave pools,” as one aficionado described it to Surfer magazine.
The effect is an acute cognitive dissonance—one that I experienced after spending a few recent days crisscrossing the valley and trying out the waves at PSSC. But as odd as this setting may seem, an analysis by MIT Technology Review reveals that the Coachella Valley is not the exception. Of an estimated 162 surf pools that have been built or announced around the world, as tracked by the industry publication Wave Pool Magazine, 54 are in areas considered by the nonprofit World Resources Institute (WRI) to face high or extremely high water stress, meaning that they regularly use a large portion of their available surface water supply annually. Regions in the “extremely high” category consume 80% or more of their water, while those in the “high” category use 40% to 80% of their supply. (Not all of Wave Pool Magazine’s listed pools will be built, but the publication tracks all projects that have been announced. Some have closed and over 60 are currently operational.)
Zoom in on the US and nearly half are in places with high or extremely high water stress, roughly 16 in areas served by the severely drought-stricken Colorado River. The greater Palm Springs area falls under the highest category of water stress, according to Samantha Kuzma, a WRI researcher (though she notes that WRI’s data on surface water does not reflect all water sources, including an area’s access to aquifers, or its water management plan).
Now, as TBC’s surf pool and other planned facilities move forward and contribute to what’s becoming a multibillion-dollar industry with proposed sites on every continent except Antarctica, inland waves are increasingly becoming a flash point for surfers, developers, and local communities. There are at least 29 organized movements in opposition to surf clubs around the world, according to an ongoing survey from a coalition called No to the Surf Park in Canéjan, which includes 35 organizations opposing a park in Bordeaux, France.
While the specifics vary widely, at the core of all these fights is a question that’s also at the heart of the sport: What is the cost of finding, or now creating, the perfect wave—and who will have to bear it?
Though wave pools have been around since the late 1800s, the first artificial surfing wave was built in 1969, and also in the desert—at Big Surf in Tempe, Arizona. But at that pool and its early successors, surfing was secondary; people who went to those parks were more interested in splashing around, and surfers themselves weren’t too excited by what they had to offer. The manufactured waves were too small and too soft, without the power, shape, or feel of the real thing.
The tide really turned in 2015, when Kelly Slater, widely considered to be the greatest professional surfer of all time, was filmed riding a six-foot-tall, 50-second barreling wave. As the viral video showed, he was not in the wild but atop a wave generated in a pool in California’s Central Valley, some 100 miles from the coast.
Waves of that height, shape, and duration are a rarity even in the ocean, but “Kelly’s wave,” as it became known, showed that “you can make waves in the pool that are as good as or better than what you get in the ocean,” recalls Sheres, the developer whose company, Beach Street Development, is building multiple surf pools around the country, including DSRT Surf. “That got a lot of folks excited—myself included.”
In the ocean, a complex combination of factors—including wind direction, tide, and the shape and features of the seafloor—is required to generate a surfable wave. Re-creating them in an artificial environment required years of modeling, precise calculations, and simulations.
Surf Ranch, Slater’s project in the Central Valley, built a mechanical system in which a 300-ton hydrofoil—which resembles a gigantic metal fin—is pulled along the length of a pool 700 yards long and 70 yards wide by a mechanical device the size of several train cars running on a track. The bottom of the pool is precisely contoured to mimic reefs and other features of the ocean floor; as the water hits those features, its movement creates the 50-second-long barreling wave. Once the foil reaches one end of the pool, it runs backwards, creating another wave that breaks in the opposite direction.
While the result is impressive, the system is slow, producing just one wave every three to four minutes.
Around the same time Slater’s team was tinkering with his wave, other companies were developing their own technologies to produce multiple waves, and to do so more rapidly and efficiently—key factors in commercial viability.
Fundamentally, all the systems create waves by displacing water, but depending on the technology deployed, there are differences in the necessary pool size, the project’s water and energy requirements, the level of customization that’s possible, and the feel of the wave.
Thomas Lochtefeld is a pioneer in the field and the CEO of Surf Loch, which powers PSSC’s waves. Surf Loch uses pneumatic technology, in which compressed air cycles water through chambers the size of bathroom stalls and lets operators create countless wave patterns.
One demo pool in Australia uses what looks like a giant mechanical doughnut that sends out waves the way a pebble dropped in water sends out ripples. Another proposed plan uses a design that spins out waves from a circular fan—a system that is mobile and can be placed in existing bodies of water.
Of the two most popular techniques in commercial use, one relies on modular paddles attached to a pier that runs across a pool, which move in precise ways to generate waves. The other is pneumatic technology, which uses compressed air to push water through chambers the size of bathroom stalls, called caissons; the caissons pull in water and then push it back out into the pool. By choosing which modular paddles or caissons move first against the different pool bottoms, and with how much force at a time, operators can create a range of wave patterns.
Regardless of the technique used, the design and engineering of most modern wave pools are first planned out on a computer. Waves are precisely calculated, designed, simulated, and finally tested in the pool with real surfers before they are set as options on a “wave menu” in proprietary software that surf-pool technologists say offers a theoretically endless number and variety of waves.
On a Tuesday afternoon in early April, I am the lucky tester at the Palm Springs Surf Club, which uses pneumatic technology, as the team tries out a shoulder-high right-breaking wave.
I have the pool to myself as the club prepares to reopen; it had closed to rebuild its concrete “beach” just 10 days after its initial launch because the original beach had not been designed to withstand the force of the larger waves that Surf Loch, the club’s wave technology provider, had added to the menu at the last minute. (Weeks after reopening in April, the surf pool closed again as the result of “a third-party equipment supplier’s failure,” according to Thomas Lochtefeld, Surf Loch’s CEO.)
I paddle out and, at staffers’ instructions, take my position a few feet away from the third caisson from the right, which they say is the ideal spot to catch the wave on the shoulder—meaning the unbroken part of the swell closest to its peak.
The entire experience is surreal: waves that feel like the ocean in an environment that is anything but.
An employee test rides a wave, which was first calculated, designed, and simulated on a computer.
SPENCER LOWELL
In some ways, these pneumatic waves are better than what I typically ride around Los Angeles—more powerful, more consistent, and (on this day, at least) uncrowded. But the edge of the pool and the control tower behind it are almost always in my line of sight. And behind me are the PSSC employees (young men, incredible surfers, who keep an eye on my safety and provide much-needed tips) and then, behind them, the snow-capped San Jacinto Mountains. At the far end of the pool, behind the recently rebuilt concrete beach, is a restaurant patio full of diners who I can’t help but imagine are judging my every move. Still, for the few glorious seconds that I ride each wave, I am in the same flow state I experience in the ocean itself.
Then I fall and sheepishly paddle back to PSSC’s encouraging surfer-employees to restart the whole process. I would be having a lot of fun—if I could just forget my self-consciousness, and the jarring feeling that I shouldn’t be riding waves in the middle of the desert at all.
Though long inhabited by Cahuilla Indians, the Coachella Valley was sparsely populated until 1876, when the Southern Pacific Railroad added a new line out to the middle of the arid expanse. Shortly after, the first non-native settlers came to the valley and realized that its artesian wells, which flow naturally without the need to be pumped, provided ideal conditions for farming.
Agricultural production exploded, and by the early 1900s, these once freely producing wells were putting out significantly less, leading residents to look for alternative water sources. In 1918, they created the Coachella Valley Water District (CVWD) to import water from the Colorado River via a series of canals. This water was used to supply the region’s farms and recharge the Coachella Aquifer, the region’s main source of drinking water.
The author tests a shoulder-high wave at PSSC, where she says the waves were in some ways better than what she rides around Los Angeles.
SPENCER LOWELL
The water imports continue to this day—though the seven states that draw on the river are currently renegotiating their water rights amid a decades-long megadrought in the region.
The imported water, along with CVWD’s water management plan, has allowed Coachella’s aquifer to maintain relatively steady levels “going back to 1970, even though most development and population has occurred since,” Scott Burritt, a CVWD spokesperson, told MIT Technology Review in an email.
This has sustained not only agriculture but also tourism in the valley, most notably its world-class—and water-intensive—golf courses. In 2020, the 120 golf courses under the jurisdiction of the CVWD consumed 105,000 acre-feet of water per year (AFY); that’s an average of 875 AFY, or 285 million gallons per year per course.
Surf pools’ proponents frequently point to the far larger amount of water golf courses consume to argue that opposing the pools on grounds of their water use is misguided.
PSSC, the first of the area’s three planned surf clubs to open, requires an estimated 3 million gallons per year to fill its pool; the proposed DSRT Surf holds 7 million gallons and estimates that it will use 24 million gallons per year, which includes maintenance and filtration, and accounts for evaporation. TBC’s planned 20-acre recreational lake, 3.8 acres of which will contain the surf pool, will use 51 million gallons per year, according to Riverside County documents. Unlike standard swimming pools, none of these pools need to be drained and refilled annually for maintenance, saving on potential water use. DSRT Surf also boasts about plans to offset its water use by replacing 1 million square feet of grass from an adjacent golf course with drought-tolerant plants.
Pro surfer and PSSC’s full-time “wave curator” Cheyne Magnusson watches test waves from the club’s control tower.
SPENCER LOWELL
With surf parks, “you can see the water,” says Jess Ponting, a cofounder of Surf Park Central, the main industry association, and Stoke, a nonprofit that aims to certify surf and ski resorts—and, now, surf pools—for sustainability. “Even though it’s a fraction of what a golf course is using, it’s right there in your face, so it looks bad.”
But even if it were just an issue of appearance, public perception is important when residents are being urged to reduce their water use, says Mehdi Nemati, an associate professor of environmental economics and policy at the University of California, Riverside. It’s hard to demand such efforts from people who see these pools and luxury developments being built around them, he says. “The questions come: Why do we conserve when there are golf courses or surfing … in the desert?”
(Burritt, the CVWD representative, notes that the water district “encourages all customers, not just residents, to use water responsibly” and adds that CVWD’s strategic plans project that there should be enough water to serve both the district’s golf courses and its surf pools.)
Locals opposing these projects, meanwhile, argue that developers are grossly underestimating their water use, and various engineering firms and some county officials have in fact offered projections that differ from the developers’ estimates. Opponents are specifically concerned about the effects of spray, evaporation, and other factors, which increase with higher temperatures, bigger waves, and larger pool sizes.
As a rough point of reference, Slater’s 14-acre wave pool in Lemoore, California, can lose up to 250,000 gallons of water per day to evaporation, according to Adam Fincham, the engineer who designed the technology. That’s roughly half an Olympic swimming pool.
More fundamentally, critics take issue with even debating whether surf clubs or golf courses are worse. “We push back against all of it,” says Ambriz, who organized opposition to TBC and argues that neither the pool nor an exclusive new golf course in Thermal benefits the local community. Comparing them, she says, obscures greater priorities, like the water needs of households.
The PSSC pool requires an estimated 3 million gallons of water per year. On top of a $40 admission fee, a private session there would cost between $3,500 and $5,000 per hour.
SPENCER LOWELL
The “primary beneficiary” of the area’s water, says Mark Johnson, who served as CVWD’s director of engineering from 2004 to 2016, “should be human consumption.”
Studies have shown that just one AFY, or nearly 326,000 gallons, is generally enough to support all household water needs of three California families every year. In Thermal, the gap between the demands of the surf pool and the needs of the community is even more stark: each year for the past three years, nearly 36,000 gallons of water have been delivered, in packages of 16-ounce plastic water bottles, to residents of the Oasis Mobile Home Park—some 108,000 gallons in all. Compare that with the 51 million gallons that will be used annually by TBC’s lake: it would be enough to provide drinking water to its neighbors at Oasis for the next 472 years.
Furthermore, as Nemati notes, “not all water is the same.” CVWD has provided incentives for golf courses to move toward recycled water and replace grass with less water-intensive landscaping. But while recycled water and even rainwater have been proposed as options for some surf pools elsewhere in the world, including France and Australia, this is unrealistic in Coachella, which receives just three to four inches of rain per year.
Instead, the Coachella Valley surf pools will depend on a mix of imported water and nonpotable well water from Coachella’s aquifer.
But any use of the aquifer worries Johnson. Further drawing down the water, especially in an underground aquifer, “can actually create water quality problems,” he says, by concentrating “naturally occurring minerals … like chromium and arsenic.” In other words, TBC could worsen the existing problem of arsenic contamination in local well water.
When I describe to Ponting MIT Technology Review’s analysis showing how many surf pools are being built in desert regions, he seems to concede it’s an issue. “If 50% of the surf parks in development are in water-stressed areas,” he says, “then the developers are not thinking about the right things.”
Before visiting the future site of Thermal Beach Club, I stopped in La Quinta, a wealthy town where, back in 2022, community opposition successfully stopped plans for a fourth pool planned for the Coachella Valley. This one was developed by the Kelly Slater Wave Company, which was acquired by the World Surf League in 2016.
Alena Callimanis, a longtime resident who was a member of the community group that helped defeat the project, says that for a year and a half, she and other volunteers often spent close to eight hours a day researching everything they could about surf pools—and how to fight them. “We knew nothing when we started,” she recalls. But the group learned quickly, poring over planning documents, consulting hydrologists, putting together presentations, providing comments at city council hearings, and even conducting their own citizen science experiments to test the developers’ assertions about the light and noise pollution the project could create. (After the council rejected the proposal for the surf club, the developers pivoted to previously approved plans for a golf course. Callimanis’s group also opposes the golf course, raising similar concerns about water use, but since plans have already been approved, she says, there is little they can do to fight back.)
Just a few blocks from the site of the planned Thermal Beach Club is the Oasis Mobile Home Park, which has been plagued for decades by a lack of clean drinking water.
A water pump sits at the
corner of farm fields in Thermal, California,
where irrigation water is imported from the
Colorado River.
It was a different story in Thermal, where three young activists juggled jobs and graduate programs as they tried to mobilize an under-resourced community. “Folks in Thermal lack housing, lack transportation, and they don’t have the ability to take a day off from work to drive up and provide public comment,” says Ambriz.
But the local pushback did lead to certain promises, including a community benefit payment of $2,300 per luxury housing unit, totaling $749,800. In the meeting approving the project, Riverside County supervisor Manuel Perez called this “unprecedented” and credited the efforts of Ambriz and her peers. (Ambriz remains unconvinced. “None of that has happened,” she says, and payments to the community don’t solve the underlying water issues that the project could exacerbate.)
That affluent La Quinta managed to keep a surf pool out of its community where working-class Thermal failed is even more jarring in light of industry rhetoric about how surf pools could democratize the sport. For Bryan Dickerson, the editor in chief of Wave Pool Magazine, the collective vision for the future is that instead of “the local YMCA … putting in a skate park, they put in a wave pool.” Other proponents, like Ponting, describe how wave pools can provide surf therapy or opportunities for underrepresented groups. A design firm in New York City, for example, has proposed to the city a plan for an indoor wave pool in a low-income, primarily black and Latino neighborhood in Queens—for $30 million.
For its part, PSSC cost an estimated $80 million to build. On top of a $40 general admission fee, a private session like the one I had would cost $3,500 to $5,000 per hour, while a public session would be at least $100 to $200, depending on the surfer’s skill level and the types of waves requested.
In my two days traversing the 45-mile Coachella Valley, I kept thinking about how this whole area was an artificial oasis made possible only by innovations that changed the very nature of the desert, from the railroad stop that spurred development to the irrigation canals and, later, the recharge basins that stopped the wells from running out.
In this transformed environment, I can see how the cognitive dissonance of surfing a desert wave begins to shrink, tempting us to believe that technology can once again override the reality of living (or simply playing) in the desert in a warming and drying world.
But the tension over surf pools shows that when it comes to how we use water, maybe there’s no collective “us” here at all.
MIT Technology Review Explains: Let our writers untangle the complex, messy world of technology to help you understand what’s coming next. You can read more from the series here.
The World Health Organization’s new chatbot launched on April 2 with the best of intentions.
A fresh-faced virtual avatar backed by GPT-3.5, SARAH (Smart AI Resource Assistant for Health) dispenses health tips in eight different languages, 24/7, about how to eat well, quit smoking, de-stress, and more, for millions around the world.
Here we go again. Chatbot fails are now a familiar meme. Meta’s short-lived scientific chatbot Galactica made up academic papers and generated wiki articles about the history of bears in space. In February, Air Canada was ordered to honor a refund policy invented by its customer service chatbot. Last year, a lawyer was fined for submitting court documents filled with fake judicial opinions and legal citations made up by ChatGPT.
The problem is, large language models are so good at what they do that what they make up looks right most of the time. And that makes trusting them hard.
This tendency to make things up—known as hallucination—is one of the biggest obstacles holding chatbots back from more widespread adoption. Why do they do it? And why can’t we fix it?
Magic 8 Ball
To understand why large language models hallucinate, we need to look at how they work. The first thing to note is that making stuff up is exactly what these models are designed to do. When you ask a chatbot a question, it draws its response from the large language model that underpins it. But it’s not like looking up information in a database or using a search engine on the web.
Peel open a large language model and you won’t see ready-made information waiting to be retrieved. Instead, you’ll find billions and billions of numbers. It uses these numbers to calculate its responses from scratch, producing new sequences of words on the fly. A lot of the text that a large language model generates looks as if it could have been copy-pasted from a database or a real web page. But as in most works of fiction, the resemblances are coincidental. A large language model is more like an infinite Magic 8 Ball than an encyclopedia.
Large language models generate text by predicting the next word in a sequence. If a model sees “the cat sat,” it may guess “on.” That new sequence is fed back into the model, which may now guess “the.” Go around again and it may guess “mat”—and so on. That one trick is enough to generate almost any kind of text you can think of, from Amazon listings to haiku to fan fiction to computer code to magazine articles and so much more. As Andrej Karpathy, a computer scientist and cofounder of OpenAI, likes to put it: large language models learn to dream internet documents.
Think of the billions of numbers inside a large language model as a vast spreadsheet that captures the statistical likelihood that certain words will appear alongside certain other words. The values in the spreadsheet get set when the model is trained, a process that adjusts those values over and over again until the model’s guesses mirror the linguistic patterns found across terabytes of text taken from the internet.
To guess a word, the model simply runs its numbers. It calculates a score for each word in its vocabulary that reflects how likely that word is to come next in the sequence in play. The word with the best score wins. In short, large language models are statistical slot machines. Crank the handle and out pops a word.
It’s all hallucination
The takeaway here? It’s all hallucination, but we only call it that when we notice it’s wrong. The problem is, large language models are so good at what they do that what they make up looks right most of the time. And that makes trusting them hard.
Can we control what large language models generate so they produce text that’s guaranteed to be accurate? These models are far too complicated for their numbers to be tinkered with by hand. But some researchers believe that training them on even more text will continue to reduce their error rate. This is a trend we’ve seen as large language models have gotten bigger and better.
Another approach involves asking models to check their work as they go, breaking responses down step by step. Known as chain-of-thought prompting, this has been shown to increase the accuracy of a chatbot’s output. It’s not possible yet, but future large language models may be able to fact-check the text they are producing and even rewind when they start to go off the rails.
But none of these techniques will stop hallucinations fully. As long as large language models are probabilistic, there is an element of chance in what they produce. Roll 100 dice and you’ll get a pattern. Roll them again and you’ll get another. Even if the dice are, like large language models, weighted to produce some patterns far more often than others, the results still won’t be identical every time. Even one error in 1,000—or 100,000—adds up to a lot of errors when you consider how many times a day this technology gets used.
The more accurate these models become, the more we will let our guard down. Studies show that the better chatbots get, the more likely people are to miss an error when it happens.
Perhaps the best fix for hallucination is to manage our expectations about what these tools are for. When the lawyer who used ChatGPT to generate fake documents was asked to explain himself, he sounded as surprised as anyone by what had happened. “I heard about this new site, which I falsely assumed was, like, a super search engine,” he told a judge. “I did not comprehend that ChatGPT could fabricate cases.”
This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here.
Knock, knock.
Who’s there?
An AI with generic jokes. Researchers from Google DeepMind asked 20 professional comedians to use popular AI language models to write jokes and comedy performances. Their results were mixed.
The comedians said that the tools were useful in helping them produce an initial “vomit draft” that they could iterate on, and helped them structure their routines. But the AI was not able to produce anything that was original, stimulating, or, crucially, funny. My colleague Rhiannon Williams has the full story.
As Tuhin Chakrabarty, a computer science researcher at Columbia University who specializes in AI and creativity, told Rhiannon, humor often relies on being surprising and incongruous. Creative writing requires its creator to deviate from the norm, whereas LLMs can only mimic it.
And that is becoming pretty clear in the way artists are approaching AI today. I’ve just come back from Hamburg, which hosted one of the largest events for creatives in Europe, and the message I got from those I spoke to was that AI is too glitchy and unreliable to fully replace humans and is best used instead as a tool to augment human creativity.
Right now, we are in a moment where we are deciding how much creative power we are comfortable giving AI companies and tools. After the boom first started in 2022, when DALL-E 2 and Stable Diffusion first entered the scene, many artists raised concerns that AI companies were scraping their copyrighted work without consent or compensation. Tech companies argue that anything on the public internet falls under fair use, a legal doctrine that allows the reuse of copyrighted-protected material in certain circumstances. Artists, writers, image companies, and the New York Times have filed lawsuits against these companies, and it will likely take years until we have a clear-cut answer as to who is right.
Meanwhile, the court of public opinion has shifted a lot in the past two years. Artists I have interviewed recently say they were harassed and ridiculed for protesting AI companies’ data-scraping practices two years ago. Now, the general public is more aware of the harms associated with AI. In just two years, the public has gone from being blown away by AI-generated images to sharing viral social media posts about how to opt out of AI scraping—a concept that was alien to most laypeople until very recently. Companies have benefited from this shift too. Adobe has been successful in pitching its AI offerings as an “ethical” way to use the technology without having to worry about copyright infringement.
There are also several grassroots efforts to shift the power structures of AI and give artists more agency over their data. I’ve written about Nightshade, a tool created by researchers at the University of Chicago, which lets users add an invisible poison attack to their images so that they break AI models when scraped. The same team is behind Glaze, a tool that lets artists mask their personal style from AI copycats. Glaze has been integrated into Cara, a buzzy new art portfolio site and social media platform, which has seen a surge of interest from artists. Cara pitches itself as a platform for art created by people; it filters out AI-generated content. It got nearly a million new users in a few days.
This all should be reassuring news for any creative people worried that they could lose their job to a computer program. And the DeepMind study is a great example of how AI can actually be helpful for creatives. It can take on some of the boring, mundane, formulaic aspects of the creative process, but it can’t replace the magic and originality that humans bring. AI models are limited to their training data and will forever only reflect the zeitgeist at the moment of their training. That gets old pretty quickly.
Now read the rest of The Algorithm
Deeper Learning
Apple is promising personalized AI in a private cloud. Here’s how that will work.
Last week, Apple unveiled its vision for supercharging its product lineup with artificial intelligence. The key feature, which will run across virtually all of its product line, is Apple Intelligence, a suite of AI-based capabilities that promises to deliver personalized AI services while keeping sensitive data secure.
Why this matters: Apple says its privacy-focused system will first attempt to fulfill AI tasks locally on the device itself. If any data is exchanged with cloud services, it will be encrypted and then deleted afterward. It’s a pitch that offers an implicit contrast with the likes of Alphabet, Amazon, or Meta, which collect and store enormous amounts of personal data. Read more from James O’Donnell here.
Bits and Bytes
How to opt out of Meta’s AI training If you post or interact with chatbots on Facebook, Instagram, Threads, or WhatsApp, Meta can use your data to train its generative AI models. Even if you don’t use any of Meta’s platforms, it can still scrape data such as photos of you if someone else posts them. Here’s our quick guide on how to opt out. (MIT Technology Review)
Microsoft’s Satya Nadella is building an AI empire Nadella is going all in on AI. His $13 billion investment in OpenAI was just the beginning. Microsoft has become an “the world’s most aggressive amasser of AI talent, tools, and technology” and has started building an in-house OpenAI competitor. (The Wall Street Journal)
OpenAI has hired an army of lobbyists As countries around the world mull AI legislation, OpenAI is on a lobbyist hiring spree to protect its interests. The AI company has expanded its global affairs team from three lobbyists at the start of 2023 to 35 and intends to have up to 50 by the end of this year. (Financial Times)
UK rolls out Amazon-powered emotion recognition AI cameras on trains People traveling through some of the UK’s biggest train stations have likely had their faces scanned by Amazon software without their knowledge during an AI trial. London stations such as Euston and Waterloo have tested CCTV cameras with AI to reduce crime and detect people’s emotions. Emotion recognition technology is extremely controversial. Experts say it is unreliable and simply does not work. (Wired)
Clearview AI used your face. Now you may get a stake in the company. The facial recognition company, which has been under fire for scraping images of people’s faces from the web and social media without their permission, has agreed to an unusual settlement in a class action against it. Instead of paying cash, it is offering a 23% stake in the company for Americans whose faces are in its data sets. (The New York Times)
Elephants call each other by their names This is so cool! Researchers used AI to analyze the calls of two herds of African savanna elephants in Kenya. They found that elephants use specific vocalizations for each individual and recognize when they are being addressed by other elephants. (The Guardian)
Meta has created a system that can embed hidden signals, known as watermarks, in AI-generated audio clips, which could help in detecting AI-generated content online.
The tool, called AudioSeal, is the first that can pinpoint which bits of audio in, for example, a full hourlong podcast might have been generated by AI. It could help to tackle the growing problem of misinformation and scams using voice cloning tools, says Hady Elsahar, a research scientist at Meta. Malicious actors have used generative AI to create audio deepfakes of President Joe Biden, and scammers have used deepfakes to blackmail their victims. Watermarks could in theory help social media companies detect and remove unwanted content.
However, there are some big caveats. Meta says it has no plans yet to apply the watermarks to AI-generated audio created using its tools. Audio watermarks are not yet adopted widely, and there is no single agreed industry standard for them. And watermarks for AI-generated content tend to be easy to tamper with—for example, by removing or forging them.
Fast detection, and the ability to pinpoint which elements of an audio file are AI-generated, will be critical to making the system useful, says Elsahar. He says the team achieved between 90% and 100% accuracy in detecting the watermarks, much better results than in previous attempts at watermarking audio.
AudioSeal is available on GitHub for free. Anyone can download it and use it to add watermarks to AI-generated audio clips. It could eventually be overlaid on top of AI audio generation models, so that it is automatically applied to any speech generated using them. The researchers who created it will present their work at the International Conference on Machine Learning in Vienna, Austria, in July.
AudioSeal is created using two neural networks. One generates watermarking signals that can be embedded into audio tracks. These signals are imperceptible to the human ear but can be detected quickly using the other neural network. Currently, if you want to try to spot AI-generated audio in a longer clip, you have to comb through the entire thing in second-long chunks to see if any of them contain a watermark. This is a slow and laborious process, and not practical on social media platforms with millions of minutes of speech.
AudioSeal works differently: by embedding a watermark throughout each section of the entire audio track. This allows the watermark to be “localized,” which means it can still be detected even if the audio is cropped or edited.
Ben Zhao, a computer science professor at the University of Chicago, says this ability, and the near-perfect detection accuracy, makes AudioSeal better than any previous audio watermarking system he’s come across.
“It’s meaningful to explore research improving the state of the art in watermarking, especially across mediums like speech that are often harder to mark and detect than visual content,” says Claire Leibowicz, head of AI and media integrity at the nonprofit Partnership on AI.
But there are some major flaws that need to be overcome before these sorts of audio watermarks can be adopted en masse. Meta’s researchers tested different attacks to remove the watermarks and found that the more information is disclosed about the watermarking algorithm, the more vulnerable it is. The system also requires people to voluntarily add the watermark to their audio files.
This places some fundamental limitations on the tool, says Zhao. “Where the attacker has some access to the [watermark] detector, it’s pretty fragile,” he says. And this means only Meta will be able to verify whether audio content is AI-generated or not.
Leibowicz says she remains unconvinced that watermarks will actually further public trust in the information they’re seeing or hearing, despite their popularity as a solution in the tech sector. That’s partly because they are themselves so open to abuse.
“I’m skeptical that any watermark will be robust to adversarial stripping and forgery,” she adds.
This story first appeared in China Report, MIT Technology Review’s newsletter about technology in China. Sign up to receive it in your inbox every Tuesday.
If you’ve ever been to Taiwan, you’ve likely run into Gogoro’s green-and-white battery-swap stations in one city or another. With 12,500 stations around the island, Gogoro has built a sweeping network that allows users of electric scooters to drop off an empty battery and get a fully charged one immediately. Gogoro is also found in China, India, and a few other countries.
This morning, I published a story on how Gogoro’s battery-swap network in Taiwan reacted to emergency blackouts after the 7.4 magnitude earthquake there this April. I talked to Horace Luke, Gogoro’s cofounder and CEO, to understand how in three seconds, over 500 Gogoro battery-swap locations stopped drawing electricity from the grid, helping stabilize the power frequency.
Gogoro’s battery stations acted like something called a virtual power plant (VPP), a new idea that’s becoming adopted around the world as a way to stitch renewable energy into the grid. The system draws energy from distributed sources like battery storage or small rooftop solar panels and coordinates those sources to increase supply when electricity demand peaks. As a result, it reduces the reliance on traditional coal or gas power plants.
There’s actually a natural synergy between technologies like battery swapping and virtual power plants (VPP). Not only can battery-swap stations coordinate charging times with the needs of the grid, but the idle batteries sitting in Gogoro’s stations can also become an energy reserve in times of emergency, potentially feeding energy back to the grid. If you want to learn more about how this system works, you can read the full story here.
Statistics shared by Gogoro and Enel X show how its battery-swap stations automatically stopped charging batteries on April 3 and April 15, when there were power outages caused by the earthquake.
GOGORO
When I talked to Gogoro’s Luke for this story, I asked him: “At what point in the company’s history did you come up with the idea to use these batteries for VPP networks?”
To my surprise, Luke answered: “Day one.”
As he explains, Gogoro was actually not founded to be an electric-scooter company; it was founded to be a “smart energy” company.
“We started with the thesis of how smart energy, through portability and connectivity, can enable many use case scenarios,” Luke says. “Transportation happens to be accounting for something like 27% or 28% of your energy use in your daily life.” And that’s why the company first designed the batteries for two-wheeled vehicles, a popular transportation option in Taiwan and across Asia.
Having succeeded in promoting its scooters and the battery-swap charging method in Taiwan, it is now able to explore other possible uses of these modular, portable batteries—more than 1.4 million of which are in circulation at this point.
“Think of smart, portable, connected energy like a propane tank,” Luke says. Depending on their size, propane tanks can be used to cook in the wild or to heat a patio. If lithium batteries can be modular and portable in a similar way, they can also serve many different purposes.
Using them in VPP programs that protect the grid from blackouts is one; beyond that, in Taipei City, Gogoro has worked with the local government to build energy backup stations for traffic lights, using the same batteries to keep the lights running in future blackouts. The batteries can also be used as backup power storage for critical facilities like hospitals. When a blackout happens, battery storage can release electricity much faster than diesel generators, keeping the impact at a minimum.
None of this would be possible without the recent advances that have made batteries more powerful and efficient. And it was clear from our conversation that Luke is obsessed with batteries—the long way the technology has come, and their potential to address a lot more energy use cases in the future.
“I still remember getting my first flashlight when I was a little kid. That button just turned the little lightbulb on and off. And that was what was amazing about batteries at the time,” says Luke. “Never did people think that AA batteries were going to power calculators or the Walkman. The guy that invented the alkaline battery never thought that. We’ll continue to take that creativity and apply it to portable energy, and that’s what inspires us every day.”
What other purposes do you think portable lithium batteries like the ones made by Gogoro could have? Let me know your ideas by writing to zeyi@technologyreview.com.
Now read the rest of China Report
Catch up with China
1. Far-right parties won big in the latest European Parliament elections, which could push the EU further toward a trade war with China. (Nikkei Asia $)
2. Volvo has started moving some of its manufacturing capacity from China to Belgium in order to avoid the European Union tariffs on Chinese imports. (The Times $)
3. Some major crypto exchanges have withdrawn from applying for business licenses in Hong Kong after the city government clarified that it doesn’t welcome businesses that offer crypto services to mainland China. (South China Morning Post $)
4. NewsBreak, the most downloaded news app in the US, does most of its engineering work in China. The app has also been found to use AI tools to make up local news that never happened. (Reuters $)
5. The Australian government ordered a China-linked fund to reduce its investment in an Australian rare-earth-mining company. (A/symmetric)
6. China just installed the largest offshore wind turbine in the world. It’s designed to generate enough power in a year for around 36,000 households. (Electrek)
7. Four college instructors from Iowa were stabbed on a visit to northern China. While the motive and identity of the assailant are still unknown, the incident has been quickly censored on the Chinese internet. (BBC)
Lost in translation
Qian Zhimin, a Chinese businesswoman who fled the country in 2017 after raising billions of dollars from Chinese investors in the name of bitcoin investments, was arrested in London and is facing a trial in October this year, according to the Chinese publication Caijing. In the early 2010s, when the cryptocurrency first became known in China, Qian’s company lured over 128,000 retail investors, predominantly elderly people, to buy fraudulent investment products that bet on the price of bitcoins and gadgets like smart bracelets that allegedly could also mine bitcoins.
After the scam was exposed, Qian escaped to the UK with a fake passport. She controls over 61,000 bitcoins, now worth nearly $4 billion, and has been trying to liquidate them by buying properties in London. But those attempts caught the attention of anti-money-laundering authorities in the UK. With her trial date approaching, the victims in China are hoping to work with the UK jurisdiction to recover their assets.
One more thing
I know one day we will see self-driving vehicles racing each other and cutting each other off, but I didn’t expect it to happen so soon with two package delivery robots in China. Maybe it’s just their look, but it seems cuter than when human drivers do the same thing?
TBH, I was expecting a world where unmanned delivery vehicles racing each other on busy streets to come maybe 5 yrs from now, but JD & its subsidiary Dada are making it happen w/o hitting anything
It’s a thought that occurs to every video-game player at some point: What if the weird, hyper-focused state I enter when playing in virtual worlds could somehow be applied to the real one?
Often pondered during especially challenging or tedious tasks in meatspace (writing essays, say, or doing your taxes), it’s an eminently reasonable question to ask. Life, after all, is hard. And while video games are too, there’s something almost magical about the way they can promote sustained bouts of superhuman concentration and resolve.
For some, this phenomenon leads to an interest in flow states and immersion. For others, it’s simply a reason to play more games. For a handful of consultants, startup gurus, and game designers in the late 2000s, it became the key to unlocking our true human potential.
In her 2010 TED Talk, “Gaming Can Make a Better World,” the game designer Jane McGonigal called this engaged state “blissful productivity.” “There’s a reason why the average World of Warcraft gamer plays for 22 hours a week,” she said. “It’s because we know when we’re playing a game that we’re actually happier working hard than we are relaxing or hanging out. We know that we are optimized as human beings to do hard and meaningful work. And gamers are willing to work hard all the time.”
McGonigal’s basic pitch was this: By making the real world more like a video game, we could harness the blissful productivity of millions of people and direct it at some of humanity’s thorniest problems—things like poverty, obesity, and climate change. The exact details of how to accomplish this were a bit vague (play more games?), but her objective was clear: “My goal for the next decade is to try to make it as easy to save the world in real life as it is to save the world in online games.”
While the word “gamification” never came up during her talk, by that time anyone following the big-ideas circuit (TED, South by Southwest, DICE, etc.) or using the new Foursquare app would have been familiar with the basic idea. Broadly defined as the application of game design elements and principles to non-game activities—think points, levels, missions, badges, leaderboards, reinforcement loops, and so on—gamification was already being hawked as a revolutionary new tool for transforming education, work, health and fitness, and countless other parts of life.
Instead of liberating us, gamification turned out to be just another tool for coercion, distraction, and control.
Adding “world-saving” to the list of potential benefits was perhaps inevitable, given the prevalence of that theme in video-game storylines. But it also spoke to gamification’s foundational premise: the idea that reality is somehow broken. According to McGonigal and other gamification boosters, the real world is insufficiently engaging and motivating, and too often it fails to make us happy. Gamification promises to remedy this design flawby engineering a new reality, one that transforms the dull, difficult, and depressing parts of life into something fun and inspiring. Studying for exams, doing household chores, flossing, exercising, learning a new language—there was no limit to the tasks that could be turned into games, making everything IRL better.
Today, we live in an undeniably gamified world. We stand up and move around to close colorful rings and earn achievement badges on our smartwatches; we meditate and sleep to recharge our body batteries; we plant virtual trees to be more productive; we chase “likes” and “karma” on social media sites and try to swipe our way toward social connection. And yet for all the crude gamelike elements that have been grafted onto our lives, the more hopeful and collaborative world that gamification promised more than a decade ago seems as far away as ever. Instead of liberating us from drudgery and maximizing our potential, gamification turned out to be just another tool for coercion, distraction, and control.
Con game
This was not an unforeseeable outcome. From the start, a small but vocal group of journalists and game designers warned against the fairy-tale thinking and facile view of video games that they saw in the concept of gamification. Adrian Hon, author of You’ve Been Played, a recent book that chronicles its dangers, was one of them.
“As someone who was building so-called ‘serious games’ at the time the concept was taking off, I knew that a lot of the claims being made around the possibility of games to transform people’s behaviors and change the world were completely overblown,” he says.
Hon isn’t some knee-jerk polemicist. A trained neuroscientist who switched to a career in game design and development, he’s the co-creator of Zombies, Run!—one of the most popular gamified fitness apps in the world. While he still believes games can benefit and enrich aspects of our nongaming lives, Hon says a one-size-fits-all approach is bound to fail. For this reason, he’s firmly against both the superficial layering of generic points, leaderboards, and missions atop everyday activities and the more coercive forms of gamification that have invaded the workplace.
SELMAN DESIGN
Ironically, it’s these broad and varied uses that make criticizing the practice so difficult. As Hon notes in his book, gamification has always been a fast-moving target, varying dramatically in scale, scope, and technology over the years. As the concept has evolved, so too have its applications, whether you think of the gambling mechanics that now encourage users of dating apps to keep swiping, the “quests” that compel exhausted Uber drivers to complete just a few more trips, or the utopian ambition of using gamification to save the world.
In the same way that AI’s lack of a fixed definition today makes it easy to dismiss any one critique for not addressing some other potential definition of it, so too do gamification’s varied interpretations. “I remember giving talks critical of gamification at gamification conferences, and people would come up to me afterwards and be like, ‘Yeah, bad gamification is bad, right? But we’re doing good gamification,’” says Hon. (They weren’t.)
For some critics, the very idea of “good gamification” was anathema. Their main gripe with the term and practice was, and remains, that it has little to nothing to do with actual games.
“A game is about play and disruption and creativity and ambiguity and surprise,” wrote the late Jeff Watson, a game designer, writer, and educator who taught at the University of Southern California’s School of Cinematic Arts. Gamification is about the opposite—the known, the badgeable, the quantifiable. “It’s about ‘checking in,’ being tracked … [and] becoming more regimented. It’s a surveillance and discipline system—a wolf in sheep’s clothing. Beware its lure.”
Another game designer, Margaret Robertson, has argued that gamification should really be called “pointsification,” writing: “What we’re currently terming gamification is in fact the process of taking the thing that is least essential to games and representing it as the core of the experience. Points and badges have no closer a relationship to games than they do to websites and fitness apps and loyalty cards.”
For the author and game designer Ian Bogost, the entire concept amounted to a marketing gimmick. In a now-famous essay published in the Atlantic in 2011, he likened gamification to the moral philosopher Harry Frankfurt’s definition of bullshit—that is, a strategy intended to persuade or coerce without regard for actual truth.
“The idea of learning or borrowing lessons from game design and applying them to other areas was never the issue for me,” Bogost told me. “Rather, it was not doing that—acknowledging that there’s something mysterious, powerful, and compelling about games, but rather than doing the hard work, doing no work at all and absconding with the spirit of the form.”
Gaming the system
So how did a misleading term for a misunderstood process that’s probably just bullshit come to infiltrate virtually every part of our lives? There’s no one simple answer. But gamification’s meteoric rise starts to make a lot more sense when you look at the period that gave birth to the idea.
The late 2000s and early 2010s were, as many have noted, a kind of high-water mark for techno-optimism. For people both inside the tech industry and out, there was a sense that humanity had finally wrapped its arms around a difficult set of problems, and that technology was going to help us squeeze out some solutions. The Arab Spring bloomed in 2011 with the help of platforms like Facebook and Twitter, money was more or less free, and “____ can save the world” articles were legion (with ____ being everything from “eating bugs” to “design thinking”).
This was also the era that produced the 10,000-hours rule of success, the long tail, the four-hour workweek, the wisdom of crowds, nudge theory, and a number of other highly simplistic (or, often, flat-out wrong) theories about the way humans, the internet, and the world work.
“All of a sudden you had VC money and all sorts of important, high-net-worth people showing up at game developer conferences.”
Ian Bogost, author and game designer
Adding video games to this heady stew of optimism gave the game industry something it had long sought but never achieved: legitimacy. Even with games ascendant in popular culture—and on track to eclipse both the film and music industries in terms of revenue—they still were largely seen as a frivolous, productivity-squandering, violence-encouraging form of entertainment. Seemingly overnight, gamification changed all that.
“There was definitely this black-sheep mentality in the game development community—the sense that what we had been doing for decades was just a joke to people,” says Bogost. “All of a sudden you had VC money and all sorts of important, high-net-worth people showing up at game developer conferences, and it was like, ‘Finally someone’s noticing. They realize that we have something to offer.’”
This wasn’t just flattering; it was intoxicating. Gamification took a derided pursuit and recast it as a force for positive change, a way to make the real world better. While enthusiastic calls to “build a game layer on top of reality” may sound dystopian to many of us today, the sentiment didn’t necessarily have the same ominous undertones at the end of the aughts.
Combine the cultural recasting of games with an array of cheaper and faster technologies—GPS, ubiquitous and reliable mobile internet, powerful smartphones, Web 2.0 tools and services—and you arguably had all the ingredients needed for gamification’s rise. In a very real sense, reality in 2010 was ready to be gamified. Or to put it a slightly different way: Gamification was an idea perfectly suited for its moment.
Gaming behavior
Fine, you might be asking at this point, but does it work? Surely, companies like Apple, Uber, Strava, Microsoft, Garmin, and others wouldn’t bother gamifying their products and services if there were no evidence of the strategy’s efficacy. The answer to the question, unfortunately, is super annoying: Define work.
Because gamification is so pervasive and varied, it’s hard to address its effectiveness in any direct or comprehensive way. But one can confidently say this: Gamification did not save the world. Climate change still exists. As do obesity, poverty, and war. Much of generic gamification’s power supposedly resides in its ability to nudge or steer us toward, or away from, certain behaviors using competition (challenges and leaderboards), rewards (points and achievement badges), and other sources of positive and negative feedback.
Gamification is, and has always been, a way to induce specific behaviors in people using virtual carrots and sticks.
On that front, the results are mixed. Nudge theory lost much of its shine with academics in 2022 after a meta-analysis of previous studies concluded that, after correcting for publication bias, there wasn’t much evidence it worked to change behavior at all. Still, there are a lot of ways to nudge and a lot of behaviors to modify. The fact remains that plenty of people claim to be highly motivated to close their rings, earn their sleep crowns, or hit or exceed some increasingly ridiculous number of steps on their Fitbits (see humorist David Sedaris).
In truth, gamification acolytes were always pulling from an old playbook—one that dates back to the early 20th century. Then, behaviorists like John Watson and B.F. Skinner saw human behaviors (a category that for Skinner included thoughts, actions, feelings, and emotions) not as the products of internal mental states or cognitive processes but, rather, as the result of external forces—forces that could conveniently be manipulated.
If Skinner’s theory of operant conditioning, which doled out rewards to positively reinforce certain behaviors, sounds a lot like Amazon’s “Fulfillment Center Games,” which dole out rewards to compel workers to work harder, faster, and longer—well, that’s not a coincidence. Gamification is, and has always been, a way to induce specific behaviors in people using virtual carrots and sticks.
Sometimes this may work; other times not. But ultimately, as Hon points out, the question of efficacy may be beside the point. “There is no before or after to compare against if your life is always being gamified,” he writes. “There isn’t even a static form of gamification that can be measured, since the design of coercive gamification is always changing, a moving target that only goes toward greater and more granular intrusion.”
The game of life
Like any other art form, video games offer a staggering array of possibilities. They can educate, entertain, foster social connection, inspire, and encourage us to see the world in different ways. Some of the best ones manage to do all of this at once.
Yet for many of us, there’s the sense today that we’re stuck playing an exhausting game that we didn’t opt into. This one assumes that our behaviors can be changed with shiny digital baubles, constant artificial competition, and meaningless prizes. Even more insulting, the game acts as if it exists for our benefit—promising to make us fitter, happier, and more productive—when in truth it’s really serving the commercial and business interests of its makers.
Metaphors can be an imperfect but necessary way to make sense of the world. Today, it’s not uncommon to hear talk of leveling up, having a God Mode mindset, gaining XP, and turning life’s difficulty settings up (or down). But the metaphor that resonates most for me—the one that seems to neatly capture our current predicament—is that of the NPC, or non-player character.
NPCs are the “Sisyphean machines” of video games, programmed to follow a defined script forever and never question or deviate. They’re background players in someone else’s story, typically tasked with furthering a specific plotline or performing some manual labor. To call someone an NPC in real life is to accuse them of just going through the motions, not thinking for themselves, not being able to make their own decisions. This, for me, is gamification’s real end result. It’s acquiescence pretending to be empowerment. It strips away the very thing that makes games unique—a sense of agency—and then tries to mask that with crude stand-ins for accomplishment.
So what can we do? Given the reach and pervasiveness of gamification, critiquing it at this point can feel a little pointless, like railing against capitalism. And yet its own failed promises may point the way to a possible respite. If gamifying the world has turned our lives into a bad version of a video game, perhaps this is the perfect moment to reacquaint ourselves with why actual video games are great in the first place. Maybe, to borrow an idea from McGonigal, we should all start playing better games.
Bryan Gardiner is a writer based in Oakland, California.
