Learning from catastrophe

The philosopher Karl Popper once argued that there are two kinds of problems in the world: clock problems and cloud problems. As the metaphor suggests, clock problems obey a certain logic. They are orderly and can be broken down and analyzed piece by piece. When a clock stops working, you’re able to take it apart, look for what’s wrong, and fix it. The fix may not be easy, but it’s achievable. Crucially, you know when you’ve solved the issue because the clock starts telling the time again. 

W.W. NORTON, 2024

Cloud problems offer no such assurances. They are inherently complex and unpredictable, and they usually have social, psychological, or political dimensions. Because of their dynamic, shape-shifting nature, trying to “fix” a cloud problem often ends up creating several new problems. For this reason, they don’t have a definitive “solved” state—only good and bad (or better and worse) outcomes. Trying to repair a broken-down car is a clock problem. Trying to solve traffic is a cloud problem.  

Engineers are renowned clock-problem solvers. They’re also notorious for treating every problem like a clock. Increasing specialization and cultural expectations play a role in this tendency. But so do engineers themselves, who are typically the ones who get to frame the problems they’re trying to solve in the first place. 

In his latest book, Wicked Problems, Guru Madhavan argues that the growing number of cloudy problems in our world demands a broader, more civic-minded approach to engineering. “Wickedness” is Madhavan’s way of characterizing what he calls “the cloudiest of problems.” It’s a nod to a now-famous coinage by Horst Rittel and Melvin Webber, professors at the University of California, Berkeley, who used the term “wicked” to describe complex social problems that resisted the rote scientific and engineering-based (i.e., clock-like) approaches that were invading their fields of design and urban planning back in the 1970s. 

Madhavan, who’s the senior director of programs at the National Academy of Engineering, is no stranger to wicked problems himself. He’s tackled such daunting examples as trying to make prescription drugs more affordable in the US and prioritizing development of new vaccines. But the book isn’t about his own work. Instead, Wicked Problems weaves together the story of a largely forgotten aviation engineer and inventor, Edwin A. Link, with case studies of man-made and natural disasters that Madhavan uses to explain how wicked problems take shape in society and how they might be tamed.

Link’s story, for those who don’t know it, is fascinating—he was responsible for building the first mechanical flight trainer, using parts from his family’s organ factory—and Madhavan gives a rich and detailed accounting. The challenges this inventor faced in the 1920s and ’30s—which included figuring out how tens of thousands of pilots could quickly and effectively be trained to fly without putting all of them up in the air (and in danger), as well as how to instill trust in “instrument flying” when pilots’ instincts frequently told them their instruments were wrong—were among the quintessential wicked problems of his time. 

To address a world full of wicked problems, we’re going to need a more expansive and inclusive idea of what engineering is and who gets to participate in it.

Unfortunately, while Link’s biography and many of the interstitial chapters on disasters, like Boston’s Great Molasses Flood of 1919, are interesting and deeply researched, Wicked Problems suffers from some wicked structural choices. 

The book’s elaborate conceptual framework and hodgepodge of narratives feel both fussy and unnecessary, making a complex and nuanced topic even more difficult to grasp at times. In the prologue alone, readers must bounce from the concept of cloud problems to that of wicked problems, which get broken down into hard, soft, and messy problems, which are then reconstituted in different ways and linked to six attributes—efficiency, vagueness, vulnerability, safety, maintenance, and resilience—that, together, form what Madhavan calls a “concept of operations,” which is the primary organizational tool he uses to examine wicked problems.

It’s a lot—or at least enough to make you wonder whether a “systems engineering” approach was the correct lens through which to examine wickedness. It’s also unfortunate because Madhavan’s ultimate argument is an important one, particularly in an age of rampant solutionism and “one neat trick” approaches to complex problems. To effectively address a world full of wicked problems, he says, we’re going to need a more expansive and inclusive idea of what engineering is and who gets to participate in it.  

MIT PRESS, 2024

While John Downer would likely agree with that sentiment, his new book, Rational Accidents, makes a strong argument that there are hard limits to even the best and broadest engineering approaches. Similarly set in the world of aviation, Downer’s book explores a fundamental paradox at the heart of today’s civil aviation industry: the fact that flying is safer and more reliable than should technically be possible.

Jetliners are an example of what Downer calls a “catastrophic technology.” These are “complex technological systems that require extraordinary, and historically unprecedented, failure rates—of the order of hundreds of millions, or even billions, of operational hours between catastrophic failures.”

Take the average modern jetliner, with its 7 million components and 170 miles’ worth of wiring—an immensely complex system in and of itself. There were over 25,000 jetliners in regular service in 2014, according to Downer. Together, they averaged 100,000 flights every single day. Now consider that in 2017, no passenger-carrying commercial jetliner was involved in a fatal accident. Zero. That year, passenger totals reached 4 billion on close to 37 million flights. Yes, it was a record-setting year for the airline industry, safety-wise, but flying remains an almost unfathomably safe and reliable mode of transportation—even with Boeing’s deadly 737 Max crashes in 2018 and 2019 and the company’s ongoing troubles. 

Downer, a professor of science and technology studies at the University of Bristol, does an excellent job in the first half of the book dismantling the idea that we can objectively recognize, understand, and therefore control all risk involved in such complex technologies. Using examples from well-known jetliner crashes, as well as from the Fukushima nuclear plant meltdown, he shows why there are simply too many scenarios and permutations of failure for us to assess or foresee such risks, even with today’s sophisticated modeling techniques and algorithmic assistance.

So how does the airline industry achieve its seemingly unachievable record of safety and reliability? It’s not regulation, Downer says. Instead, he points to three unique factors. First is the massive service experience the industry has amassed. Over the course of 70 years, manufacturers have built tens of thousands of jetliners, which have failed (and continue to fail) in all sorts of unpredictable ways. 

This deep and constantly growing data set, combined with the industry’s commitment to thoroughly investigating each and every failure, lets it generalize the lessons learned across the entire industry—the second key to understanding jetliner reliability. 

Finally is what might be the most interesting and counterintuitive factor: Downer argues that the lack of innovation in jetliner design is an essential but overlooked part of the reliability record. The fact that the industry has been building what are essentially iterations of the same jetliner for 70 years ensures that lessons learned from failures are perpetually relevant as well as generalizable, he says. 

That extremely cautious relationship to change flies in the face of the innovate-or-die ethos that drives most technology companies today. And yet it allows the airline industry to learn from decades of failures and continue to chip away at the future “failure performance” of jetliners.

The bad news is that the lessons in jetliner reliability aren’t transferable to other catastrophic technologies. “It is an irony of modernity that the only catastrophic technology with which we have real experience, the jetliner, is highly unrepresentative, and yet it reifies a misleading perception of mastery over catastrophic technologies in general,” writes Downer.

For instance, to make nuclear reactors as reliable as jetliners, that industry would need to commit to one common reactor design, build tens of thousands of reactors, operate them for decades, suffer through thousands of catastrophes, slowly accumulate lessons and insights from those catastrophes, and then use them to refine that common reactor design.  

This obviously won’t happen. And yet “because we remain entranced by the promise of implausible reliability, and implausible certainty about that reliability, our appetite for innovation has outpaced our insight and humility,” writes Downer. With the age of catastrophic technologies still in its infancy, our continued survival may very well hinge not on innovating our way out of cloudy or wicked problems, but rather on recognizing, and respecting, what we don’t know and can probably never understand.  

If Wicked Problems and Rational Accidents are about the challenges and limits of trying to understand complex systems using objective science- and engineering-based methods, Georgina Voss’s new book, Systems Ultra, provides a refreshing alternative. Rather than dispassionately trying to map out or make sense of complex systems from the outside, Voss—a writer, artist, and researcher—uses her book to grapple with what they feel like, and ultimately what they mean, from the inside.

VERSO, 2024

“There is something rather wonderful about simply feeling our way through these enormous structures,” she writes before taking readers on a whirlwind tour of systems visible and unseen, corrupt and benign, ancient and new. Stops include the halls of hype at Las Vegas’s annual Consumer Electronics Show (“a hot mess of a Friday casual hellscape”), the “memetic gold mine” that was the container ship Ever Given and the global supply chain it broke when it got stuck in the Suez Canal, and the payment systems that undergird the porn industry. 

For Voss, systems are both structure and behavior. They are relational technologies that are “defined by their ability to scale and, perhaps more importantly, their peculiar relationship to scale.” She’s also keenly aware of the pitfalls of using an “experiential” approach to make sense of these large-scale systems. “Verbal attempts to neatly encapsulate what a system is can feel like a stoner monologue with pointed hand gestures (‘Have you ever thought about how electricity is, like, really big?’),” she writes. 

Nevertheless, her written attempts are a delight to read. Voss manages to skillfully unpack the power structures that make up, and reinforce, the large-scale systems we live in. Along the way, she also dispels many of the stories we’re told about their inscrutability and inevitability. That she does all this with humor, intelligence, and a boundless sense of curiosity makes Systems Ultra both a shining example of the “civic engagement as engineering” approach that Madhavan argues for in Wicked Problems, and proof that his argument is spot on. 

Bryan Gardiner is a writer based in Oakland, California.

Hong Kong is targeting Western Big Tech companies in its new ban of a popular protest song

It wasn’t exactly surprising when, on Wednesday May 8, a Hong Kong appeals court sided with the city government to take down “Glory to Hong Kong” from the internet. The trial, in which no one represented the defense, was the culmination of a years-long battle over the song that has become the unofficial anthem for protesters fighting China’s tightening control and police brutality in the city. But it remains an open question how exactly Western Big Tech companies will respond. Even as the injunction is narrowly designed to make it easier for them to comply, the companies may still be seen as aiding authoritarian control and obstructing internet freedom if they do so.  

Google, Apple, Meta, Spotify, and more have spent the last several years largely refusing to cooperate with previous efforts by the Hong Kong government to prevent the spread of the song, which the government has claimed is a threat to national security. But the government has also hesitated to leverage criminal law to force them to comply with requests for removal of content, which could risk international uproar and have a negative effect on the city’s economy. 

Now, the new ruling seemingly finds a third option: By providing the platforms with a civil injunction that doesn’t invoke criminal prosecution—which is similar to how copyright violations are enforced—the platforms can theoretically face less reputational blowback when they comply with the court order.

“If you look closely at the judgment, it’s basically tailor-made for the tech companies at stake,” says Chung Ching Kwong, a senior analyst at the Inter-Parliamentary Alliance on China, an advocacy organization that connects legislators from over 30 countries to try to hold China accountable. She believes the language in the judgment suggests the tech companies will now be ready to comply with the government’s request.

A Google spokesperson says the company is reviewing the court’s judgment and didn’t respond to specific questions sent by MIT Technology Review. A Meta spokesperson pointed to a statement from Jeff Paine, the managing director of the Asia Internet Coalition, a trade group representing many tech companies in the Asia-Pacific region: The AIC “is assessing the implications of the decision made today, including how the injunction will be implemented, to determine its impact on businesses. We believe that a free and open internet is fundamental to the city’s ambitions to become an international technology and innovation hub.” The AIC did not immediately reply to questions sent via email. Apple and Spotify didn’t immediately respond to requests for comment.

But no matter what these companies do next, the ruling is already having an effect: Just over 24 hours after the court order, some of the 32 YouTube videos that are explicitly named in the injunction as requiring removal were inaccessible for users worldwide, not just in Hong Kong. 

While it’s unclear whether the videos were removed by the platform or by their creators, experts say the court decision will almost certainly set a precedent for more content to be censored from Hong Kong’s internet in the future.

“Censorship of the song would be a clear violation of internet freedom and freedom of expression,” says Yaqiu Wang, the research director for China, Hong Kong, and Taiwan at Freedom House, a human rights advocacy group. “Google and other internet companies should use all available channels to challenge the decision.” 

Erasing a song from the internet

Since “Glory to Hong Kong” was first uploaded to YouTube in August 2019 by an anonymous group called Dgx Music, it’s been adored by protesters and applauded as their anthem. Its popularity only grew after China passed the harsh Hong Kong national security law in 2020. 

It also unsurprisingly became a major flashpoint. With lyrics like, “Liberate Hong Kong, revolution of our times,” the city and national Chinese governments were wary of its spread. 

Their fears escalated when the song was repeatedly mistaken for China’s national anthem at international events, and was broadcast in sporting events after Hong Kong athletes won. By mid 2023, the mistake, intentional or not, had happened 887 times, according to the Hong Kong government’s request for the content’s removal; the request to the court credits YouTube videos and Google search results referring to the song as the “Hong Kong National Anthem” as the reason. 

The government has been arresting people for performing the song on the ground in Hong Kong, but it has been harder to prosecute the online activity since most of the videos and music were uploaded anonymously, and Hong Kong, unlike mainland China, has historically had a free internet. This meant officials needed to explore new approaches to content removal. 

To comply or not to comply

Using the controversial 2020 national security law as legal justification to make requests for removal of certain content deemed threatening, the Hong Kong government has been able to exert pressure on local companies, like internet service providers (ISPs). “In Hong Kong, all the major internet service providers are locally owned or Chinese-owned. For business reasons, probably within the last 20 years, most of the foreign investors like Verizon left on their own,” says Charles Mok, a researcher at Stanford University’s Cyber Policy Center and a former legislator in Hong Kong. “So right now, the government is focusing on telling the customer-facing internet service providers to do the blocking.” And it seems to have been somewhat effective, with a few websites for human rights activist organizations becoming inaccessible locally.

But the city government can’t get its way as easily when the content is on foreign-owned platforms like YouTube or Facebook. Back in 2020, most major Western companies declared they would pause processing data requests from the Hong Kong government while they assessed the law. Over time, some of them have started answering government requests again. But they’ve largely remained firm: Over the first six months of 2023, for example, Meta received 41 requests from the Hong Kong government to obtain user data and answered 0; during the same period, Google received requests for removing 164 items from Google services and ended up removing 82 of them, according to both companies’ transparency reports. Google specifically mentioned that it chose to not remove two YouTube videos and one Google Drive file related to “Glory to Hong Kong.”

Both sides are in tight spots. Tech companies don’t want to lose the Hong Kong market or endanger their local staff, but they are also worried about being seen as complying with authoritarian government actions. And the Hong Kong government doesn’t want to be seen as openly fighting Western platforms while trust in the region’s financial markets is already in decline. In particular, officials fear international headlines if the government invokes criminal law to force tech companies to remove certain content. 

“I think both sides are navigating this balancing act. So the government finally figured out a way that they thought might be able to solve the impasse: by going to the court and narrowly seeking an injunction,” Mok says.

That happened in June 2023, when Hong Kong’s government requested a court injunction to ban the distribution of the song online with the purpose of “inciting others to commit secession.” It named 32 YouTube videos explicitly, including the original version and live performances, translations in other languages, instrumental and opera versions, and an interview of the original creators. But the order would also cover “any adaptation of the song, the melody and/or lyrics of which are substantially the same as the song,” according to court documents. 

The injunction went through a year of back-and-forth hearings, including a lower court ruling that briefly swatted down the ban. But now, the Court of Appeal has granted the government approval. The case can theoretically be appealed one last time, but with no defendants present, that’s unlikely to happen.

The key difference between this action and previous attempts to remove content is that this is a civil injunction, unlike a criminal prosecution—meaning it is, at least legally speaking, closer to a copyright takedown request. In turn, a platform could arguably be less likely to take a reputational hit as long as it removes the content upon request. 

Kwong believes this will indeed make platforms more likely to cooperate and there have already been pretty clear signs to that effect. In one hearing in December, the government was asked by the court to consult online platforms for the feasibility of the injunction. The final judgment this week says that while the platforms “have not taken part in these proceedings, they have indicated that they are ready to accede to the Government’s request if there is a court order.”

“The actual targets in this case, mainly the tech giants, may have less hesitation to comply with a civil court order than a national security order because if it’s the latter, they may also face backfire from the US,” says Eric Yan-Ho Lai, a research fellow at Georgetown Center for Asian Law. 

Lai also says now that the injunction is granted, it will be easier to prosecute an individual based on violating a civil injunction rather than prosecuting someone based on criminal offenses, since the government won’t need to prove criminal intent.

The chilling effect

Immediately after the injunction, human rights advocates called on tech companies to remain committed to their values. “Companies like Google and Apple have repeatedly claimed that they stand by the universal right to freedom of expression. They should put their ideals into practice,” says Freedom House’s Wang. “Google and other tech companies should thoroughly document government demands, and publish detailed transparency reports on content takedowns, both for those initiated by the authorities and those done by the companies themselves.”

Without making their plans clear, it’s too early to know just how tech companies will react. But right after the injunction was granted, the song largely remained available on most platforms, including YouTube, iTunes, and Spotify, for Hong Kong users, according to the South China Morning Post. On iTunes, the song even returned to the top of the download rankings a few hours after the injunction.

One key factor that may still determine corporate cooperation is how far the content removal requests go. There will surely be more videos of the song that are uploaded to YouTube, not to mention independent websites hosting the videos and music for more people to access. Will the government go after each of them too?

The Hong Kong government has previously said in court hearings that it only seeks a local restriction of the online content, meaning content will only be inaccessible to users physically in the city, which large platforms like YouTube can do so without difficulty. 

Theoretically, this allows local residents to still circumvent the ban by using VPN software, but not everyone would be technologically savvy enough to do so. And that wouldn’t do much to minimize the larger chilling effect on free speech, says Kwong from the Inter-Parliamentary Alliance on China. 

“As a Hong Konger living abroad, I do rely on Hong Kong services or international services based in Hong Kong to get a hold of what’s happening in the city. I do use YouTube Hong Kong to see certain things, and I do use Spotify Hong Kong or Apple Music because I want access to Cantopop,” she says. “At the same time, you worry about what you can share with friends in Hong Kong and whatnot. We don’t want to put them into trouble by sharing things that they are not supposed to see, which they should be able to see.”

The court made at least two explicit exemptions to the song’s ban, for “lawful activities conducted in connection with the song, such as those for the purpose of academic activity and news activity.” But even the implementation of these could be incredibly complex and confusing in practice. “In the current political context in Hong Kong, I don’t see anyone willing to take the risk,” Kwong says. 

The government has already arrested prominent journalists in the name of endangering national security, and a new law passed in 2024 has expanded the crimes that can be prosecuted on national security grounds. As with all efforts to suppress free speech, the impact of vague boundaries that encourage self-censorship on potentially sensitive topics is often sprawling and hard to measure. 

“Nobody knows where the actual red line is,” Kwong says.

A brief, weird history of brainwashing

On an early spring day in 1959, Edward Hunter testified before a US Senate subcommittee investigating “the effect of Red China Communes on the United States.” It was the kind of opportunity he relished. A war correspondent who had spent considerable time in Asia, Hunter had achieved brief media stardom in 1951 after his book Brain-Washing in Red China introduced a new concept to the American public: a supposedly scientific system for changing people’s minds, even making them love things they once hated. 

But Hunter wasn’t just a reporter, objectively chronicling conditions in China. As he told the assembled senators, he was also an anticommunist activist who served as a propagandist for the OSS, or Office of Strategic Services—something that was considered normal and patriotic at the time. His reporting blurred the line between fact and political mythology.

WIKIMEDIA COMMONS

When a senator asked about Hunter’s work for the OSS, the operative boasted that he was the first to “discover the technique of mind-attack” in mainland China, the first to use the word “brainwashing” in writing in any language, and “the first, except for the Chinese, to use the word in speech in any language.” 

None of this was true. Other operatives associated with the OSS had used the word in reports before Hunter published articles about it. More important, as the University of Hong Kong legal scholar Ryan Mitchell has pointed out, the Chinese word Hunter used at the hearing—xinao (), translated as “wash brain”—has a long history going back to scientifically minded Chinese philosophers of the late 19th century, who used it to mean something more akin to enlightenment. 

Yet Hunter’s sensational tales still became an important part of the disinformation and pseudoscience that fueled a “mind-control race” during the Cold War, much like the space race. Inspired by new studies on brain function, the US military and intelligence communities prepared themselves for a psychic war with the Soviet Union and China by spending millions of dollars on research into manipulating the human brain. But while the science never exactly panned out, residual beliefs fostered by this bizarre conflict continue to play a role in ideological and scientific debates to this day.

Coercive persuasion and pseudoscience

Ironically, “brainwashing” was not a widely used term among communists in China. The word xinao, Mitchell told me in an email, is actually a play on an older word, xixin, or washing the heart, which alludes to a Confucian and Buddhist ideal of self-awareness. In the late 1800s, Chinese reformists such as Liang Qichao began using xinao—replacing the character for “heart” with “brain”—in part because they were trying to modernize Chinese philosophy. “They were eager to receive and internalize as much as they could of Western science in general, and discourse about the brain as the seat of consciousness was just one aspect of that set of imported ideas,” Mitchell said. 

For Liang and his circle, brainwashing wasn’t some kind of mind-wiping process. “It was a sort of notion of epistemic virtue,” Mitchell said, “or a personal duty to make oneself modern in order to behave properly in the modern world.”

Meanwhile, scientists outside China were investigating “brainwashing” in the sense we usually think of, with experiments into mind clearing and reprogramming. Some of the earliest research into the possibility began in the 1890s, when Ivan Pavlov, the Russian physiologist who had famously conditioned dogs to drool at the sound of a bell, worked on Soviet-funded projects to investigate how trauma could change animal behavior. He found that even the most well-conditioned dogs would forget their training after intensely stressful experiences such as nearly drowning, especially when those were combined with sleep deprivation and isolation. It seemed that Pavlov had hit upon a quick way to wipe animals’ memories. Scientists on both sides of the Iron Curtain subsequently wondered whether it might work on humans. And once memories were wiped, they wondered, could something else be installed their place? 

During the 1949 show trial of the Hungarian anticommunist József Mindszenty, American officials worried that the Russians might have found the answer. A Catholic cardinal, Mindszenty had protested several government policies of the newly formed, Soviet-backed Hungarian People’s Republic. He was arrested and tortured, and he eventually made a series of outlandish confessions at trial: that he had conspired to steal the Hungarian crown jewels, start World War III, and make himself ruler of the world. In his book Dark Persuasion, Joel Dimsdale, a psychiatry professor at the University of California, San Diego, argues that the US intelligence community saw these implausible claims as confirmation that the Soviets had made some kind of scientific breakthrough that allowed them to control the human mind through coercive persuasion.

This question became more urgent when, in 1953, a handful of American POWs in China and Korea switched sides, and a Marine named Frank Schwable was quoted on Chinese radio validating the communist claim that the US was testing germ warfare in Asia. By this time, Hunter had already published a book about brainwashing in China, so the Western public quickly gravitated toward his explanation that the prisoners had been brainwashed, just like Mindszenty. People were terrified, and this was a reassuring explanation for how nice American GIs could go Red. 

Over the following years, in the wake of the Korean War, “brainwashing” grew into a catchall explanation for any kind of radical or nonconformist behavior in the United States. Social scientists and politicians alike latched onto the idea. The Dutch psychologist Joost Meerloo warned that television was a brainwashing machine, for example, and the anticommunist educator J. Merrill Root claimed that high schools brainwashed kids into being weak-willed and vulnerable to communist influence. Meanwhile, popular movies like 1962’s The Manchurian Candidate, starring Frank Sinatra, offered thrilling tales of Chinese communists whose advanced psychological techniques turned unsuspecting American POWs into assassins. 

For the military and intelligence communities, mind control hovered between myth and science. Nowhere is this more obvious than in the peculiar case of an anonymously published 1955 pamphlet called Brain-Washing: A Synthesis of the Russian Textbook on Psychopolitics, which purported to be a translation of work by the Soviet secret-police chief Lavrentiy Beria. Full of wild claims about how the Soviets used psychology and drugs to control the masses, the pamphlet has a peculiar section devoted to the ways that Dianetics—a pseudoscience invented by the founder of Scientology, L. Ron Hubbard—could prevent brainwashing. As a result, it is widely believed that Hubbard himself wrote the pamphlet as black propaganda, or propaganda that masquerades as something produced by a foreign adversary. 

ALAMY

Still, US officials apparently took it seriously. David Seed, a cultural studies scholar at the University of Liverpool, plumbed the National Security Council papers at the Dwight D. Eisenhower Library, where he discovered that the NSC’s Operations Coordinating Board had analyzed the pamphlet as part of an investigation into enemy capabilities. A member of the board wrote that it might be “fake” but contained so much accurate information that it was clearly written by “experts.” When it came to brainwashing, government operatives made almost no distinction between black propaganda and so-called expertise.

This gobbledygook may also have struck the NSC investigator as legitimate because Hubbard borrowed lingo from the same sources as many scientists of the era. Hubbard chose the name Dianetics, for instance, specifically to evoke the computer scientist Norbert Wiener’s idea of cybernetics, an influential theory about information control systems that heavily informed both psychology and the burgeoning field of artificial intelligence. Cybernetics suggested that the brain functioned like a machine, with inputs and outputs, feedback and control. And if machines could be optimized, then why not brains?

An excuse for government abuse 

The fantasy of brainwashing was always one of optimization. Military experts knew that adversaries could be broken with torture, but it took months and was often a violent, messy process. A fast, scientifically informed interrogation method would save time and could potentially be deployed on a mass scale. In 1953, that dream led the CIA to invest millions of dollars in MK-Ultra, a project that injected cash into university and research programs devoted to memory wiping, mind control, and “truth serum” drugs. Worried that their rivals in the Soviet Union and China were controlling people’s minds to spread communism throughout the world, the intelligence community was willing to try almost anything to fight back. No operation was too weird. 

One of MK-Ultra’s most notorious projects was “Operation Midnight Climax” in San Francisco, where sex workers lured random American men to a safe house and dosed them with LSD while CIA agents covertly observed their behavior. At McGill University in Montreal, the CIA funded the work of the psychologist Donald Cameron, who used a combination of drugs and electroconvulsive therapy on patients with mental illness, attempting to erase and “repattern” their minds. Though many of his victims did wind up suffering from amnesia for years, Cameron never successfully injected new thoughts or memories. Marcia Holmes, a science historian who researched brainwashing for the Hidden Persuaders project at Birkbeck, University of London, told me that the CIA used Cameron’s data to develop new kinds of torture, which the US adopted as  “enhanced interrogation” techniques in the wake of 9/11. “You could put a scientific spin on it and claim that’s why it worked,” she said. “But it always boiled down to medieval tactics that people knew from experience worked.”

MK-Ultra remained secret until the mid-1970s, when the US Senate Select Committee to Study Governmental Operations with Respect to Intelligence Activities, commonly known as the Church Committee after its chair, Senator Frank Church, opened hearings into the long-­running project. The shocking revelations that the CIA was drugging American citizens and paying for the torment of vulnerable Canadians changed the public’s understanding of mind control. “Brainwashing” came to seem less like a legitimate threat from overseas enemies and more like a ruse or excuse for almost any kind of bad behavior. When Patty Hearst, granddaughter of the newspaper publisher William Randolph Hearst, was put on trial in 1976 for robbing a bank after being kidnapped by the Symbionese Liberation Army, an American militant organization, the judge refused to believe experts who testified that she had been tortured and brainwashed by her captors. She was convicted and spent 22 months in jail. This marked the end of the nation’s infatuation with brainwashing, and experts began to debunk the idea that there was a scientific basis for mind control.

GIFT OF TIME MAGAZINE

Still, the revelations about MK-Ultra led to new cultural myths. Communists were no longer the baddies—instead, people feared that the US government was trying to experiment on its citizens. Soon after the Church Committee hearings were over, the media was gripped by a crime story of epic proportions: nearly two dozen Black children had been murdered in Atlanta, and the police had no leads other than a vague idea that maybe it could be a serial killer. Wayne Williams, a Black man who was eventually convicted of two of the murders, claimed at various points that he had been trained by the CIA. This led to popular conspiracy theories that MK-Ultra had been experimenting on Black people in Atlanta.

Colin Dickey, author of Under the Eye of Power: How Fear of Secret Societies Shapes American Democracy, told me these conspiracy theories became “a way of making sense of an otherwise mystifying and terrifying reality, [which is that America is] a country where Black people are so disenfranchised that their murders aren’t noticed.” Dickey added that this MK-Ultra conspiracy theory “gave a shape to systemic racism,” placing blame for the Atlanta child murders on the US government. In the process, it also suggested that Black people had been brainwashed to kill each other. 

No evidence ever surfaced that MK-Ultra was behind the children’s deaths, but the idea of brainwashing continues to be a powerful metaphor for the effects of systemic racism. It haunts contemporary Black horror films like Get Out, where white people take over Black people’s bodies through a fantastical version of hypnosis. And it provides the analytical substrate for the scathing indictment of racist marketing in the book Brainwashed: Challenging the Myth of Black Inferiority, by the Black advertising executive Tom Burrell. He argues that advertising has systematically pushed stereotypes of Black people as second-class citizens, instilling a “slave mindset” in Black audiences.

A social and political phenomenon

Today, even as the idea of brainwashing is often dismissed as pseudoscience, Americans are still spellbound by the idea that people we disagree with have been psychologically captured by our enemies. Right-wing pundits and politicians often attribute discussions of racism to infections by a “woke mind virus”—an idea that is a direct descendant of Cold War panics over communist brainwashing. Meanwhile, contemporary psychology researchers like UCSD’s Dimsdale fear that social media is now a vector for coercive persuasion, just as Meerloo worried about television’s mind-control powers in the 1950s. 

Cutting-edge technology is also altering how we think about mind control. In a 2017 open letter published in Nature, an international group of researchers and ethicists warned that neurotechnologies like brain-computer interfaces “mean that we are on a path to a world in which it will be possible to decode people’s mental processes and directly manipulate the brain mechanisms underlying their intentions, emotions and decisions.” It sounds like MK-Ultra’s wish list. Hoping to head off a neuro-dystopia, the group outlined several key ways that companies and universities could guard against coercive uses of this technology in the future. They suggested that we need laws to prevent companies from spying on people’s private thoughts, for example, as well as regulations that bar anyone from using brain implants to change people’s personalities or make them more neurotypical. 

Many neuroscientists feel that these concerns are overblown; one of them, the University of Maryland cognitive scientist R. Douglas Fields, summed up the naysayers’ position with a column in Quanta magazine arguing that the brain is more plastic than we realize, and that neurotech mind control will never be as simple as throwing a switch. Kathleen Taylor, another neuroscientist who studies brainwashing, takes a more measured view; in her book Brainwashing: The Science of Thought Control, she acknowledges that neurotech and drugs could change people’s thought processes but ultimately concludes that “brainwashing is above all a social and political phenomenon.” 

COURTESY OF THE CIA

Perhaps that means the anonymous National Security Council examiner was right to call Hubbard’s black propaganda the work of an “expert.” If brainwashing is politics, then disinformation might be as effective (or ineffective) as a brain implant in changing someone’s mind. Still, scholars have learned that political efforts at mind control do not have predictable results. Online disinformation leads to what Juliette Kayyem, a former assistant secretary of the Department of Homeland Security, identifies as stochastic terrorism, or acts of violence that cannot be predicted precisely but can be analyzed statistically. She writes that stochastic terrorism is inspired by online rhetoric that demonizes groups of people, but it’s hard to know which people consuming that rhetoric will actually become terrorists, and which of them will just rage at their computer screens—the result of coercive persuasion that works on some targets and misses others. 

American operatives may never have found the perfect system for brainwashing foreign adversaries or unsuspecting citizens, but the US managed to win the mind-control wars in one small way. Mitchell, the legal scholar at Hong Kong University, told me that the American definition of brainwashing, or xinao, is now the dominant way the word is used in modern Chinese speech. “People refer to aggressive advertising campaigns or earworm pop songs as having a xinao effect,” he said. The Chinese government, Mitchell added, uses the term exactly the way the US military did back in the 1950s. State media, for example, “described many Hong Kong protesters in 2019 as having undergone xinao by the West.”

Annalee Newitz is the author of Stories Are Weapons: Psychological Warfare and the American Mind, coming in June 2024.