Algorithms are everywhere

Like a lot of Netflix subscribers, I find that my personal feed tends to be hit or miss. Usually more miss. The movies and shows the algorithms recommend often seem less predicated on my viewing history and ratings, and more geared toward promoting whatever’s newly available. Still, when a superhero movie starring one of the world’s most famous actresses appeared in my “Top Picks” list, I dutifully did what 78 million other households did and clicked.

As I watched the movie, something dawned on me: recommendation algorithms like the ones Netflix pioneered weren’t just serving me what they thought I’d like—they were also shaping what gets made. And not in a good way. 

DOUBLEDAY

The movie in question wasn’t bad, necessarily. The acting was serviceable, and it had high production values and a discernible plot (at least for a superhero movie). What struck me, though, was a vague sense of déjà vu—as if I’d watched this movie before, even though I hadn’t. When it ended, I promptly forgot all about it. 

That is, until I started reading Kyle Chayka’s recent book, Filterworld: How Algorithms Flattened Culture. A staff writer for the New Yorker, Chayka is an astute observer of the ways the internet and social media affect culture. “Filterworld” is his coinage for “the vast, interlocking … network of algorithms” that influence both our daily lives and the “way culture is distributed and consumed.” 

Music, film, the visual arts, literature, fashion, journalism, food—Chayka argues that algorithmic recommendations have fundamentally altered all these cultural products, not just influencing what gets seen or ignored but creating a kind of self-reinforcing blandness we are all contending with now.

That superhero movie I watched is a prime example. Despite my general ambivalence toward the genre, Netflix’s algorithm placed the film at the very top of my feed, where I was far more likely to click on it. And click I did. That “choice” was then recorded by the algorithms, which probably surmised that I liked the movie and then recommended it to even more viewers. Watch, wince, repeat.  

“Filterworld culture is ultimately homogenous,” writes Chayka, “marked by a pervasive sense of sameness even when its artifacts aren’t literally the same.” We may all see different things in our feeds, he says, but they are increasingly the same kind of different. Through these milquetoast feedback loops, what’s popular becomes more popular, what’s obscure quickly disappears, and the lowest-­common-denominator forms of entertainment inevitably rise to the top again and again. 

This is actually the opposite of the personalization Netflix promises, Chayka notes. Algorithmic recommendations reduce taste—traditionally, a nuanced and evolving opinion we form about aesthetic and artistic matters—into a few easily quantifiable data points. That oversimplification subsequently forces the creators of movies, books, and music to adapt to the logic and pressures of the algorithmic system. Go viral or die. Engage. Appeal to as many people as possible. Be popular.  

A joke posted on X by a Google engineer sums up the problem: “A machine learning algorithm walks into a bar. The bartender asks, ‘What’ll you have?’ The algorithm says, ‘What’s everyone else having?’” “In algorithmic culture, the right choice is always what the majority of other people have already chosen,” writes Chayka. 

One challenge for someone writing a book like Filterworld—or really any book dealing with matters of cultural import—is the danger of (intentionally or not) coming across as a would-be arbiter of taste or, worse, an outright snob. As one might ask, what’s wrong with a little mindless entertainment? (Many asked just that in response to Martin Scorsese’s controversial Harper’s essay  in 2021, which decried Marvel movies and the current state of cinema.) 

Chayka addresses these questions head on. He argues that we’ve really only traded one set of gatekeepers (magazine editors, radio DJs, museum curators) for another (Google, Facebook, TikTok, Spotify). Created and controlled by a handful of unfathomably rich and powerful companies (which are usually led by a rich and powerful white man), today’s algorithms don’t even attempt to reward or amplify quality, which of course is subjective and hard to quantify. Instead, they focus on the one metric that has come to dominate all things on the internet: engagement.

There may be nothing inherently wrong (or new) about paint-by-numbers entertainment designed for mass appeal. But what algorithmic recommendations do is supercharge the incentives for creating only that kind of content, to the point that we risk not being exposed to anything else.

“Culture isn’t a toaster that you can rate out of five stars,” writes Chayka, “though the website Goodreads, now owned by Amazon, tries to apply those ratings to books. There are plenty of experiences I like—a plotless novel like Rachel Cusk’s Outline, for example—that others would doubtless give a bad grade. But those are the rules that Filterworld now enforces for everything.”

Chayka argues that cultivating our own personal taste is important, not because one form of culture is demonstrably better than another, but because that slow and deliberate process is part of how we develop our own identity and sense of self. Take that away, and you really do become the person the algorithm thinks you are. 

Algorithmic omnipresence

As Chayka points out in Filterworld, algorithms “can feel like a force that only began to exist … in the era of social networks” when in fact they have “a history and legacy that has slowly formed over centuries, long before the Internet existed.” So how exactly did we arrive at this moment of algorithmic omnipresence? How did these recommendation machines come to dominate and shape nearly every aspect of our online and (increasingly) our offline lives? Even more important, how did we ourselves become the data that fuels them?

W.W. NORTON

These are some of the questions Chris Wiggins and Matthew L. Jones set out to answer in How Data Happened: A History from the Age of Reason to the Age of Algorithms. Wiggins is a professor of applied mathematics and systems biology at Columbia University. He’s also the New York Times’ chief data scientist. Jones is now a professor of history at Princeton. Until recently, they both taught an undergrad course at Columbia, which served as the basis for the book.

They begin their historical investigation at a moment they argue is crucial to understanding our current predicament: the birth of statistics in the late 18th and early 19th century. It was a period of conflict and political upheaval in Europe. It was also a time when nations were beginning to acquire both the means and the motivation to track and measure their populations at an unprecedented scale.

“War required money; money required taxes; taxes required growing bureaucracies; and these bureaucracies needed data,” they write. “Statistics”may have originally described “knowledge of the state and its resources, without any particularly quantitative bent or aspirations at insights,” but that quickly began to change as new mathematical tools for examining and manipulating data emerged.

One of the people wielding these tools was the 19th-century Belgian astronomer Adolphe Quetelet. Famous for, among other things, developing the highly problematic body mass index (BMI), Quetelet had the audacious idea of taking the statistical techniques his fellow astronomers had developed to study the position of stars and using them to better understand society and its people. This new “social physics,” based on data about phenomena like crime and human physical characteristics, could in turn reveal hidden truths about humanity, he argued.

“Quetelet’s flash of genius—whatever its lack or rigor—was to treat averages about human beings as if they were real quantities out there that we were discovering,” write Wiggins and Jones. “He acted as if the average height of a population was a real thing, just like the position of a star.” 

From Quetelet and his “average man” to Francis Galton’s eugenics to Karl Pearson and Charles Spearman’s “general intelligence,” Wiggins and Jones chart a depressing progression of attempts—many of them successful—to use data as a scientific basis for racial and social hierarchies. Data added “a scientific veneer to the creation of an entire apparatus of discrimination and disenfranchisement,” they write. It’s a legacy we’re still contending with today. 

Another misconception that persists? The notion that data about people are somehow objective measures of truth. “Raw data is an oxymoron,” observed the media historian Lisa Gitelman a number of years ago. Indeed, all data collection is the result of human choice, from what to collect to how to classify it to who’s included and excluded. 

Whether it’s poverty, prosperity, intelligence, or creditworthiness, these aren’t real things that can be measured directly, note Wiggins and Jones. To quantify them, you need to choose an easily measured proxy. This “reification” (“literally, making a thing out of an abstraction about real things”) may be necessary in many cases, but such choices are never neutral or unproblematic. “Data is made, not found,” they write, “whether in 1600 or 1780 or 2022.”

“We don’t need to build systems that learn the stratifications of the past and present and reinforce them in the future.”

Perhaps the most impressive feat Wiggins and Jones pull off in the book as they continue to chart data’s evolution throughout the 20th century and the present day is dismantling the idea that there is something inevitable about the way technology progresses. 

For Quetelet and his ilk, turning to numbers to better understand humans and society was not an obvious choice. Indeed, from the beginning, everyone from artists to anthropologists understood the inherent limitations of data and quantification, making some of the same critiques of statisticians that Chayka makes of today’s algorithmic systems (“Such statisticians ‘see quality not at all, but only quantity’”).

Whether they’re talking about the machine-learning techniques that underpin today’s AI efforts or an internet built to harvest our personal data and sell us stuff, Wiggins and Jones recount many moments in history when things could have just as likely gone a different way.

“The present is not a prison sentence, but merely our current snapshot,” they write. “We don’t have to use unethical or opaque algorithmic decision systems, even in contexts where their use may be technically feasible. Ads based on mass surveillance are not necessary elements of our society. We don’t need to build systems that learn the stratifications of the past and present and reinforce them in the future. Privacy is not dead because of technology; it’s not true that the only way to support journalism or book writing or any craft that matters to you is spying on you to service ads. There are alternatives.” 

A pressing need for regulation

If Wiggins and Jones’s goal was to reveal the intellectual tradition that underlies today’s algorithmic systems, including “the persistent role of data in rearranging power,” Josh Simons is more interested in how algorithmic power is exercised in a democracy and, more specifically, how we might go about regulating the corporations and institutions that wield it.

PRINCETON UNIVERSITY PRESS

Currently a research fellow in political theory at Harvard, Simons has a unique background. Not only did he work for four years at Facebook, where he was a founding member of what became the Responsible AI team, but he previously served as a policy advisor for the Labour Party in the UK Parliament. 

In Algorithms for the People: Democracy in the Age of AI, Simons builds on the seminal work of authors like Cathy O’Neil, Safiya Noble, and Shoshana Zuboff to argue that algorithmic prediction is inherently political. “My aim is to explore how to make democracy work in the coming age of machine learning,” he writes. “Our future will be determined not by the nature of machine learning itself—machine learning models simply do what we tell them to do—but by our commitment to regulation that ensures that machine learning strengthens the foundations of democracy.”

Much of the first half of the book is dedicated to revealing all the ways we continue to misunderstand the nature of machine learning, and how its use can profoundly undermine democracy. And what if a “thriving democracy”—a term Simons uses throughout the book but never defines—isn’t always compatible with algorithmic governance? Well, it’s a question he never really addresses. 

Whether these are blind spots or Simons simply believes that algorithmic prediction is, and will remain, an inevitable part of our lives, the lack of clarity doesn’t do the book any favors. While he’s on much firmer ground when explaining how machine learning works and deconstructing the systems behind Google’s PageRank and Facebook’s Feed, there remain omissions that don’t inspire confidence. For instance, it takes an uncomfortably long time for Simons to even acknowledge one of the key motivations behind the design of the PageRank and Feed algorithms: profit. Not something to overlook if you want to develop an effective regulatory framework. 

“The ultimate, hidden truth of the world is that it is something that we make, and could just as easily make differently.”

Much of what’s discussed in the latter half of the book will be familiar to anyone following the news around platform and internet regulation (hint: that we should be treating providers more like public utilities). And while Simons has some creative and intelligent ideas, I suspect even the most ardent policy wonks will come away feeling a bit demoralized given the current state of politics in the United States. 

In the end, the most hopeful message these books offer is embedded in the nature of algorithms themselves. In Filterworld, Chayka includes a quote from the late, great anthropologist David Graeber: “The ultimate, hidden truth of the world is that it is something that we make, and could just as easily make differently.” It’s a sentiment echoed in all three books—maybe minus the “easily” bit. 

Algorithms may entrench our biases, homogenize and flatten culture, and exploit and suppress the vulnerable and marginalized. But these aren’t completely inscrutable systems or inevitable outcomes. They can do the opposite, too. Look closely at any machine-learning algorithm and you’ll inevitably find people—people making choices about which data to gather and how to weigh it, choices about design and target variables. And, yes, even choices about whether to use them at all. As long as algorithms are something humans make, we can also choose to make them differently. 

Bryan Gardiner is a writer based in Oakland, California.

How Antarctica’s history of isolation is ending—thanks to Starlink

“This is one of the least visited places on planet Earth and I got to open the door,” Matty Jordan, a construction specialist at New Zealand’s Scott Base in Antarctica, wrote in the caption to the video he posted to Instagram and TikTok in October 2023. 

In the video, he guides viewers through an empty, echoing hut, pointing out where the men of Ernest Shackleton’s 1907 expedition lived and worked—the socks still hung up to dry and the provisions still stacked neatly in place, preserved by the cold. 

Jordan, who started making TikToks to keep family and friends up to date with his life in Antarctica, has now found himself at the center of a phenomenon. His channels have over a million followers. The video of Shackleton’s hut alone has racked up millions of views from all over the world. It’s also kind of a miracle: until very recently, those who lived and worked on Antarctic bases had no hope of communicating so readily with the outside world. 

Antarctica has long been a world apart. In the 19th and early 20th centuries, when dedicated expeditions began, explorers were cut off from home for years at a time, reliant on ships sailing back and forth from civilization to carry physical mail. They were utterly alone, the only humans for thousands of miles.

This made things difficult, emotionally and physically. With only the supplies they had on hand, explorers were limited in the scientific experiments they could conduct. They couldn’t send an SOS if they needed help (which was fairly often). And also—importantly, because many relied on publicity for funding—they couldn’t let the world know what was going on. 

In 1911, an Australian expedition led by Douglas Mawson was the first to take an antenna to the continent and attempt to transmit and receive wireless signals. But while Mawson was able to send a few messages during the team’s first season, he never received any back, so he didn’t know if his had been successful.

The winds at their base at Cape Denison, on the Antarctic coast directly south of Australia, raged at 70 kilometers an hour—every day, every night, for months on end. They finally succeeded in raising the mast during their second winter, only to be faced with a different problem: their radio operator was unable to work, having suffered psychosis during the six months of darkness. So the expedition was left isolated again. 

While Antarctic telecommunications have been steadily improving ever since the first permanent bases were established, many decades after Mawson’s ill-fated trip, life on the ice has always been characterized by some level of disconnection. And as life at home has become ever more dependent on constant connection, instant updates, streaming, and algorithms, living in Antarctica has been seen as a break—for better and for worse—from all the digital hustle-bustle. 

But the end of that long-standing disparity is now in sight. Starlink, the satellite constellation developed by Elon Musk’s company SpaceX to service the world with high-speed broadband internet, has come to Antarctica, finally bringing with it the sort of connectivity enjoyed by the world beyond the ice. 

Stories from the first Antarctic expeditions were so scarce they were a hot commodity: newspapers would pay top dollar to be told the news the minute explorers like Mawson and Shackleton arrived back in port. Now videos, posts, and FaceTime calls are common from people stationed at Antarctic bases and field camps, and from the surging numbers of tourists on ships. 

Suddenly, after more than a century as one of the least connected parts of the world, the seventh continent feels a lot closer to the others. For those whose lives and livelihoods take them there regularly, it’s been a long time coming. 

Taking the public with you

People have always been hungry for news of life in Antarctica. In the early days, regular updates about derring-do in the polar winds were the perfect way to capture the attention of the press—a key to securing the funding necessary for the outsize private expeditions of the early 20th century.

No one exemplified the close relationship between exploration and attention better than Admiral Richard E. Byrd, a charismatic self-publicist who named his succession of bases on the Ross Ice Shelf “Little America” and brought along a Boy Scout to represent America’s youth. Byrd was the quintessential explorer-­celebrity, constantly making headlines with his daring feats.

His first privately funded expedition, in 1929, aimed to reach the South Pole by plane. Byrd gave frequent updates on his progress to the press via radiotelegraphy, using wireless signals to send messages in Morse code directly from Little America to coastal stations in San Francisco and Long Island. A New York Times reporter embedded with the expedition filed stories almost daily over the radiotelegraph, and the public followed Byrd’s every move, culminating in the historic flight over the South Pole on November 29, 1929.

By the time of Byrd’s next expedition, in 1933, technology had progressed enough to allow the first audio broadcasting station in Antarctica. The station made use of shortwave radio’s long-distance capabilities to transmit official mission reports, and it was also able to receive messages for the expedition members. A weekly variety program organized by the expedition’s journalist, Charles Murphy, was broadcast live to the public on AM stations. 

This innovative program allowed audiences at home to feel they were taking part in the expedition themselves. Like popular radio shows of the day such as The Shadow and The Lone Ranger, Adventures with Admiral Byrd was an action-packed serial, featuring updates on the progress of the expedition directly from the hardy explorers themselves. The scientists also gave weather reports and talks, and they performed songs and skits. 

US NAVY/UNITED STATES ANTARCTIC PROGRAM

In the program’s most popular segment, Americans were able to chat with the men in Little America live on air. Little America’s postmaster spoke to his wife on their 21st anniversary; Al Carbone, the eccentric cook for the expedition, spoke to the chef at New York’s Waldorf-Astoria hotel. 

“The spirited narratives of real-life adventure are making interesting program fare for the world’s radio listeners who have been accustomed to the make-believe studio dramatizations usually available on the broadcast channels,” read a 1934 review in Radio News. The program indelibly associated its main sponsor, General Foods’ Grape-Nuts cereal, with the bravery of Admiral Byrd and his companions, and brought Antarctica vividly into the lives of millions of listeners on a wider scale than ever before. 

Helpful hams and secret codes

By 1957,Admiral Byrd was recognized as the world’s foremost expert in Antarctic exploration and was leading America’s Operation Deep Freeze, a mission to build a permanent American presence on the continent. The US Naval Construction Battalions, known as the Seabees, were deployed to build McMurdo Station on the solid ground of Ross Island, close to the first hut built by Captain Robert Scott in 1901. 

Deep Freeze brought a massive military presence to Antarctica, including the most complex and advanced communications array the Navy could muster. Still, men who wanted to speak to loved ones at home had limited options. Physical mail could come and go on ships a few times a year, or they could send expensive telegrams over wireless—limited to 100 or 200 words per month each way. At least these methods were private, unlike the personal communications over radio on Byrd’s expedition, which everyone else could listen in to by default.

In the face of these limitations, another option soon became popular among the Navy men. The licensed operators of McMurdo’s amateur (ham) station were assisted by hams back at home. Seabees would call from McMurdo to a ham in America, who would patch them straight through to their destination through the US phone system, free of charge. 

Some of these helpful hams became legendary. Jules Madey and his brother John, two New Jersey teenagers with the call sign K2KGJ, had built a 110-foot-tall radio tower in their backyard, with a transmitter that was more than capable of communicating to and from McMurdo Sound. 

To save money, a code known as “WYSSA” offered a broad variety of set phrases for common topics. WYSSA itself stood for “All my love, darling.”

From McMurdo, the South Pole, and the fifth Little America base on the Ross Ice Shelf, ham operators could ring Jules at nearly any time of day or night, and he’d connect them to home. Jules became an Antarctic celebrity and icon. A few of the engaged couples he helped to link up even invited him and his brother to their weddings, after the men returned from their tours of duty in Antarctica. Many Deep Freeze men still remembered the Madey brothers decades later. 

In the early 1960s, continued Deep Freeze operations, including support ships, were improving communication across American outposts in Antarctica. Bigger antennas, more powerful receivers and transmitters, and improvements to ground-to-air communication systems were installed, shoring up the capacity for scientific activity, transport, and construction.  

Around this time, the Australian National Antarctic Research Expeditions were improving their communications capacity as well. Like other Antarctic programs, they used telex machines, sending text out over radio waves to link up with a phone-line-based system on land. Telex, a predecessor to fax technology, text messaging, and email, was in use from the 1960s onwards as an alternative to Morse code and voice over HF and VHF radio. On the other side of the line, a terminal would receive the text and print it out.

MALCOLM MACFARLANE ©ANTARCTICA NEW ZEALAND PICTORIAL COLLECTION

In order to save money on the expensive per-word rates, a special code known as “WYSSA” (pronounced, in an Australian accent, “whizzer”) was constructed. This creative solution became legendary in Antarctic history. WYSSA itself stood for “All my love, darling,” and the code offered a broad variety of predetermined phrases for common topics, from the inconveniences of Antarctic life (YAYIR—“Fine snow has penetrated through small crevices in the huts”) to affectionate sentiments (YAAHY—“Longing to hear from you again, darling”) and personal updates (YIGUM—“I have grown a beard which is awful”). 

Changing times

Visit stations in Antarctica today, and you will see massive radio domes that now dot the landscape. Inside are the dishes that track the satellites the stations rely on. Satellite schedules are published weekly on the US Antarctic Program’s website, showing the windows when connectivity is available. 

The first satellites, run by Inmarsat, came online in the early 1980s and were a huge improvement on radio transmission. Inmarsat’s network provided coverage up to 75° latitude, 9° south of the Antarctic circle—which meant it now covered some, although not all, Antarctic bases. (McMurdo Station and Scott Base, down at 77° S, missed out.) It also allowed for high-quality service at any time of day or night, unaffected by atmospheric disturbances.

In the ’90s, the Iridium constellation of low-orbit satellites became operational. These satellites were launched into polar orbits and provided continuous service to all parts of Antarctica. Widespread satellite phone and email access quickly replaced radio as the best way to speak to those back home. But a thousand seasonal workers at McMurdo still had to share an internet link with a paltry capacity of 17 megabits per second, accessible only by a few in-­demand Ethernet cables. Phone calls, while possible, were still inconveniently expensive, and video calls thanklessly difficult.

Now the satellite revolution has taken its next step. The 2022–’23 season in Antarctica brought an exciting development for those on the ice: the first trial of SpaceX’s Starlink satellite connection.

Its introduction means even the most remote parts of the region—where a great deal of important scientific work takes place—are becoming more connected. When Peter Neff, a glaciologist at the University of Minnesota, first went out in the field in the Antarctic summer season of 2009–’10, he had to send a physical thumb drive back to McMurdo to share photos of his field camps.

Now Neff is director of field research and data at the Center for Oldest Ice Exploration, or COLDEX, a National Science Foundation–funded project aiming to seek out the oldest possible ice-core records in Antarctica to solve mysteries about Earth’s past climate. He spearheaded the installation of a Starlink connection at COLDEX’s field site in the 2022–’23 summer season, making it among the first field camps to have access to high-speed internet connectivity. The test was free thanks to Neff’s connections at the NSF, but for the 2023–’24 summer season, his camp is paying just $250 a month for 50 gigabytes or $1,000 for a terabyte, along with a $2,500 flat fee for the terminal. 

The team now has an easy link with the world, allowing participants to, among other things, text photos of the weather to transport pilots and trade spreadsheets with remote logistics managers. Starlink “helps with just complete ease of communications, in the form that we’re all used to,” Neff says. Matty Jordan, the TikTok creator at Scott Base, agrees that Starlink has made doing science there easier. “Scientists are able to transmit larger amounts of data back to New Zealand, which makes their work much faster and more efficient,” he says.

And crucially, it has helped scientists at the bases communicate their work to the public.

“Social media is an easy way for people to see what happens on a research station and helps people engage with the work that happens there,” Jordan says, emphasizing the opportunity for people at home to learn more about the importance of climate research.

“The first thing that people took advantage of it for was outreach,” agrees Ed Brook, COLDEX’s director, “and the possibility of talking to television or radio or print journalists in the field.” Just as with Byrd and his radio show, the first use for communication upgrades is often to satisfy the worldwide appetite for stories straight from the ice. 

High bandwidth in the ice

The next step to open up Antarctic communications could be a proposed fiber-optic undersea cable linking New Zealand and McMurdo Station. It would cost upwards of $200 million, but as an NSF workshop determined, it would be a huge boon not just to the US and New Zealand but to all countries with Antarctic programs, and the science they conduct.

According to the report released by the workshop in 2021 (before the Starlink deployment), the entire US Antarctic scientific community shared the same connective capacity available to many individual American households or small businesses—well under 100 Mbps per individual end user. A fiber link capable of anywhere from 100 gigabits per second to 300 terabits per second would bring Antarctic research up to the level of connectivity enjoyed by scientists in many other places.

No telecommunications upgrades can change the nature of Antarctica, or the emotions it can stir up.

It would also make Antarctica an even more popular place for tourists. In 2023, the number of visitors was 40% over pre-­pandemic levels. More than 100,000 people were expected to visit in the 2023–’24 winter, mainly on expedition cruise ships to the Antarctic Peninsula. 

Lizzie Williams, a product manager at the travel agency Swoop Antarctica, has seen firsthand the changes Starlink and improved internet connectivity have brought to tourist excursions. “It’s now possible to send through compressed photos and videos,” she told me over email. “We have even seen some people FaceTiming their families, though it can be a bit glitchy.”

According to Williams, daily inquiries about accessing the internet on board have increased. Growing numbers of people even try to work remotely from Antarctic cruises. But she warns that connection on most ships is far too expensive and unreliable for Zoom calls, and that Swoop’s cruise staff urges a low-technology environment for guests. “We encourage them to be out on deck enjoying the icebergs and wildlife, really making the most of their precious time in Antarctica,” she says. 

Some who live on the ice also favor keeping the experience as low-tech as possible. While social media may have its tentacles wrapped around the rest of the world, Antarctica has been, up to a point, safe until now. 

Luckily, so far, it seems as if Starlink hasn’t upset the special, close-knit atmosphere at Antarctica’s outposts. Demie Huffman, a PhD student in land and atmospheric science at the University of Minnesota, surveyed participants in COLDEX’s 2022–’23 field year about their experiences with the service. “People generally ended up being pleasantly surprised with the minimal impact that it had on group cohesion,” she says. “They would have a movie night together, instead of just having to read a book or rely on the things that a couple of people had thought to bring.” 

Still, Antarctica’s regular residents want to make sure the arrival of high-speed internet doesn’t change things too much as time goes on. “Over the winter, we made a rule that no phones were allowed at the dinner table to ensure that people built personal relationships with others on base,” says Jordan.

In any case, Antarctica will always be a magical place, even if it is no longer isolated from communication with the rest of the world. Ever since Scott and Shackleton published their best-selling books, one of its greatest natural resources has been its stories.People just can’t get enough of penguins, crevasses, tales of adventure, and natural spectacle. There’s a reason reporters swarmed returning explorers at train stations—the same reason Jordan’s TikToks rack up views by the million. 

And while going there no longer means stepping out of time and into another world entirely, no telecommunications upgrades can change the nature of the place, or the emotions it can stir up. 

The same intense winds still blow where Shackleton’s men once lived, the same sun still hangs in the sky for six months of endless daylight, and the icy landscapes still exert an inexplicable pull on human hearts and minds. 

Only now, you can share the magic freely with everyone at home, without delay. 

Allegra Rosenberg covers media, the environment, and technology. Her book Fandom Forever (and Ever) is forthcoming from W.W. Norton.

Inside the hunt for new physics at the world’s largest particle collider

In 1977, Ray and Charles Eames released a remarkable film that, over the course of just nine minutes, spanned the limits of human knowledge. Powers of Ten begins with an overhead shot of a man on a picnic blanket inside a one-square-­meter frame. The camera pans out: 10, then 100 meters, then a kilometer, and eventually all the way to the then-known edges of the observable universe—10²⁴ meters. There, at the farthest vantage, it reverses. The camera zooms back in, flying through galaxies to arrive at the picnic scene, where it plunges into the man’s skin, digging down through successively smaller scales: tissues, cells, DNA, molecules, atoms, and eventually atomic nuclei—10^-14 meters. The narrator’s smooth voice-over ends the journey: “As a single proton fills our scene, we reach the edge of present understanding.” 

During the intervening half-century, particle physicists have been exploring the subatomic landscape where Powers of Ten left off. Today, much of this global effort centers on CERN’s Large Hadron Collider (LHC), an underground ring 17 miles (27 kilometers) around that straddles the border between Switzerland and France. There, powerful magnets guide hundreds of trillions of protons as they do laps at nearly the speed of light underneath the countryside. When a proton headed clockwise plows into a proton headed counterclockwise, the churn of matter into energy transmutes the protons into debris: electrons, photons, and more exotic subatomic bric-a-brac. The newly created particles explode radially outward, where they are picked up by detectors. 

In 2012, using data from the LHC, researchers discovered a particle called the Higgs boson. In the process, they answered a nagging question: Where do fundamental particles, such as the ones that make up all the protons and neutrons in our bodies, get their mass? A half-­century earlier, theorists had cautiously dreamed the Higgs boson up, along with an accompanying field that would invisibly suffuse space and provide mass to particles that interact with it. When the particle was finally found, scientists celebrated with champagne. A Nobel for two of the physicists who predicted the Higgs boson soon followed.

But now, more than a decade after the excitement of finding the Higgs, there is a sense of unease, because there are still unanswered questions about the fundamental constituents of the universe. 

Perhaps the most persistent of these questions is the identity of dark matter, a mysterious substance that binds galaxies together and makes up 27% of the cosmos’s mass. We know dark matter must exist because we have astronomical observations of its gravitational effects. But since the discovery of the Higgs, the LHC has seen no new particles—of dark matter or anything else—despite nearly doubling its collision energy and quintupling the amount of data it can collect. Some physicists have said that particle physics is in a “crisis,” but there is disagreement even on that characterization: another camp insists the field is fine and still others say that there is indeed a crisis, but that crisis is good. “I think the community of particle phenomenologists is in a deep crisis, and I think people are afraid to say those words,” says Yoni Kahn, a theorist at the University of Illinois Urbana-Champaign. 

The anxieties of particle physicists may, at first blush, seem like inside baseball. In reality, they concern the universe, and how we can continue to study it—of interest if you care about that sort of thing. The past 50 years of research have given us a spectacularly granular view of nature’s laws, each successive particle discovery clarifying how things really work at the bottom. But now, in the post-Higgs era, particle physicists have reached an impasse in their quest to discover, produce, and study new particles at colliders. “We do not have a strong beacon telling us where to look for new physics,” Kahn says. 

So, crisis or no crisis, researchers are trying something new. They are repurposing detectors to search for unusual-looking particles, squeezing what they can out of the data with machine learning, and planning for entirely new kinds of colliders. The hidden particles that physicists are looking for have proved more elusive than many expected, but the search is not over—nature has just forced them to get more creative. 

An almost-complete theory

As the Eameses were finishing Powers of Ten in the late ’70s, particle physicists were bringing order to a “zoo” of particles that had been discovered in the preceding decades. Somewhat drily, they called this framework, which enumerated the kinds of particles and their dynamics, the Standard Model.

Roughly speaking, the Standard Model separates fundamental particles into two types: fermions and bosons. Fermions are the bricks of matter—two kinds of fermions called up and down quarks, for example, are bound together into protons and neutrons. If those protons and neutrons glom together and find an electron (or electrons) to orbit them, they become an atom. Bosons, on the other hand, are the mortar between the bricks. Bosons are responsible for all the fundamental forces besides gravity: electromagnetism; the weak force, which is involved in radioactive decay; and the strong force, which binds nuclei together. To transmit a force between one fermion and another, there must be a boson to act as a messenger. For example, quarks feel the attractive power of the strong force because they send and receive bosons called gluons. 

The Standard Model

This framework unites three out of four fundamental forces and tamed an unruly zoo into just 17 elementary particles.

Quarks are bound together by gluons. They form composite particles called hadrons, the most stable of which are protons and neutrons, the components of atomic nuclei.

Leptons can be charged or neutral. The charged leptons are the electron, muon, and tau. Each of these has a neutral neutrino counterpart.

Gauge bosons convey forces. Gluons carry the strong force; photons carry the electromagnetic force; and W and Z bosons carry the weak force, which is involved in radioactive processes.

The Higgs boson is the fundamental particle associated with the Higgs field, a field that permeates the entire universe and gives mass to other fundamental particles.

Nearly 50 years later, the Standard Model remains superbly successful; even under stress tests, it correctly predicts fundamental properties of the universe, like the magnetic properties of the electron and the mass of the Z boson, to extremely high accuracy. It can reach well past where Powers of Ten left off, to the scale of 10^-20 meters, roughly a 10,000th the size of a proton. “It’s remarkable that we have a correct model for how the world works down to distances of 10^-20 meters. It’s mind blowing,” says Seth Koren, a theorist at the University of Notre Dame, in Indiana. 

Despite its accuracy, physicists have their pick of questions the Standard Model doesn’t answer—what dark matter actually is, why matter dominates over antimatter when they should have been made in equal amounts in the early universe, and how gravity fits into the picture. 

Over the years, thousands of papers have suggested modifications to the Standard Model to address these open questions. Until recently, most of these papers relied on the concept of supersymmetry, abbreviated to the friendlier “SUSY.” Under SUSY, fermions and bosons are actually mirror images of one another, so that every fermion has a boson counterpart, and vice versa. The photon would have a superpartner dubbed a “photino” in SUSY parlance, while an electron would have a “selectron.” If these particles were high in mass, they would be “hidden,” unseen unless a sufficiently high-energy collision left them as debris. In other words, to create these heavy superpartners, physicists needed a powerful particle collider.

It might seem strange, and overly complicated, to double the number of particles in the universe without direct evidence. SUSY’s appeal was in its elegant promise to solve two tricky problems. First, superpartners would explain the Higgs boson’s oddly low mass. The Higgs is about 100 times more massive than a proton, but the math suggests it should be 100 quadrillion times more massive. (SUSY’s quick fix is this: every particle that interacts with the Higgs contributes to its mass, causing it to balloon. But each superpartner would counteract its ordinary counterpart’s contribution, getting the mass of the Higgs under control.) The second promise of SUSY: those hidden particles would be ideal candidates for dark matter. 

SUSY was so nifty a fix to the Standard Model’s problems that plenty of physicists thought they would find superpartners before they found the Higgs boson when the LHC began taking data in 2010. Instead, there has been resounding silence. Not only has there been no evidence for SUSY, but many of the most promising scenarios where SUSY particles would solve the problem of the Higgs mass have been ruled out.

At the same time, many non-collider experiments designed to directly detect the kind of dark matter you’d see if it were made up of superpartners have come up empty. “The lack of evidence from both direct detection and the LHC is a really strong piece of information the field is still kind of digesting,” Kahn says. 

SAMUELJOSEPH HERTZOG/CERN

Many younger researchers—like Sam Homiller, a theorist at Harvard University—are less attached to the idea. “[SUSY] would have been a really pretty story,” says Homiller. “Since I came in after it … it’s just kind of like this interesting history.” 

Some theorists are now directing their search away from particle accelerators and toward other sources of hidden particles. Masha Baryakhtar, a theorist at the University of Washington, uses data from stars and black holes. “These objects are really high density, often high temperature. And so that means that they have a lot of energy to give up to create new particles,” Baryakhtar says. In their nuclear furnaces, stars might produce loads and loads of another dark matter candidate called the axion. There are experiments on Earth that aim to detect such particles as they reach us. But if a star is expending energy to create axions, there will also be telltale signs in astronomical observations. Baryakhtar hopes these celestial bodies will be a useful complement to detectors on Earth. 

Other researchers are finding ways to give new life to old ideas like SUSY. “I think SUSY is wonderful—the only thing that’s not wonderful is that we haven’t found it,” quips Karri DiPetrillo, an experimentalist at the University of Chicago. She points out that SUSY is far from being ruled out. In fact, some promising versions of SUSY that account for dark matter (but not the Higgs mass) are completely untested. 

After initial investigations did not find SUSY in the most obvious places, many researchers began looking for “long-lived particles” (LLPs), a generic class of potential particles that includes many possible superpartners. Because detectors are primarily designed to see particles that decay immediately, spotting LLPs challenges researchers to think creatively. 

“You need to know the details of the experiment that you’re working on in a really intimate way,” DiPetrillo says. “That’s the dream—to really be using your experiment and pushing it to the max.”

The two general-purpose detectors at the LHC, ATLAS and CMS, are a bit like onions, with concentric layers of particle-­tracking hardware. Most of the initial mess from proton collisions—jets and showers of quarks—decays immediately and gets absorbed by the inner layers of the onion. The outermost layer of the detector is designed to spot the clean, arcing paths of muons, which are heavier versions of electrons. If an LLP created in the collision made it to the muon tracker and then decayed, the particle trajectory would be bizarre, like a baseball hit from first base instead of home plate. A recent search by the CMS collaboration used this approach to search for LLPs but didn’t spot any evidence for them. 

Researchers scouring the data often don’t have any faith that any particular search will turn up new physics, but they feel a responsibility to search all the same. “We should do everything in our power to make sure we leave no stone unturned,” DiPetrillo says. “The worst thing about the LHC would be if we were producing SUSY particles and we didn’t find them.” 

Needles in high-energy haystacks

Searching for new particles isn’t just a matter of being creative with the hardware; it’s also a software problem. While it’s running, the LHC generates about a petabyte of collision data per second—a veritable firehose of information. Less than 1% of that gets saved, explains Ben Nachman, a data physicist at Lawrence Berkeley National Lab: “We just can’t write a petabyte per second to tape right now.” 

Dealing with that data will only become more important in the coming years as the LHC receives its “high luminosity” upgrade. Starting at the end of the decade, the HL-LHC will operate at the same energy, but it will record about 10 times more data than the LHC has accumulated so far. The boost will come from an increase in beam density: stuffing more protons into the same space leads to more collisions, which translates to more data. As the frame fills with dozens of collisions, the detector begins to look like a Jackson Pollock painting, with splashes of particles that are impossible to disentangle.

To handle the increasing data load and search for new physics, particle physicists are borrowing from other disciplines, like machine learning and math. “There’s a lot of room for creativity and exploration, and really just kind of thinking very broadly,” says Jessica Howard, a phenomenologist at the University of California, Santa Barbara. 

One of Howard’s projects involves applying optimal transport theory, an area of mathematics concerned with moving stuff from one place to the next, to particle detection. (The field traces its roots to the 18th century, when the French mathematician Gaspard Monge was thinking about the optimal way to excavate earth and move it.) Conventionally, the “shape” of a particle collision—roughly, the angles at which the particles fly out—has been described by simple variables. But using tools from optimal transport theory, Howard hopes to help detectors be more sensitive to new kinds of particle decays that have unusual shapes, and better able to handle the HL-LHC’s higher rates of collisions.

As with many new approaches, there are doubts and kinks to work out. “It’s a really cute idea, but I have no idea what it’s useful for at the moment,” Nachman says of optimal transport theory. He is a proponent of novel machine-learning approaches, some of which he hopes will allow researchers to do entirely different kinds of searches and “look for patterns that we couldn’t have otherwise found.”

Though particle physicists were early adopters and have been using machine learning since the late 1990s, the past decade of advances in deep learning has dramatically changed the landscape. 

Packing more power

The energy of particle colliders (as measured by the combined energy of two colliding particles) has risen over the decades, opening up new realms of physics to explore.

Collisions between leptons, such as electrons and positrons, are efficient and precise, but limited in energy. Among potential future projects is the possibility of colliding muons, which would give a big jump in collision energy.

Collisions between hadrons, such as protons and antiprotons, have high energy but limited precision. Although it would start with electrons (rightmost point), a possible Future Circular Collider could reach 100,000 (10⁵) GeV by colliding protons.

“[Machine learning] can almost always improve things,” says Javier Duarte, an experimentalist at the University of California, San Diego. In a hunt for needles in haystacks, the ability to change the signal-to-noise ratio is crucial. Unless physicists can figure out better ways to search, more data might not help much—it might just be more hay. 

One of the most notable but understated applications for this kind of work is refining the picture of the Higgs. About 60% of the time, the Higgs boson decays into a pair of bottom quarks. Bottom quarks are tricky to find amid the mess of debris in the detectors, so researchers had to study the Higgs through its decays into an easy-to-spot photon pair, even though that happens only about 0.2% of the time. But in the span of a few years, machine learning has dramatically improved the efficiency of bottom-quark tagging, which allows researchers another way to measure the Higgs boson. “Ten years ago, people thought this was impossible,” Duarte says. 

The Higgs boson is of central importance to physicists because it can tell them about the Higgs field, the phenomenon that gives mass to all the other elementary particles. Even though some properties of the Higgs boson have been well studied, like its mass, others—like the recursive way it interacts with itself—remain unknown with any kind of precision. Measuring those properties could rule out (or confirm) theories about dark matter and more. 

What’s truly exciting about machine learning is its potential for a completely different class of searches called anomaly detection. “The Higgs is kind of the last thing that was discovered where we really knew what we were looking for,” Duarte says. Researchers want to use machine learning to find things they don’t know to look for.

In anomaly detection, researchers don’t tell the algorithm what to look for. Instead, they give the algorithm data and tell it to describe the data in as few bits of information as possible. Currently, anomaly detection is still nascent and hasn’t resulted in any strong hints of new physics, but proponents are eager to try it out on data from the HL-LHC. 

Because anomaly detection aims to find anything that is sufficiently out of place, physicists call this style of search “model agnostic”—it doesn’t depend on any real assumptions. 

Not everyone is fully on board. Some theorists worry that the approach will only yield more false alarms from the collider—more tentative blips in the data like “two-sigma bumps,” so named for their low level of statistical certainty. These are generally flukes that eventually disappear with more data and analysis. Koren is concerned that this will be even more the case with such an open-ended technique: “It seems they want to have a machine that finds more two-sigma bumps at the LHC.” 

Nachman told me that he received a lot of pushback; he says one senior physicist told him, “If you don’t have a particular model in mind, you’re not doing physics.” Searches based on specific models, he says, have been amazingly productive—he points to the discovery of the Higgs boson as a prime example—but they don’t have to be the end of the story. “Let the data speak for themselves,” he says.

Building bigger machines

One thing particle physicists would really like in the future is more precision. The problem with protons is that each one is actually a bundle of quarks. Smashing them together is like a subatomic food fight. Ramming indivisible particles like electrons (and their antiparticles, positrons) into one another results in much cleaner collisions, like the ones that take place on a pool table. Without the mess, researchers can make far more precise measurements of particles like the Higgs. 

An electron-positron collider would produce so many Higgs bosons so cleanly that it’s often referred to as a “Higgs factory.” But there are currently no electron-­positron colliders that have anywhere near the energies needed to probe the Higgs. One possibility on the horizon is the Future Circular Collider (FCC). It would require digging an underground ring with a circumference of 55 miles (90 kilometers)—three times the size of the LHC—in Switzerland. That work would likely cost tens of billions of dollars, and the collider would not turn on until nearly 2050. There are two other proposals for nearer-term electron-positron colliders in China and Japan, but geopolitics and budgetary issues, respectively, make them less appealing prospects. 

D. LUCCHESI ET AL.

Physicists would also like to go to higher energies. “The strategy has literally never failed us,” Homiller says. “Every time we’ve gone to higher energy, we’ve discovered some new layer of nature.” It will be nearly impossible to do so with electrons; because they have such a low mass, they radiate away about a trillion times more energy than protons every time they loop around a collider. But under CERN’s plan, the FCC tunnel could be repurposed to collide protons at energies eight times what’s possible in the LHC—about 50 years from now. “It’s completely scientifically sound and great,” Homiller says. “I think that CERN should do it.” 

Could we get to higher energies faster? In December, the alliteratively named Particle Physics Project Prioritization Panel (P5) put forward a vision for the near future of the field. In addition to addressing urgent priorities like continued funding for the HL-LHC upgrade and plans for telescopes to study the cosmos, P5 also recommended pursuing a “muon shot”—an ambitious plan to develop technology to collide muons. 

The idea of a muon collider has tantalized physicists because of its potential to combine both high energies and—since the particles are indivisible—clean collisions. It seemed well out of reach until recently; muons decay in just 2.2 microseconds, which makes them extremely hard to work with. Over the past decade, however, researchers have made strides, showing that, among other things, it should be possible to manage the roiling cloud of energy caused by decaying muons as they’re accelerated around the machine. Advocates of a muon collider also tout its smaller size (10 miles), its faster timeline (optimistically, as early as 2045), and the possibility of a US site (specifically, Fermi National Laboratory, about 50 miles west of Chicago).

There are plenty of caveats: a muon collider still faces serious technical, financial, and political hurdles—and even if it is built, there is no guarantee it will discover hidden particles. But especially for younger physicists, the panel’s endorsement of muon collider R&D is more than just a policy recommendation; it is a bet on their future. “This is exactly what we were hoping for,” Homiller says. “This opens a pathway to having this exciting, totally different frontier of particle physics in the US.” It’s a frontier he and others are keen to explore. 

Dan Garisto is a freelance physics journalist based in New York City.