Scientists have come up with a new way to introduce odors into virtual reality via small, wireless interfaces.
Creating smells in virtual reality is a vexing problem that has prevented consumer VR devices from offering a full sensory experience in most settings. “People can touch in VR,” says Xinge Yu, a professor at the department of biomedical engineering at the City University of Hong Kong and the lead author of the new paper, published today in Nature Communications. “And of course, you can see and hear in VR. But how about smell and taste?”
Previous efforts to create odors in VR have involved multiple wires, messy liquids, and bulky contraptions that don’t lend themselves to home use.
To tackle the problem, Yu and his co-author, Yuhang Li of Beihang University in Beijing (both of whom have backgrounds designing flexible electronics), developed two wearable interfaces. One can adhere to the skin between the nose and mouth like a bandage; the other straps on beneath a headset like a face mask.
Both types of interfaces use miniaturized odor generators, a grid of tiny containers filled with perfumed paraffin wax. When a heat source beneath the wax is activated, the wax heats up, essentially becoming a scented candle capable of reproducing multiple odors within 1.44 seconds, according to Yu and Li. When the experience is over, a copper coil kicks a magnet to tamp down on the wax and cool down, ending the scent.
The higher the heat, the stronger the odor and the more easily identifiable the smell, Yu says. That means that interfaces can get very hot—up to 60° C (140° F), which is dangerous for human skin. But Yu says the interface is safe because of an “open” design that lets hot air escape, along with a piece of silicone that forms a barrier between the skin and the actual device.
In a test with 11 volunteers, the interface that goes between nose and mouth was judged safe so long as it was at least 1.5 millimeters from the nose, with a temperature at the skin’s surface of 32.2° C, or 90° F—less than human body temperature. Yu realizes, though, that a scalding hot interface attached to your face might not feel safe enough to use, and he said he and Li were testing ways to make the interface run at lower temperatures or cool down more efficiently.
Yu and Li are not alone in trying to create seamless olfactory experiences in VR. At this year’s CES, OVR Technology announced that it would release a headset containing a cartridge with eight “primary” aromas designed to mix and match.
“This is quite an exciting development,” says Jas Brooks, a PhD candidate at the University of Chicago’s Human-Computer Integration Lab who has studied chemical interfaces and smell. “It’s tackling a core problem with smell in VR: How do we miniaturize this, make it not messy, and not use liquid?”
Artists have long attempted to bring scents into entertainment. In 1960, “Smell-O-Vision” made its first and only appearance with the film Scent of Mystery, which released odors during key plot points via air conditioning. But the effort bombed: during screenings, scents were either delayed or too faint to notice.
These new interfaces are a notable development that could change how we experience VR. Olfaction is a powerful sense and a prerequisite for our mouths to detect flavor. The possibilities range from the obvious—sniffing a virtual flower field or inhaling VR food—to some less obvious applications. For example, perfumeries could test fragrances virtually.
Medically, scent-equipped VR could be helpful for people who have anosmia, or an inability to smell, according to Yu. Scents can also be therapeutic for patients with memory issues and might even help with mood. Yu told me he noticed he felt happier when he used the green tea scent in his tests. He realized that the smell was nostalgic: “When I was little, I’d have some chocolate with a green tea flavor,” he recalled. “I still remember peeling off the wrapper, and how I loved the smell.”
What stands out about these new interfaces is that they are light, small, and wireless. While the device wasn’t tested directly with a VR game, platform, or specific device, the fact that it can be used without clunky wires should mean fewer tangles, less bulk, and a more immersive experience.
One drawback is that the interfaces remain limited in scope. Yu used 30 different scents chosen for their distinctive, recognizable odors, like rosemary and durian. But many real-life scents are less memorable and might not be as recognizable. Additionally, the miniaturized odor generators have yet to be programmed to work smoothly with existing VR headsets. “It’s hard to say how this would work in a commercial interface,” Brooks says.
Yu says next steps include testing mechanisms to release scents at the right moment. He also wants to start incorporating what he’s learned about smell into figuring out how to introduce taste in VR. Perhaps one day he can replicate the experience of biting into a green- tea-flavored chocolate candy.
Lo-fi chill music was playing in the distance. Shooting stars sliced through the sparkling galaxy overhead. I was defying physics, hovering in space, on my back. Relaxed, I yawned and stretched, my fist punching a pillow that I had forgotten about.
I was, of course, not in space. Physically, I was on a chaise in my home. Virtually, I was in one of many “sleep rooms” on the virtual-reality platform VRChat—virtual spaces where people can relax, and even sleep, with their headsets on. VR sleep rooms are becoming popular among people who suffer from insomnia or loneliness, offering cozy enclaves where strangers can safely find relaxation and company—most of the time.
Each VR sleep room is created to induce calm. Some imitate beaches and campsites with bonfires, while others re-create hotel rooms or cabins. Soundtracks vary from relaxing beats to nature sounds to absolute silence, while lighting can range from neon disco balls to pitch-black darkness. The opportunity to sleep in groups can be particularly appealing to isolated or lonely people who want to feel less alone.
That’s the case for Mydia Garcia, who began social sleeping almost a year ago: “I’d go dancing [in VR] till 3 a.m., and I was tired but I didn’t want to leave VR or my friends.” Garcia and their friends would visit secluded worlds and then cuddle together, finding the experience therapeutic and bonding.
Likewise, Jeff Schwerd discovered sleep rooms during the pandemic and found an antidote to loneliness. He likes to snuggle with strangers and often uses full-body tracking, which allows avatars to move in sync with IRL bodies, to imitate the feeling of being cuddled and held. Schwerd says it makes him feel protected and so more able to sleep. He finds the atmosphere of sleep rooms relaxing, too.
“My favorite place to relax alone is this grassy hill with a campfire,” he says. “I like hearing the sound of the fire.”
The company is not the only reason people fall asleep in VR. Scott Davis uses VRChat sleep rooms multiple times a week to fight his insomnia. “It’s so much easier to sleep in VR for me, and it has helped me get sleep more reliably,” he says. “Normally, outside of VR, I need to be quite fatigued to fall asleep. But in VR, I can go and lie down and fall asleep faster, even if I’m not tired at first.”
It’s why he returns to sleep rooms. “I can feel confident that I am controlling my sleep as an insomniac,” Davis says.
That feeling of control is a huge reason why VR can have a therapeutic effect for people with insomnia, says Massimiliano de Zambotti, a neuroscientist who researches sleep at the nonprofit SRI International.
“If you have insomnia, you go to bed and your brain starts spinning. You have worries and ruminations and your heart is racing. You’re not relaxed and in an elevated state of arousal, which prevents you from falling asleep,” de Zambotti says. “Neuroscientifically, VR works because you can modulate the environment you are in, but you have an anchor to reality and can feel safe enough to fall asleep.”
The trouble is, what if the experience doesn’t make you feel that way?
Feeling safe is crucial for relaxation and sleep, even if you are alone in your own bed at home.
I entered a sleep room one day and immediately heard the voice of a child in my ear. The kid, who had a robot avatar, tried and failed to engage me and a medieval knight in conversation. (My avatar was a stick of butter with a tiny top hat, because why not?) Exasperated, the robot floated over to the corner where about seven avatars were peacefully lying together, seemingly asleep. The child’s voice then taunted them: “I will kill you. I will literally kill you.”
It’s well known that the metaverse is full of underage users, and my journey through sleep rooms confirmed that kids pop up disturbingly often in these adult spaces. Another sleep room I visited was overrun with childlike voices speaking Spanish and French. I took an elevator up to a “roof” where I found a corner illuminated in red lights with plush, velvety couches. “Hi, I like your avi [avatar],” a kid’s voice said behind me. I swiveled around to find another robot avatar talking to what appeared to be a scarecrow. “I like yours too,” a man’s voice said. “Wanna cuddle?” The child floated away and I followed suit, unnerved.
Schwerd told me that he’d seen kids in sleep rooms, too. “You definitely get underage people being a nuisance,” he says. But he insisted that most sleep rooms were quiet and “respectful.”
As I roamed around, I mostly found this to be true. Some sleep rooms I stumbled into were empty and silent. Others had avatars nestled against each other, fast asleep. Still others had groups of avatars huddled together, awake but quiet, some whispering, others just relaxing. I often felt the need to mutter “Excuse me” and tiptoe, forgetting that since I was a drifting stick of butter in a room full of avatars, few would hear me or care.
I couldn’t fall asleep in VR. I was extremely aware of my surroundings and found the headset on my face uncomfortable. But while I found some rooms to be disturbing, I did discover sleep rooms that were hushed and peaceful, places to simply sit and be. In the real world, I struggle to find quiet places to relax in, and if nothing else, virtual sleep rooms offered me space and time to lie back and stare at the stars.
Last month, Sheel Mohnot and Amruta Godbole got married. This was no ordinary wedding, though. It was hosted on Decentraland, a virtual platform, and sponsored by Taco Bell.
I tried to attend. As a reporter covering virtual spaces and a fellow Indian-American, I was intrigued. Weddings are very important in Indian culture, and I wanted to see how that would play out digitally.
Unfortunately, I couldn’t get past the initial sign-in, and my screen kept crashing. It was so glitchy that I had to give up trying to watch the ceremony just a few minutes in. In fairness, that might have been just me. Others were able to watch the entire experience, including Mohnot’s grandmother in India.
Still, it left me wondering: Why would people opt to have a metaverse wedding? And will these sorts of ceremonies—especially sponsored ones—stick around, or will they fade away if virtual reality doesn’t live up to the hype?
“It’s crazy and definitely not what we had in mind,” Mohnot says. But the couple say they wanted to do something different from the usual. And beyond the novelty, Mohnot and Godbole’s motivations were straightforward: they got a free wedding out of the bargain. Mohnot is a big fan of Taco Bell, so they entered a competition for the company to pay for the technical aspects of a virtual wedding—the avatars, the production, and more. They won. In return, it plastered its brand everywhere.
For Taco Bell, it was not only a marketing opportunity but an outgrowth of what its fans wanted. The chapel at the company’s Taco Bell Cantina restaurant in Las Vegas has married 800 couples so far. There were copycat virtual weddings, too. “Taco Bell saw fans of the brand interact in the metaverse and decided to meet them quite literally where they were,” a spokesperson said. That meant dancing hot sauce packets, a Taco Bell–themed dance floor, a turban for Mohnot, and the famous bell branding everywhere.
Sheel Mohnot and Amruta Godbole’s Taco Bell metaverse wedding reception. Courtesy Taco Bell
COURTESY OF TACO BELL
If you look past the splashy branding—a trade-off some couples are willing to make for corporate help building and customizing a digital platform—virtual weddings let you do things you can’t in normal ones. For example, Mohnot rode into the ceremony in avatar form atop an elephant for his baraat, a pre-wedding procession for the groom. It’s a fun touch that would be far harder to arrange for an in-person party, especially in San Francisco, where they live.
Making it count was less straightforward. They had to set up a simultaneous livestream of themselves on YouTube in order to meet a legal requirement for their real faces to be visible. That’s because some jurisdictions—including Utah, where their officiant was based—recognize remote weddings as legally binding only if the participants are viewable on video.
A lot of couples won’t be willing to jump through that many hoops. The pandemic created an urgent need for virtual weddings, but traditional in-person ceremonies have roared back in the last year. Roughly 2.5 million weddings were held in 2022, up from 1.3 million in 2020, according to a trade group called the Wedding Report.
So why get married in the metaverse? Some are attracted to the lower cost, according to Klaus Bandisch, who runs Just Maui Weddings in Hawaii. He says the company, which also organizes real-world weddings, is booked several months in advance with metaverse ceremonies.
“We have 120 people on standby and perform at least two metaverse weddings a week,” Bandisch says. “Typically, my vow renewal package is almost $1,000, and if the couple wants avatars, we charge $300 each [person].”
That’s very affordable compared with the standard wedding held in the US, which cost an average of$30,000 in 2022, according to wedding publication The Knot.
Metaverse weddings also allow loved ones to participate without having to go anywhere. For Traci Gagnon, a particularly emotional part of her virtual wedding was having a dear friend, who had terminal cancer and was unable to travel, walk her down the aisle. “She was dancing all night long,” she says. “It was so fun and beautiful.”
One clear downside of metaverse weddings, though, is their lack of, well … realness. Weddings can be deeply sensory experiences: the smell of flowers, the sound of music, the hugs and kisses, the laughter and tears. Much of that is impossible to replicate in a virtual environment. As a result, a metaverse wedding can feel less like a wedding and more like an interactive video game.
But the couples I spoke to say that simply having loved ones “there” outweighed this drawback. Traci Gagnon spoke at length about feeling a sense of connection with her guests, despite the fact that they weren’t sharing the same physical space.
Even the distracting parts of VR were endearing to Godbole and Mohnot. “A kid would run across the screen [during the ceremony] and it was fine,” Godbole says. “It was more interactive than a normal wedding, where you are sitting silently and nothing is happening. In this case you could be expressing your own emotions through your avatar at the same time and not interrupt anything.”
The one remaining obstacle many couples and families might contend with before considering a metaverse wedding is the emotional aspect. Do you really feel married after your virtual avatars share vows and kiss?
Mohnot and Godbole said they were surprised by the intensity of their emotions after their virtual ceremony. “I thought this was going to be some fun, random thing to add to our list of unique experiences,” Godbole says. “But this was a lot more real than I expected it to be.”
Days after learning that her husband, Ted, had only months to live, Claire Matte found herself telling strangers about it in VR.
The 62-year-old retiree had bought a virtual-reality headset in 2021 as a social getaway. Ted had late-stage cancer, and the intense responsibility of caring for him had shrunk her daily reality. With the Oculus, she’d travel the world in VR and sing karaoke.
But last January, after 32 failed rounds of radiation, a doctor had told Matte and her husband that it was time to give up on treating his cancer.
“[Ted] did not want to know how long he had,” she tells me. “He left the room.” But Matte felt that, as his caretaker, she had to know. When Ted was out of earshot, the doctors told her he had four to six months to live.
On the car ride home, Ted asked if he had at least six months left. Matte decided “yes” was an honest enough answer.
Ted took his prognosis in stride—he stayed excited for the next football season, and Matte caught him laughing in front of the TV hours after the news. But he grew too sick to leave the house or, given his fragile immune system, to see guests. Their isolation deepened.
Matte still had the virtual world, though she says, “After the death sentence, I didn’t exactly feel like singing.” Later that month, as she checked out a calendar of live meetups to attend in VR, one event caught her attention: “What’s this Death Q&A?”
A virtual destination where conversation can veer from the abstract to the incredibly intimate, Death Q&A is a weekly hour-long session built around grappling with mortality, where attendees often open up about experiences and feelings they’ve shared with no one else. Bright, cartoon-like avatars represent the dozen or so people who attend each meetup, freed by VR’s combination of anonymity and togetherness to engage strangers with an earnestness we typically reserve for rare moments, if we reveal it at all.
During my four months sitting in on Death Q&A and similar sessions, I’ve heard people process cancer diagnoses, question their marriages, share treasured memories of parents and friends who’d passed hours before, turn over childhood traumas, and question openly how we can stare down our own mortality.
Despite the perception that they’re just for gaming, more people like Matte are putting on VR headsets to talk through deep pain in their day-to-day lives. The people attending VR meetups like Death Q&A are test-driving a new type of 360° digital community: one much more visceral and consuming than Zoom or the online forums that came before, and untethered to the complex social network that grounds and creates tension in traditional, face-to-face experiences.
“These relationships that we make in VR can become very intimate and deep and vulnerable,” says Tom Nickel, the 73-year-old former hospice volunteer who runs the virtual meetups with co-host Ryan Astheimer. “But they’re not complicated. Our lives don’t depend on each other.”
These people don’t share a bathroom. They don’t need to get out of bed or look presentable. They just have to listen. Many people call the meetups a lifeline—one that was particularly needed during the pandemic but seems poised to persist long after, as money continues to be pumped into building out the metaverse and loneliness crushes more people than ever.
Building an intimate VR community
Entering Death Q&A plops you in front of an inviting reproduction of a Tibetan Buddhist temple, surrounded by images from a different real-life graveyard each week. People arrange their virtual selves to face Nickel, who stands at the front by an altar. He begins most sessions by asking in a warm, neighborly voice if anyone has come with something specific to share.
About 20% log on from computers, which deliver only a 2D experience; the rest attend using VR headsets, so I put one on too. Wearing it, you hear other attendees so close up—the tremble in their voices, and a bouquet of accents. It’s as if they’re in your ear, whispering. Laughter and tears seem equally common.
The atmosphere in the sessions feels nostalgic and confessional—spectating has often felt like crashing a church service or family reunion. The crowd brings a palpable curiosity about the lives of the other attendees. Before Nickel kicks off each session, regulars often clump together to catch up; after the hour, most attendees strike up unmoderated conversations and choose to linger.
Claire and Ted Matte
COURTESY PHOTO
Matte attended her first Death Q&A right after she learned how soon her husband would die. Though Ted didn’t want to know, “those people, I could tell how long he had left,” she says.
After Matte shared, someone raised their hand to empathize, describing how they’d grieved and recovered from losing their spouse. This is one of the most striking things about Death Q&A—sharing almost always inspires someone else to talk about an experience so similar that participants feel they’ve found a person who actually understands what they’re going through.
“I knew by the end of it I was going to attend these every Tuesday at one o’clock Eastern,” Matte says.
At Death Q&A, Matte met Paul Waiyaki, a 38-year-old man living in Kenya. Matte, who lives in Georgia, now calls him one of her closest friends. “It’s just like back when you were in kindergarten, and you would look at someone and go—‘Hi, I want to be friends,’”she says. “As an adult, you don’t make friends like that. But on Oculus, with an avatar, you sure can.”
Waiyaki says he didn’t allow himself to process his sister’s death until he did it through VR. “Men, in my society, can’t be seen breaking down,” he explains. “At Death Q&A, I was able to put the baggage down. I was able to mourn and cry the tears I hadn’t cried before. It hurt to, but I could feel a wound heal as I did.”
Saying goodbye during a pandemic
Death Q&A and a similar evening session called Saying Goodbye, which is focused on loss, are just two of the 40 or so live events offered each week by EvolVR, a virtualspiritual community that was founded in 2017 by Tom Nickel’s son, Jeremy.
Before starting EvolVR, Jeremy Nickel led an interfaith church congregation in the Bay Area that was “very liberal in theology,” he says. He was looking for new ways to minister, untethered to the conventions of mainstream religion, when he first tried on a VR headset in 2015.
“The lightbulb went off in my head—people feel like they’re really together in VR,” Jeremy says. That feeling of true presence, as if avatars were really sharing a room together, convinced him that a spiritual community could form among people wearing headsets. He left the physical pulpit to host live group meditations in VR.
Then the pandemic hit. Both Saying Goodbye and Death Q&A began in early 2020—“our response to understanding that people would be losing a lot,” Tom Nickel says. They knew “that maybe people would need places to talk about it,” especially as covid precautions took away hospital-bed goodbyes and shrank people’s social circles.
Nickel, a cancer survivor himself, had spent years helping the dying depart comfortably as a hospice caregiver. That helped him gently moderate crowded Saying Goodbye and Death Q&A sessions as people joined to mourn friends and family, lament canceled graduations and closed beaches, and air anxiety about the fragility of elderly family members.
Covid-19 also triggered a wave of what psychologists call mortality salience—the realization that death isn’t only possible, but inevitable.
Elena Lister, a psychiatrist at Columbia University who specializes in grief, says a healthy level of denial about death is necessary. But now, Lister says, her colleagues are talking about a pandemic of loss that’s being felt across society—the product of mass death compounded by stunted mourning.
“What those people are doing is having an experience where they’re putting what’s deeply, deeply painful inside of them into words.”
In particular, doctors like Lister worry about complicated grief, a psychiatric disorder diagnosed when, a year after a loss, the pain of acute grief hasn’t begun muting. About 10% of the bereaved have it; they remain severely socially withdrawn and despairing, incapable of resuming the activities of their life.
The pandemic created particularly fertile ground for complicated grief. Funerals are meant to kick-start the process of integrating loss into our new reality, but for two years, “we couldn’t be together to hug and cry and sob,” she says. Lister thinks experiencing the pandemic has actually left peoplemore avoidant of discussing death.
To explain the promise of processing grief in VR, Lister paraphrases wisdom from Mr. Rogers: “What’s mentionable is manageable.” When avatars file into Death Q&A, “what those people are doing is having an experience where they’re putting what’s deeply, deeply painful inside of them into words,” Lister says, turning raw torment into something workable.
Social isolation makes it more likely that loss will harden into complicated grief. But mourning invites estrangement. Everyday conversation can feel unbearably trite when your loss feels so much more piercing, but “after a while people don’t want to hear it because they can’t fix it for you,” Nickel says. Death Q&A hands a mic to that pain and supplies an eager audience; Lister says having that community is great for promoting a healthy progression through grief.
A VR support group might suit you better than a traditional one because “there’s protection,” she says. “You can control what’s seen about you.” Sharing through an avatar, to people you never have to see again, creates a digital veil that liberates people to be shockingly honest and vulnerable.
Indeed, this echoes how Matte describes her VR experiences. “I would come and say some pretty bad things in a matter-of-fact voice, and often [Nickel] would say—‘Whoa, you know, let’s stay with this a while,’” Matte says, noting howTed worried about being a burden. “Some days I really don’t know how I went without walking around the house bawling all the time … so I told myself: Get your shit together.” Airing her devastation in VR helped her focus on making his death as comfortable as possible.
By 2021, Jeremy Nickel felt his nonprofit organization had reached an inflection point. EvolVR says 40,000 people had participated in its events since 2017. At that point, “we can either stay this sweet little thing that’s serving a couple hundred people,” he figured—or “we could make a play and try to share this with a whole lot more.”
He opted to create spaces where people can practice this new way to mourn and process in huge numbers.
In February 2022, he sold EvolVR to TRIPP, a Los Angeles–based company, for an undisclosed amount. TRIPP, which raised over $11 million in funding from backers including Amazon the previous year, has offered VR-guided meditations since 2017; the sessions have people do things like visualize their breath as stardust, coming in and out at the ideal pace to meditate.
But TRIPP’s VR meditations were solo experiences. By acquiring EvolVR, the company got a chance to tap into the unstructured, relationship-driven world of social VR, which provides a gathering space where anyone can attend events or meet people at virtual destinations open 24/7.
A “paradigm shift” for the sick and elderly
Saying Goodbye is Death Q&A’s nighttime counterpart, which Tom Nickel also runs on Tuesdays. Avatars gather around a firepit that’s lit at the end of each session.
Tom Nickel, next to his avatar
COURTESY IMAGES
Most attendees dress casually, while a few choose unnatural skin tones like bright blue. I dressed my own avatar in drab business casual, hoping to be inconspicuous. But after taking raised hands, Nickel calls on quiet attendees, asking if there’s anything on their minds that they’d like to share. During two Saying Goodbye sessions, I surprised myself by answering yes—once to talk about a painful breakup and the next time to share my mom’s cancer diagnosis. I’d spoken to friends about both, but venting in VR gave me permission to air the anxieties that their consolations couldn’t shake, without worrying about being melodramatic.
The age of participants varies, but most are over 30, and many are over 60. This initially surprised me, though in hindsight, the particular appeal of VR for older people is obvious.
A regular at Saying Goodbye, a user with a British accent and the screen name Esoteric Student, tells me he bought an Oculus on a whim in 2020. That year he lived with his nan, who was seriously ill. He watched her world shrink.
“Imagine being an 80-year-old lady and seeing your circle get smaller,” he says. “So you start off with the boundaries of the house. And it just keeps getting smaller, until you’re in one spot. And that’s it.”
He showed her the Oculus and asked, “Want to go on a spacewalk?”
They tried out a popular experience from NASA that lets you view Earth from the International Space Station. It made him sick, but his grandmother loved it. She’d never left the country.
Before she died, she saw more of the world and parts of Mars through real, crystal-clear, immersive images rendered in VR.
I’d spoken to friends, but venting in VR gave me permission to air the anxieties that their consolations couldn’t shake, without worrying about being melodramatic.
“Coming from the Great Depression to running to bomb shelters in Birmingham to eventually spending her last days being able to ascend, in a way?” he explains, crying a little bit. “It’s a paradigm shift.”
Some familiar faces at Saying Goodbye and Death Q&A are terminally ill or disabled. VR can offer a path to friendship and fresh experiences that cuts through people’s physical limits. It can also help the elderly avoid the loneliness they might feel as they watch friends die and children move away, and as retirement removes them from the working world.
Matte experiences mobility issues herself. “So I can go in VR and run, jump off a building—you know, everything under the sun,” she says. “Be young again, really.”
How far virtual support can go
Despite all its promise, at least one thing about processing emotions in VR makes Lister nervous: How do you know if people are so distressed they are at risk of harming themselves?
“It allows for more hiding,” she notes. When people interact as avatars, the nonverbal communication that psychiatrists are trained to notice, like hand gestures and fidgeting feet, is simply lost.
And the name Death Q&A can particularly attract people in crisis. Toward the end of one Death Q&A session I attended in September, an avatar in a lime green snapback, who sounded young, asked if he could speak. He’d tried to kill himself a few weeks before and said he’d found immense peace in the decision. But having survived, he told us, his behavior had changed—he was flirting with girls nonstop and found everything funny. He came off as strikingly light and unbothered. His question was: I’m still here. Now what?
Nickel sprang into action—offering, with a gentle urgency, to connect him to other survivors of suicide and asking if the young man could talk one on one after the session.
“I have to do my best to understand: Are you in a safe place right now?” Nickel says he asks himself when an attendee shares something that worries him. In addition to working in hospice, Nickel also previously worked as the director of continuing education at the California School for Professional Psychology, where he took and helped develop workshops on suicide awareness and response. But he says these trainings all need updating and rethinking for VR.
“I think that the best I can do is to offer a daily, hearing, non-judging, non-trying-to-save-anybody contact,” he says. When people in the meetup seem “shaky,” Nickel DMs them and shares his personal email. The boy in the snapback never replied. But some people do. “And in a couple of cases, I called every day.”
Lister agrees that anyone expressing suicidal ideation needs repeated support from someone highly trained. She says that if you’re going to do grief work virtually, there needs to be “a full understanding of how to reach this person, and what the follow-up is”—though, even in person, you can’t make anyone return to get help.
The more muscular tools of suicide prevention, like constant monitoring and physical restraints, are also not available in VR. “If somebody came to me in person and said they were suicidal or had tried to end their life last week, I would have great pause about having them leave my office until I felt like I could secure their safety,” Lister says.
“All I had to do was put on a headset”
In the months after Ted’s prognosis, Matte updated her new friends and fellow avatars as Ted’s voice gave out and his legs shrank from sturdy to emaciated.
Then, two nights before Ted died, he suddenly awoke, full of energy, and asked his wife if they could order Chinese food.
“At Death Q&A, I was able to put the baggage down. I was able to mourn and cry the tears I hadn’t cried before. It hurt to, but I could feel a wound heal as I did.”
He’d slept through the day and hadn’t eaten or taken his medicine, which terrified Matte. That night they enjoyed pork fried rice together on the couch; Ted ate more than he had in weeks. He put the Cubs game on in the background—he was a loyal fan, despite being from New York. “He loved an underdog,” Matte says.
It was his last solid meal. Ted Matte died June 11, 2022, at age 77.
Matte decided to attend Death Q&A and Saying Goodbye two days later. “I sort of surprised myself, being able to go,” she says. “But all I had to do was put on a headset.”
Unlike most sessions, which move from person to person, the meetings were mostly spent on Matte. Attendance at Saying Goodbye that night doubled; people said they’d come to support Matte. Through months of meetups, they’d come to feel like they knew Ted. She told them about the process of his death and their conversations in hospice. “I said that I would be okay. And I knew he loved me. And I loved him dearly,” Matte says. “And so you give the person permission to die, really.”
Attendees offered condolences and asked questions. Matte says people are interested “to compare and learn” about how peers experience a similar loss differently.
On the EvolVR Discord a month after Ted’s death, Matte shared that she’d gotten four straight nights of good sleep: “I’m onto something.” Three months out, I joined Matte in a Death Q&A session where she shared the frustration of handling an earache without Ted: “I just want someone to commiserate with!” That prompted a first-time attendee to speak, through sobs, about her husband’s death a year and a half earlier. Matte invited her to Saying Goodbye that night and stayed after to comfort her.
It’s now been six months since Ted passed. Matte feels she’s reached a turning point; she says the edges of her grief have softened. But it saddens her to move further from that anniversary. She still spends a few hours in virtual reality each day. Some days she’ll do a meditation session, or play a game with friends. But her Tuesdays remain bookended by grief meetups.
Matte acknowledges Death Q&A isn’t for everyone. She says close friends have questioned whether the meetups are cultish. But sharing her grief in VR and offering what she’s learned has “felt like a warm blanket, to be honest.”
“I don’t know what my journey would have been like without it,” she says. “But I have to envision it as much worse.”
Hana Kiros is a former Emerging Journalism Fellow at MIT Technology Review. As a freelancer, she covers science, human rights, and technology.
For a moment on Friday, Biggie Smalls was the only man on stage. A spotlight shone on him and his red velvet suit, and amid pre-recorded cheers, he rapped the lyrics to “Mo Money Mo Problems,” his orange sneakers swiveling to the beat.
You wouldn’t be wrong to be confused. Smalls died in 1997 when he was shot at the age of 24, leaving an outsize musical and cultural legacy as one of the greatest rappers of all time. But Smalls—whose real name was Christopher Wallace—was in full form on Meta’s Horizon Worlds metaverse platform on Friday: heaving between stanzas, pumping his fist rhythmically, and seeming very much alive. The performance can be seen here but may require logging into Facebook.
Smalls’s hyperrealistic avatar is not just an impressive technical feat. It is also a crucial test of two big questions we’ll soon face if metaverse platforms gain traction: whether people will pay to see an avatar of a dead artist perform, and whether that business is ethical.
Smalls isn’t the first dead artist to be resurrected. Hologram performances have long been a controversial but popular way of reanimating musicians who have passed away: Buddy Holly, Whitney Houston, Michael Jackson, and Amy Winehouse have all been turned into holograms for gigs held after they died. One of the most notable hologram shows was by Smalls’s rival Tupac Shakur, who died in 1996 but “performed” at Coachella in 2012.
Holograms, however, are inherently limited. They require audiences to sit at a specific angle to get the illusion of the artist performing in 3D. The metaverse offers a way for people to see a more lifelike avatar and even potentially interact with it—something the team behind Smalls’s gig hopes to be able to offer in the near future.
What’s remarkable about Smalls’s performance on Friday was the realism. His moves, mannerisms, and facial expressions were stunningly lifelike.
But there were some hiccups to remind viewers that Smalls was an avatar. In scenes with live rappers, Smalls seemed to stumble into his co-performers. When other rappers supported his lyrics, Smalls would sometimes wander out of the central circle where he was performing, not responding to his fellow rappers the way a living human performer would.
Smalls’s avatar was more “natural” off-screen, in pre-recorded digital segments where his likeness roamed through ’90s-era Brooklyn. His movements weren’t unnatural, his clothes werewrinkled,andhis head turned and hands moved in ways that made it hard to tell this person was a digital creation.
The technology behind this visual feat has been years in the making, says Remington Scott, the VFX director responsible for creating the Smalls avatar. Scott is the founder of Hyperreal, the studio behind the motion capture that made Andy Serkis’s Gollum character come to life in TheLord of the Rings. (In this new case, an actor was used, but the avatar incorporated the same techniques.) “When we used this technology in feature films, it would take six months and millions of dollars,” Scott says. “Now, we can do it in six weeks and at a much lower cost.”
The team gathered dozens of hours of footage from home videos and family photos to help create Smalls’s avatar, Scott says. This reference imagery was used to incorporate minuscule details into the avatar, down to the corners of Smalls’s eyes or the way his skin furrowed when he made certain expressions.
The team created a database of “micro-expression reference materials,” analyzed “pore-level resolution imagery,” and tracked the elasticity of sub-skin layers to understand how Smalls’s facial skin moved, Scott explains. Those minute changes in facial expression were crucial to creating as real an avatar as possible.
All that research paid off. “I have seen the avatar throughout the process of building … and it looks very real to me. I see my son’s characteristics in the detailing,” his mother, Voletta Wallace, said via email. “The avatar turned out to be all that I hoped for.” Scott says that when the team unveiled Smalls’s avatar to Wallace, she said, “That’s my Christopher.”
“There wasn’t a dry eye in the room,” Scott recalls. “At that moment, we surpassed any technical achievements we were striving for and were in the realm of emotionally real simulations.”
Part of the reason Smalls was a prime contender for a VR concert was that he was a star with no live recorded performances. “Biggie lived through two albums and never went on tour,” says Elliot Osagie, founder of Willingie, a digital media company that collaborated on the event. The virtual performance was an opportunity for fans to finally see their hero live—and introduce a new generation to a legendary rapper.
That’s where Wallace, who is also executor of his estate (estimated to be worth around $160 million), comes in. Although it was an emotional project, there’s no question that it was also a business opportunity: Scott says that Wallace and her son’s estate had been searching for “opportunities to bring him back to reengage with his fans and build a new fan base.” The latter part is particularly important: Smalls’s peers are Gen Xers who are only getting older. Putting Smalls in the metaverse, an arena that is dominated by younger generations, could expand his audience. Wallace confirms this: “I envision more concerts, videos of his music, commercials, animation, films, and more opportunities in the metaverse.”
Wallace, Hyperreal, Willingie, and Meta refused to disclose how much Wallace’s estate paid for the avatar, or how much Meta paid for exclusively hosting the VR concert. Meta also did not respond to MIT Technology Review’s questions about its role in the concert but did insist that the event—which was held on the company’s flagship metaverse platform, Horizon Worlds—was not held in the metaverse, but rather in virtual reality. When asked to clarify what the metaverse was, Meta did not respond.
However, Scott says that what differentiates his company’s avatars from traditional ones is ownership. With other avatars, “the actors and performers don’t subsequently have rights,” he says. “But our model is to flip that. We create digital identities for talent and then move forward.” In Smalls’s case, his estate had full input into creating his digital twin.
But how do you ensure an artist has a say in what can or cannot be reproduced? “That’s the million—or should I say billions-of-dollars question,” says Theo Tzanidis, a senior lecturer in digital marketing at the University of the West of Scotland, where he has written about the hologram and metaverse music business.
For the most part, celebrities and artists do not currently include clauses in contracts or wills about how they would like their likeness used in the metaverse or by artificial intelligence, but Tzanidis would not be surprised if the practice were to begin soon.
We have no possible way to know if Smalls would have consented to this use of his likeness, though—and there is no way he could have conceived of a platform like Horizon Worlds.
To Osagie, it’s important to make sure an avatar remains true to a given artist’s era and doesn’t do anything that person couldn’t have conceived of. He uses an upcoming metaverse project with a jazz legend as an example: “Miles Davis had a career that lasted decades. If you wanted to tell a story about his music, that’s cool. If you wanted to animate his avatar and have him playing cards with Drake—well, that’s not something that could have happened. The real line for me is that the artist is doing what they were doing.”
That may make sense. But in a future where avatars become increasingly lifelike, business expands, and the line between the metaverse and real life is blurred, it may be entirely possible for Miles Davis to play cards with Drake, with or without the approval of either person’s estate.
Even the creators of Smalls’s concert took creative liberties. One scene showed Smalls’s avatar on the balcony of what is presumablyhis apartment; the camera pans over a portrait of former president Barack Obama embracing Smalls, an event that could not have happened because Obama was elected more than 10 years after the musician’s death. At least twice, Smalls is shown answering a smartphone, a product that wasn’t available during his lifetime.
Tzanidis thinks the lack of legal framework is problematic. And it goes far beyond the traditional confines of art, in his opinion: “What if you could return back and ask people [historical figures] what they did? What if you could get training from people in your field? What will happen when we can re-create previous timelines?”
That vision is already happening: a digital version of the American golfer Jack Nicklaus is set to launch soon on an as-yet-undisclosed virtual platform. Fans will be able to interact with him, and he’ll offer golfing tips and stories recounting his wins.
Nicklaus was fully involved in the creation of his avatar. But Smalls wasn’t. And there is no way to confirm that his wishes matched his mother’s. “For the metaverse, there is no rulebook, no rules,” Tzanidis says. “There should be.”
Osagie says that Thursday’s concert is not the end for Smalls’s avatar. He and Scott are exploring expanding into other gigs and games, as well as putting on a Coachella performance by Smalls. Scott is excited by the prospect. “The metaverse is another reality, and within this one, Biggie is still alive, and I love that world,” he says. “I think a lot of fans will love that world.”
When I met Jenni Svoboda, she was in the midst of designing a beanie with a melted cupcake top, sprinkles, and doughnuts for ears.
“It’s something you’d probably never wear in real life,” she said with a laugh. But Svoboda isn’t designing for the physical world. She’s designing for the metaverse. Svoboda is working in a burgeoning, if bizarre, new niche: fashion stylists who create or curate outfits for people in virtual spaces.
Keep an eye out for #Forever21 dropping the FIRST of their line of LIMITED beanies starting today at 3 PM EST!
Grab em while you can! They WONT be on sale for 24 hours!
You can’t touch digital fabric, and if you’re not on virtual platforms like Decentraland and Roblox, you can’t even see these outfits. Nevertheless, metaverse stylists are increasingly being sought after as frequent users seek help dressing their avatars—often in experimental, wildly creative looks that defy personal expectations, societal standards, and sometimes even physics.
Most digital stylists balance their metaverse clients with real-world gigs. Michaela Leitz-Askalan, for example, runs a plus-size styling business in the real world but decided to start selling her services as a metaverse fashion stylist after hanging out in the 3D virtual world Decentraland, where her outfits got her compliments from strangers.
Another stylist, British reality television fashion expert Gemma Sheppard, made the jump to styling people in digital spaces after her goddaughter asked her to buy a pair of $60 sparkly shoes for her Roblox avatar three Christmases ago.
But not all metaverse stylists started out doing a real-world version of the job. Svoboda spends her days designing digital clothing and accessories on Roblox, where her unique fashion sense has made her an it-girl. People are lining up to pay to learn from her.
Being a metaverse fashion stylist isn’t currently a gig that can pay all the bills on its own. Leitz-Askalan says that metaverse styling accounts for about 20% of her income in a good month, and both she and Sheppard juggle multiple jobs in real life.
They say it’s still worth it, though, because the job offers the unique opportunity to work in a new medium and learn new skills. Leitz-Askalan launched her metaverse styling business a couple of years ago, meeting with clients on Discord, a chat platform popular with gamers. She designed lookbooks to help them dress their avatars on platforms like Decentraland, DressX, and Auroboros.
Her clients get an expertly curated outfit; she gets $49 in cryptocurrency. To Leitz-Askalan’s clients, it’s well worth the money. “People are like, ‘I want to try crazy things,’” she says. “And I love that.”
Svoboda is primarily a creator designing clothing and accessories for Roblox avatars, but she has begun to style clients’ avatars as well, and she’s meticulous about working out how to do it.
“We have to trial-and-error it,” she says. Svoboda will often look through users’ history of outfits, ask who their favorite artists and influencers are, and then create looks that fit their aesthetic.
“People give me notes and I go into the [Roblox] catalogue and pick out stuff that represents them,” she says. Svoboda also helps people snag their favorite influencers’ outfits, creating detailed “what they wore” pages linking to products.
None of them say it out loud, but it’s likely that some stylists are at least partly attracted by the potential to jump into what’s potentially a very lucrative market early in the game.
The metaverse fashion industry is growing rapidly, and companies like Roblox are already raking in hundreds of millions of dollars on digital clothes. In 2022, over 11.5 million creators made 62 million clothing and accessory items on Roblox alone. DressX, an online digital fashion marketplace, has raised $4.2 million in seed funding since its launch in 2020 and is one of a few brands Meta is working with to launch its own avatar fashion marketplace for its virtual platform, Horizon Worlds. And the world of haute couture is experimenting with independent metaverse projects after successful runs on other platforms, such as Gucci’s “vault” where people can browse exclusive digital fashions and play games.
Not all these outfits are pricey; indeed, many can be obtained for free. But there’s a growing market of super-exclusive outfits released in collaboration with designers that cost hundreds, even thousands, of dollars on Roblox, whose demographic veers young but is increasingly diverse in terms of age and socioeconomic status, according to the three stylists I spoke to.
If people can’t snag the stuff they want, a vibrant secondhand market exists: “In the metaverse, you can sell things for often greater prices than what you initially purchased them for,” says Sheppard, who has styled Charli XCX and the Grammys red carpet on Roblox. A perfect example is a Carolina Herrera Spring/Summer 2023 dress. The sunny, floral gown was modeled by Karlie Kloss in the designer’s New York Fashion Week showcase. Svoboda created a digital version of the dress with Herrera’s backing and Kloss promoted it, releasing the design for 500 Robux, or $5. The 432 units sold out in four hours; today, the dress is worth upwards of $5,000.
To the casual observer it can seem outlandish and even obscene to spend so much money on virtual clothes, but there are deeper reasons why people are hiring professionals to curate their outfits in the metaverse, says Sheppard. It’s all to do with experimenting in a safe, social space online.
Leitz-Askalan agrees that the metaverse offers a chance for people to go totally wild with their avatars’ outfits. “Society tells you to look a certain way, but in the metaverse, you can be anything,” she says. Her clients are willing to try avant-garde, eccentric fashions that they might consider too risky or implausible to pull off in real life, she says. As a stylist, Leitz-Askalan loves that freedom, and she can’t help but contrast it with the experiences she has in the real world, where fashion is much more restricted and restrained.
People are even using virtual clothing to play with and blur gender boundaries or explore a side of themselves they might have previously felt was inaccessible. A few weeks ago, Leitz-Askalan met with a male client on Discord who gave her free rein to dress him in whatever she wanted, gender and conventions be damned. The result was an iridescent blue-winged fairy dress, with netted sleeves, a crown of roses twisted with blue vines, and lavender kitten heels. Her client didn’t expect it—and loved it.
MICHAELA LEITZ / CONFIDENCE-STYLE.COM
Svoboda, who is trans, says exploring digital fashion can also help people escape body dysmorphia and feelings of discomfort around their appearance. It allows people to focus purely on the clothes and how they look on a virtual platform.
“When I’m working on a dress, it’s going to fit on the [avatar’s] body, no matter if they are a man or a woman, and that’s beautiful,” she says. “They can be a man, a woman, pre-op, post-op, whatever—it’s still going to be a dress, and it’s still going to fit them.” It’s a point that’s echoed by Leitz-Askalan, who often works with curvier or larger people who may have societally-driven insecurities about body image.
In theory, anyone can wear anything in the metaverse. Someone’s digital self doesn’t have to take human form or even have a body, allowing for expression that simply can’t exist in the physical world. Both Svodoba and Leitz-Askalan have styled non-human avatars, and it’s an area of experimentation that excites them both. People are realizing that in the metaverse, clothes don’t have to follow the rules. Want to be a centaur? Sure. How about a vampire with spidery legs? Why not!
That lack of restrictions is something Svoboda particularly enjoys. She describes her signature style as Barbie-core, Y2K, “fantasy pink.“ But when I called her, she was working on a completely different look for a client: “Sort of Zoe Saldana in Avatar, with blue skin—sci-fi.“
The star of Tuesday’s Meta Connect, the so-called “state of the union” for the company formerly known as Facebook, was Meta Quest Pro. Meta’s newest virtual-reality headset clocks in at a whopping $1,499.99. That’s a significant price jump from its previous iteration, Meta Quest 2, which could be yours for $399.99—not exactly cheap, but still in triple-digit territory.
That price hike, coupled with Meta’s insistence throughout the virtual event that the company envisioned the metaverse as a “next-generation social platform” accessible to everyone, sort of feels like a blatant contradiction. Even if you are among the lucky few who can shell out a grand and a half for a virtual-reality headset, would you really want to?
That’s the question Meta seems to be grappling with. While the headset price jumped, nearly all the company’s other big moves are aimed at a common and simple baseline: making the metaverse something people actually want to use.
Meta’s metaverse hasn’t exactly had a smooth year. Less than a year ago, founder Mark Zuckerberg rebranded what was then Facebook in an effort to show that the company was pivoting to what he believed was the future of our digital lives. Since then, Meta has been saddled with hiccups and gaffes, including a much-ballyhooed avatar of Zuckerberg that got memed to oblivion, a report suggesting that the company’s employees were less than enthused about the metaverse, and allegations of virtual sexual assault.
So its current strategy seems to be to release a string of updates to see what might get people interested—a “throw spaghetti at the wall and see what sticks” approach, if you will.
Besides the Meta Quest Pro, the company also announced at the event that it was going to open up Horizon Worlds, the social media platform within Meta’s metaverse, to mobile and desktop users, so people without a headset will be able to access the virtual world.
That’s a notable step: it’s a tacit admission that VR headsets aren’t taking off as quickly as the company would like. Without a critical mass of people who understand what the metaverse feels like or even is, Meta can’t hope to have its products adopted. Opening its virtual worlds to the formats consumers are comfortable with (their text messages, their browsers, the company’s beleaguered Instagram platform) gives people who aren’t open to shelling out $399.99—much less $1,499.99—a way to experience the new world.
What’s also made the metaverse a hard sell is the disorienting experience of being a floating, legless torso, and Meta announced that it won’t be that way anymore. Previously, Andrew Bosworth, Meta’s CTO, said in an Instagram AMA that full-body avatars were difficult to implement, particularly because VR tracking usually comes from someone’s real-life eyes and hands. “Tracking your own legs accurately is super hard and basically not workable just from a physics standpoint with existing headsets,” he said in February.
“legs are hard” says Meta’s Mark Zuckerberg of avatars… But they’re on the case… pic.twitter.com/mwSDagmrGm
But Zuckerberg (or, rather, his leggy avatar) announced at the event that the company was going to use artificial intelligence to map out legs in the metaverse, allowing avatars the ability not only to walk and run but also to wear digital clothing for their legs (a marketplace that Zuckerberg has said he is eager to participate in; Roblox, a gaming platform I’ve written about before, currently has a comfortable share of the market). This would be a huge step to improving how users think about movement in the metaverse and how they decide to represent themselves there.
META
But even with legs, and even with the ability to roam the metaverse without a headset strapped to your face, the key question remains: Is Meta’s metaverse something people will actually buy into? It’s worth noting that even employees at Meta are skeptical about the company’s vision, with one going so far as to say the amount spent on these projects to date made him “sick to [his] stomach.”
A free, shareable version of the metaverse accessible via weblink will open the previously closed world up to people who may not have hundreds of dollars to burn, and it’s a huge move toward democratizing the space. It might lead people to buy Meta’s claim that talking to a cartoon version of your boss is totally cool—and, more broadly, that the metaverse really is the next digital plane on which we’ll conduct our lives.
But it might also do the opposite: people might hop on the link and find that even in its now full-bodied state, the metaverse, er, doesn’t have legs.
