Making data matter in real time

As the world becomes increasingly networked and connected devices proliferate, organizations are producing a plethora of data. The potential to collect data is growing exponentially. From smart grids to mobile phones and from connected cars to the industrial internet of things, tens of billions of devices will act as sensors, delivering data to networks.

Whether data is created intentionally or as a by-product of operations, companies that want to grow beyond digitization and become data-driven enterprises must harness it to inform decisions and streamline operations. As experts predict that nearly every employee in the modern enterprise will use data to support their work, targeted curation and analysis of real-time data will be increasingly critical. While the fintech and health-care sectors are leaders in using real-time data, the trend is also picking up in manufacturing, city infrastructure, and other sectors where the push for smart technology is growing.

This report examines how organizations can rethink their data management strategies to leverage real-time data to develop personalized real-time digital experiences or applications that customers want, boost operational efficiency, and increase business performance.

The following are the key findings of this report:

• Collecting and analyzing data in real time allows companies to create innovative applications. When done right, real-time data can be used to gauge customer sentiment and engagement, accurately deliver targeted content, optimize supply chains, fight fraud, and offer alternatives to missed flights, among other things. However, experts say only a fraction of data from connected devices is collected and processed in real time due to hurdles including legacy technology architectures, signaling untapped potential for enterprises.

• Organizations need to determine which business processes would benefit most from real-time data collection and analysis, because capturing all data and processing it in real time is expensive. Creating architectures to process captured data also becomes exponentially more difficult as the number of devices grows. Companies that create the right data architectures will be able to process the captured data in real time and store the information in a way that is most useful for future decisions.

Open-source solutions, cloud, and artificial intelligence systems could help organizations overcome the challenges of managing real-time data. Experts we spoke to for this report recommend developing initial data pipelines with open-source technologies. That will help organizations to develop in-house expertise, avoid vendor lock-in, and establish a baseline for expectations. In addition, artificial intelligence (AI) is the next natural progression: Operational data platforms need to support real-time analytic functionality as AI and machine learning (ML) become increasingly embedded in enterprise operations. Real-time, AI-powered apps with the right data, at the right time, will deliver the biggest business impact for leading organizations.

Download the full report.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff.

Asia spurs transparency and digital economy growth 

From generating artworks to creating sustainable supply chains, artificial intelligence (AI) has become a critical tool in a myriad of economic sectors worldwide. As global enterprises increasingly use AI to gain a competitive edge, governments are also working hard to fuel innovation and growth with AI. 

In recent years, Asian countries have stepped up efforts to support the rapid growth of their digital economies. These include measures to equip businesses with the necessary tools and infrastructure to use emerging technologies, such as AI, and support innovation and foster global confidence in them. 

Singapore has unveiled the world’s first AI governance testing framework and toolkit. Named AI Verify, it is currently a minimum viable product at a pilot stage.

The ‘innovative regulator’

“We have a belief that being an innovative regulator is not an oxymoron,” says Lew Chuen Hong, chief executive of Singapore’s Infocomm Media Development Authority (IMDA). “And the real role of the regulator is to build the foundations for trust, so that businesses, governments, and consumers have the trust to innovate and co-create in the digital domain.” 

As AI fast becomes ubiquitous in day-to-day activities, calls for more robust governance to ensure AI systems are fair, transparent, and safe are increasing. For example, the European Union is negotiating a new AI Act and the U.S. Federal Trade Commission is working on new legislation to allow it to rule on issues of AI discrimination, fraud, and related data misuse. 

In Asia, countries such as Korea, India, and Singapore are trying to chart their own paths in AI ethics and governance. Among them, Singapore is taking a balanced approach by working with various stakeholders to build a more trusted AI environment. 

In 2020, Singapore released its Model AI Governance Framework to provide detailed guidance—with implementable measures and practices—to help companies deploy AI responsibly. Besides showcasing use cases from different industries, IMDA also collaborated with the World Economic Forum’s Centre for the Fourth Industrial Revolution to release a guide to help organizations align their AI governance practices with the framework. 

Putting AI governance to the test

In 2022, Singapore took another step forward to help companies validate the implementation of responsible AI. The island nation unveiled the world’s first AI governance testing framework and toolkit, named AI Verify, designed to provide a standardized method to verify AI systems’ performance in relation to internationally recognized ethical principles. 

Currently a minimum viable product at a pilot stage, AI Verify is a testing framework that comprises process checks and technical tests. For a start, the technical tests will focus on verifying the fairness, robustness, and explainability of some supervised learning models. Companies that test with AI Verify can use the reports it generates to improve their AI models and demonstrate how their AI systems align with their claimed performance. Rather than setting ethical standards, AI Verify helps companies be more transparent about their AI implementation. 

Following feedback and preliminary testing with partners such as Singapore-based bank DBS, Google, Meta, Microsoft, Singapore Airlines, and Standard Chartered Bank, AI Verify is available for international pilot. Policymakers, regulators, AI system developers, and business owners can participate and provide feedback on the global viability of the framework. 

Robust growth, high digital penetration 

The role of AI governance will become even more significant as Asia’s digital economy continues to grow. While a tech slowdown has dogged the U.S.—with more than 91,000 workers laid off in 2022—Asia seems unfazed. According to a Google, Temasek, and Bain & Company report in October 2022, Southeast Asia’s leading digital economies likely amounted to S$ 200 billion (US$ 149 billion) in 2022, marking a 20% increase from 2021. Far from this being a short-term growth spurt, the region’s digital economy is forecast to reach S$ 300 billion (US$ 224 billion) by 2025.

Asia’s ability to defy a digital downturn that has plagued others lies in “big shifts both on the demand side and the supply side,” says Simon Chesterman, senior director of AI governance at AI Singapore. On the demand side, a combination of high internet usage, high penetration of digital devices, such as smartphones, and population-level comfort with technological innovation has seen many Asian individuals and businesses embrace the digital economy at speed, explains Chesterman. 

As of February 2023, 93% of companies in Singapore had adopted some form of digital technology, marking an increase of 19 percentage points from 2018, according to IMDA. This explains a key point of differentiation with some western economies, says Chesterman. “When you’ve got fast-developing economies, people are more willing to embrace change because they can see the benefit,” he says. “Whereas the more comfortable you are, the more resistant you may be to change.” 

This willingness to embrace digital technologies has only increased with the global pandemic. Three quarters (76%) of the population in Southeast Asia viewed technology as an enabler rather than an impediment during the peak of covid-19, according to an August 2022 report by VMware—surpassing the global average by four percentage points—and 77% say digitalization improves both their work and lifestyles. 

Compounding strong demand in the region has been a steady supply of innovation from the region’s vast network of enterprises, underpinned by direct support from government. Increased public funding in Hong Kong, for example, resulted in the creation of 3,755 start-ups in 2021, a 12% boost over the previous year, marking a record high for the Special Administrative Region. The Singapore government has committed S$ 25 billion (US$ 18 billion) to research, innovation, and enterprise from 2021 to 2025, and growing the digital economy was identified as one of the key pillars of that initiative.

Building a digital ecosystem

Meanwhile, Singapore’s IMDA, which bills itself as the “architect” of the island’s digital future, has introduced a series of initiatives to entrench the city-state as a global and regional technology hub. It has made strategic investments in both hard and soft infrastructure to accelerate digital economic growth in the country. Singapore has achieved nationwide standalone 5G coverage (over 95%) three years ahead of schedule, and IMDA has rolled out digital utilities such as TradeTrust, which streamlines the exchange of electronic documents. 

IMDA also plays a central role in creating a strong digital talent pipeline and a progressive regulatory framework to foster innovation. By enhancing the credibility and trustworthiness of digital products and services, it aims to spur growth in the digital economy. In June 2022, for instance, it launched a US$ 36.3 million Digital Trust Centre as part of the country’s R&D efforts focused on enhancing the legitimacy of digital systems.

A fine balance

Government intervention often takes a two-pronged approach, Chesterman explains: “Governments should regulate to avoid market failures, because it’s inefficient to expect individual consumers to negotiate this themselves. The second reason governments regulate, though, is, even if it’s not geared toward efficiency, we have certain values and principles that we hold to.” 

There are challenges though, Chesterman adds. To develop globally accepted standards, a delicate balance between a viable framework and overregulation needs to be achieved via mechanisms such as digital economy agreements. Finding a way to develop a framework that can evolve at the same rapid pace as the technology itself is also crucial.

Singapore has risen to the challenge with AI Verify. Its pioneering work in AI governance, coupled with ongoing infrastructure investments to support the country’s digital economy, indicates the significance of the sector to the region’s growth prospects. It also shows the need for authorities in the region to collaborate with international partners to ensure any such digital economy is both open and interoperable. This is particularly true for cities like Hong Kong and geographically small countries like Singapore.

“Small states like Singapore require open trade for survival,” says Lew. “A strong and robust digital economy means companies here can thrive globally, where size or geography does not matter. Investing in such technology and innovation is critical for long-term competitiveness and value capture.”

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff.

Big Tech could help Iranian protesters by using an old tool

After the Iranian government took extreme measures to limit internet use in response to the pro-democracy protests that have filled Iranian streets since mid-September, Western tech companies scrambled to help restore access to Iranian citizens. 

Signal asked its users to help run proxy servers with support from the company. Google offered credits to help Iranians get online using Outline, the company’s own VPN. And in response to a post by US Secretary of State Antony Blinken on Iran’s censorship, Elon Musk quickly tweeted: “Activating Starlink …

But these workarounds aren’t enough. Though the first Starlink satellites have been smuggled into Iran, restoring the internet will likely require several thousand more. Signal tells MIT Technology Review that it has been vexed by “Iranian telecommunications providers preventing some SMS validation codes from being delivered.” And Iran has already detected and shut down Google’s VPN, which is what happens when any single VPN grows too popular (plus, unlike most VPNs, Outline costs money).

What’s more, “there’s no reliable mechanism for Iranian users to find these proxies,” Nima Fatemi, head of global cybersecurity nonprofit Kandoo, points out. They’re being promoted on social media networks that are themselves banned in Iran. “While I appreciate their effort,” he adds, “it feels half-baked and half-assed.”

There is something more that Big Tech could do, according to some pro-democracy activists and experts on digital freedom. But it has received little attention—even though it’s something several major service providers offered until just a few years ago.

“One thing people don’t talk about is domain fronting,” says Mahsa Alimardani, an internet researcher at the University of Oxford and Article19, a human rights organization focused on freedom of expression and information. It’s a technique developers used for years to skirt internet restrictions like those that have made it incredibly difficult for Iranians to communicate safely. In essence, domain fronting allows apps to disguise traffic directed toward them; for instance, when someone types a site into a web browser, this technique steps into that bit of browser-to-site communication and can scramble what the computer sees on the back end to disguise the end site’s true identity.

In the days of domain fronting, “cloud platforms were used for circumvention,” Alimardani explains. From 2016 to 2018, secure messaging apps like Telegram and Signal used the cloud hosting infrastructure of Google, Amazon, and Microsoft—which most of the web runs on—to disguise user traffic and successfully thwart bans and surveillance in Russia and across the Middle East.

But Google and Amazon discontinued the practice in 2018, following pushback from the Russian government and citing security concerns about how it could be abused by hackers. Now activists who work at the intersection of human rights and technology say reinstating the technique, with some tweaks, is a tool Big Tech could use to quickly get Iranians back online.

Domain fronting “is a good place to start” if tech giants really want to help, Alimardani says. “They need to be investing in helping with circumvention technology, and having stamped out domain fronting is really not a good look.”

Domain fronting could be a critical tool to help protesters and activists stay in touch with each other for planning and safety purposes, and to allow them to update worried family and friends during a dangerous period. “We recognize the possibility that we might not come back home every time we go out,” says Elmira, an Iranian woman in her 30s who asked to be identified only by her first name for security reasons.

Still, no major companies have publicly said they will consider launching or restoring the anti-censorship tool. Two of the three major service providers that previously allowed domain fronting, Google and Microsoft, could not be reached for comment. The third, Amazon, directed MIT Technology Review to a 2019 blog post in which a product manager described steps the company has taken to minimize the “abusive use of domain fronting practices.”

“A cat-and-mouse game”

By now, Iranian citizens largely expect that their digital communications and searches are being combed through by the powers of the state. “They listen and control almost all communications in order to counter demonstrations,” says Elmira. “It’s like we’re being suffocated.”

This isn’t, broadly speaking, a new phenomenon in the country. But it’s reached a crisis point over the past two months, during a growing swell of anti-government protests sparked by the death of 22-year-old Mahsa Amini on September 16 after Iran’s Guidance Patrol—more commonly known as the morality police—arrested her for wearing her hijab improperly.

“The world realized that the matter of hijab, which I myself believe is a personal choice, could become an incident over which a young girl can lose her life,” Elmira says. 

According to rights groups, over 300 people, including at least 41 children, have been killed since protests began. The crackdown has been especially brutal in largely Kurdish western Iran, where Amini was from and Elmira now lives. Severely restricting internet access has been a way for the regime to further crush dissent. “This is not the first time that the internet services have been disrupted in Iran,” Elmira says. “The reason for this action is the government’s fear, because there is no freedom of speech here.”

The seeds of today’s digital repression trace back to 2006, when Iran announced plans to craft its own intranet—an exclusive, national network designed to keep Iranians off the World Wide Web. 

“This is really hard to do,” says Kian Vesteinsson, a senior analyst for the global democracy nonprofit Freedom House. That’s because it requires replicating global infrastructure with domestic resources while pruning global web access.

The payoff is “digital spaces that are easier to monitor and to control,” Vesteinsson says. Of the seven countries trying to isolate themselves from the global internet, Iran is the furthest along today.

Iran debuted its National Information Network in 2019, when authorities hit a national kill switch on the global web amid protests over gas prices. During a week when the country was electronically cut off from the rest of the world, the regime killed 1,500 people. The Iranian economy, which relies on broader connectivity to do business, lost over a billion US dollars during the bloody week. 

While recently Iran has intermittently cut access to the entire global internet in some regions, it hasn’t instituted another total global web shutdown. Instead, it is largely pursuing censorship strategies designed to crush dissent while sparing the economy. Rolling “digital curfews” are in place from about 4 p.m. into the early morning hours—ensuring that the web becomes incredibly difficult to access during the period when most protests occur.

The government has blocked most popular apps, including Twitter, Instagram, Facebook, and WhatsApp, in favor of local copycat apps where no message or search is private.

“The messaging apps we use, like WhatsApp, have a certain level of protection embedded in their coding,” Elmira says. “We feel more comfortable using them. [The government] cannot have control over them, and as a result, they restrict access.”

The Iranian regime is also aggressively shutting down VPNs, which were a lifeline for many Iranians and the country’s most popular censorship workaround. About 80% of Iranians use tools to bypass censorship and use apps they prefer. “Even my grandpa knows how to install a VPN app,” an Iranian woman who requested anonymity for safety reasons tells me. 

To crush VPN use, Iran’s government has invested heavily in “deep packet inspection,” a technology that peers into the fine print of internet traffic and can recognize and shut down nearly any VPN with time.

That’s created a “cat-and-mouse game,” says Alimardani, the internet researcher. “You need to be offering, like, thousands of VPNs,” she says, so that some will remain available as Iran diligently recognizes and blocks others. Without enough VPNs, activists aren’t left with many secure communication options, making it much harder for Iranians to coordinate protests and communicate with the outside world as death tolls climb.

Domain fronting to beat censors

Domain fronting works by concealing the app or website a user ultimately wants to reach. It’s sort of like putting a correctly addressed postcard in an envelope with a different, innocuous destination—then having someone at the fake-out address hand-deliver it.

The technique is attractive because it’s implemented by service providers rather than individuals, who may or may not be tech savvy. It also makes censorship more painful for governments to pursue. The only way to ban a domain-fronted app is to shut down the entire web hosting provider the app uses—bringing an avalanche of other apps and sites down with it. And since Microsoft, Amazon, and Google provide hosting services for most of the digital world, domain fronting by those companies would force countries to crash much of the internet in order to deny access to an undesired app.

“There’s no way to just pick out Telegram. That’s the power of it,” says Erik Hunstad, a security expert and CTO of the cybersecurity company SixGen.

Nevertheless, in April 2018, Russia blocked Amazon, Google, and a host of other popular services in order to ban the secure-messaging app Telegram, which initially used domain fronting to beat censors. These disruptions made the ban broadly unpopular with average Russians, not just activists who favored the app. 

The Russian government, in turn, exerted pressure on Amazon and Google to end the practice.

In April 2018, the companies terminated support for domain fronting altogether. “Amazon and Google just completely disabled this potentially extremely useful service,” Alimardani says. 

Google made the change quietly, but soon afterwards, it described domain fronting to the Verge as a “quirk” of its software. In its own announcement, Amazon said domain fronting could help malware masquerade as standard traffic. Hackers could also abuse the technique—the Russian hacker group APT29 has used domain fronting, alongside other means, to access classified data.

Still, Signal, which began using domain fronting in 2016 to operate in several Middle Eastern countries attempting to block the app, issued a statement at the time: “The censors in these countries will have (at least temporarily) achieved their goals.”

“While domain fronting still works with domains on smaller networks, this greatly limits the current utility of the technique,” says Simon Migliano, a digital privacy expert and head of research at Top10VPN, an independent VPN review website.

(Microsoft announced a ban on domain fronting in 2021, but the cloud infrastructure that enables the technique is intact. Earlier this week, Microsoft wrote that, going forward, it will “block any HTTP request that exhibits domain fronting behavior.”)

Migliano echoes Google in describing domain fronting as “essentially a bug,” and he admits it has “very real security risks.” It is “certainly a shame” that companies are revoking it, he says, “but you can understand their position.”

But Hunstad, who also works in cybersecurity, says there are ways to minimize the cybersecurity risks of domain fronting while preserving its use as an anti-censorship tool. He explains that the way networks process user requests means Google, Amazon, or Microsoft could easily greenlight the use of domain fronting for certain apps, like WhatsApp or Telegram, while otherwise banning the tactic.

Rather than technical limitations, Hunstad says, it’s a “prisoner’s dilemma situation [for] the big providers” that is keeping them from re-enabling domain fronting—they’re stuck between pressure from authoritarian governments and an outcry from activists. He speculates that financial imperatives are part of the calculus as well. 

“If I’m hosting my website with Google, and they decide to enable this for Signal and Telegram, or maybe across the board, and multiple countries decide to remove access to all of Google because of that—then I have potentially less reach,” Hunstad says. “I’ll just go to the provider that’s not doing it, and Google is going to have a business impact.” 

The likelihood that Amazon or Google will reinstate domain fronting depends on “how cynical you are about their profit motives versus their good intentions for the world,” Hunstad adds. 

What’s next

While Fatemi, from Kandoo, argues that restoring domain fronting would be helpful for Iranian protesters, he emphasizes that it wouldn’t be a silver bullet. 

“In the short term, if they can relax domain fronting so that people, for example, can use Signal, or people can connect to VPN connections, that would be phenomenal,” he says. He adds that to move solutions along more quickly, companies like Google could collaborate with nonprofits that specialize in deploying tech in vulnerable situations. 

But Big Tech companies also need to commit a bigger slice of their resources and talent to developing technologies that can beat internet censorship, he says: “[Domain fronting is] a Band-Aid on a much larger problem. If we want to go at a much larger problem, we have to dedicate engineers.” 

Until the world finds an enduring solution to authoritarian attempts to splinter the global web, tech companies that want to help people will be left scrambling for reactive tactics. 

“There needs to be a whole toolkit of different kinds of VPNs and circumvention tools right now, because what they are doing is highly sophisticated,” Alimardani says. “Google is one of the richest and most powerful companies in the world. And offering one VPN is really not enough.”

So for now, seven weeks into Iran’s protests, internet and VPN access remain throttled, restrictions show no sign of slowing, and domain fronting remains dead. And it’s the citizens on the front lines who have to carry the biggest burden.

“The conditions are dire here,” Elmira tells me. The lack of connectivity has made massacres difficult to verify and has complicated efforts to sustain protests and other activism. 

“To counter the demonstrations, they cut off our access to the internet and social media,” she says. 

But Elmira is resolute. “I, myself, and many of my friends now go out with no fear,” she says. “We know that they might shoot us. But it is worth taking this risk and to go out and try our best instead of staying home and continuing taking this.”