Why & How To Track Google Algorithm Updates via @sejournal, @lorenbaker

Google constantly evaluates and updates its algorithms. There can be hundreds or even thousands of individual changes per year.

Google does confirm some of the major updates, such as site reputation abuse, the March 2024 core update, and the November 2023 reviews update.

But, often, Google will not officially confirm an update, and these are only picked up through high volatility in the SERPs.

For example, in May of 2024, Lily Ray observed huge changes in traffic to a dozen publisher sites using rank tracking tools. Google rejected the idea of an algorithm update.

Google rejects the idea of an algorithm update observed by SEO tools.Google rejects the idea of an algorithm update observed by SEO tools.

The volatility mentioned in the tweet was observed around May 7, a day after Google announced that it rolled out a reputation abuse update with manual actions, with the algorithmic part following later.

Since it didn’t mention specific dates, many assumed that those websites were hit by the reputation abuse algorithmic rollout. However, SearchLiaison responded and refuted that assumption, leaving many SEO pros in a state of confusion.

A lot of common SEO advice you’ll see (especially from Google) amounts to “don’t chase algorithms, just do what’s best for the user” – but algorithms can have a catastrophic impact on SEO performance (sometimes unjustly).

For this reason, if you are managing a site for a brand, you need to act quickly if there is an update.

Knowing when an update hits and understanding each update will help you to adjust your strategy as needed, to avoid being impacted in future updates and also to try and recover quickly if you do have a negative impact.

Why You Should Track Google Updates

Understanding algorithms and updates is a core SEO skill. Occasionally, Google releases an update that is consequential enough to get a name (e.g., FloridaPanda, Penguin, RankBrain), and significantly impact how Google search works. You don’t want to get caught out by a big update, which means you should analyze the history of the algorithms to understand their future trajectory.

An algorithm change or update primarily impacts your website’s organic visibility in Google Search. Mostly, that comes in the form of rankings. But updates can cause disruption in other ways, too, for example by adding Search features to a particular search engine results page (SERP) that reduce click-through rates and traffic.

Tracking and understanding Google updates helps you adjust for sudden performance instability. It also helps you create SEO strategies that will be effective in the long term. Understanding where the algorithms have been helps you project where they might go. This will help you avoid risky SEO practices and reduce the risk of an update significantly impacting your website.

Recovering from updates that impact you negatively takes work and time. If you track updates, you can understand why your site’s ranking might have changed and take the necessary steps to recover as quickly as possible.

Resources For Tracking Google Algorithm Updates

Here are resources that can make your life easier and help you keep track of Google algorithm updates.

Google Search Status Dashboard

Google search dashboardImage from Google search dashboard, June 2024

The advantage of this dashboard is that it also tracks indexing and crawling incidents alongside algorithm updates in the Ranking section.

You can subscribe to updates using this RSS feed it provides.

Keep an eye on this resource to stay updated on the latest changes and incidents straight from Google.

X (Formerly Twitter)

Ten years ago, Matt Cutts was the best person at Google to follow as he regularly kept the SEO community informed about changes to search.

This role is now performed by Google SearchLiaison, which is managed by Danny Sullivan.

Google SearchLiaison's Page on XGoogle SearchLiaison’s Page on X

Make sure you follow the real @searchliaison page that has a verified badge on the profile.

If you have questions regarding Google’s algorithm updates, you may post your question on X by tagging @searchliaison, and you may be lucky to get answers directly from Google. Try to be comprehensive and provide as many details about your issue as possible to increase your chances of getting a response.

Other than the official page, you may want to follow Barry Schwartz (@rustybrick) and Marie Haynes (@Marie_Haynes), who are always on the lookout for news about algorithm changes.

Search Engine Journal

History of Google Algorithm UpdatesHistory of Google Algorithm Updates

Search Engine Journal has a dedicated page about the history of Google’s algorithm updates – from 2003 to the present. It includes the following information:

  • Algorithm name.
  • The rollout date.
  • A brief overview of the impact.
  • Whether it is confirmed or unconfirmed.
  • Related publications and official announcements so you can dive deeper and understand the changes.

You can also sign up for SEJ’s newsletters, and we’ll keep you posted on every major algorithm update.

Google isn’t a fan of third-party tools that track algorithm updates. It warns the SEO community that they are prone to errors and may have false positive detections.

I can see why Google disliked them, as they are crawling Google SERPs regularly to gather data – which, of course, Google doesn’t like. 😀

It is often true that they report volatile changes in search result pages. Still, in most cases, these tools report accurately by providing “volatility scores,” representing how much the SERP has changed.

Below is a table detailing the SERP volatility levels for various tools:

Now, let’s review a few tools you can use to track Google’s algorithm updates.

1. MozCast

Screenshot of MozCast from moz.comScreenshot of MozCast from moz.com

MozCast makes rank tracking fun in the style of a weather report.

It compares the rankings of the same set of keywords on two consecutive days and calculates how much the positions of these keywords have moved up or down, translating into a temperature scale. Per their specifications, 70°F represents a normal, stable day, and higher temperatures indicate more drastic changes.

To get an idea of what temperature is considered high, I want to note that during the March core update, MozCast’s temperature was 108°F-115°F. On May 7, its score was 90°F, then went up to 111°F, indicating that MozCast could detect movements.

So, when MozCast temperature is close to 100°F, it is quite high.

2. Semrush Sensor

Screenshot from Semrush.comScreenshot from Semrush.com

Semrush Sensor is a powerful tool designed to help you understand and track fluctuations in rankings.

Similar to MozCast, it monitors a fixed set of keywords and how much the search results for these keywords change by the end of each day. But it provides richer information by industries and locations.

Another highly valuable feature of the Semrush sensor is the report of winners and losers, which can help you run a quick competitive analysis to see websites benefiting or suffering from recent changes.

Its scale varies from 0 to 10. Usually, during core algorithm updates, the score is between 8 and 10. On May 7, its score was around 9.5 out of 10, which means there was an earthquake in SERPs.

3. Similarweb

Screenshot from Similarweb.comScreenshot from Similarweb.com

Similarweb monitors more than 10,000 domains and keywords on a daily basis to identify ranking patterns and track volatility in Google’s desktop and mobile search results.

Here is how to read its graphs:

  • The numbers on the graph indicate the level of ranking fluctuations on specific dates.
  • A higher number means more significant changes in rankings.
  • Orange signals a moderate risk.
  • Red indicates a high risk.

Again, to give you an idea of what risk level is considered high, I want to mention that during Google’s March core update, the risk level metric was 65. On May 11, the risk level metric was 71, which is high. We can conclude that Similarweb was able to detect the anomaly observed by the SEO community.

4. Accuranker ‘Grump’ Rating

Screenshot from accuranker.comScreenshot from accuranker.com

Accuranker is another great tool for observing Google SERP volatilities.

They have a fun scoring scale:

  • Grumpy (0-10): Google is chilled.
  • Cautious (10-12): Normal activity.
  • Grumpy (12-15): More than usual.
  • Furious (15+): High fluctuations in SERP.

One advantage over others is that they let you go back as far as you want, providing historical data back to 2016, and the data is updated in real-time.

In contrast to several other sensors, it provides details on how Accuranker calculates its rating:

  • It monitors a set of 30,000 randomly selected keywords.
  • It splits the keyword selection set between mobile and desktop searches (15,000 each).
  • For each keyword, it analyzes the top 100 search results.
  • The final index number for the keyword is the total sum of the position differences for each keyword divided by the number of results (typically 100).

A higher index number means more significant fluctuations in the rankings. For example, during the core update, it is in the order of ~14, which is more than usual. On May 7-9, the tool scored “Google is chilled” and ‘Cautious’ with a score of ~9.

5. Advanced Web Rankings Google Algorithm Changes

Screenshot from advancedwebranking.comScreenshot from advancedwebranking.com

Advanced Web Rankings monitors the ranking changes of approximately 400,000 desktop keywords and 200,000 mobile keywords across various countries.

You can segment the data countries, devices, and industries, and look up historical data by going back as much as you want by selecting a custom date period.

The tool calculates the Volatility (KPI), which has the following areas:

  • Low Volatility: Indicates insignificant changes.
  • Medium Volatility: Represents moderate changes in SERPs, which could be due to minor algorithm updates or other factors.
  • High Volatility: This means high fluctuations in SERP often correlated with major Google algorithm updates.

During the March core update, it detected high volatility with a score of 7.3 and medium volatility with a score of 4-5 on May 7-9.

6. CognitiveSEO Signals

Screenshot from cognitiveseo.comScreenshot from cognitiveseo.com

CognitiveSEO Signals monitors over 170,000 keywords. These keywords are randomly selected to track ranking fluctuations in desktop, mobile, and local search results.

Again, it doesn’t disclose how it calculates the volatility score, but it has a nice chart showing days with high fluctuations in red.

During the recent March core update and on May 7-9, it detected high volatility, with scores of 70 and 75, respectively.

7. Algoroo

Screenshot from algoroo.comScreenshot from algoroo.com

Algoroo is another tool to track Google’s algorithm updates, which is built and maintained by Dejan.

It doesn’t disclose how tracking works. What we know is that it tracks selected keywords and calculates their ranking movements.

Reading data is really simple; when bars are in red, it means high fluctuations.

During the recent March core update, medium volatility was detected, and nothing unusual but normal activity on May 7-9.

What To Do After An Algorithm Update

There are six things you should always remember when algorithm updates (whether confirmed or unconfirmed) negatively impact your website:

  • Don’t jump and perform sitewide changes in panic mode.
  • Check the website’s technical setup to ensure that your traffic didn’t drop due to the server being down or your developer accidentally blocking it via robots.txt or noindexing mistakenly.
  • Be patient and collect data.
  • Observe how your competitors are affected by the update to find any patterns.
  • Read credible sources (like Search Engine Journal) to gain insights and see what the SEO experts have to say.
  • Make adjustments to your SEO strategy and tactics as necessary.

It’s also important to remember that Google’s algorithms are constantly changing.

What impacts your rankings today could change in a few days, a week, or a month.

For more in-depth information, check out our guides:


Featured Image: salarko/Shutterstock

The Expert SEO Guide To URL Parameter Handling via @sejournal, @jes_scholz

In the world of SEO, URL parameters pose a significant problem.

While developers and data analysts may appreciate their utility, these query strings are an SEO headache.

Countless parameter combinations can split a single user intent across thousands of URL variations. This can cause complications for crawling, indexing, visibility and, ultimately, lead to lower traffic.

The issue is we can’t simply wish them away, which means it’s crucial to master how to manage URL parameters in an SEO-friendly way.

To do so, we will explore:

What Are URL Parameters?

url parameter elementsImage created by author

URL parameters, also known as query strings or URI variables, are the portion of a URL that follows the ‘?’ symbol. They are comprised of a key and a value pair, separated by an ‘=’ sign. Multiple parameters can be added to a single page when separated by an ‘&’.

The most common use cases for parameters are:

  • Tracking – For example ?utm_medium=social, ?sessionid=123 or ?affiliateid=abc
  • Reordering – For example ?sort=lowest-price, ?order=highest-rated or ?so=latest
  • Filtering – For example ?type=widget, colour=purple or ?price-range=20-50
  • Identifying – For example ?product=small-purple-widget, categoryid=124 or itemid=24AU
  • Paginating – For example, ?page=2, ?p=2 or viewItems=10-30
  • Searching – For example, ?query=users-query, ?q=users-query or ?search=drop-down-option
  • Translating – For example, ?lang=fr or ?language=de

SEO Issues With URL Parameters

1. Parameters Create Duplicate Content

Often, URL parameters make no significant change to the content of a page.

A re-ordered version of the page is often not so different from the original. A page URL with tracking tags or a session ID is identical to the original.

For example, the following URLs would all return a collection of widgets.

  • Static URL: https://www.example.com/widgets
  • Tracking parameter: https://www.example.com/widgets?sessionID=32764
  • Reordering parameter: https://www.example.com/widgets?sort=latest
  • Identifying parameter: https://www.example.com?category=widgets
  • Searching parameter: https://www.example.com/products?search=widget

That’s quite a few URLs for what is effectively the same content – now imagine this over every category on your site. It can really add up.

The challenge is that search engines treat every parameter-based URL as a new page. So, they see multiple variations of the same page, all serving duplicate content and all targeting the same search intent or semantic topic.

While such duplication is unlikely to cause a website to be completely filtered out of the search results, it does lead to keyword cannibalization and could downgrade Google’s view of your overall site quality, as these additional URLs add no real value.

2. Parameters Reduce Crawl Efficacy

Crawling redundant parameter pages distracts Googlebot, reducing your site’s ability to index SEO-relevant pages and increasing server load.

Google sums up this point perfectly.

“Overly complex URLs, especially those containing multiple parameters, can cause a problems for crawlers by creating unnecessarily high numbers of URLs that point to identical or similar content on your site.

As a result, Googlebot may consume much more bandwidth than necessary, or may be unable to completely index all the content on your site.”

3. Parameters Split Page Ranking Signals

If you have multiple permutations of the same page content, links and social shares may be coming in on various versions.

This dilutes your ranking signals. When you confuse a crawler, it becomes unsure which of the competing pages to index for the search query.

4. Parameters Make URLs Less Clickable

parameter based url clickabilityImage created by author

Let’s face it: parameter URLs are unsightly. They’re hard to read. They don’t seem as trustworthy. As such, they are slightly less likely to be clicked.

This may impact page performance. Not only because CTR influences rankings, but also because it’s less clickable in AI chatbots, social media, in emails, when copy-pasted into forums, or anywhere else the full URL may be displayed.

While this may only have a fractional impact on a single page’s amplification, every tweet, like, share, email, link, and mention matters for the domain.

Poor URL readability could contribute to a decrease in brand engagement.

Assess The Extent Of Your Parameter Problem

It’s important to know every parameter used on your website. But chances are your developers don’t keep an up-to-date list.

So how do you find all the parameters that need handling? Or understand how search engines crawl and index such pages? Know the value they bring to users?

Follow these five steps:

  • Run a crawler: With a tool like Screaming Frog, you can search for “?” in the URL.
  • Review your log files: See if Googlebot is crawling parameter-based URLs.
  • Look in the Google Search Console page indexing report: In the samples of index and relevant non-indexed exclusions, search for ‘?’ in the URL.
  • Search with site: inurl: advanced operators: Know how Google is indexing the parameters you found by putting the key in a site:example.com inurl:key combination query.
  • Look in Google Analytics all pages report: Search for “?” to see how each of the parameters you found are used by users. Be sure to check that URL query parameters have not been excluded in the view setting.

Armed with this data, you can now decide how to best handle each of your website’s parameters.

SEO Solutions To Tame URL Parameters

You have six tools in your SEO arsenal to deal with URL parameters on a strategic level.

Limit Parameter-based URLs

A simple review of how and why parameters are generated can provide an SEO quick win.

You will often find ways to reduce the number of parameter URLs and thus minimize the negative SEO impact. There are four common issues to begin your review.

1. Eliminate Unnecessary Parameters

remove unnecessary parametersImage created by author

Ask your developer for a list of every website’s parameters and their functions. Chances are, you will discover parameters that no longer perform a valuable function.

For example, users can be better identified by cookies than sessionIDs. Yet the sessionID parameter may still exist on your website as it was used historically.

Or you may discover that a filter in your faceted navigation is rarely applied by your users.

Any parameters caused by technical debt should be eliminated immediately.

2. Prevent Empty Values

no empty parameter valuesImage created by author

URL parameters should be added to a URL only when they have a function. Don’t permit parameter keys to be added if the value is blank.

In the above example, key2 and key3 add no value, both literally and figuratively.

3. Use Keys Only Once

single key usageImage created by author

Avoid applying multiple parameters with the same parameter name and a different value.

For multi-select options, it is better to combine the values after a single key.

4. Order URL Parameters

order url parametersImage created by author

If the same URL parameter is rearranged, the pages are interpreted by search engines as equal.

As such, parameter order doesn’t matter from a duplicate content perspective. But each of those combinations burns crawl budget and split ranking signals.

Avoid these issues by asking your developer to write a script to always place parameters in a consistent order, regardless of how the user selected them.

In my opinion, you should start with any translating parameters, followed by identifying, then pagination, then layering on filtering and reordering or search parameters, and finally tracking.

Pros:

  • Ensures more efficient crawling.
  • Reduces duplicate content issues.
  • Consolidates ranking signals to fewer pages.
  • Suitable for all parameter types.

Cons:

  • Moderate technical implementation time.

Rel=”Canonical” Link Attribute

rel=canonical for parameter handlingImage created by author

The rel=”canonical” link attribute calls out that a page has identical or similar content to another. This encourages search engines to consolidate the ranking signals to the URL specified as canonical.

You can rel=canonical your parameter-based URLs to your SEO-friendly URL for tracking, identifying, or reordering parameters.

But this tactic is not suitable when the parameter page content is not close enough to the canonical, such as pagination, searching, translating, or some filtering parameters.

Pros:

  • Relatively easy technical implementation.
  • Very likely to safeguard against duplicate content issues.
  • Consolidates ranking signals to the canonical URL.

Cons:

  • Wastes crawling on parameter pages.
  • Not suitable for all parameter types.
  • Interpreted by search engines as a strong hint, not a directive.

Meta Robots Noindex Tag

meta robots noidex tag for parameter handlingImage created by author

Set a noindex directive for any parameter-based page that doesn’t add SEO value. This tag will prevent search engines from indexing the page.

URLs with a “noindex” tag are also likely to be crawled less frequently and if it’s present for a long time will eventually lead Google to nofollow the page’s links.

Pros:

  • Relatively easy technical implementation.
  • Very likely to safeguard against duplicate content issues.
  • Suitable for all parameter types you do not wish to be indexed.
  • Removes existing parameter-based URLs from the index.

Cons:

  • Won’t prevent search engines from crawling URLs, but will encourage them to do so less frequently.
  • Doesn’t consolidate ranking signals.
  • Interpreted by search engines as a strong hint, not a directive.

Robots.txt Disallow

robots txt disallow for parameter handlingImage created by author

The robots.txt file is what search engines look at first before crawling your site. If they see something is disallowed, they won’t even go there.

You can use this file to block crawler access to every parameter based URL (with Disallow: /*?*) or only to specific query strings you don’t want to be indexed.

Pros:

  • Simple technical implementation.
  • Allows more efficient crawling.
  • Avoids duplicate content issues.
  • Suitable for all parameter types you do not wish to be crawled.

Cons:

  • Doesn’t consolidate ranking signals.
  • Doesn’t remove existing URLs from the index.

Move From Dynamic To Static URLs

Many people think the optimal way to handle URL parameters is to simply avoid them in the first place.

After all, subfolders surpass parameters to help Google understand site structure and static, keyword-based URLs have always been a cornerstone of on-page SEO.

To achieve this, you can use server-side URL rewrites to convert parameters into subfolder URLs.

For example, the URL:

www.example.com/view-product?id=482794

Would become:

www.example.com/widgets/purple

This approach works well for descriptive keyword-based parameters, such as those that identify categories, products, or filters for search engine-relevant attributes. It is also effective for translated content.

But it becomes problematic for non-keyword-relevant elements of faceted navigation, such as an exact price. Having such a filter as a static, indexable URL offers no SEO value.

It’s also an issue for searching parameters, as every user-generated query would create a static page that vies for ranking against the canonical – or worse presents to crawlers low-quality content pages whenever a user has searched for an item you don’t offer.

It’s somewhat odd when applied to pagination (although not uncommon due to WordPress), which would give a URL such as

www.example.com/widgets/purple/page2

Very odd for reordering, which would give a URL such as

www.example.com/widgets/purple/lowest-price

And is often not a viable option for tracking. Google Analytics will not acknowledge a static version of the UTM parameter.

More to the point: Replacing dynamic parameters with static URLs for things like pagination, on-site search box results, or sorting does not address duplicate content, crawl budget, or internal link equity dilution.

Having all the combinations of filters from your faceted navigation as indexable URLs often results in thin content issues. Especially if you offer multi-select filters.

Many SEO pros argue it’s possible to provide the same user experience without impacting the URL. For example, by using POST rather than GET requests to modify the page content. Thus, preserving the user experience and avoiding SEO problems.

But stripping out parameters in this manner would remove the possibility for your audience to bookmark or share a link to that specific page – and is obviously not feasible for tracking parameters and not optimal for pagination.

The crux of the matter is that for many websites, completely avoiding parameters is simply not possible if you want to provide the ideal user experience. Nor would it be best practice SEO.

So we are left with this. For parameters that you don’t want to be indexed in search results (paginating, reordering, tracking, etc) implement them as query strings. For parameters that you do want to be indexed, use static URL paths.

Pros:

  • Shifts crawler focus from parameter-based to static URLs which have a higher likelihood to rank.

Cons:

  • Significant investment of development time for URL rewrites and 301 redirects.
  • Doesn’t prevent duplicate content issues.
  • Doesn’t consolidate ranking signals.
  • Not suitable for all parameter types.
  • May lead to thin content issues.
  • Doesn’t always provide a linkable or bookmarkable URL.

Best Practices For URL Parameter Handling For SEO

So which of these six SEO tactics should you implement?

The answer can’t be all of them.

Not only would that create unnecessary complexity, but often, the SEO solutions actively conflict with one another.

For example, if you implement robots.txt disallow, Google would not be able to see any meta noindex tags. You also shouldn’t combine a meta noindex tag with a rel=canonical link attribute.

Google’s John Mueller, Gary Ilyes, and Lizzi Sassman couldn’t even decide on an approach. In a Search Off The Record episode, they discussed the challenges that parameters present for crawling.

They even suggest bringing back a parameter handling tool in Google Search Console. Google, if you are reading this, please do bring it back!

What becomes clear is there isn’t one perfect solution. There are occasions when crawling efficiency is more important than consolidating authority signals.

Ultimately, what’s right for your website will depend on your priorities.

url parameter handling option pros and consImage created by author

Personally, I take the following plan of attack for SEO-friendly parameter handling:

  • Research user intents to understand what parameters should be search engine friendly, static URLs.
  • Implement effective pagination handling using a ?page= parameter.
  • For all remaining parameter-based URLs, block crawling with a robots.txt disallow and add a noindex tag as backup.
  • Double-check that no parameter-based URLs are being submitted in the XML sitemap.

No matter what parameter handling strategy you choose to implement, be sure to document the impact of your efforts on KPIs.

More resources: 


Featured Image: BestForBest/Shutterstock

Meet the radio-obsessed civilian shaping Ukraine’s drone defense

Serhii “Flash” Beskrestnov hates going to the front line. The risks terrify him. “I’m really not happy to do it at all,” he says. But to perform his particular self-appointed role in the Russia-Ukraine war, he believes it’s critical to exchange the relative safety of his suburban home north of the capital for places where the prospect of death is much more immediate. “From Kyiv,” he says, “nobody sees the real situation.”

So about once a month, he drives hundreds of kilometers east in a homemade mobile intelligence center: a black VW van in which stacks of radio hardware connect to an array of antennas on the roof that stand like porcupine quills when in use. Two small devices on the dash monitor for nearby drones. Over several days at a time, Flash studies the skies for Russian radio transmissions and tries to learn about the problems facing troops in the fields and in the trenches.

He is, at least in an unofficial capacity, a spy. But unlike other spies, Flash does not keep his work secret. In fact, he shares the results of these missions with more than 127,000 followers—including many soldiers and government officials—on several public social media channels. Earlier this year, for instance, he described how he had recorded five different Russian reconnaissance drones in a single night—one of which was flying directly above his van.

“Brothers from the Armed Forces of Ukraine, I am trying to inspire you,” he posted on his Facebook page in February, encouraging Ukrainian soldiers to learn how to recognize enemy drone signals as he does. “You will spread your wings, you will understand over time how to understand distance and, at some point, you will save the lives of dozens of your colleagues.”

Drones have come to define the brutal conflict that has now dragged on for more than two and a half years. And most rely on radio communications—a technology that Flash has obsessed over since childhood. So while Flash is now a civilian, the former officer has still taken it upon himself to inform his country’s defense in all matters related to radio.

As well as the frontline information he shares on his public channels, he runs a “support service” for almost 2,000 military communications specialists on Signal and writes guides for building anti-drone equipment on a tight budget. “He’s a celebrity,” one special forces officer recently shouted to me over the thump of music in a Kyiv techno club. He’s “like a ray of sun,” an aviation specialist in Ukraine’s army told me. Flash tells me that he gets 500 messages every day asking for help.

Despite this reputation among rank-and-file service members—and maybe because of it—Flash has also become a source of some controversy among the upper echelons of Ukraine’s military, he tells me. The Armed Forces of Ukraine declined multiple requests for comment, but Flash and his colleagues claim that some high-ranking officials perceive him as a security threat, worrying that he shares too much information and doesn’t do enough to secure sensitive intel. As a result, some refuse to support or engage with him. Others, Flash says, pretend he doesn’t exist. Either way, he believes they are simply insecure about the value of their own contributions—“because everybody knows that Serhii Flash is not sitting in Kyiv like a colonel in the Ministry of Defense,” he tells me in the abrasive fashion that I’ve come to learn is typical of his character. 

But above all else, hours of conversations with numerous people involved in Ukraine’s defense, including frontline signalmen and volunteers, have made clear that even if Flash is a complicated figure, he’s undoubtedly an influential one. His work has become greatly important to those fighting on the ground, and he recently received formal recognition from the military for his contributions to the fight, with two medals of commendation—one from the commander of Ukraine’s ground forces, the other from the Ministry of Defense. 

With a handheld directional antenna and a spectrum analyzer, Flash can scan for hostile signals.
EMRE ÇAYLAK

Despite a small number of semi-autonomous machines with a reduced reliance on radio communications, the drones that saturate the skies above the battlefield will continue to largely depend on this technology for the foreseeable future. And in this race for survival—as each side constantly tries to best the other, only to start all over again when the other inevitably catches up—Ukrainian soldiers need to develop creative solutions, and fast. As Ukraine’s wartime radio guru, Flash may just be one of their best hopes for doing that. 

“I know nothing about his background,” says “Igrok,” who works with drones in Ukraine’s 110th Mechanized Brigade and whom we are identifying by his call sign, as is standard military practice. “But I do know that most engineers and all pilots know nothing about radios and antennas. His job is definitely one of the most powerful forces keeping Ukraine’s aerial defense in good condition.”

And given the mounting evidence that both militaries and militant groups in other parts of the world are now adopting drone tactics developed in Ukraine, it’s not only his country’s fate that Flash may help to determine—but also the ways that armies wage war for years to come.

A prescient hobby

Before I can even start asking questions during our meeting in May, Flash is rummaging around in the back of the Flash-mobile, pulling out bits of gear for his own version of show-and-tell: a drone monitor with a fin-shaped antenna; a walkie-talkie labeled with a sticker from Russia’s state security service, the FSB; an approximately 1.5-meter-long foldable antenna that he says probably came from a US-made Abrams tank.

Flash has parked on a small wooded road beside the Kyiv Sea, an enormous water reservoir north of the capital. He’s wearing a khaki sweat-wicking polo shirt, combat trousers, and combat boots, with a Glock 19 pistol strapped to his hip. (“I am a threat to the enemy,” he tells me, explaining that he feels he has to watch his back.) As we talk, he moves from one side to the other, as if the electromagnetic waves that he’s studied since childhood have somehow begun to control the motion of his body.

Now 49, Flash grew up in a suburb of Kyiv in the ’80s. His father, who was a colonel in the Soviet army, recalls bringing home broken radio equipment for his preteen son to tinker with. Flash showed talent from the start. He attended an after-school radio club, and his father fixed an antenna to the roof of their apartment for him. Later, Flash began communicating with people in countries beyond the Iron Curtain. “It was like an open door to the big world for me,” he says.

Flash recalls with amusement a time when a letter from the KGB arrived at his family home, giving his father the fright of his life. His father didn’t know that his son had sent a message on a prohibited radio frequency, and someone had noticed. Following the letter, when Flash reported to the service’s office in downtown Kyiv, his teenage appearance confounded them. Boy, what are you doing here? Flash recalls an embarrassed official saying. 

Ukraine had been a hub of innovation as part of the Soviet Union. But by the time Flash graduated from military communications college in 1997, Ukraine had been independent for six years, and corruption and a lack of investment had stripped away the armed forces’ former grandeur. Flash spent just a year working in a military radio factory before he joined a private communications company developing Ukraine’s first mobile network, where he worked with technologies far more advanced than what he had used in the military. The  project was called “Flash.” 

A decade and a half later, Flash had risen through the ranks of the industry to become head of department at the progenitor to the telecommunications company Vodafone Ukraine. But boredom prompted him to leave and become an entrepreneur. His many projects included a successful e-commerce site for construction services and a popular video game called Isotopium: Chernobyl, which he and a friend based on the “really neat concept,” according to a PC Gamer review, of allowing players to control real robots (fitted with radios, of course) around a physical arena. Released in 2019, it also received positive reviews from Reuters and BBC News.

But within just a few years, an unexpected attack would hurl his country into chaos—and upend Flash’s life. 

“I am here to help you with technical issues,” Flash remembers writing to his Signal group when he first started offering advice. “Ask me anything and I will try to find the answer for you.”
EMRE ÇAYLAK

By early 2022, rumors were growing of a potential attack from Russia. Though he was still working on Isotopium, Flash began to organize a radio network across the northern suburbs of Kyiv in preparation. Near his home, he set up a repeater about 65 meters above ground level that could receive and then rebroadcast transmissions from all the radios in its network across a 200-square-kilometer area. Another radio amateur programmed and distributed handheld radios.

When Russian forces did invade, on February 24, they took both fiber-optic and mobile networks offline, as Flash had anticipated. The radio network became the only means of instant communications for civilians and, critically, volunteers mobilizing to fight in the region, who used it to share information about Russian troop movements. Flash fed this intel to several professional Ukrainian army units, including a unit of special reconnaissance forces. He later received an award from the head of the district’s military administration for his part in Kyiv’s defense. The head of the district council referred to Flash as “one of the most worthy people” in the region.

Yet it was another of Flash’s projects that would earn him renown across Ukraine’s military.

Despite being more than 100 years old, radio technology is still critical in almost all aspects of modern warfare, from secure communications to satellite-guided missiles. But the decline of Ukraine’s military, coupled with the movement of many of the country’s young techies into lucrative careers in the growing software industry, created a vacuum of expertise. Flash leaped in to fill it.

Within roughly a month of Russia’s incursion, Flash had created a private group called “Military Signalmen” on the encrypted messaging platform Signal, and invited civilian radio experts from his personal network to join alongside military communications specialists. “I am here to help you with technical issues,” he remembers writing to the group. “Ask me anything and I will try to find the answer for you.”

The kinds of questions that Flash and his civilian colleagues answered in the first months were often basic. Group members wanted to know how to update the firmware on their devices, reset their radios’ passwords, or set up the internal communications networks for large vehicles. Many of the people drafted as communications specialists in the Ukrainian military had little relevant experience; Flash claims that even professional soldiers lacked appropriate training and has referred to large parts of Ukraine’s military communications courses as “either nonsense or junk.” (The Korolov Zhytomyr Military Institute, where many communications specialists train, declined a request for comment.)

After Russia’s invasion of Ukraine, Flash transformed his VW van into a mobile radio intelligence center.
EMRE ÇAYLAK

He demonstrates handheld spectrum analyzers with custom Ukrainian firmware.

News of the Signal group spread by word of mouth, and it soon became a kind of 24-hour support service that communications specialists in every sector of Ukraine’s frontline force subscribed to. “Any military engineer can ask anything and receive the answer within a couple of minutes,” Flash says. “It’s a nice way to teach people very quickly.” 

As the war progressed into its second year, Military Signalmen became, to an extent, self-sustaining. Its members had learned enough to answer one another’s questions themselves. And this is where several members tell me that Flash has contributed the most value. “The most important thing is that he brought together all these communications specialists in one team,” says Oleksandr “Moto,” a technician at an EU mission in Kyiv and an expert in Motorola equipment, who has advised members of the group. (He asked to not be identified by his surname, due to security concerns.) “It became very efficient.”

Today, Flash and his partners continue to answer occasional questions that require more advanced knowledge. But over the past year, as the group demanded less of his time, Flash has begun to focus on a rapidly proliferating weapon for which his experience had prepared him almost perfectly: the drone.  

A race without end

The Joker-10 drone, one of Russia’s latest additions to its arsenal, is equipped with a hibernation mechanism, Flash warned his Facebook followers in March. This feature allows the operator to fly it to a hidden location, leave it there undetected, and then awaken it when it’s time to attack. “It is impossible to detect the drone using radio-electronic means,” Flash wrote. “If you twist and turn it in your hands—it will explode.” 

This is just one example of the frequent developments in drone engineering that Ukrainian and Russian troops are adapting to every day. 

Larger strike drones similar to the US-made Reaper have been familiar in other recent conflicts, but sophisticated air defenses have rendered them less dominant in this war. Ukraine and Russia are developing and deploying vast numbers of other types of drones—including the now-notorious “FPV,” or first-person view, drone that pilots operate by wearing goggles that stream video of its perspective. These drones, which can carry payloads large enough to destroy tanks, are cheap (costing as little as $400), easy to produce, and difficult to shoot down. They use direct radio communications to transmit video feeds, receive commands, and navigate.

A Ukrainian soldier prepares an FPV drone equipped with dummy ammunition for a simulated flight operation.
MARCO CORDONE/SOPA IMAGES/SIPA USA VIA AP IMAGES

But their reliance on radio technology is a major vulnerability, because enemies can disrupt the signals that the drones emit—making them far less effective, if not inoperable. This form of electronic warfare—which most often involves emitting a more powerful signal at the same frequency as the operator’s—is called “jamming.”

Jamming, though, is an imperfect solution. Like drones, jammers themselves emit radio signals that can enable enemies to locate them. There are also effective countermeasures to bypass jammers. For example, a drone operator can use a tactic called “frequency hopping,” rapidly jumping between different frequencies to avoid a jammer’s signal. But even this method can be disrupted by algorithms that calculate the hopping patterns.

For this reason, jamming is a frequent focus of Flash’s work. In a January post on his Telegram channel, for instance, which people viewed 48,000 times, Flash explained how jammers used by some Ukrainian tanks were actually disrupting their own communications. “The cause of the problems is not direct interference with the reception range of the radio station, but very powerful signals from several [electronic warfare] antennae,” he wrote, suggesting that other tank crews experiencing the same problem might try spreading their antennas across the body of the tank. 

It is all part of an existential race in which Russia and Ukraine are constantly hunting for new methods of drone operation, drone jamming, and counter-jamming—and there’s no end in sight. In March, for example, Flash says, a frontline contact sent him photos of a Russian drone with what looks like a 10-kilometer-long spool of fiber-optic cable attached to its rear—one particularly novel method to bypass Ukrainian jammers. “It’s really crazy,” Flash says. “It looks really strange, but Russia showed us that this was possible.”

Flash’s trips to the front line make it easier for him to track developments like this. Not only does he monitor Russian drone activity from his souped-up VW, but he can study the problems that soldiers face in situ and nurture relationships with people who may later send him useful intel—or even enemy equipment they’ve seized. “The main problem is that our generals are located in Kyiv,” Flash says. “They send some messages to the military but do not understand how these military people are fighting on the front.”

Besides the advice he provides to Ukrainian troops, Flash also publishes online his own manuals for building and operating equipment that can offer protection from drones. Building their own tools can be soldiers’ best option, since Western military technology is typically expensive and domestic production is insufficient. Flash recommends buying most of the parts on AliExpress, the Chinese e-commerce platform, to reduce costs.

While all his activity suggests a close or at least cooperative relationship between Flash and Ukraine’s military, he sometimes finds himself on the outside looking in. In a post on Telegram in May, as well as during one of our meetings, Flash shared one of his greatest disappointments of the war: the military’s refusal of his proposal to create a database of all the radio frequencies used by Ukrainian forces. But when I mentioned this to an employee of a major electronic warfare company, who requested anonymity to speak about the sensitive subject, he suggested that the only reason Flash still complains about this is that the military hasn’t told him it already exists. (Given its sensitivity, MIT Technology Review was unable to independently confirm the existence of this database.) 

Flash believes that generals in Kyiv “do not understand how these military people are fighting on the front.” So even though he doesn’t like the risks they involve, he takes trips to the frontline about once a month.
EMRE ÇAYLAK

This anecdote is emblematic of Flash’s frustration with a military complex that may not always want his involvement. Ukraine’s armed forces, he has told me on several occasions, make no attempt to collaborate with him in an official manner. He claims not to receive any financial support, either. “I’m trying to help,” he says. “But nobody wants to help me.”

Both Flash and Yurii Pylypenko, another radio enthusiast who helps Flash manage his Telegram channel, say military officials have accused Flash of sharing too much information about Ukraine’s operations. Flash claims to verify every member of his closed Signal groups, which he says only discuss “technical issues” in any case. But he also admits the system is not perfect and that Russians could have gained access in the past. Several of the soldiers I interviewed for this story also claimed to have entered the groups without Flash’s verification process. 

It’s ultimately difficult to determine if some senior staff in the military hold Flash at arm’s length because of his regular, often strident criticism—or whether Flash’s criticism is the result of being held at arm’s length. But it seems unlikely either side’s grievances will subside soon; Pylypenko claims that senior officers have even tried to blackmail him over his involvement in Flash’s work. “They blame my help,” he wrote to me over Telegram, “because they think Serhii is a Russian agent reposting Russian propaganda.” 

Is the world prepared?

Flash’s greatest concern now is the prospect of Russia overwhelming Ukrainian forces with the cheap FPV drones. When they first started deploying FPVs, both sides were almost exclusively targeting expensive equipment. But as production has increased, they’re now using them to target individual soldiers, too. Because of Russia’s production superiority, this poses a serious danger—both physical and psychological—to Ukrainian soldiers. “Our army will be sitting under the ground because everybody who goes above ground will be killed,” Flash says. Some reports suggest that the prevalence of FPVs is already making it difficult for soldiers to expose themselves at all on the battlefield.

To combat this threat, Flash has a grand yet straightforward idea. He wants Ukraine to build a border “wall” of jamming systems that cover a broad range of the radio spectrum all along the front line. Russia has already done this itself with expensive vehicle-based systems, but these present easy targets for Ukrainian drones, which have destroyed several of them. Flash’s idea is to use a similar strategy, albeit with smaller, cheaper systems that are easier to replace. He claims, however, that military officials have shown no interest.

Although Flash is unwilling to divulge more details about this strategy (and who exactly he pitched it to), he believes that such a wall could provide a more sustainable means of protecting Ukrainian troops. Nevertheless, it’s difficult to say how long such a defense might last. Both sides are now in the process of developing artificial-intelligence programs that allow drones to lock on to targets while still outside enemy jamming range, rendering them jammer-proof when they come within it. Flash admits he is concerned—and he doesn’t appear to have a solution.

Flash admits he is worried about Russia overwhelming Ukrainian forces with the cheap FPV drones: “Our army will be sitting under the ground because everybody who goes above ground will be killed.”
EMRE ÇAYLAK

He’s not alone. The world is entirely unprepared for this new type of warfare, says Yaroslav Kalinin, a former Ukrainian intelligence officer and the CEO of Infozahyst, a manufacturer of equipment for electronic warfare. Kalinin recounts talking at an electronic-warfare-focused conference in Washington, DC, last December where representatives from some Western defense companies weren’t able to recognize the basic radio signals emitted by different types of drones. “Governments don’t count [drones] as a threat,” he says. “I need to run through the streets like a prophet—the end is near!”

Nevertheless, Ukraine has become, in essence, a laboratory for a new era of drone warfare—and, many argue, a new era of warfare entirely. Ukraine’s and Russia’s soldiers are its technicians. And Flash, who sometimes sleeps curled up in the back of his van while on the road, is one of its most passionate researchers. “Military developers from all over the world come to us for experience and advice,” he says. Only time will tell whether their contributions will be enough to see Ukraine through to the other side of this war. 

Charlie Metcalfe is a British journalist. He writes for magazines and newspapers, including Wired, the Guardian, and MIT Technology Review.

Meet 2024’s climate innovators under 35

This article is from The Spark, MIT Technology Review’s weekly climate newsletter. To receive it in your inbox every Wednesday, sign up here.

One way to know where a field is going? Take a look at what the sharpest new innovators are working on.

Good news for all of us: MIT Technology Review’s list of 35 Innovators Under 35 just dropped. And a decent number of the people who made the list are working in fields that touch climate and energy in one way or another.

Looking through, I noticed a few trends that might provide some hints about the future of climate tech. Let’s dig into this year’s list and consider what these innovators’ work might mean for efforts to combat climate change.

Power to the people

Perhaps unsurprisingly, quite a few innovators on this list are working on energy—and many of them have an interest in making energy consistently available where and when it’s needed. Wind and solar are getting cheap, but we need solutions for when the sun isn’t shining and the wind isn’t blowing.

Tim Latimer cofounded Fervo Energy, a geothermal company hoping to provide consistently available, carbon-free energy using Earth’s heat. You may be familiar with his work, since Fervo was on our list of 15 Climate Tech Companies to Watch in 2023.

Another energy-focused innovator on the list is Andrew Ponec of Antora Energy, a company working to build thermal energy storage systems. Basically, the company’s technology heats up blocks when cheap renewables are available, and then stores that heat and delivers it to industrial processes that need constant power. (You, the readers, named thermal energy storage the readers’ choice on this year’s 10 Breakthrough Technologies list.)

Rock stars

While new ways of generating electricity and storing energy can help cut our emissions in the future, other people are focused on how to clean up the greenhouse gases already in the atmosphere. At this point, removing carbon dioxide from the atmosphere is basically required for any scenario where we limit warming to 1.5 °C over preindustrial levels. A few of the new class of innovators are turning to rocks for help soaking up and locking away atmospheric carbon. 

Noah McQueen cofounded Heirloom Carbon Technologies, a carbon removal company. The technology works by tweaking the way minerals soak up carbon dioxide from the air (before releasing it under controlled conditions, so they can do it all again). The company has plans for facilities that could remove hundreds of thousands of tons of carbon dioxide each year. 

Another major area of research focuses on how we might store captured carbon dioxide. Claire Nelson is the cofounder of Cella Mineral Storage, a company working on storage methods to better trap carbon dioxide underground once it’s been mopped up.  

Material world

Finally, some of the most interesting work on our new list of innovators is in materials. Some people are finding new ones that could help us address our toughest problems, and others are trying to reinvent old ones to clean up their climate impacts.

Julia Carpenter found a way to make a foam-like material from metal. Its high surface area makes it a stellar heat sink, meaning it can help cool things down efficiently. It could be a huge help in data centers, where 40% of energy demand goes to cooling.

And I spoke with Cody Finke, cofounder and CEO of Brimstone, a company working on cleaner ways of making cement. Cement alone is responsible for nearly 7% of global greenhouse-gas emissions, and about half of those come from chemical reactions necessary to make it. Finke and Brimstone are working to wipe out the need for these reactions by using different starting materials to make this crucial infrastructural glue.

Addressing climate change is a sprawling challenge, but the researchers and founders on this list are tackling a few of the biggest issues I think about every day. 

Ensuring that we can power our grid, and all the industrial processes that we rely on for the stuff in our daily lives, is one of the most substantial remaining challenges. Removing carbon dioxide from the atmosphere in an efficient, cheap process could help limit future warming and buy us time to clean up the toughest sectors. And finding new materials, and new methods of producing old ones, could be a major key to unlocking new climate solutions. 

To read more about the folks I mentioned here and other innovators working in climate change and beyond, check out the full list.


Now read the rest of The Spark

Related reading

Fervo Energy (cofounded by 2024 innovator Tim Latimer) showed last year that its wells can be used like a giant underground battery.

A growing number of companies—including Antora Energy, whose CEO Andrew Ponec is a 2024 innovator—are working to bring thermal energy storage systems to heavy industry.

Cement is one of our toughest challenges, as Brimstone CEO and 2024 innovator Cody Finke will tell you. I wrote about Brimstone and other efforts to reinvent cement earlier this year.

A plant with yellow flowers

Another thing

We need a whole lot of metals to address climate change, from the copper in transmission lines to the nickel in lithium-ion batteries that power electric vehicles. Some researchers think plants might be able to help. 

Roughly 750 species of plants are so-called hyperaccumulators, meaning they naturally soak up and tolerate relatively high concentrations of metal. A new program is funding research into how we might use this trait to help source nickel, and potentially other metals, in the future. Read the full story here.

Keeping up with climate  

A hurricane that recently formed in the Gulf of Mexico is headed for Louisiana, ending an eerily quiet few weeks of the season. (Scientific American)

→ After forecasters predicted a particularly active season, the lull in hurricane activity was surprising. (New Scientist)

Rising sea levels are one of the symptoms of a changing climate, but nailing down exactly what “sea level” means is more complicated than you might think. We’ve gotten better at measuring sea level over the past few centuries, though. (New Yorker)

The US Department of Energy’s Loan Programs Office has nearly $400 million in lending authority. This year’s election could shift the focus of that office drastically, making it a bellwether of how the results could affect energy priorities. (Bloomberg)

What if fusion power ends up working, but it’s too expensive to play a significant role on the grid? Some modelers think the technology will remain expensive and could come too late to make a dent in emissions. (Heatmap)

Electric-vehicle sales are up overall, but some major automakers are backing away from goals on zero-emissions vehicles. Even though sales are increasing, uptake is slower than many thought it would be, contributing to the nervous energy in the industry. (Canary Media)

It’s a tough time to be in the business of next-generation batteries. The woes of three startups reveal that difficult times are here, likely for a while. (The Information)

The Download: Ukraine’s drone defenses, and today’s climate heroes

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

Meet the radio-obsessed civilian shaping Ukraine’s drone defense

Drones have come to define the brutal conflict in Ukraine that has now dragged on for more than two and a half years. And most rely on radio communications—a technology that Serhii “Flash” Beskrestnov has obsessed over since childhood.

While Flash is now a civilian, the former officer has still taken it upon himself to inform his country’s defense in all matters related to radio. Once a month, he studies the skies for Russian radio transmissions and tries to learn about the problems facing troops in the fields and in the trenches.

In this race for survival—as each side constantly tries to best the other, only to start all over again when the other inevitably catches up—Ukrainian soldiers need to develop creative solutions, and fast. As Ukraine’s wartime radio guru, Flash may just be one of their best hopes for doing that. Read the full story.

—Charlie Metcalfe

Meet 2024’s climate innovators under 35

One way to know where a field is going? Take a look at what the sharpest new innovators are working on.

Good news for all of us: MIT Technology Review’s list of 35 Innovators Under 35 just dropped. A decent number of the people who made the list are working in fields that touch climate and energy in one way or another. And our senior climate reporter Casey Crownhart noticed a few trends that might provide some hints about the future. Read the full story.

This year’s list is available exclusively to MIT Technology Review subscribers. If you’re not a subscriber already, you sign up here with a 25% discount on the usual price.

This story is from The Spark, our weekly climate and energy newsletter. Sign up to receive it in your inbox every Wednesday.

The must-reads

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 The first commercial spacewalk by private citizens is underway
And, thus far, it’s been a success. (CNN)
+ Take a look at the long and illustrious history of spacewalks. (BBC)

2 Silicon Valley is divided over California’s AI safety bill
Big Tech is waiting anxiously for the state’s governor to make a decision. (FT $)
+ What’s next for AI regulation? (MIT Technology Review)

3 Wildfires are raging across southern California
The state has weathered nearly three times as much acreage burn this year so far compared to the whole of 2023. (The Guardian)
+ Canada’s 2023 wildfires produced more emissions than fossil fuels in most countries. (MIT Technology Review)

4 Broken wind turbines have major repercussions
Multiple offshore wind projects have run into serious trouble. (NYT $)

5 The percentage of women in tech has hardly changed in 20 year
Women and people of color face an uphill battle to get hired. (WP $)
+ Why can’t tech fix its gender problem? (MIT Technology Review)

6 Google’s new app can turn your research into an AI podcast
Please don’t do this, though. (The Verge)

7 Human drivers keep crashing into Waymo robotaxis
The company has launched a new website to put the incidents into perspective.(Ars Technica)
+ What’s next for robotaxis in 2024. (MIT Technology Review)

8 This tiny SpaceX rival is poised to launch its first satellites
AST SpaceMobile’s star appears to be on the rise—but for how long?(Bloomberg $)

9 You’ve got a fax 📠
Pagers, fax machines and dumbphones are all the rage these days. (WSJ $)

10 Have we reached peak emoji? 😲
The little pictograms are an illustrative language, not an ideographic one. (The Atlantic $)

Quote of the day

“A beautiful world.”

—Billionaire businessman Jared Isaacman’s reaction as he saw Earth from space during the first privately funded spacewalk today, the BBC reports.

The big story

What does GPT-3 “know” about me?

August 2022

One of the biggest stories in tech is the rise of large language models that produce text that reads like a human might have written it.

These models’ power comes from being trained on troves of publicly available human-created text hoovered up from the internet. If you’ve posted anything even remotely personal in English on the internet, chances are your data might be part of some of the world’s most popular LLMs.

Melissa Heikkilä, MIT Technology Review’s senior AI reporter, wondered what data these models might have on her—and how it could be misused. So she put OpenAI’s GPT-3 to the test. Read about what she found.

In this section yesterday we stated that Amazon had acquired iRobot. This was incorrect—the acquisition never completed. We apologize for the error.

We can still have nice things

A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line or tweet ’em at me.)

+ These photos of London taken on a Casio camera watch are a snapshot of bygone times.
+ If you’ve noticed elaborate painted nails making their way into your cookbooks, it’s part of a wider trend. 💅
+ Painting Paint, now that’s meta.
+ Wow, enthusiastic skeletons are already limbering up for next month!

Google’s new tool lets large language models fact-check their responses

As long as chatbots have been around, they have made things up. Such “hallucinations” are an inherent part of how AI models work. However, they’re a big problem for companies betting big on AI, like Google, because they make the responses it generates unreliable. 

Google is releasing a tool today to address the issue. Called DataGemma, it uses two methods to help large language models fact-check their responses against reliable data and cite their sources more transparently to users. 

The first of the two methods is called Retrieval-Interleaved Generation (RIG), which acts as a sort of fact-checker. If a user prompts the model with a question—like “Has the use of renewable energy sources increased in the world?”—the model will come up with a “first draft” answer. Then RIG identifies what portions of the draft answer could be checked against Google’s Data Commons, a massive repository of data and statistics from reliable sources like the United Nations or the Centers for Disease Control and Prevention. Next, it runs those checks and replaces any incorrect original guesses with correct facts. It also cites its sources to the user.

The second method, which is commonly used in other large language models, is called Retrieval-Augmented Generation (RAG). Consider a prompt like “What progress has Pakistan made against global health goals?” In response, the model examines which data in the Data Commons could help it answer the question, such as information about access to safe drinking water, hepatitis B immunizations, and life expectancies. With those figures in hand, the model then builds its answer on top of the data and cites its sources.

“Our goal here was to use Data Commons to enhance the reasoning of LLMs by grounding them in real-world statistical data that you could source back to where you got it from,” says Prem Ramaswami, head of Data Commons at Google. Doing so, he says, will “create more trustable, reliable AI.”

It is only available to researchers for now, but Ramaswami says access could widen further after more testing. If it works as hoped, it could be a real boon for Google’s plan to embed AI deeper into its search engine.  

However, it comes with a host of caveats. First, the usefulness of the methods is limited by whether the relevant data is in the Data Commons, which is more of a data repository than an encyclopedia. It can tell you the GDP of Iran, but it’s unable to confirm the date of the First Battle of Fallujah or when Taylor Swift released her most recent single. In fact, Google’s researchers found that with about 75% of the test questions, the RIG method was unable to obtain any usable data from the Data Commons. And even if helpful data is indeed housed in the Data Commons, the model doesn’t always formulate the right questions to find it. 

Second, there is the question of accuracy. When testing the RAG method, researchers found that the model gave incorrect answers 6% to 20% of the time. Meanwhile, the RIG method pulled the correct stat from Data Commons only about 58% of the time (though that’s a big improvement over the 5% to 17% accuracy rate of Google’s large language models when they’re not pinging Data Commons). 

Ramaswami says DataGemma’s accuracy will improve as it gets trained on more and more data. The initial version has been trained on only about 700 questions, and fine-tuning the model required his team to manually check each individual fact it generated. To further improve the model, the team plans to increase that data set from hundreds of questions to millions.

Chatbots can persuade people to stop believing in conspiracy theories

The internet has made it easier than ever before to encounter and spread conspiracy theories. And while some are harmless, others can be deeply damaging, sowing discord and even leading to unnecessary deaths.

Now, researchers believe they’ve uncovered a new tool for combating false conspiracy theories: AI chatbots. Researchers from MIT Sloan and Cornell University found that chatting about a conspiracy theory with a large language model (LLM) reduced people’s belief in it by about 20%—even among participants who claimed that their beliefs were important to their identity. The research is published today in the journal Science.

The findings could represent an important step forward in how we engage with and educate people who espouse such baseless theories, says Yunhao (Jerry) Zhang, a postdoc fellow affiliated with the Psychology of Technology Institute who studies AI’s impacts on society.

“They show that with the help of large language models, we can—I wouldn’t say solve it, but we can at least mitigate this problem,” he says. “It points out a way to make society better.” 

Few interventions have been proven to change conspiracy theorists’ minds, says Thomas Costello, a research affiliate at MIT Sloan and the lead author of the study. Part of what makes it so hard is that different people tend to latch on to different parts of a theory. This means that while presenting certain bits of factual evidence may work on one believer, there’s no guarantee that it’ll prove effective on another.

That’s where AI models come in, he says. “They have access to a ton of information across diverse topics, and they’ve been trained on the internet. Because of that, they have the ability to tailor factual counterarguments to particular conspiracy theories that people believe.”

The team tested its method by asking 2,190 crowdsourced workers to participate in text conversations with GPT-4 Turbo, OpenAI’s latest large language model.

Participants were asked to share details about a conspiracy theory they found credible, why they found it compelling, and any evidence they felt supported it. These answers were used to tailor responses from the chatbot, which the researchers had prompted to be as persuasive as possible.

Participants were also asked to indicate how confident they were that their conspiracy theory was true, on a scale from 0 (definitely false) to 100 (definitely true), and then rate how important the theory was to their understanding of the world. Afterwards, they entered into three rounds of conversation with the AI bot. The researchers chose three to make sure they could collect enough substantive dialogue.

After each conversation, participants were asked the same rating questions. The researchers followed up with all the participants 10 days after the experiment, and then two months later, to assess whether their views had changed following the conversation with the AI bot. The participants reported a 20% reduction of belief in their chosen conspiracy theory on average, suggesting that talking to the bot had fundamentally changed some people’s minds.

“Even in a lab setting, 20% is a large effect on changing people’s beliefs,” says Zhang. “It might be weaker in the real world, but even 10% or 5% would still be very substantial.”

The authors sought to safeguard against AI models’ tendency to make up information—known as hallucinating—by employing a professional fact-checker to evaluate the accuracy of 128 claims the AI had made. Of these, 99.2% were found to be true, while 0.8% were deemed misleading. None were found to be completely false. 

One explanation for this high degree of accuracy is that a lot has been written about conspiracy theories on the internet, making them very well represented in the model’s training data, says David G. Rand, a professor at MIT Sloan who also worked on the project. The adaptable nature of GPT-4 Turbo means it could easily be connected to different platforms for users to interact with in the future, he adds.

“You could imagine just going to conspiracy forums and inviting people to do their own research by debating the chatbot,” he says. “Similarly, social media could be hooked up to LLMs to post corrective responses to people sharing conspiracy theories, or we could buy Google search ads against conspiracy-related search terms like ‘Deep State.’”

The research upended the authors’ preconceived notions about how receptive people were to solid evidence debunking not only conspiracy theories, but also other beliefs that are not rooted in good-quality information, says Gordon Pennycook, an associate professor at Cornell University who also worked on the project. 

“People were remarkably responsive to evidence. And that’s really important,” he says. “Evidence does matter.”

New Ecommerce Tools: September 12, 2024

Every week we publish a rundown of new products from companies offering services to ecommerce and omnichannel merchants. This installment includes updates on biometric security, robotics, product images, ecommerce personalization, AI-powered sales assistants, cross-border payments, and selling in the metaverse.

Got an ecommerce product release? Email releases@practicalecommerce.com.

New Tools for Merchants: September 12, 2024

Shopify integrates Checkout into gaming platform Roblox. Shopify announced an alliance with Roblox as a commerce integration partner on the gaming platform. Shopify will add its Checkout solution into Roblox, with a larger launch in early 2025. Developers, creators, and brands on Shopify can sell physical items directly within their Roblox games without leaving that platform. Creators and brands can engage with a global audience of nearly 80 million daily active Roblox users.

Web page with logos of Roblox and Shopify

Roblox and Shopify

Amazon partners with Covariant on AI models for robots. Amazon has entered into an agreement with Covariant, a Bay Area-based company that builds advanced AI models to help robots automate warehouse operations. Amazon receives a non-exclusive license to Covariant’s robotic foundation models (to develop new ways for Amazon’s robotic systems to learn and work for customers). A group of Covariant’s research scientists and engineers will join Amazon’s fulfillment technologies and robotics team for the implementation.

Mastercard selects India to launch its Payment Passkey Service. Mastercard has launched its Payment Passkey Service, debuting first in India as a pilot with payment aggregators, online merchants, and leading banks. Payment passkeys use device-based biometric authentication methods such as fingerprints or facial scans for secure online shopping. Mastercard’s Payment Passkey Service uses tokenization to secure a consumer’s payment details and biometric data, ensuring that no data is shared with third parties and is useless to fraudsters.

Bloomreach integrates generative AI into ecommerce search with Nvidia NeMo. Bloomreach, an ecommerce personalization platform, has announced the integration of generative AI, using Nvidia NIM embedding microservices, into its search and merchandising platform. According to Bloomreach, the NIM microservices (which leverage the Nvidia NeMo platform, part of the Nvidia AI Enterprise software stack) offer deeper search relevance with a flexible platform that allows for customizations by commerce teams. The integration into Bloomreach’s search and merchandising platform enables customized online shopping with flexibility, performance, and relevance.

Home page of Bloomreach

Bloomreach

Alibaba enables payment through Tencent’s WeChat Pay app. China-based ecommerce giant Alibaba has announced that its core Taobao and Tmall ecommerce platforms will allow payment through Tencent’s WeChat app. WeChat has more than 1.3 billion users globally, primarily in China. WeChat Pay is one of the biggest mobile payments apps in that country. By allowing users to transact through WeChat Pay on Taobao and Tmall, Alibaba seeks to increase its market share in less-developed Chinese regions.

ChannelEngine launches Vendor Hub to help Amazon merchants recover revenue. ChannelEngine, an ecommerce marketplace integrator, has launched Vendor Hub featuring a recovery management solution. This tool automates the recapture of revenue lost to operational discrepancies in Amazon’s first-party selling model. According to ChannelEngine, for brands selling through Amazon Vendor, approximately 3.5% of revenue remains unpaid due to unresolved disputes over shortages and pricing discrepancies. In addition to recovery management, Vendor Hub enhances operational efficiency with solutions to manage purchase orders and catalogs.

U.K.-based Warehow secures £2.1 million funding to enhance fulfillment services. Warehow, a U.K.-based ecommerce fulfillment platform, has received £2.1 million (USD 2.8 million) in a Series A funding round led by the Midlands Engine Investment Fund II. Warehow provides a solution for managing online orders, specifically for fashion and homeware businesses, and helps brands sell on popular marketplaces and platforms such as Shopify and TikTok, handling everything from storing inventory to shipping orders.

Home page of Warehow

Warehow

Amazon launches generative AI-powered shopping assistant Rufus in the U.K. Amazon is launching Rufus in beta to customers in the U.K. Rufus is an AI-powered shopping assistant trained on Amazon’s product catalog and information from across the web. Rufus can answer customer questions on shopping needs, products, comparisons, recommendations, and discovery. Rufus is launching in Amazon’s mobile app for a subset of U.K. customers and will add more in the coming weeks.

JD.com offers enhanced shipping options and free delivery in the U.S., Japan, and Singapore. JD Global Sales, JD.com’s international ecommerce division that caters to Chinese consumers globally, has announced enhancements to its shipping services for customers in the U.S., Japan, and Singapore. Shoppers will have access to improved logistics, including consolidated shipping and three direct shipping models: Air Express, Air Standard, and Sea Economical. JD Global Sales is also introducing free regional shipping services for qualifying orders.

Cymbio launches Cymagery AI to expedite product content uploads. Cymbio, a marketplace and dropship automation platform, has launched Cymagery AI, an image upload and editing suite. Cymagery AI automates product content marketing, from image upload to verifying copyright compliance. Users can (i) automatically adjust image size, margins, and file names, (ii) edit and remove backgrounds, (iii) expand images, (iv) access instant AI-generated swatches, and (v) convert image formats. According to Cymbio, Cymagery AI automates the order synchronization process for online marketplaces, expediting time-to-market for sellers.

DataFeedWatch by Cart.com introduces native AI to its feed management. DataFeedWatch by Cart.com, a platform for managing and optimizing product feeds, has integrated AI into its feed marketing engine. The AI uses all available product data to create keyword-rich titles and descriptions based on best practices for 200 product types. Marketers retain complete control and can decide whether to use AI-generated content.

Home page of DataFeedWatch by Cart.com

DataFeedWatch by Cart.com

OpenAI Claims New “o1” Model Can Reason Like A Human via @sejournal, @MattGSouthern

OpenAI has unveiled its latest language model, “o1,” touting advancements in complex reasoning capabilities.

In an announcement, the company claimed its new o1 model can match human performance on math, programming, and scientific knowledge tests.

However, the true impact remains speculative.

Extraordinary Claims

According to OpenAI, o1 can score in the 89th percentile on competitive programming challenges hosted by Codeforces.

The company insists its model can perform at a level that would place it among the top 500 students nationally on the elite American Invitational Mathematics Examination (AIME).

Further, OpenAI states that o1 exceeds the average performance of human subject matter experts holding PhD credentials on a combined physics, chemistry, and biology benchmark exam.

These are extraordinary claims, and it’s important to remain skeptical until we see open scrutiny and real-world testing.

Reinforcement Learning

The purported breakthrough is o1’s reinforcement learning process, designed to teach the model to break down complex problems using an approach called the “chain of thought.”

By simulating human-like step-by-step logic, correcting mistakes, and adjusting strategies before outputting a final answer, OpenAI contends that o1 has developed superior reasoning skills compared to standard language models.

Implications

It’s unclear how o1’s claimed reasoning could enhance understanding of queries—or generation of responses—across math, coding, science, and other technical topics.

From an SEO perspective, anything that improves content interpretation and the ability to answer queries directly could be impactful. However, it’s wise to be cautious until we see objective third-party testing.

OpenAI must move beyond benchmark browbeating and provide objective, reproducible evidence to support its claims. Adding o1’s capabilities to ChatGPT in planned real-world pilots should help showcase realistic use cases.


Featured Image: JarTee/Shutterstock

Google Expands YouTube First Position Ad Availability via @sejournal, @MattGSouthern

Google has announced the expansion of its First Position ad offering on YouTube, making it available across all content types through Display & Video 360.

This marks a change from the previous limitation of First Position ads to YouTube Select inventory.

What Are First Position Ads?

First Position ads are in-stream advertisements that appear at the beginning of YouTube videos, ensuring they are the first ad viewers see.

This placement is designed to capture audience attention when engagement is at its highest.

Key Changes to First Position:

  • Availability: Now accessible for all YouTube content, not just YouTube Select inventory
  • Pricing: Shifted from a fixed-rate CPM to a dynamic pricing model through Display & Video 360
  • Targeting: Allows advertisers to reach target audiences across a broader range of content

This feature is now available in all markets where First Position ads were previously offered.

Ad Formats & Placement

First Position targeting is available for both in-stream and Shorts ad formats, expanding the potential reach of these ads.

However, it’s worth noting that in-stream line items targeting the first position are not guaranteed to serve in the first position of a user’s session on YouTube TV.

This may affect strategies for connected TV advertising.

Instant Reserve & Implementation

Advertisers can use Instant Reserve, a Display & Video 360 feature, to get a quote and reserve YouTube inventory immediately without negotiations.

This aligns with the new dynamic pricing model, offering more flexibility in ad purchasing.

For implementation, advertisers should note that YouTube videos used in First Position ads must be set to “Public” or “Unlisted” visibility. Private videos cannot be used in these campaigns.

Reporting & Measurement

To assess the performance of First Position ads, advertisers can utilize Basic report templates and YouTube-specific reports available in Display & Video 360.

These tools allow for detailed analysis of ad performance across various metrics.

Case Studies Provided

Google cited two examples in its announcement:

  1. Booking.com reportedly saw a 21% relative lift in ad recall during a holiday campaign.
  2. IHG Hotels & Resorts claimed to achieve twice the YouTube benchmark for ad recall and brand awareness when combining First Position ads with Content Takeovers.

Context

The move may affect how brands allocate their video advertising budgets and could impact competition for prime ad placements on YouTube.

Here are the potential implications of these changes for advertisers:

  • Flexible Budgeting: Dynamic pricing allows for more adaptable spending strategies.
  • Expanded Reach: First Position ads are now available across all YouTube content, not just Select inventory.
  • Increased Competition: Wider availability may drive up costs for premium placements.
  • Strategic Planning: Advertisers may need to be more selective about using First Position ads.

Advertisers interested in leveraging First Position ads should consult Google’s Help Center for information on Instant Reserve in Display & Video 360 and Reservations in Google Ads to understand the implementation process and best practices.


Featured Image: Rokas Tenys/Shutterstock