Apr 25 2024
Google’s New Infini-Attention And SEO via @sejournal, @martinibuster

Google has published a research paper on a new technology called Infini-attention that allows it to process massively large amounts of data with “infinitely long contexts” while also being capable of being easily inserted into other models to vastly improve their capabilities

That last part should be of interest to those who are interested in Google’s algorithm. Infini-Attention is plug-and-play, which means it’s relatively easy to insert into other models, including those in use b Google’s core algorithm. The part about “infinitely long contexts” may have implications for how some of Google’s search systems may work.

The name of the research paper is: Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

Memory Is Computationally Expensive For LLMs

Large Language Models (LLM) have limitations on how much data they can process at one time because the computational complexity and memory usage can spiral upward significantly. Infini-Attention gives the LLM the ability to handle longer contexts while keeping the down memory and processing power needed.

The research paper explains:

“Memory serves as a cornerstone of intelligence, as it enables efficient computations tailored to specific contexts. However, Transformers …and Transformer-based LLMs …have a constrained context-dependent memory, due to the nature of the attention mechanism.

Indeed, scaling LLMs to longer sequences (i.e. 1M tokens) is challenging with the standard Transformer architectures and serving longer and longer context models becomes costly financially.”

And elsewhere the research paper explains:

“Current transformer models are limited in their ability to process long sequences due to quadratic increases in computational and memory costs. Infini-attention aims to address this scalability issue.”

The researchers hypothesized that Infini-attention can scale to handle extremely long sequences with Transformers without the usual increases in computational and memory resources.

Three Important Features

Google’s Infini-Attention solves the shortcomings of transformer models by incorporating three features that enable transformer-based LLMs to handle longer sequences without memory issues and use context from earlier data in the sequence, not just data near the current point being processed.

The features of Infini-Attention

  • Compressive Memory System
  • Long-term Linear Attention
  • Local Masked Attention

Compressive Memory System

Infini-Attention uses what’s called a compressive memory system. As more data is input (as part of a long sequence of data), the compressive memory system compresses some of the older information in order to reduce the amount of space needed to store the data.

Long-term Linear Attention

Infini-attention also uses what’s called, “long-term linear attention mechanisms” which enable the LLM to process data that exists earlier in the sequence of data that’s being processed which enables to retain the context. That’s a departure from standard transformer-based LLMs.

This is important for tasks where the context exists on a larger plane of data. It’s like being able to discuss and entire book and all of the chapters and explain how the first chapter relates to another chapter closer to the end of the book.

Local Masked Attention

In addition to the long-term attention, Infini-attention also uses what’s called local masked attention. This kind of attention processes nearby (localized) parts of the input data, which is useful for responses that depend on the closer parts of the data.

Combining the long-term and local attention together helps solve the problem of transformers being limited to how much input data it can remember and use for context.

The researchers explain:

“The Infini-attention incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block.”

Results Of Experiments And Testing

Infini-attention was tested with other models for comparison across multiple benchmarks involving long input sequences, such as long-context language modeling, passkey retrieval, and book summarization tasks. Passkey retrieval is a test where the language model has to retrieve specific data from within a extremely long text sequence.

List of the three tests:

  1. Long-context Language Modeling
  2. Passkey Test
  3. Book Summary

Long-Context Language Modeling And The Perplexity Score

The researchers write that the Infini-attention outperformed the baseline models and that increasing the training sequence length brought even further improvements in the Perplexity score. The Perplexity score is a metric that measures language model performance with lower scores indicating better performance.

The researchers shared their findings:

“Infini-Transformer outperforms both Transformer-XL …and Memorizing Transformers baselines while maintaining 114x less memory parameters than the Memorizing Transformer model with a vector retrieval-based KV memory with length of 65K at its 9th layer. Infini-Transformer outperforms memorizing transformers with memory length of 65K and achieves 114x compression ratio.

We further increased the training sequence length to 100K from 32K and trained the models on Arxiv-math dataset. 100K training further decreased the perplexity score to 2.21 and 2.20 for Linear and Linear + Delta models.”

Passkey Test

The passkey test is wherea random number is hidden within a long text sequence with the task being that the model must fetch the hidden text. The passkey is hidden either near the beginning, middle or the end of the long text. The model was able to solve the passkey test up to a length of 1 million.

“A 1B LLM naturally scales to 1M sequence length and solves the passkey retrieval task when injected with Infini-attention. Infini-Transformers solved the passkey task with up to 1M context length when fine-tuned on 5K length inputs. We report token-level retrieval accuracy for passkeys hidden in a different part (start/middle/end) of long inputs with lengths 32K to 1M.”

Book Summary Test

Infini-attention also excelled at the book summary test by outperforming top benchmarks achieving new state of the art (SOTA) performance levels.

The results are described:

“Finally, we show that a 8B model with Infini-attention reaches a new SOTA result on a 500K length book summarization task after continual pre-training and task fine-tuning.

…We further scaled our approach by continuously pre-training a 8B LLM model with 8K input length for 30K steps. We then fine-tuned on a book summarization task, BookSum (Kry´sci´nski et al., 2021) where the goal is to generate a summary of an entire book text.

Our model outperforms the previous best results and achieves a new SOTA on BookSum by processing the entire text from book. …There is a clear trend showing that with more text provided as input from books, our Infini-Transformers improves its summarization performance metric.”

Implications Of Infini-Attention For SEO

Infini-attention is a breakthrough in modeling long and short range attention with greater efficiency than previous models without Infini-attention. It also supports “plug-and-play continual pre-training and long-context adaptation
by design” which means that it can easily be integrated into existing models.

Lastly, the “continual pre-training and long-context adaptation” makes it exceptionally useful for scenarios where it’s necessary to constantly train the model on new data. This last part is super interesting because it may make it useful for applications on the back end of Google’s search systems, particularly where it is necessary to be able to analyze long sequences of information and understand the relevance from one part near the beginning of the sequence and another part that’s closer to the end.

Other articles focused on the “infinitely long inputs” that this model is capable of but where it’s relevant to SEO is how that ability to handle huge input and “Leave No Context Behind” is what’s relevant to search marketing and how some of Google’s systems might work if Google adapted Infini-attention to their core algorithm.

Read the research paper:

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

Featured Image by Shutterstock/JHVEPhoto

Ecommerce MGMT 0 Comments
Feb 1 2024
Could This Be The Navboost Patent? via @sejournal, @martinibuster

There’s been a lot of speculation of what Navboost is but to my knowledge nobody has pinpointed an adequate patent that could be the original Navboost patent. This patent from 2004 closely aligns with Navboost

So I took the few clues we have about it and identified a couple likely patents.

The clues I was working with are that Google Software Engineer Amit Singhal was involved with Navboost and had a hand in inventing it. Another clue is that Navboost dated to 2005. Lastly, the court documents indicate that Navboost was updated later on so there may be other patents in there about that, which we’ll get to at some point but not in this article.

So I deduced that if Amit Singhal was the inventor then there would be a patent with his name on it and indeed there is, dating from 2004.

Out of all the patents I saw, the two most interesting were these:

  • Systems and methods for correlating document topicality and popularity 2004
  • Interleaving Search Results 2007

This article will deal with the first one, Systems and methods for correlating document topicality and popularity dating from 2004, which aligns with the known timeline of Navboost dating to 2005.

Patent Does Not Mention Clicks

An interesting quality of this patent is that it doesn’t mention clicks and I suspect that people looking for the Navboost patent may have ignored it because it doesn’t mention clicks.

But the patent discusses concepts related to user interactions and navigational patterns which are references to clicks.

Instances Where User Clicks Are Implied In The Patent

Document Selection and Retrieval:
The patent describes a process where a user selects documents (which can be inferred as clicking on them) from search results. These selections are used to determine the documents’ popularity.

Mapping Documents to Topics:
After documents are selected by users (through clicks), they are mapped to one or more topics. This mapping is a key part of the process, as it associates documents with specific areas of interest or subjects.

User Navigational Patterns:
The patent frequently refers to user navigational patterns, which include how users interact with documents, such as the documents they choose to click on. These patterns are used to compute popularity scores for the documents.

It’s clear that user clicks are a fundamental part of how the patent proposes to assess the popularity of documents.

By analyzing which documents users choose to interact with, the system can assign popularity scores to these documents. These scores, in combination with the topical relevance of the documents, are then used to enhance the accuracy and relevance of search engine results.

Patent: User Interactions Are A Measure Of Popularity

The patent US8595225 makes implicit references to “user clicks” in the context of determining the popularity of documents. Heck, popularity is so important to the patent that it’s in the name of the patent: Systems and methods for correlating document topicality and popularity

User clicks, in this context, refers to the interactions of users with various documents, such as web pages. These interactions are a critical component in establishing the popularity scores for these documents.

The patent describes a method where the popularity of a document is inferred from user navigational patterns, which can only be clicks.

I’d like to stop here and mention that Matt Cutts has discussed in a video that Popularity and PageRank are two different things. Popularity is about what users tend to prefer and PageRank is about authority as evidenced by links.

Matt defined popularity:

“And so popularity in some sense is a measure of where people go whereas PageRank is much more a measure of reputation.”

That definition from about 2014 fits what this patent is talking about in terms of popularity being about where people go.

See Matt Cutts Explains How Google Separates Popularity From True Authority

Watch the YouTube Video: How does Google separate popularity from authority?

How The Patent Uses Popularity Scores

The patent describes multiple ways that it uses popularity scores.

Assigning Popularity Scores:
The patent discusses assigning popularity scores to documents based on user interactions such as the frequency of visits or navigation patterns (Line 1).

Per-Topic Popularity:
It talks about deriving per-topic popularity information by correlating the popularity data associated with each document to specific topics (Line 5).

Popularity Scores in Ranking:
The document describes using popularity scores to order documents among one or more topics associated with each document (Line 13).

Popularity in Document Retrieval:
In the context of document retrieval, the patent outlines using popularity scores for ranking documents (Line 27).

Determining Popularity Based on User Navigation:
The process of determining the popularity score for each document, which may involve using user navigational patterns, is also mentioned (Line 37).

These instances demonstrate the patent’s focus on incorporating the popularity of documents, as determined by user interaction (clicks), into the process of ranking and correlating them to specific topics.

The approach outlined in the patent suggests a more dynamic and user-responsive method of determining the relevance and importance of documents in search engine results.

Navboost Assigns Scores To Documents

I’m going to stop here to also mention that this patent mentions assigning scores to documents, which is how Google executive Eric Lehman described in the trial how Navboost worked:

Speaking about the situation where there wasn’t a lot of click data, Lehman testified:

“And so I think Navboost does kind of the natural thing, which is, in the face of that kind of uncertainty, you take gentler measures. So you might modify the score of a document but more mildly than if you had more data.”

That’s another connection to Navboost in that the trial description and the patent describe using User Interaction for scoring webpages.

The more this patent is analyzed, the more it looks like what the trial documents described as Navboost.

Read the patent here:

Systems and methods for correlating document topicality and popularity

Featured Image by Shutterstock/Sabelskaya

Ecommerce MGMT 0 Comments
Jan 29 2024
Why Google SGE Is Stuck In Google Labs And What’s Next via @sejournal, @martinibuster

Google Search Generative Experience (SGE) was set to expire as a Google Labs experiment at the end of 2023 but its time as an experiment was quietly extended, making it clear that SGE is not coming to search in the near future. Surprisingly, letting Microsoft take the lead may have been the best perhaps unintended approach for Google.

Google’s AI Strategy For Search

Google’s decision to keep SGE as a Google Labs project fits into the broader trend of Google’s history of preferring to integrate AI in the background.

The presence of AI isn’t always apparent but it has been a part of Google Search in the background for longer than most people realize.

The very first use of AI in search was as part of Google’s ranking algorithm, a system known as RankBrain. RankBrain helped the ranking algorithms understand how words in search queries relate to concepts in the real world.

According to Google:

“When we launched RankBrain in 2015, it was the first deep learning system deployed in Search. At the time, it was groundbreaking… RankBrain (as its name suggests) is used to help rank — or decide the best order for — top search results.”

The next implementation was Neural Matching which helped Google’s algorithms understand broader concepts in search queries and webpages.

And one of the most well known AI systems that Google has rolled out is the Multitask Unified Model, also known as Google MUM.  MUM is a multimodal AI system that encompasses understanding images and text and is able to place them within the contexts as written in a sentence or a search query.

SpamBrain, Google’s spam fighting AI is quite likely one of the most important implementations of AI as a part of Google’s search algorithm because it helps weed out low quality sites.

These are all examples of Google’s approach to using AI in the background to solve different problems within search as a part of the larger Core Algorithm.

It’s likely that Google would have continued using AI in the background until the transformer-based large language models (LLMs) were able to step into the foreground.

But Microsoft’s integration of ChatGPT into Bing forced Google to take steps to add AI in a more foregrounded way with  their Search Generative Experience (SGE).

Why Keep SGE In Google Labs?

Considering that Microsoft has integrated ChatGPT into Bing, it might seem curious that Google hasn’t taken a similar step and is instead keeping SGE in Google Labs. There are good reasons for Google’s approach.

One of Google’s guiding principles for the use of AI is to only use it once the technology is proven to be successful and is implemented in a way that can be trusted to be responsible and those are two things that generative AI is not capable of today.

There are at least three big problems that must be solved before AI can successfully be integrated in the foreground of search:

  1. LLMs cannot be used as an information retrieval system because it needs to be completely retrained in order to add new data. .
  2. Transformer architecture is inefficient and costly.
  3. Generative AI tends to create wrong facts, a phenomenon known as hallucinating.

Why AI Cannot Be Used As A Search Engine

One of the most important problems to solve before AI can be used as the backend and the frontend of a search engine is that LLMs are unable to function as a search index where new data is continuously added.

In simple terms, what happens is that in a regular search engine, adding new webpages is a process where the search engine computes the semantic meaning of the words and phrases within the text (a process called “embedding”), which makes them searchable and ready to be integrated into the index.

Afterwards the search engine has to update the entire index in order to understand (so to speak) where the new webpages fit into the overall search index.

The addition of new webpages can change how the search engine understands and relates all the other webpages it knows about, so it goes through all the webpages in its index and updates their relations to each other if necessary. This is a simplification for the sake of communicating the general sense of what it means to add new webpages to a search index.

In contrast to current search technology, LLMs cannot add new webpages to an index because the act of adding new data requires a complete retraining of the entire LLM.

Google is researching how to solve this problem in order create a transformer-based LLM search engine, but the problem is not solved, not even close.

To understand why this happens, it’s useful to take a quick look at a recent Google research paper that is co-authored by Marc Najork and Donald Metzler (and several other co-authors). I mention their names because both of those researchers are almost always associated with some of the most consequential research coming out of Google. So if it has either of their names on it, then the research is likely very important.

In the following explanation, the search index is referred to as memory because a search index is a memory of what has been indexed.

The research paper is titled: “DSI++: Updating Transformer Memory with New Documents” (PDF)

Using LLMs as search engines is a process that uses a technology called Differentiable Search Indices (DSIs). The current search index technology is referenced as a dual-encoder.

The research paper explains:

“…index construction using a DSI involves training a Transformer model. Therefore, the model must be re-trained from scratch every time the underlying corpus is updated, thus incurring prohibitively high computational costs compared to dual-encoders.”

The paper goes on to explore ways to solve the problem of LLMs that “forget” but at the end of the study they state that they only made progress toward better understanding what needs to be solved in future research.

They conclude:

“In this study, we explore the phenomenon of forgetting in relation to the addition of new and distinct documents into the indexer. It is important to note that when a new document refutes or modifies a previously indexed document, the model’s behavior becomes unpredictable, requiring further analysis.

Additionally, we examine the effectiveness of our proposed method on a larger dataset, such as the full MS MARCO dataset. However, it is worth noting that with this larger dataset, the method exhibits significant forgetting. As a result, additional research is necessary to enhance the model’s performance, particularly when dealing with datasets of larger scales.”

LLMs Can’t Fact Check Themselves

Google and many others are also researching multiple ways to have AI fact check itself in order to keep from giving false information (referred to as hallucinations). But so far that research is not making significant headway.

Bing’s Experience Of AI In The Foreground

Bing took a different route by incorporating AI directly into its search interface in a hybrid approach that joined a traditional search engine with an AI frontend. This new kind of search engine revamped the search experience and differentiated Bing in the competition for search engine users.

Bing’s AI integration initially created significant buzz, drawing users intrigued by the novelty of an AI-driven search interface. This resulted in an increase in Bing’s user engagement.

But after nearly a year of buzz, Bing’s market share saw only a marginal increase. Recent reports, including one from the Boston Globe, indicate less than 1% growth in market share since the introduction of Bing Chat.

Google’s Strategy Is Validated In Hindsight

Bing’s experience suggests that AI in the foreground of a search engine may not be as effective as hoped. The modest increase in market share raises questions about the long-term viability of a chat-based search engine and validates Google’s cautionary approach of using AI in the background.

Google’s focusing of AI in the background of search is vindicated in light of Bing’s failure to cause users to abandon Google for Bing.

The strategy of keeping AI in the background, where at this point in time it works best, allowed Google to maintain users while AI search technology matures in Google Labs where it belongs.

Bing’s approach of using AI in the foreground now serves as almost a cautionary tale about the pitfalls of rushing out a technology before the benefits are fully understood, providing insights into the limitations of that approach.

Ironically, Microsoft is finding better ways to integrate AI as a background technology in the form of useful features added to their cloud-based office products.

Future Of AI In Search

The current state of AI technology suggests that it’s more effective as a tool that supports the functions of a search engine rather than serving as the entire back and front ends of a search engine or even as a hybrid approach which users have refused to adopt.

Google’s strategy of releasing new technologies only when they have been fully tested explains why Search Generative Experience belongs in Google Labs.

Certainly, AI will take a bolder role in search but that day is definitely not today. Expect to see Google adding more AI based features to more of their products and it might not be surprising to see Microsoft continue along that path as well.

Featured Image by Shutterstock/ProStockStudio

Ecommerce MGMT 0 Comments
Jan 18 2024
How To Read Google Patents In 5 Easy Steps via @sejournal, @martinibuster

Reading and understanding patents filed by Google can be challenging but this guide will help you to understand what the patents are about and to as avoid the many common mistakes that lead to misunderstandings.

How To Understand Google Patents

Before starting to read a patent it’s important to understand how to read the patents. The following rules will form the foundation upon which you can build a solid understanding of what patents mean.

Step #1 Do Not Scan Patents

One of the biggest mistakes I see people make when reading patents is to approach the task as if it’s a treasure hunt. They scan the patents looking for tidbits and secrets about Google’s algorithms.

I know people do this because I’ve seen so many wrong conclusions made by SEOs who I can tell didn’t read the patent because they only speak about the one or two sentences that jump out at them.

Had they read the entire patent they would have understood that the passage they got excited about had nothing to do with ranking websites.

Reading a patent is not like a treasure hunt with a metal detector where the treasure hunter scans an entire field and then stops in one spot to dig up a cache of gold coins.

Don’t scan a patent. Read it.

Step #2 Understand The Context Of The Patent

A patent is like an elephant. An elephant has a trunk, big ears, a little tail and legs thick as trees. Similarly, a patent is made up of multiple sections that are each very important because they create the context of what the patent is about. Each section of a patent is important.

And just like how each part of an elephant in the context of the entire animal helps to better understand the creature, so it is with patents that every section only makes sense within the context of the entire patent.

In order to understand the patent it’s important to read the entire patent several times in order to be able step back and see the entire patent, not just one part of the patent.

Reading the entire patent reveals what the context of the entire patent is, which is the most important thing about the patent, what the entire thing means.

Step #3 Not Every Patent Is About Ranking

If there’s any one thing I wish the reader to take away from this article is this rule. When I read tweets or articles by people who don’t know how to read patents, this is the rule that they haven’t understood. Consequently, the interpretation of the patent is wrong.

Google Search is not just one ranking algorithm. There are many algorithms that comprise different parts of Search. The Ranking Engine and the Indexing Engine are just two parts of Search.

Other elements of search that may be referred to are:

  • Ranking engine
  • Modification engine
  • Indexing engine
  • Query reviser engine

Those are just a few of the kinds of software engines that are a part of a typical search engine. While the different software engines are not necessarily a part of the ranking part of Google’s algorithms, that does not minimize their importance.

Back in 2020 Gary Illyes of Google tweeted that Search consists of thousands of different systems working together.

He tweeted about the indexing engine:

“The indexing system, Caffeine, does multiple things:
1. ingests fetchlogs,
2. renders and converts fetched data,
3. extracts links, meta and structured data,
4. extracts and computes some signals,
5. schedules new crawls,
6. and builds the index that is pushed to serving.”

He followed up with another tweet about the thousands of systems in search:

“Don’t oversimplify search for it’s not simple at all: thousands of interconnected systems working together to provide users high quality and relevant results…

…the last time i did this exercise I counted off the top of my head about 150 different systems from crawling to ranking, so thousands is likely not an exaggeration. Yes, some things are micro services”

Here’s The Important Takeaway:

There are many parts of Search. But not all parts of Search are a part of the ranking systems.

A very important habit to cultivate when reading a patent is to let the patent tell you what it’s about.

Equally important is to not make assumptions or assume that something is implied. Patents don’t generally imply. They may be broad and and they may seem to be so repetitive that it almost feels like a deliberate attempt obfuscate (make it hard to understand) and they consistently describe the inventions in extremely broad terms, but they don’t really imply what they are describing.

Patents, for legal purposes, are actually quite specific about what the patents are about.

If something is used for ranking then it will not be implied, the patent will say so because that’s an important quality to describe in a patent application.

Step #4 Entity & Entities: Understand The Use Of Abstraction

One of the biggest mistakes that happens to people who read patents is to overlook the context of where the invention can be used. For example, let’s review a specific patent called “Identifying subjective attributes by analysis of curation signals.”

This patent mentions entities 52 times and the word “entity” is mentioned in the patent itself 124 times. One can easily guess that this patent is probably about entities, right? It makes sense that if the patent mentions the words “entities” and “entity” nearly 200 times that the patent is about entities.

But that would be an unfortunate assumption because the patent is not about entities at all because the context of the use of the words “entity” and “entities” in this patent is to refer to a broad and inclusive range of items, subjects, or objects to which the invention can be applied.

Patents often cast a wide net in terms of how the invention can be used, which helps to ensure that the patent’s claims aren’t limited to one type of use but can be applied in many ways.

The word “entity” in this patent is used as a catch-all term that allows the patent to cover a wide range of different types of content or objects. It is used in the sense of an abstraction so that it can be applied to multiple objects or forms of content. This frees the patent to focus on the functionality of the invention and how it can be applied.

The use of abstraction keeps a patent from being tied down to the specifics of what it is being applied to because in most cases the patent is trying to communicate how it can be applied in many different ways.

In fact, the patent places the invention in the context of different forms of content entities such as videos, images, and audio clips. The patent also refers to text-based content (like articles, blog posts), as well as more tangible entities (like products, services, organizations, or even individuals).

Here is an example from the patent where it explicitly refers to video clips as one of the entities that the patent is concerned with:

“In one implementation, the above procedure is performed for each entity in a given set of entities (e.g., video clips in a video clip repository, etc.), and an inverse mapping from subjective attributes to entities in the set is generated based on the subjective attributes and relevancy scores.”

In this context, “video clips” are explicitly mentioned as an example of the entities to which the invention can be applied. The passage indicates that the procedure described in the patent (identifying and scoring subjective attributes of entities) is applicable to video clips.”

Here is another passage where the word entity is used to denote a type of content:

“Entity store 120 is a persistent storage that is capable of storing entities such as media clips (e.g., video clips, audio clips, clips containing both video and audio, images, etc.) and other types of content items (e.g., webpages, text-based documents, restaurant reviews, movie reviews, etc.), as well as data structures to tag, organize, and index the entities.”

That part of the patent describes “content items” as entities and gives examples like webpages, text-based documents, restaurant reviews, and movie reviews, alongside media clips such as video and audio clips. This and other similar passages show that the term “entity” within the context of this patent broadly encompasses multiple forms of digital content.

That patent,  titled Identifying subjective attributes by analysis of curation signals, is actually related to a recommender system or search that leverages User Generated Content like comments for the purpose of tagging digital content with the subjective opinions of those users.

The patent specifically uses the example of users describing an entity (like an image or a video) as funny, which can then be used to surface a video that has the subjective quality of funny as a part of a recommender system.

The most obvious application of this patent is for finding videos on YouTube that users and authors have described as funny. The use of this patent isn’t limited to just YouTube videos, it can also be used in other scenarios that intersect with user generated content.

The patent explicitly mentions the application of the invention in the context of a recommender system in the following passage:

“In one implementation, the above procedure is performed for each entity in a given set of entities (e.g., video clips in a video clip repository, etc.), and an inverse mapping from subjective attributes to entities in the set is generated based on the subjective attributes and relevancy scores.

The inverse mapping can then be used to efficiently identify all entities in the set that match a given subjective attribute (e.g., all entities that have been associated with the subjective attribute ‘funny’, etc.), thereby enabling rapid retrieval of relevant entities for processing keyword searches, populating playlists, delivering advertisements, generating training sets for the classifier, and so forth.”

Some SEOs, because the patent mentions authors three times have claimed that this patent has something to do with ranking content authors and because of that they also associate the patent it with E-A-T.

Others, because the patent mentions the words “entity” and “entities” so many times have come to believe it has something to do with natural language processing and semantic understanding of webpages.

But neither of those are true and now that I’ve explained some of this patent it should be apparent how a lack of understanding of how to read a patent plus approaching patents with the mindset of treasure hunting for spicy algorithm clues can lead to unfortunate and misleading errors in understanding what the patents are actually about.

In a future article I will walk through different patents and I think doing that will help readers understand how to read a patent. If that’s something you are interested in then please share this article on social media and let me know!

I’m going to end this article with a description of the different parts of a patent, which should go some way to building an understanding of patents.

Step #5 Know The Parts Of A Patent

Every patent is comprised of multiple parts, a beginning, a middle and an end that each have a specific purpose. Many patents are also accompanied by illustrations that are helpful for understanding what the patent is about.

Patents typically follow this pattern:

Abstract:
A concise summary of the patent, giving a quick overview of what the invention is and what it does. It’s provides a brief explanation. This part is actually important because it tells what the patent is about. Do not be one of those SEOs who skip this part to go treasure hunting in the middle parts for clues about the algorithm. Pay attention to the Abstract.

Background:
This section offers context for the invention. It typically gives an overview of the field related to the invention and in a direct or indirect way explains how the invention fits into the context. This is another important part of the patent. It doesn’t give up clues about the algorithm but it tells what part of the system it belongs to and what it’s trying to do.

Summary:
The Summary provides a more detailed overview of the invention than the Abstract. We often say you can step back and view the forest, can step closer and see the trees. The Summary can be said to be stepping forward to see the leaves and just like a tree has a lot of leaves, a Summary can contain a lot of details.

The Summary outlines the invention’s primary objectives, features, and the minutiae of how it does it and all the variations of how it does it. It is almost always an eye-wateringly comprehensive description.

The very first paragraph though can often be the most descriptive and understandable part, after which the summary deep-dives into fine detail. One can feel lost in the seemingly redundant descriptions of the invention. It can be boring but read it at least twice, more if you need to.

Don’t be dismayed if you can’t understand it all because this part isn’t about finding the spicy bits that make for good tweets. This part of reading a patent is sometimes more about kind of absorbing the ideas and getting a feel for it.

Brief Description Of The Drawings:
In patents where drawings are included, this section explains what each drawing represents, sometimes with just a single sentence. It can be as brief as this:

“FIG. 1 is a diagram that illustrates obtaining an authoritative search result.
FIG. 2 is a diagram that illustrates resources visited during an example viewing session.
FIG. 3 is a flow chart of an example process for adjusting search result scores.”

The descriptions provide valuable information and are just as important as the illustrations themselves. They both can communicate a sharper understanding of the function of the patent invention.

What may seem like an invention about choosing authoritative sites for search results might in the illustrations turn out to be about finding the right files on a mobile phone and not have anything to do with information retrieval.

This where my advice to let the patent tell you what it’s about pays off. People too often skip these parts because they don’t contain spicy details. What happens next is that they miss the context for the entire patent and reach completely mistaken conclusions.

Detailed Description Of The Patent:
This is an in-depth description of the invention that uses the illustrations (figure 1, figure 2, etc.) as the organizing factor. This section may include technical information, how the invention works, how it is organized in relation to other parts, and how it can be used.

This section is intended to be thorough enough that someone skilled in the field could replicate the invention but also general enough so that it can be broadly applied in different ways.

Embodiment Examples:
Here is where specific examples of the invention are provided. The word “embodiment” refers to a particular implementation or an example of the invention. It is a way for the inventor to describe specific ways the invention can be used.

There are different contexts of the word embodiment that make it clear what the inventor considers a part of the invention, it is used in the context of illustrating the real-world use of the invention, define technical aspects and to show different ways the invention can be made or used.

That last one you’ll see a lot of paragraphs describing “in another embodiment the invention can bla bla bla…”

So when you see that word “embodiment” try to think of the word “body” and then “embody” in the sense of making something tangible and that will help you to better understand the “Embodiment” section of a patent.

Claims:
The Claims are the legal part of the patent. This section defines the scope of protection that the patent is looking for and it also offers insights into what the patent is about because this section often talks about what’s new and different about the invention. So don’t skip this part.

Citations:
This part lists other patents that are relevant to the invention. It’s used to acknowledge similar inventions but also to show how this invention is different from them and how it improves on what came before.

Firm Starting Point For Reading Patents

You should by this point have a foundation for practicing how to read a patent. Don’t be discouraged if the patent seems opaque and hard to understand. That’s normal.

I asked Jeff Coyle (LinkedIn), cofounder of MarketMuse (LinkedIn) for tips about reading patents because he’s filed some patent applications.

Jeff  offered this advice:

“Use Google Patent’s optional ‘non-patent literature’ Google Scholar search to find articles that may reference or support your knowledge of a patent.

Also understand that sometimes understanding a patent in isolation is nearly impossible, which is why it’s important to build context by collecting and reviewing connected patent and non-patent citations, child/priority patents/applications.

Another way that helps me to understand patents is to research other patents filed by the same authors. These are my core methods for understanding patents.”

That last tip is super important because some inventors tend to invent one kind of thing. So if you’re in doubt about whether a patent is about a certain thing, take a look at other patents that the inventor has filed to see if they tend to file patents on what you think a patent is about.

Patents have their own kind of language, with a formal structure and purpose to each section. Anyone who has learned a second language knows how important it is to look up words and to understand the structure that’s inherent in what’s written.

So don’t be discouraged because with practice you will be able to read patents better than many in the SEO industry are currently able to.

I intend at some point to walk through several patents with the hope that this will help you improve on reading patents. And remember to let me know on social media if this is something you want me to write!

Ecommerce MGMT 0 Comments
Nov 29 2023
Google SGE & Generative Summaries For Search Results Patent via @sejournal, @kristileilani

Google patent US11769017B1, shared by X user @seostratega, focusing on “Generative Summaries for Search Results,” appears to lay the groundwork for Google’s SGE with the integration of generative AI search summaries.

This article aims to provide a quick analysis of the patent, how it relates to Google’s SGE, and its implications for SEO professionals.

What Is Patent US11769017B1?

The patent describes a method for creating summaries of search results using large language models (LLMs).

These models are designed to understand the context and content of web pages, generating concise and relevant summaries.

It reimagines the search results page, allowing for more complex queries and delivering AI-powered overviews with links to further information.

How Does It Relate To Google SGE?

Using LLMs, Google SGE can generate AI-powered snapshots for queries, offering users an immediate understanding of a topic and avenues to delve deeper.

These snapshots are not isolated pieces of information but are corroborated by links to high-quality web sources, ensuring reliability and breadth in the information provided.

The technology underpinning US11769017B1 is integral to SGE, facilitating these concise overviews and ensuring reliable web sources back them.

Google SGE & Generative Summaries For Search Results PatentScreenshot from Google Patents, November 2023

Key Takeaways For SEO

Integrating generative AI into search suggests a shift towards more nuanced and contextually rich content.

SEO strategies must adapt to prioritize content that aligns with these AI-driven summaries.

Start by focusing on creating comprehensive content. Ensure your content thoroughly covers topics and addresses users’ top questions. This holistic approach increases the likelihood of being featured in AI-generated summaries.

Write in a clear, concise manner while providing context. Content that is easily digestible and contextually rich is more likely to be favored by LLMs for summarization.

Optimize content for conversational queries and voice searches beyond keywords and phrases.

With SGE’s emphasis on reliable sources and the latest updates to E-E-A-T, building the authority and trustworthiness of your content is vital. This includes citing reputable sources and maintaining factual accuracy.

Stay current on emerging trends in generative AI and search, testing the latest updates to SGE in Labs and how it could influence search behavior. Adapting to these changes promptly can give you a competitive edge.

Creating multiple content formats, including text, audio, videos, and images, may increase visibility across different generative search experiences.

Most importantly, tailor your content to align with the potential intent behind search queries, as SGE is likely to prioritize content that best matches user intent.

Learn More About Google SGE With AI

Want to learn more about the patent’s technical details and how they relate to the inner workings of Google SGE?

Poe users can try the SGEPatentBot for free.

poe explains google sge patentScreenshot from Poe, November 2023

ChatGPT Plus subscribers can try the SGEPatentReader, a custom GPT.

gpt explains google sge patentScreenshot from ChatGPT, November 2023

Conclusion

Google’s patent US11769017B1 and continued experimentation with SGE mark a shift towards more AI-driven, contextually aware search experiences.

For SEO professionals, adapting to these changes is crucial. By focusing on comprehensive, clear, and authoritative content and optimizing for conversational queries and user intent, SEO strategies can align with the evolving landscape of Google search, potentially leading to greater visibility and engagement in the SGE era.

Featured image: sdx15/Shutterstock

Ecommerce MGMT 0 Comments