Google Cautions On Blocking GoogleOther Bot via @sejournal, @martinibuster

Google’s Gary Illyes answered a question about the non-search features that the GoogleOther crawler supports, then added a caution about the consequences of blocking GoogleOther.

What Is GoogleOther?

GoogleOther is a generic crawler created by Google for the various purposes that fall outside of those of bots that specialize for Search, Ads, Video, Images, News, Desktop and Mobile. It can be used by internal teams at Google for research and development in relation to various products.

The official description of GoogleOther is:

“GoogleOther is the generic crawler that may be used by various product teams for fetching publicly accessible content from sites. For example, it may be used for one-off crawls for internal research and development.”

Something that may be surprising is that there are actually three kinds of GoogleOther crawlers.

Three Kinds Of GoogleOther Crawlers

1. GoogleOther
   Generic crawler for public URLs
2. GoogleOther-Image
   Optimized to crawl public image URLs
3. GoogleOther-Video
   Optimized to crawl public video URLs

All three GoogleOther crawlers can be used for research and development purposes. That’s just one purpose that Google publicly acknowledges that all three versions of GoogleOther could be used for.

What Non-Search Features Does GoogleOther Support?

Google doesn’t say what specific non-search features GoogleOther supports, probably because it doesn’t really “support” a specific feature. It exists for research and development crawling which could be in support of a new product or an improvement in a current product, it’s a highly open and generic purpose.

This is the question asked that Gary narrated:

“What non-search features does GoogleOther crawling support?”

Gary Illyes answered:

“This is a very topical question, and I think it is a very good question. Besides what’s in the public I don’t have more to share.

GoogleOther is the generic crawler that may be used by various product teams for fetching publicly accessible content from sites. For example, it may be used for one-off crawls for internal research and development.

Historically Googlebot was used for this, but that kind of makes things murky and less transparent, so we launched GoogleOther so you have better controls over what your site is crawled for.

That said GoogleOther is not tied to a single product, so opting out of GoogleOther crawling might affect a wide range of things across the Google universe; alas, not Search, search is only Googlebot.”

It Might Affect A Wide Range Of Things

Gary is clear that blocking GoogleOther wouldn’t have an affect on Google Search because Googlebot is the crawler used for indexing content. So if blocking any of the three versions of GoogleOther is something a site owner wants to do, then it should be okay to do that without a negative effect on search rankings.

But Gary also cautioned about the outcome that blocking GoogleOther, saying that it would have an effect on other products and services across Google. He didn’t state which other products it could affect nor did he elaborate on the pros or cons of blocking GoogleOther.

Pros And Cons Of Blocking GoogleOther

Whether or not to block GoogleOther doesn’t necessarily have a straightforward answer. There are several considerations to whether doing that makes sense.

Pros

Inclusion in research for a future Google product that’s related to search (maps, shopping, images, a new feature in search) could be useful. It might be helpful to have a site included in that kind of research because it might be used for testing something good for a site and be one of the few sites chosen to test a feature that could increase earnings for a site.

Another consideration is that blocking GoogleOther to save on server resources is not necessarily a valid reason because GoogleOther doesn’t seem to crawl so often that it makes a noticeable impact.

If blocking Google from using site content for AI is a concern then blocking GoogleOther will have no impact on that at all. GoogleOther has nothing to do with crawling for Google Gemini apps or Vertex AI, including any future products that will be used for training associated language models. The bot for that specific use case is Google-Extended.

Cons

On the other hand it might not be helpful to allow GoogleOther if it’s being used to test something related to fighting spam and there’s something the site has to hide.

It’s possible that a site owner might not want to participate if GoogleOther comes crawling for market research or for training machine learning models (for internal purposes) that are unrelated to public-facing products like Gemini and Vertex.

Allowing GoogleOther to crawl a site for unknown purposes is like giving Google a blank check to use your site data in any way they see fit outside of training public-facing LLMs or purposes related to named bots like GoogleBot.

Takeaway

Should you block GoogleOther? It’s a coin toss. There are possible potential benefits but in general there isn’t enough information to make an informed decision.

Listen to the Google SEO Office Hours podcast at the 1:30 minute mark:

Featured Image by Shutterstock/Cast Of Thousands

Study Backs Google’s Claims: AI Search Boosts User Satisfaction via @sejournal, @MattGSouthern

A new study finds that despite concerns about AI in online services, users are more satisfied with search engines and social media platforms than before.

The American Customer Satisfaction Index (ACSI) conducted its annual survey of search and social media users, finding that satisfaction has either held steady or improved.

This comes at a time when major tech companies are heavily investing in AI to enhance their services.

Search Engine Satisfaction Holds Strong

Google, Bing, and other search engines have rapidly integrated AI features into their platforms over the past year. While critics have raised concerns about potential negative impacts, the ACSI study suggests users are responding positively.

Google maintains its position as the most satisfying search engine with an ACSI score of 81, up 1% from last year. Users particularly appreciate its AI-powered features.

Interestingly, Bing and Yahoo! have seen notable improvements in user satisfaction, notching 3% gains to reach scores of 77 and 76, respectively. These are their highest ACSI scores in over a decade, likely due to their AI enhancements launched in 2023.

The study hints at the potential of new AI-enabled search functionality to drive further improvements in the customer experience. Bing has seen its market share improve by small but notable margins, rising from 6.35% in the first quarter of 2023 to 7.87% in Q1 2024.

Customer Experience Improvements

The ACSI study shows improvements across nearly all benchmarks of the customer experience for search engines. Notable areas of improvement include:

• Ease of navigation
• Ease of using the site on different devices
• Loading speed performance and reliability
• Variety of services and information
• Freshness of content

These improvements suggest that AI enhancements positively impact various aspects of the search experience.

Social Media Sees Modest Gains

For the third year in a row, user satisfaction with social media platforms is on the rise, increasing 1% to an ACSI score of 74.

TikTok has emerged as the new industry leader among major sites, edging past YouTube with a score of 78. This underscores the platform’s effective use of AI-driven content recommendations.

Meta’s Facebook and Instagram have also seen significant improvements in user satisfaction, showing 3-point gains. While Facebook remains near the bottom of the industry at 69, Instagram’s score of 76 puts it within striking distance of the leaders.

Challenges Remain

Despite improvements, the study highlights ongoing privacy and advertising challenges for search engines and social media platforms. Privacy ratings for search engines remain relatively low but steady at 79, while social media platforms score even lower at 73.

Advertising experiences emerge as a key differentiator between higher- and lower-satisfaction brands, particularly in social media. New ACSI benchmarks reveal user concerns about advertising content’s trustworthiness and personal relevance.

Why This Matters For SEO Professionals

This study provides an independent perspective on how users are responding to the AI push in online services. For SEO professionals, these findings suggest that:

1. AI-enhanced search features resonate with users, potentially changing search behavior and expectations.
2. The improving satisfaction with alternative search engines like Bing may lead to a more diverse search landscape.
3. The continued importance of factors like content freshness and site performance in user satisfaction aligns with long-standing SEO best practices.

As AI becomes more integrated into our online experiences, SEO strategies may need to adapt to changing user preferences.

Featured Image: kate3155/Shutterstock

OpenAI Launches SearchGPT: AI-Powered Search Prototype via @sejournal, @MattGSouthern

OpenAI has announced the launch of SearchGPT, a prototype AI-powered search engine.

This move marks the company’s entry into the competitive search market, potentially challenging established players.

Key Features & Functionality

SearchGPT aims to directly answer user queries by combining AI language models with real-time web information.

Rather than offering a list of links, SearchGPT attempts to deliver concise responses with citations to source material.

Here’s an example of a search results page for the query: “music festivals in boone north carolina in august.”

Screenshot from openai.com/index/searchgpt-prototype/, July 2024.

The SearchGPT prototype includes:

• A conversational interface allowing follow-up questions
• Real-time information retrieval from web sources
• In-line attributions and links to original content

Publisher Controls & Content Management

OpenAI is also introducing tools for publishers to manage how their content appears in SearchGPT, giving them more control over their presence in AI-powered search results.

Key points about the publisher controls include:

1. Separate from AI training: OpenAI emphasizes that SearchGPT is distinct from the training of their generative AI models. Sites can appear in search results even if they opt out of AI training data.
2. Content management options: Publishers can influence how their content is displayed and used within SearchGPT.
3. Feedback mechanism: OpenAI has provided an email (publishers-feedback@openai.com) for publishers to share their thoughts and concerns.
4. Performance insights: The company plans to share information with publishers about their content’s performance within the AI search ecosystem.

These tools are OpenAI’s response to ongoing debates about AI’s use of web content and concerns over intellectual property rights.

Publisher Partnerships & Reactions

OpenAI reports collaborating with several publishers during the development of SearchGPT.

Nicholas Thompson, CEO of The Atlantic, provided a statement supporting the initiative, emphasizing the importance of valuing and protecting journalism in AI search development.

Robert Thomson, News Corp’s chief executive, also commented on the project, stressing the need for a symbiotic relationship between technology and content and the importance of protecting content provenance.

Limited Availability & Future Plans

Currently, SearchGPT is available to a restricted group of users and publishers.

OpenAI describes it as a temporary prototype, indicating plans to integrate features into their existing ChatGPT product eventually.

Why This Matters

The introduction of SearchGPT represents a potential shakeup to the search engine market.

This development could have far-reaching implications for digital marketing, content creation, and user behavior on the internet.

Potential effects include:

• Changes in content distribution and discovery mechanisms
• New considerations for search engine optimization strategies
• Evolving relationships between AI companies and content creators

Remember, this is still a prototype, and we have yet to see its capabilities.

There’s a waitlist available for those trying to get their hands on it early.

What This Means For You

AI-powered search might offer users more direct access to information. However, the accuracy and comprehensiveness of results may depend on publisher participation and content management choices.

For content creators and publishers, these new tools provide opportunities to have more say in how their work is used in AI search contexts.

While it may increase content visibility and engagement, it also requires adapting to new formats and strategies to ensure content is AI-friendly and easily discoverable.

As SearchGPT moves from prototype to integration with ChatGPT, it will be vital to stay informed about these developments and adapt your strategies.

The future of search is evolving, and AI is at the forefront of this transformation.

Bing’s Updated AI Search Will Make Site Owners Happy via @sejournal, @martinibuster

Bing is rolling out a new version of Generative Search that displays information in an intuitive way that encourages exploration but also prioritizes clicks from the search results to websites.

Microsoft introduced their new version of AI search:

“After introducing LLM-powered chat answers on Bing in February of last year, we’ve been hard at work on the ongoing revolution of search. …Today, we’re excited to share an early view of our new generative search experience which is currently shipping to a small percentage of user queries.”

New Layout

Bing’s announcement discusses new features that not only make it easy for users to find information, Bing also makes it easy for users to see the organic search results and click through and browse websites.

On the desktop view, Bing shows three panels:

• A table of content on the left
• AI answers in the center (with links to website sources)
• Traditional organic search results on the right hand side
• Even more organic search results beneath “the fold”

The table of contents that is on the right hand side is invites exploration. It has the main topic at the top, with directly related subtopics beneath it. This is so much better than a People Also Asked type of navigation because it invites the user to explore and click on an organic search result to keep on exploring.

Screenshot: Table Of Contents

This layout is the result of a conscious decision at Bing to engineer it so that that it preserves and encourages clicks to websites.

Below is a screenshot of the new generative AI search experience. What’s notable is how Bing surrounds the AI answers with organic search results.

Screenshot Of The New Bing AI Search Results

Bing makes a point to explain that they have tested the new interface to make sure that the search results will send the same amount of traffic and to avoid creating a layout that results in an increase in zero click search results.

When other search engines talk about search quality it is always from the context of user satisfaction. Bing’s announcement makes it clear that sustaining traffic to websites was an important context that guided the design of the new layout.

Below is a screenshot of a typical Bing AI search result for a query about the life span of elephants.

Note that all the areas that I bounded with blue boxes are AI answers while everything outside of the blue boxes are organic search results.

Screenshot Of Mix of AI And Organic Results

The screenshot makes it clear that there is a balance of organic search results and AI answers. In addition to those contextually relevant organic search results there are also search results on the right hand side (not shown in the above screenshot).

Microsoft’s blog post explained:

“We are continuing to look closely at how generative search impacts traffic to publishers. Early data indicates that this experience maintains the number of clicks to websites and supports a healthy web ecosystem. The generative search experience is designed with this in mind, including retaining traditional search results and increasing the number of clickable links, like the references in the results.”

Bing’s layout is a huge departure from the zero-click style of layouts seen in other search engines. Bing has purposely designed their generative AI layout to maintain clicks to websites. It cannot be overstated how ethical Bing’s approach to the web ecosystem is.

Bing Encourages Browsing And Discovery

An interesting feature of Bing’s implementation of generative AI search is that it shows the answer to the initial question first, and it also anticipates related questions. This is similar to a technique called “information gain” where an AI search assistant will rank an initial set of pages that answers a search query, but will also rank a second, third and fourth set of search results that contain additional information that a user may be interested in, information on related topics.

What Bing does differently from the Information Gain technique is that Bing displays all the different search results on a single page and then uses a table of contents on the left hand side that makes it easy for a user to click and go straight to the additional AI answers and organic search results.

Bing’s Updated AI Search Is Rolling Out Now

Bing’s newly updated AI search engine layout is slowly rolling out and they are observing the feedback from users. Microsoft has already tested it and is confident that it will continue to send clicks to websites. Search engines have a relationship with websites, what is commonly referred to as the web ecosystem. Every strong relationship is based on giving, not taking. When both sides give it creates a situation where both sides receive.

More search engines should take Bing’s approach of engineering their search results to satisfy users in a way that encourages discovery on the websites that originate the content.

Read Bing’s announcement:

Introducing Bing generative search

Featured Image by Shutterstock/Primakov

Google Search Revenue Grows 14% In Q2 2024 via @sejournal, @MattGSouthern

Alphabet Inc., Google’s parent company, released its second quarter 2024 financial results, revealing a 14% year-over-year increase in revenue for its core Google Search business.

Key Financial Data:

• Google Search revenue: $48.5 billion (up from $42.6 billion in Q2 2023)
• Total Alphabet revenue: $84.7 billion (14% increase year-over-year)
• Operating income: $27.4 billion
• Net income: $23.6 billion
• Earnings per share: $1.89

Strong performances in Search and Cloud services primarily drove the company’s overall revenue growth.

Google Cloud surpassed $10 billion in quarterly revenue for the first time, reaching $10.3 billion with $1.2 billion in operating profit.

YouTube ad revenue increased from $7.7 billion in Q2 2023 to $8.7 billion in Q2 2024.

Alphabet CEO Sundar Pichai commented on the results, emphasizing the company’s focus on AI innovation. The report also noted a recent reorganization of AI teams, combining elements of Google Research with Google DeepMind.

Pichai stated:

“Our strong performance this quarter highlights ongoing strength in Search and momentum in Cloud. We are innovating at every layer of the AI stack.”

The report also noted a recent reorganization of AI teams, combining elements of Google Research with Google DeepMind.

While the results indicate strong performance, Alphabet faces challenges, including regulatory scrutiny and evolving competition in the tech sector.

The company’s CFO, Ruth Porat, mentioned ongoing efforts to optimize cost structures.

Regarding the company’s financial strategy, Porat stated:

“As we invest to support our highest growth opportunities, we remain committed to creating investment capacity with our ongoing work to durably re-engineer our cost base.”

Why This Matters

The performance of Google Search and Alphabet has implications for the digital marketing industry.

As the dominant search engine, Google’s revenue growth indicates continued strength in search advertising, which remains an essential channel for many businesses.

Additionally, the growth in Cloud services and YouTube advertising suggests evolving digital trends and potential opportunities for marketers.

What Does This Mean For You?

For digital marketers and SEO professionals, these are the key takeaways from Alphabet’s earnings call:

• Search remains vital: The growth in Google Search revenue shows that SEO and search advertising remain key components of marketing strategies.
• Cloud and AI focus: Alphabet’s emphasis on Cloud services and AI development may lead to new tools and platforms for marketers to leverage.
• Video advertising potential: The growth in YouTube ad revenue indicates the ongoing importance of video content in digital marketing strategies.
• Competitive landscape: While Google maintains its market position, the focus on AI development across the tech industry may lead to new challenges and opportunities in search and digital advertising.
• Potential changes ahead: As Alphabet continues to invest in AI and reorganize its teams, marketers should stay alert for potential changes in search algorithms or new AI-driven features that could impact SEO and PPC strategies

Featured Image: sdx15/Shutterstock

Google Shares Tips To Improve SEO Through Internal Links via @sejournal, @MattGSouthern

In a new installment of its “SEO Made Easy” video series, Google provides three simple guidelines for utilizing internal linking to improve SEO.

The video, presented by Google’s Martin Splitt, offers valuable insights for improving site structure and user experience.

Strategic internal linking highlights your most valuable content, ensuring users and search engines can identify them quickly.

Additionally, internal linking can help search engines understand the relationships between pages, potentially leading to better rankings.

3 Tips For Internal Linking

Splitt emphasized three main points regarding the effective use of internal links:

1. User Navigation: Internal links guide users through a website, helping them find related content and understand the site’s structure.
2. Search Engine Crawling: Google’s web crawler, Googlebot, uses internal links to discover new pages and understand the relationships between different pages on a site.
3. HTML Best Practices: Properly using HTML elements, particularly the < a> tag with an href attribute, is essential for creating effective links.

The Importance Of Meaningful Anchor Text

One of Google’s key recommendations is to use descriptive, meaningful anchor text for links.

Splitt demonstrated how clear anchor text improves user experience by allowing visitors to quickly scan a page and understand where each link will lead them.

He stated:

“Users and Bots alike prefer meaningful anchor text. Here on the left you see what that looks like each link has meaningful words as anchor text and you can easily spot what the link will take you to.”

See the examples he’s referring to in the image below:

Screenshot from: YouTube.com/GoogleSearchCentral, July 2024.

Splitt continues:

“On the right you see a page that doesn’t use meaningful anchor text and that isn’t a good user experience especially when you try to quickly scan the page and find the right link to use.”

Balancing Link Quantity

While internal linking is vital, Splitt cautioned against overdoing it.

He advises applying critical judgment when adding links and creating logical connections between related content without overwhelming the user or diluting the page’s focus.

Technical Considerations For Links

The video also touched on the technical aspects of link implementation.

Splitt discouraged using non-standard elements like spans, divs, or buttons to create links, saying if an element behaves like a link, it should be coded as one using the proper HTML structure.

Screenshot from: YouTube.com/GoogleSearchCentral, July 2024.

In Summary

These are the key takeaways from Google’s video on internal linking:

• Internal linking is a fundamental aspect of SEO and user experience.
• Focus on creating meaningful, descriptive anchor text for links.
• Use internal links strategically to guide users and search engines through your site.
• Balance the number of links to avoid overwhelming users or diluting page focus.
• Stick to proper HTML structure when implementing links.

See the full video below:

Featured Image: Screenshot from YouTube.com/GoogleSearchCentral, July 2024.