Google has added two new crawlers to the Googlebot family of web crawlers:
GoogleOther-Image GoogleOther-Video
Google said these two new trackers are versions of GoogleOther optimized for image and video bytes. GoogleOther was added in April 2023 to be used internally by Google teams to crawl the public web and free up some resources for the main Googlebot crawlers.
GoogleOther-Image. GoogleOther-Image, according to the documentation, is the version of GoogleOther optimized for publicly accessible image URLs. It will pass under the user agent tokens of GoogleOther-Image and GoogleOther and the full user agent string will be GoogleOther-Image/1.0.
GoogleOther-Video. GoogleOther-Video, according to the documentation, is GoogleOther’s version optimized for publicly accessible video URLs. It will go under the GoogleOther-Video and GoogleOther user-agent tokens and the full user-agent string will be GoogleOther-Video/1.0.
Why these new trackers. Google said“the new trackers were released to better support the tracking of binary data that can be used for research and development.”
Learn more about Google trackers. Types of Googlebot crawlers include:
Googlebot – The main tracker for Google search products. Google says this crawler always respects robots.txt rules.
Special Case Crawlers – Crawlers that perform specific functions (such as AdsBot), which may or may not respect robots.txt rules.
Sensors activated by the user: Product tools and features where the end user triggers a recovery. For example, Google Site Verifier acts on a user’s request, or some Google Search Console tools will send Google to search the page based on an action a user takes.
Google has also listed the IP address ranges and reverse DNS mask for each type:
Why we care Many of you check your crawling activities and bot activity on your website and log files. When you see this new GoogleOther tracker, don’t be alarmed. It’s a real Googlebot.
Add Search Engine Land to your Google News feed.
New in Search Engine Land
About the author
Barry Schwartz is a contributing editor at Search Engine Land and a member of the SMX event programming team. He owns RustyBrick, a web consulting firm based in New York. He also runs Round table of search engines, a popular search blog about very advanced SEM topics. Barry can be followed on Twitter here.
[ad_2]
Source link
