How does Googlebot behave? Find out

This is a series of tests to analyse how Googlebot behaves.

The test pages log Googlebot activity in the html so I can analyse real Googlebot activity from the Google Search Console. It also logs server side activity which can be viewed in a looker studio report.

You can see a visible version of this log at the end of this page.

Unfortunately, this site has not been promoted, so it gets few crawls from Google. I will open it up, to hopefully get more love from Google.

• Image Tests - Which responsive images are loaded?
• Image Tests 2 - Images in Structured Data?
• Image Tests 3 - Images in a link?
• No Index - Does Googlebot render noindex pages?
• To No Index - Does Googlebot index JS based noindex pages?
• To No Index, Smartphone only - Does Googlebot index the desktop version?
• robots.txt - Does Googlebot honour it?
• Canonical tags - What if a page is canonicalised out?
• Missing Page - Does Googlebot render 404 pages?
• GoogleBot Navigator - How does Googlebot identify itself?
• GoogleBot Random - What do random numbers look like?
• GoogleBot JS Timeout - How long does Googlebot render for?
• GoogleBot Date.Now() - How long does it take to run code or timeouts?
• GoogleBot Date.Now() Excessive - Just to prove a point, and test timeouts.
• GoogleBot 5 Second Timeout - How long does it take in real time?
• GoogleBot 5 Second Request - How long does it take in Google time?
• GoogleBot Resources - How long do they take in Google time?
• GoogleBot Resources Chained - How long do they take in Google time?
• GoogleBot Resources Chained for 1 minute - Will Googlebot go on forever?
• GoogleBot Viewport - How tall can it be?
• GoogleBot Infinite Scroll - Does Googlebot see the dynamically added content?
• GoogleBot Caching - How much and how long?
• Large Page - Does Googlebot have a limit?
• GoogleBot Mixed - Randomised request loading with different types of delays.
• 0 Second Timeouts - Yielding.
• 301 Redirects - How long?
• 302 Redirects - Any different?
• JavaScript Redirects - Are they followed?
• noindex JavaScript Redirects - Does the noindex matter?
• Pinging when unloading the page - Is it reliable?
• Many redirects - How many will Googlebot follow?

Can you think of any other Googlebot tests?

I'm working on an article to go deeper into what I have discovered about Googlebot. It's become a massive task starting in 2025. Hopefully, I will complete it this year.

Log