Did you know that sometimes these bot protection services will serve the noindex guidelines of Google and other search engines? Google’s John Mueller had this to say Xsaying “Sometimes there’s a bot protection (or login, interstitial, etc.) running that doesn’t have an index.”
He also added that it would be better if they provided 503 server status codes. “It would be better to use 503 for server-side blocks like bot protection,” he wrote.
In the end the person he replied to had a different problem causing the noindex, it wasn’t bot protection. He said it was “some shenanigans with NextJS doing client-side rendering instead of server-side rendering.” “The HTML had a noindex tag, which was removed after rendering in the browser. The HTML displayed in GSC is AFTER the JavaScript rendering, so I couldn’t find it and thought it was a bug of GSC,” he added.
This was the error I was seeing:
Here are those posts:
wtf is going on here? why isn’t google indexing this one?
They say there is a “noindex” tag, but there isn’t one. robots.txt looks good too.
No one? #seo help pic.twitter.com/DYzJsLQ4Yk
— Guilherme (@goenning) December 26, 2023
Thanks John, I finally found it thanks to @HamilcDev
Some cheats with NextJS doing client-side rendering instead of server-side rendering
The HTML had a noindex tag, which was removed after rendering to the browser. The HTML displayed in GSC is AFTER rendering the JavaScript,… https://t.co/NffU7suVLM
— Guilherme (@goenning) December 26, 2023
Discussion in the forum a X.
[ad_2]
Source link
