We create digital solutions that work for businesses
SEO and marketing
You may also like
Why a page is not getting into Google.
We check everything that can keep a page out of search: robots.txt rules for Googlebot, the robots meta tag, the X-Robots-Tag header, the status code, the redirect chain and the canonical.
Seven causes, most common first. The top three account for the majority of cases.
They get confused constantly, and the difference matters.
The crawler visits the page, reads it and sees the instruction to keep it out of search. Links on the page still count.
The crawler never opens the page. But if links point to it, the URL can still surface in results — with no description.
The classic trap is using both at once. Disallow keeps the crawler out, so it never sees the noindex — and the URL stays in results. To remove a page from search you need noindex and open access to it.