Deindexing is the removal of a page, a section or a whole website from a search engine’s index, so it can no longer appear in results. It can be deliberate, when you tell Google to drop a page, or something that happens to you through a technical error, a quality decision or a penalty.
How deindexing works
Google keeps an index of the pages it has crawled and judged worth storing. A page leaves that index for one of a handful of reasons:
- A noindex directive. When Googlebot next crawls the page and finds a noindex meta tag or HTTP header, it drops the page.
- The page has gone. A 404 or 410 status tells Google the page no longer exists, and it is removed once repeated crawls confirm it.
- Consolidation. If Google decides another URL is the canonical version, it keeps that one and leaves the duplicate out.
- Quality. Google may index a page and later drop it if it judges the content not useful enough. Search Console labels many of these “Crawled – currently not indexed”.
- A manual action. For serious spam, a reviewer at Google can remove a whole site. This is rare and is always reported in Search Console.
- A removal request. The Removals tool in Search Console hides a URL quickly but only for about six months; legal and personal-information requests go through separate Google forms.
One widespread misunderstanding: blocking a page in robots.txt does not remove it. Google cannot crawl a blocked page, so it never sees a noindex tag on it, and the URL can stay indexed with no description.
Why it matters
Accidental deindexing is one of the fastest ways to lose search traffic. The cause I see most often is a launch where a staging site’s noindex setting, or WordPress’s “Discourage search engines” option, travels to the live site. Nothing looks broken, so nobody notices until enquiries dry up weeks later.
Deliberate deindexing is just as useful. Removing thin tag archives, internal search results, expired offers and old campaign pages keeps Google’s attention on the pages that earn money and reduces index bloat. For UK businesses there is a legal side too: individuals can ask Google to delist results about them under data protection law, and Google weighs those requests against the public interest.
Common mistakes
- Using robots.txt to remove pages. Use noindex or a 404 or 410, and let Google crawl the page to see it.
- Blocking and noindexing together. A Disallow rule stops Google from ever reading the noindex.
- Treating the Removals tool as permanent. When the request expires, the page can return unless it has also been removed or noindexed.
- Panicking over a site: search. The site: operator gives a rough sample, not a count. Use the indexing report instead.
- Noindexing pages that still do a job. Check what a page does for visitors and internal links before dropping it from search.
How to act on it
If pages have disappeared, open the Page indexing report in Google Search Console and read the reasons listed for excluded URLs. Inspect a few affected URLs with the URL Inspection tool, which shows whether Google found a noindex, a redirect, a different canonical or a crawl block. Check the Manual actions and Security issues pages in the same account, since a hacked site can be flagged too.
If you want pages removed, pick the right mechanism for each group: noindex for pages visitors still need, a 410 for pages that should not exist, and a 301 redirect where there is a clear replacement. Keep those URLs crawlable until Google has processed the change. Any launch checklist should include confirming that indexing is allowed on the live site. Diagnosing problems like these is a core part of technical SEO work.