# Noindex Tag: When to Use It, How to Add It and Common Mistakes
> How the noindex tag works, when to use it instead of robots.txt or canonicals, how to add it by meta tag or X-Robots-Tag header and the mistakes to avoid.
URL: https://seoconsultantlondon.online/guides/noindex-tag-when-to-use-it/
Updated: 2026-10-24
## Quick answer
The noindex directive tells Google to keep a page out of search results. Add it with a robots meta tag or an X-Robots-Tag header, and let Googlebot crawl the page so it can see the rule. Never combine noindex with a robots.txt block, and never leave it on pages you want ranking.
## Key takeaways
- Noindex is a page-level instruction that removes a URL from Google's results once Googlebot recrawls the page and sees the rule.
- It only works if Google can crawl the page, so a robots.txt block cancels it out.
- Google does not support noindex inside robots.txt, so the directive belongs in the page head or an HTTP header.
- Removal is not instant: Google has to recrawl, and for unimportant pages that can take months.
- The costliest noindex mistakes are accidental ones, such as a staging setting that survives launch.
## What does the noindex tag actually do?
The noindex tag does one thing, and it does it bluntly: it tells a search engine not to include a page in its results. That's it. The page can still load for anyone who has the link, still get visits from email and social, still be crawled. It just won't show up when someone types a query into Google.
People tend to reach for it when they want something hidden, and that's fine, as long as hiding is really what they want. The trouble starts when a page that should rank carries the tag by accident, or when someone assumes noindex works in situations where it can't.
> Why it matters: noindex is one of the few SEO settings that can make a page vanish from Google entirely. Used well it keeps your index tidy. Used carelessly it deletes your best pages.
### How does Google treat it?
Google's documentation on [blocking Search indexing with noindex](https://developers.google.com/search/docs/crawling-indexing/block-indexing), last updated on 10 December 2025, is plain about the effect. When Googlebot crawls a page and sees the noindex rule, Google drops that page from Search results entirely, regardless of whether other sites link to it.
Two details in that sentence deserve underlining. First, Googlebot has to crawl the page. Second, it has to see the rule. Everything else in this guide follows from those two conditions.
#### The two forms of the directive
You can express noindex as a meta tag inside the page's head, or as an X-Robots-Tag in the HTTP response header. Both do the same job. The header form is the one that works for files that have no HTML head, such as PDFs, images and videos.
### Is noindex a ranking factor?
Not in the way people sometimes imagine. Noindex doesn't push a page down. It removes it. If you want a page to rank lower, noindex is the wrong tool. If you want it gone from results while it stays live for people, it's the right one.
It also doesn't make a page secret. Anyone with the address can open it, and other search engines may treat the directive differently, so a page might still appear elsewhere. Genuine privacy needs a login, not a tag.
## How do you add a noindex tag?
There are two standard methods, and a third route through your CMS settings. Pick the one that matches the kind of file and the access you have.
None of them is complicated. The risk is in where you apply them.
### How do you use the robots meta tag?
Place a meta tag in the head section of the page. The general form, ``, speaks to every search engine that supports it. If you want to address Google alone, `` does that. You can combine noindex with other directives, such as nofollow, in the same tag.
Most business sites never touch raw HTML for this. WordPress, Wix, Blogger and similar platforms usually expose a search engine visibility setting per page or per page type, and SEO plugins add a simple toggle. Use those controls and check the rendered result, rather than editing templates by hand.
#### What to check in the source
Open the page, view the source and search for "robots". Confirm exactly one robots meta tag and that it says what you intended. Duplicate or conflicting tags are a common cause of surprises.
### How do you use the X-Robots-Tag header?
The server sends `X-Robots-Tag: noindex` as part of the HTTP response. It's the right choice for non-HTML files, because a PDF has no head to put a tag in, and it's handy for applying a rule to a whole folder or file type at server level.
It needs access to server configuration, which many small business owners don't have, so it's often a job for a developer or hosting provider. Because it's invisible in the page source, it's also easy to forget it exists. Document every header rule you add.
#### Finding out whether a header is set
Use the URL Inspection tool in Search Console, or a browser's developer tools, to read the response headers for a URL. A header-level noindex can sit unnoticed for years.
## Why must Google be able to crawl the page?
Here's the rule that catches people out more than any other. Noindex is an instruction written on the page. If Googlebot can't open the page, it can't read the instruction.
Google's documentation says so directly: if robots.txt blocks the page, Googlebot never sees the noindex rule, and the URL can still appear in results if other pages link to it.
### What happens when you block and noindex together?
You get a page that Google knows about but can't read. It may keep showing the URL, often as a bare address with no description, because the only things Google can see are the links pointing to it. That's the opposite of what you wanted.
The fix is counterintuitive but reliable. Remove the robots.txt block so Googlebot can fetch the page, leave the noindex in place, wait for Google to recrawl and drop the page, and only then, if you still want to, block crawling again. The full story of how that status looks in reports is covered in [pages blocked by robots.txt](/guides/blocked-by-robots-txt-search-console/).
#### The order of operations
Noindex first. Wait. Block second, and only if you need to. Reversing that order leaves stale entries in the index.
### Does Google support noindex in robots.txt?
No. Google's documentation states that it doesn't support a noindex rule in the robots.txt file. Old advice from years ago sometimes mentions it, and some third-party tools still suggest it. Ignore them. Put the directive on the page or in the header.
This is also a good moment to remember what robots.txt is for. It manages crawler traffic. It doesn't decide what appears in search results. Treating it as an indexing switch is a source of constant confusion.
## When should you use noindex?
Use it for pages that need to exist for people but have no business showing up in search. That's a short, specific list.
Everything on it shares one property: a visitor who lands there from Google would be confused or underwhelmed.
### Which pages are good noindex candidates?
Internal search result pages, which generate near-endless low-value combinations. Thank-you and confirmation pages that only make sense after an action. Login, account and cart or checkout pages. Printer-friendly or filtered versions of content you already have. Thin tag or archive pages that add nothing beyond a list of links. Staging and test environments, though a password is the real protection there. Gated or members-only previews where a search listing would be misleading.
Each of those can stay in your site's navigation or workflow, serve a real purpose and still be kept out of the index. If you'd like to see how these pages show up in reports, the statuses explained under [soft 404 errors](/guides/soft-404-errors-seo/) often point to the same pages.
#### A quick test before you apply it
Would I be happy for this URL to appear as a search result? If the honest answer is no, and the page must stay live, noindex is a sensible choice.
### Which pages should never carry it?
Your service pages, product and category pages, key articles, location pages with real content and your homepage. Any page you hope to earn enquiries or links from. If you find noindex on one of those, treat it as a bug until proven otherwise.
Also be wary of applying it to a whole section "to be safe." A broad rule is easy to write and hard to audit, and it can quietly remove dozens of valuable URLs. Apply it narrowly and verify each pattern.
## What's the difference between noindex, canonical, robots.txt and redirects?
These four tools get lumped together as "SEO directives," and mixing them up is behind a surprising share of indexing problems. Each answers a different question.
Pick the tool that matches the goal, not the one you remember first.
### How do the four compare?
| Tool | Question it answers | Effect on search results | Typical use |
|---|---|---|---|
| Noindex | Should this page appear in results? | Removes it | Internal search, thank-you pages |
| Canonical | Which version is preferred? | Consolidates duplicates into one | Parameter and duplicate URLs |
| Robots.txt | May crawlers fetch this? | Controls crawling, not listing | Low-value URL patterns, crawl load |
| Redirect | Where has this moved? | Sends users and signals elsewhere | Moved or merged pages |
Google's guidance on [canonical tags and duplicate URLs](/guides/canonical-tags/) makes one point worth repeating here: don't use noindex to choose a canonical within your own site, because noindex removes the page from search entirely instead of consolidating it.
#### The common misuse
Noindexing a duplicate page throws away whatever links and signals it had earned. A canonical or a redirect keeps them working for the preferred version.
### When is a redirect better than noindex?
When the page has been replaced or merged. If a visitor or a link lands on an old URL, noindex leaves them on a dead-end page. A 301 redirect sends them somewhere useful and passes the signals along. Reach for noindex only when the page genuinely needs to stay accessible in its current form.
The mechanics of doing that cleanly are in the guide to [301 redirects and redirect mapping](/guides/301-redirects/), and they matter most during redesigns and migrations.
## How long does noindex take, and how do you speed it up?
Patience is part of the deal. Noindex works on Google's schedule, because the rule only takes effect when Googlebot recrawls the page.
Google's documentation notes that depending on a page's importance, recrawling can take months.
### Why can removal be slow?
Google decides how often to revisit each URL based on factors like how important and how fresh the page seems. A busy, well-linked page gets recrawled often. An obscure archive page may not be revisited for a long time, so a new noindex tag can sit unread. During that period the page stays in the index.
That delay is normal and not a sign that the tag is broken. It does mean you shouldn't count on noindex for anything urgent.
#### Don't test by searching
Searching for the page to see whether it's gone is unreliable. Use Search Console to see what Google last crawled and what it recorded.
### What can you do to speed it up?
Request a recrawl using the URL Inspection tool, which Google's documentation points to for exactly this purpose. Make sure the page is linked from somewhere Google visits often, and that it appears in your sitemap while you wait, so Google has reason to return. For a genuine emergency, such as exposed private information, use Google's removals tooling, which hides a URL quickly while you apply the permanent fix.
Treat the removals tool as a temporary measure. It buys time. The permanent solution, a noindex tag, authentication or deletion, still has to be in place.
## How do you verify that noindex is working?
Don't assume. Check what Google actually sees, because the gap between your intentions and the live page is where mistakes hide.
Search Console gives you everything you need.
### How do you test one URL?
Paste the address into the URL Inspection tool. The tool reports whether indexing is allowed, and the live test shows the HTML Googlebot receives, so you can confirm the noindex tag is visible. Google's own cautions on this page say exactly that: verify the rule is visible to Googlebot using URL Inspection, which shows the HTML Googlebot received.
If "Indexing allowed?" says no and the page is not indexed, the tag is doing its job. If the page is still indexed after a reasonable wait, check the date of the last crawl.
#### A subtle trap with robots.txt
When robots.txt blocks crawling, the "Indexing allowed?" field always says yes, because Google can't see the directive. A reassuring answer here can be misleading, so check "Crawl allowed?" first. The mechanics of the inspection tool are covered in [how to check whether a page is indexed by Google](/guides/how-to-check-if-a-page-is-indexed-by-google/).
### How do you monitor a whole section?
Use the page indexing report. Pages that Google excluded because of a noindex tag appear under the reason "Excluded by 'noindex' tag." Review that list regularly. It should contain only the pages you chose to exclude. Any surprise entry is worth investigating the same day.
A crawl with a tool like Screaming Frog or Sitebulb can list every page with a noindex directive, in the meta tag or the header. Compare that list against your intentions after every release.
#### Build it into releases
Add "check noindex on key templates" to the launch checklist. It takes a minute and catches the costliest mistakes.
## Which noindex mistakes do the most damage?
Most noindex disasters have the same shape: a harmless setting that outlived its purpose, or a broad rule that caught more than expected.
Knowing the patterns helps you spot them early.
### What causes accidental noindex?
A CMS option such as "discourage search engines from indexing this site," switched on during development and never switched off. A staging site's template copied to production. An SEO plugin setting applied to a whole post type. A header rule added at server level and forgotten. A migration that carried over test settings.
The symptoms are alarming but recognisable: pages falling out of the index, impressions collapsing and a spike in "Excluded by noindex tag" in the page indexing report. The fix is to remove the directive, confirm it with URL Inspection and request recrawls for the most important pages.
#### The post-launch ritual
On launch day, check the live robots.txt, a few key templates for meta robots tags and a response header on a representative page. If anything says noindex that shouldn't, fix it before Google notices.
### What mistakes come from misunderstanding?
Blocking and noindexing the same URL. Expecting noindex to consolidate duplicates. Using it on pages that earn links, which wastes their value. Adding it to paginated pages without thinking through how Google discovers deeper content. Assuming that noindex also stops crawling, which it doesn't: Google still requests the page, which on large sites can matter for crawl efficiency.
On that last point, Google's crawl budget guidance for large sites notes that noindex doesn't prevent requests, because Google still has to fetch the page to see the rule. For huge sets of low-value URLs, blocking crawling can be the better long-term approach, after the pages have been removed.
#### The reasoning behind the choice
Ask what you want: fewer crawls, fewer listings or fewer duplicates? Then pick the matching tool.
## What about nofollow, noarchive and other directives?
Noindex is the headline directive, but it shares a family with several others. Understanding the family helps you avoid combining them carelessly.
Keep it simple: use only what you need.
### How does noindex relate to nofollow?
Nofollow asks search engines not to follow the links on a page. You can combine it with noindex in one tag, but there's rarely a reason to. A noindexed page that still links to your good content passes useful signals, and cutting those links can isolate the pages beyond it. Unless you have a specific reason, leave nofollow off.
Google has also changed how it treats long-term noindex pages and links over time, so avoid designing a strategy that depends on subtle behaviour. Keep the architecture simple, and make sure important pages are reachable through normal indexed pages.
#### Don't hide your navigation
If an important section is only linked from noindexed pages, Google may find it harder to discover. Check internal links when you apply noindex at scale.
### What other directives exist?
There are directives for snippets, cached copies, image indexing and more, and each affects how a page appears rather than whether it appears. Use them sparingly, and only with a clear purpose. They're covered in Google's robots meta tag documentation. For most business sites, noindex and, rarely, a snippet control are all that's ever needed.
Remember that other search engines may interpret directives differently, so a rule that works as intended in Google might behave differently in Bing. If another engine matters to you, check its documentation too.
## How should small business sites approach noindex?
Most small sites need very little of it. A short, deliberate list of noindexed page types, kept under review, is the healthy pattern.
The temptation is to over-engineer.
### What's a sensible small-site policy?
Index your service pages, key articles, location pages and contact page. Noindex your internal search results, thank-you pages, account areas and any thin archive pages. Leave everything else alone until you have evidence that a page type causes trouble. Write the policy down in a single page so anyone who edits the site understands it.
Review it twice a year and after every redesign. Policies drift as plugins and templates change, and a written list gives you something to compare against. For help turning that into a proper audit, a specialist [technical SEO consultant](/technical-seo-consultant/) can check directives, templates and server headers in one pass.
#### What to avoid
Applying noindex "just in case," and leaving test settings on. Both are how good pages disappear.
### How do you decide about a borderline page?
Ask three questions. Does it help a searcher? Does it duplicate something stronger? Could it be improved instead? If the page could become useful, improve it. If it duplicates a better page, consolidate it with a canonical or redirect. If it's useful only once someone is already on your site, noindex it.
Borderline pages are also where the statuses described in [crawled, currently not indexed](/guides/crawled-currently-not-indexed/) tend to appear. Sometimes Google has already made the call, and your job is just to make the signals clear.
## What does a noindex clean-up look like in practice?
A story helps make this concrete. It's a hypothetical composite built from common patterns, with no real names or figures.
It shows the reasoning rather than predicting any specific result.
### What was the situation?
A small professional services firm launched a redesigned site. Three weeks later, enquiries from search dropped. Search Console showed a jump in "Excluded by 'noindex' tag," covering most of the service pages. The cause turned out to be a single checkbox in the CMS, "discourage search engines," left on from the build phase.
At the same time, the old site's thank-you and search pages, which had been correctly noindexed, had lost their tags during the template change and begun to appear in results. Two opposite problems, one release.
#### The pattern
A launch that flipped the wrong switches in both directions. It happens often enough that every launch checklist should include a directive check.
### What did the fix involve?
The team switched off the sitewide setting, then verified a sample of service pages with URL Inspection and requested recrawls for the most important ones. They reapplied noindex to the thank-you, search and account templates through the SEO plugin, and confirmed the header and tag on each. They added a post-launch check, a short list of ten URLs to inspect, and documented the policy.
Over the following weeks the service pages returned to the index and the stray utility pages dropped out. The lesson wasn't about the tag. It was about process: write down which pages should and shouldn't be indexed, and check it every time the site changes. If you'd like a second pair of eyes on that process, an independent [SEO audit](/seo-audit/) reviews directives alongside crawling, canonicals and sitemaps.
#### Keep the habit
Directives are tiny and powerful. A two-minute check after every release is the cheapest insurance in technical SEO.
## How does noindex fit into a wider indexing strategy?
Directives work best when they're part of a plan. A site where every page type has a deliberate status, indexed, noindexed, canonicalised or redirected, is easy to audit and hard to break. A site where directives accumulate by accident is a maintenance headache.
The aim is a short, written policy that matches what you want searchers to find.
### How do you build a page-type inventory?
List every distinct kind of page your site produces: homepage, service pages, location pages, blog posts, category archives, tag archives, author pages, search results, thank-you pages, account areas, attachments, pagination and any parameter-based variants. For each, write down whether it should be indexed and why. Then check what the site does today.
Most owners are surprised by the number of page types they find. Crawl tools help, and a sample of URLs per type is enough. The table you end up with becomes the reference for developers, plugin settings and future audits.
| Page type | Index? | Method if not | Reason |
|---|---|---|---|
| Service and location pages | Yes | n/a | Core search value |
| Blog posts | Yes | n/a | Informational demand |
| Tag and date archives | Usually no | Noindex | Thin lists of links |
| Internal search results | No | Noindex, then block crawling | Endless low-value URLs |
| Thank-you pages | No | Noindex | Only meaningful after an action |
| Parameter duplicates | No | Canonical to the clean URL | Same content, different address |
#### Keep the inventory short and honest
If you can't explain why a page type is indexed, ask whether it should be. Clean, intentional structure is easier for Google to understand, and it keeps crawl attention on the pages that earn money.
### How does this connect to crawl efficiency?
On small sites, barely. On large ones, quite a lot. Noindexed pages still get crawled, which uses requests that could go to valuable URLs. For big catalogues, the long-term plan is often to stop generating pointless URLs, block crawling of whole patterns once they've left the index and keep links to them out of the navigation.
That's a decision for sites with thousands of URLs and visible crawl pressure, and it ties in with the patterns discussed under [discovered, currently not indexed](/guides/discovered-currently-not-indexed/). For most business sites, a clean noindex policy and healthy internal linking are enough.
## What should you tell developers and editors about noindex?
Directives are changed by people, so the people need to understand them. A short briefing prevents most accidents, and it's one of the cheapest habits to adopt.
The message is simple: here's what noindex does, here's where it's allowed and here's who to ask.
### What belongs in a one-page briefing?
What noindex does in plain language. The list of page types that carry it on purpose. Where it's controlled, such as the SEO plugin, the template or the server config. A rule that nobody applies it to service, product or key content pages without approval. And a note that robots.txt and noindex shouldn't be combined for removal.
Share it with whoever edits templates, installs plugins or manages hosting. If an agency or freelancer works on the site, include them. A clear briefing costs an hour and saves weeks of lost visibility.
#### Add a review step to releases
Before any release goes live, someone checks the live site's robots.txt, a handful of key templates and the response headers on a few URLs. It's a two-minute job once it's routine.
### How do you handle plugin and platform updates?
Updates can change defaults. A plugin may add a new setting, reset an old one or alter how archives are handled. After significant updates, spot-check your noindex status on the key page types. Subscribe to release notes for the plugins that control indexing, and treat updates to those as small risk events.
If something changes unexpectedly, the page indexing report in Search Console usually shows it within days. Setting up an alert on sudden jumps in "Excluded by 'noindex' tag" gives you early warning.
### When should you review the noindex policy?
At three moments: before any redesign or migration, after any major plugin or platform change, and once or twice a year as routine maintenance. At each point, rerun the page-type inventory, crawl the site for directives, compare the results with the written policy and fix any differences. It's an hour of work that keeps the index clean and the site predictable.
Include the people who change the site in that review. Directives usually go wrong at the handoff between the person who understands them and the person who edits a template, and a short conversation closes that gap before it costs you traffic.
## Frequently asked questions
### What does a noindex tag do?
It tells search engines that support it, including Google, not to show the page in search results. When Googlebot crawls the page and sees the directive, it drops the URL from its index, even if other sites link to it. The page stays accessible to visitors who have the address.
### Is noindex the same as blocking a page in robots.txt?
No. Robots.txt controls crawling and noindex controls indexing. A page blocked in robots.txt can still appear in results if other pages link to it, because Google never sees a noindex tag on a page it can't fetch. To remove a page, allow crawling and use noindex.
### How long does it take for noindex to work?
It takes as long as Google needs to recrawl the page. Google's documentation says that can run to months for less important pages. You can ask for a recrawl with the URL Inspection tool, and use the removals tool if something must disappear urgently.
### Can I use noindex in my robots.txt file?
No. Google does not support a noindex rule in robots.txt. Put the directive in a robots meta tag in the page head, or send it as an X-Robots-Tag HTTP header, which also works for PDFs, images and other non-HTML files.
### Should I noindex thin or duplicate pages?
Sometimes. Noindex suits pages that must exist for visitors but have no search value, such as internal search results or thank-you pages. For true duplicates, a canonical tag or a redirect is usually better, because those consolidate signals instead of removing the page entirely.