Indexed, though blocked by robots.txt
The warning where a blocked URL is indexed anyway — why it happens, and the counterintuitive unblock-then-noindex fix most people miss.
“Indexed, though blocked by robots.txt” is a warning about a contradiction: the URL is in Google’s index, but your robots.txt tells Google not to crawl it. Google added the URL from links pointing at it, yet it was never allowed to read the page — so it does not know the title, the content, or whether you wanted it indexed at all. That gap is why this status is worth resolving.
How a blocked page ends up indexed
robots.txt stops crawling, not indexing. When other pages — yours or someone else’s — link to a blocked URL, Google can decide the address is worth listing even though it cannot fetch the content. The result is a search listing with no real description, often the line “No information is available for this page,” because Google is indexing a URL it has never been allowed to read.
Why it is a problem
Three reasons. The snippet is empty or unhelpful, so the listing earns few clicks. The page you actually want ranking for that topic may be competing with this blind entry. And most importantly, if you added a noindex tag to get rid of the page, Google cannot see it — reading the noindex requires crawling, which robots.txt forbids — so the page stays stuck in the index. This is the single most common reason a noindex “does not work.”
Decide what you want, then fix it
If you want the page indexed: remove the robots.txt rule blocking it. Once Google can crawl the page, it reads the real content and replaces the blind listing with a proper title and snippet, and the warning clears. This is the right path for a page that was blocked by mistake.
If you do not want the page indexed: this is the trap, and the fix is counterintuitive. You must first unblock the URL in robots.txt and add a noindex tag to the page. Google then crawls it, sees the noindex, and drops it from the index. Only after it has fallen out is it safe to re-block it in robots.txt. Blocking alone will never remove it. For something urgent, Search Console’s Removals tool hides the URL temporarily while the noindex takes effect.
Find the block and the inbound links
Use the robots.txt checker to confirm which rule is blocking the URL, and the indexability checker to see whether a noindex is present but unreadable behind the block. If you do not want the page indexed, also reduce the internal links pointing at it — those links are part of why Google considered it worth indexing in the first place.
Not the same as “Blocked by robots.txt”
The plain Blocked by robots.txt status means the URL is blocked and not indexed — the block is doing its job. This warning means the URL is blocked and indexed anyway, which is the block failing to do the job people expect of it. Same rule, opposite outcome, different fix. For the full set of statuses, see the Page indexing report reference, or run a full site check.
Frequently asked questions
- What does 'Indexed, though blocked by robots.txt' mean?
- It means the URL is in Google's index even though your robots.txt blocks Google from crawling it. Google added the address from links pointing at it but was never allowed to read the page, so it has no real title or description for it. It is a warning about a contradiction between your robots.txt and what is actually in the index.
- How can a page be indexed if it is blocked by robots.txt?
- Because robots.txt stops crawling, not indexing. When other pages link to a blocked URL, Google can index the address based on those links alone, without ever fetching the content. The listing usually shows no description — often the line 'No information is available for this page' — because Google indexed a URL it was never allowed to read.
- How do I remove a page that is indexed though blocked by robots.txt?
- You have to let Google crawl it first. Remove the robots.txt block and add a noindex tag to the page. Google then crawls it, sees the noindex, and drops it from the index. Only after it has fallen out is it safe to re-block it in robots.txt. Blocking alone can never remove it, because Google cannot read a noindex on a page it is not allowed to crawl.
- Why does my noindex tag not work on a blocked page?
- Because Google cannot see it. Reading a noindex tag requires crawling the page, but robots.txt forbids that crawl, so the tag is never read and the page stays indexed. Unblock the URL in robots.txt so Google can crawl it and see the noindex; once the page is deindexed you can block it again if you want to save crawl budget.
- Should I just delete the robots.txt rule?
- If you want the page indexed, yes — removing the block lets Google crawl it and build a proper listing, and the warning clears. If you do not want it indexed, remove the block only long enough to add a noindex and let Google process it, then you can re-block. Either way, leaving the block in place while the page is indexed is the one option that never resolves the warning.