The canonical tag that points at a redirect

Search Console reported twelve of our pages as Discovered – currently not indexed, every one with Last crawled: N/A. Google knew the URLs existed and had not spent a request on any of them.

The pages were fine. They returned 200, they were fast, they had content, they were in the sitemap. The problem was that every URL we were advertising was an address that redirects.

The shape of it

The site serves from www. Everything the application emitted named the apex, and the apex redirects to www:

sitemap   <loc>https://example.com/about</loc>              -> 308 -> www
robots    Host: https://example.com                           -> 308 -> www
robots    Sitemap: https://example.com/sitemap.xml            -> 308 -> www
page      <link rel="canonical" href="https://example.com/about">  -> 308 -> www

The last line is the one that matters. The page served at www.example.com/about returned 200 and declared its canonical to be the apex — which redirects back to www. The single tag whose job is to say this is the real address of this content was pointing at a redirect that leads back to the page doing the pointing.

Google resolves this. It follows the redirect, works out that www is the page, and eventually indexes something. But “eventually” is doing real work in that sentence, and in the meantime the sitemap — the one place you get to tell a crawler exactly what to fetch — is a list of addresses that all answer with a redirect. For a new site with no authority and a small crawl allocation, that is enough to keep pages in the discovered pile.

Why it survived

One environment variable held the app’s public origin, and it was set to the apex while the platform served www. Everything downstream — canonicals, Open Graph URLs, the sitemap, the robots host line, links in outgoing email — derived from that one value and was therefore consistently, uniformly wrong.

Nothing in the codebase could detect it. The variable was a valid URL. The pages rendered. The tests passed. The build succeeded. The mismatch existed only between the application and the platform’s domain configuration, which no test has visibility into, and it was reported back to us months later by a crawler.

We had even written a script to check for exactly this, after an earlier audit found it. Then nothing ran the script. A diagnostic that exists and is never scheduled is a diagnostic that does not exist, and that was the more useful lesson of the two.

How to check yours

Three commands, no tooling. Pick a real page, not the homepage.

curl -sI https://example.com/about | head -1
curl -s https://www.example.com/about | grep -o '<link rel="canonical"[^>]*>'
curl -s https://example.com/sitemap.xml | grep -m1 '<loc>'

You are looking for one thing: that the host in the canonical tag, the host in the sitemap, and the host that answers with a 200 are all the same string. Which host you choose does not matter. That they agree matters enormously.

The failure is easy to miss by hand because each check passes in isolation. The apex redirects correctly. The canonical tag is well-formed and present on every page. The sitemap is valid XML with the right URLs in it. Only the combination is wrong.

Fixing it

Pick the host that serves content and make every emitted URL use it. Usually that is one setting in whatever holds your public origin, and a redeploy. If your framework builds canonicals from a base URL — Next.js does this with metadataBase — changing that one value corrects the tag, the Open Graph URLs and the sitemap together.

Then check what else derived from it. Ours also fed the URLs in outgoing email and the return addresses for checkout, which followed the change automatically and were worth verifying rather than assuming. Anything a third party has stored a copy of — an OAuth redirect allowlist, a webhook target — will not follow, and needs updating by hand.

Afterwards, resubmit the sitemap and use Search Console’s validate step. And if you have a URL-prefix property for the host that does not serve content, you have been looking at a property covering nothing. A domain property covers both hosts and every protocol, which is what you want while you are still deciding.

The general version

A redirect is not a bug. It is the correct answer to a request for the wrong address, and one hop from apex to www or http to https is normal and fine.

The defect is naming a redirecting address as canonical. Sitemaps, canonical tags, internal links, structured data and Open Graph URLs are all statements about where content lives, and every one of them should name the address that answers. A crawler that has to resolve a redirect to act on your instruction is a crawler you have given a task instead of an answer — and crawlers are working to a budget.