I Ran A WordPress Crawl Audit On Our Own Site And Found A Broken Link On Every Page.

Short answer: a wordpress crawl audit checks the relationships between pages, which is the one thing an SEO plugin sitting inside a single page cannot do. It finds pages nothing links to, pages buried too deep to matter, and links pointing at URLs that no longer exist. Every individual page can return 200 while the site as a whole is broken.

I ran one against protuno.com while writing this. It found two things I did not know, and one of them is on every page of the site.

What a crawl actually produces

A crawler starts at the homepage, follows every internal link, and records what it finds. The useful column is not the status code. It is how many other pages point at each URL.

single image missing an alt attribute. One row is red. It is linked from all 24 pages.*

The good news first, because a crawl should tell you what is working: no orphan pages, no missing alt attributes, nothing buried deeper than three clicks.

Then the row I did not expect.

The defect that was on every single page

link to an endpoint that is not being served. Every page carrying that link returns 200, so nothing anywhere reports a problem.*

Click the contact email in our footer and you land on a 404.

Nothing told us. The CMS is happy, because it does not know what Cloudflare rewrote. Analytics is happy, because a 404 on a link nobody clicks generates no events. An uptime monitor is happy, because every page it checks returns 200.

This is the entire category in one example. The pages are fine. The relationship between them is broken. No per-page check will ever find it, because the problem does not live in any single page.

Why your SEO plugin will not catch this

Not a criticism of SEO plugins. They are answering a different question.

own link graph. Everything in the bottom half of that table is a relationship, which is exactly what code running inside one page cannot see.*

The second finding, which I had just written a post about

The crawl cross-references robots.txt against the robots meta tag on each page. Ours disagree.

/blog as its canonical, so it cannot compete with itself. The first two rows are the mismatch.*

/auth/login is linked from 40 pages, disallowed in robots.txt, and tells search engines to index it. Because it is disallowed, that instruction can never be read. The URL can sit in results as a bare link with no description.

I wrote about that exact failure mode a couple of days ago, and then found it on our own site by crawling it. Which is the argument for crawling rather than assuming.

Running a WordPress crawl audit yourself

You do not need a subscription. The check that matters most is inbound link counts, and any crawler will give you that. What to look at, in order:

Anything returning a non-200 with inbound links above zero. This is the highest-value output of any crawl. Fix these first, always.

Pages with zero inbound links. If nothing links to a page, you are asking search engines to value something you do not value enough to link to.

Click depth above three. Depth is a rough proxy for how important your own site says a page is. A commercially important page at depth five is a structural problem.

robots.txt against the meta robots tag. They should agree. When they disagree, the one that can actually be read wins, and it is usually not the one you intended.

Canonical tags on parameterised URLs. Sorting, filtering and pagination parameters should canonicalise back to the clean URL, as our ?order=asc does.

One practical note on crawling a headless site, because it caught me out mid-write. Two of the posts I published this week returned 404 to the crawler for a few minutes after going live, then 200 once the front end revalidated. If you crawl a site that renders pages on demand and caches them, a fresh URL can look broken purely because nothing has asked for it yet. Re-crawl before you report a 404 on a page you have just published, or you will file a bug against yourself.

Crawl as a logged-out visitor. It is easy to run a crawl in a browser where you are signed in and see admin-only links, draft content and navigation that no visitor gets. The crawler should carry no cookies at all. What you want is the site a stranger sees.

Why this ends up never being done

A crawl takes minutes and needs redoing after every content change, every migration, every theme update and every time a third party rewrites your HTML, which is what happened to us.

It produces nothing to show a client when it passes. So it gets run once during onboarding and then never again, exactly like the backup file left in the web root and the scheduled jobs that stopped.

Iris, Protuno’s SEO agent, is built to crawl a connected site the way a search engine does and report what changed since last time. Straight with you, as on every post here: Iris is built and named but not live yet.

The free audit checks robots.txt, the sitemap and the indexing signals from the domain today, which catches the second finding above but not the first. The broken-link finding needs a real crawl.

Run one on your own site before you run one on a client’s. I did, and it found something on every page.