The Technical SEO Checklist I Run Before Every Client Engagement

Before I write a single content brief for a new client, I work through the same technical pass. The reason is simple: content and link work compound, but only on top of pages that search engines can actually reach, render, index, and load quickly. Skip that layer and you spend months producing pages that never get a fair evaluation.

The checklist below is grouped the way search engines process a page: crawl, index, render, then experience. Within each group I have noted what the check catches and the usual fix, so you can run it yourself rather than just tick boxes.

One practical note before you start. Do not audit URLs, audit templates. Almost every issue worth fixing on a real site is a template issue that repeats across hundreds or thousands of pages, so I sample two or three live URLs per template (home, category, product or service, article, paginated listing, search results, and any locale variant) and run the whole pass on each sample. Fixing one template fixes the whole set.

1. Crawl

Crawl problems are the cheapest to fix and the most expensive to ignore, because everything downstream depends on them. The goal of this group is simple: one address per piece of content, reachable in a few hops, returning honest status codes.

• robots.txt sanity. Confirm nothing important is disallowed, and that CSS and JS paths are crawlable, and blocking them breaks rendering. Fix: narrow the disallow rules.
• Sitemap accuracy. Every URL should return 200, be canonical, and be absolute. Redirects, 404s, and noindexed URLs inside a sitemap waste crawl budget and cause fetch warnings in Search Console.
• Internal link reachability. Any page that only exists in the sitemap, with no internal link pointing at it, is effectively an orphan. Fix: link it from a relevant hub page.
• Redirect chains and loops. Every hop loses a little and slows crawling. Fix: point redirects straight at the final destination.
• Host and protocol consolidation. One canonical host, HTTPS only, one trailing-slash convention. Everything else should redirect once to it.
• Server response codes on error paths. Missing pages must return 404 or 410, not a 200 with a "not found" message. Soft 404s keep dead URLs in the index.
• Faceted and parameter URLs. Filter and sort parameters can generate thousands of near-duplicate URLs. Fix: canonicalise to the clean URL, and avoid linking to parameterised variants internally.

A robots.txt that does not cause damage

Most robots.txt files I inherit either block something important or try to solve an indexing problem that robots.txt cannot solve. Remember the distinction: disallow controls crawling, not indexing. A blocked URL can still appear in results, and a blocked URL cannot be seen to carry a noindex tag. If you want a page out of the index, allow the crawl and serve noindex.

User-agent: *
Allow: /

# Block only genuinely useless paths
Disallow: /cart
Disallow: /checkout
Disallow: /*?sort=
Disallow: /*?sessionid=

# Never block the assets needed to render the page
Allow: /assets/
Allow: /*.css$
Allow: /*.js$

Sitemap: https://example.com/sitemap.xml

Two checks after any edit: fetch the file over HTTPS on the canonical host and confirm it returns 200 with a plain text content type, then run a handful of important URLs through the robots tester. A robots.txt that 500s is treated as a full disallow by some crawlers, which is the worst possible failure mode.

What a healthy log or crawl profile looks like

If you have access to server logs, look at where crawl activity is going. On a stalled site the pattern is usually the same: a large share of requests hitting parameter URLs, expired listings, or paginated tails, while the commercial pages get visited rarely. You do not need to increase crawl budget, you need to stop spending it on pages that will never rank.

2. Index

Indexing is where most of the value hides. This group is about making sure each piece of content has exactly one indexable address, and that the address you chose is the one search engines agree on.

• Indexation gap. Compare pages you want indexed against Search Console's Pages report. Persistent "Crawled, currently not indexed" clusters usually signal thin or duplicate content, not a technical bug.
• Canonical correctness. Every page should self-reference unless it is a deliberate duplicate. Cross-domain or homepage-pointing canonicals on inner pages are one of the most damaging mistakes I find, because they hand every page's signals somewhere else.
• Accidental noindex. Check both the meta tag and the X-Robots-Tag header. Staging noindex directives surviving a launch is a classic.
• Duplicate titles and descriptions. Usually a template problem rather than a writing problem. Fix the template variables and hundreds of pages improve at once.
• Pagination handling. Paginated pages should be crawlable, self-canonical, and linked with real anchors rather than JavaScript-only controls.
• Language and region signals. If the site has multiple locales, hreflang must be reciprocal and reference indexable URLs only.

The canonical patterns worth memorising

Canonical mistakes are the single most damaging category I find, because a wrong canonical does not look broken. The page loads, the content is there, and the signals quietly go somewhere else. These are the patterns that are almost always correct:

Standard page, self referencing, absolute:
<link rel="canonical" href="https://example.com/services/aeo" />

Parameter variant points at the clean URL:
/services/aeo?utm_source=news  ->  https://example.com/services/aeo

Paginated page is self canonical, not pointing at page 1:
/blog?page=3  ->  https://example.com/blog?page=3

Locale variants, reciprocal hreflang plus self canonical:
<link rel="alternate" hreflang="en-in" href="https://example.com/in/" />
<link rel="alternate" hreflang="en-gb" href="https://example.com/uk/" />
<link rel="alternate" hreflang="x-default" href="https://example.com/" />

Then verify with URL Inspection, not with view-source. Search Console reports both the canonical you declared and the canonical Google selected. When those two differ, you have found a real duplication problem that the markup alone will not tell you about.

Reading the Pages report properly

• Crawled, currently not indexed. Usually a quality or duplication judgement. Consolidate, expand, or accept that the page should not exist.
• Discovered, currently not indexed. A crawl capacity or priority signal. Improve internal linking to those URLs and reduce the low value URLs competing for attention.
• Duplicate, Google chose a different canonical. Your consolidation is not working. Align internal links, sitemap entries, and the canonical tag on the same URL.
• Alternate page with proper canonical tag. Normal and healthy when it matches your intent. Only worrying if the affected URLs are pages you expected to rank.
• Soft 404. A thin page or an error path returning 200. Fix the status code or give the page real content.

3. Render

• Main content in the initial HTML. View source, not the inspector. If the body copy, headings, and internal links only appear after JavaScript runs, treat that as a priority fix, server rendering or static generation.
• Blocked resources. Use the URL Inspection rendered-HTML view to confirm no critical CSS or JS is blocked or failing.
• Real anchors for navigation. Click handlers on divs are not links. Navigation and pagination need <a href>.
• Heading structure. One H1 per page, headings that describe the section beneath them. This matters more now that answer engines extract passages by heading.
• Structured data validity. Schema should match what is visible on the page and parse without errors. Invalid JSON-LD is ignored silently.
• Image and media semantics. Descriptive alt text, real dimensions, and no content that exists only inside an image.

The two minute rendering test

Open view-source on a key page and search the raw HTML for a distinctive sentence from the middle of the body copy, then for one of the internal link URLs. If both are present, rendering is not your problem. If neither is, the page depends on client side JavaScript and you are relying on a second pass that is neither guaranteed nor timely, and that most AI retrieval systems do not run at all. Server rendering or static generation is the fix, not more markup.

Repeat the same test with JavaScript disabled in DevTools to see the worst case a limited client experiences. You are not checking that the page looks good, only that the content, headings, and links exist.

Structured data and internationalisation

• Match markup to visible content. Every value in the JSON-LD should be readable on the page. Invented ratings and invisible FAQ answers are the fastest route to a manual action.
• Use the types that apply, typically Organization sitewide, then Article, Product, Service, BreadcrumbList, and FAQPage where they genuinely fit. More types are not better.
• Keep identifiers absolute. URLs inside schema should be full https URLs so the entity resolves consistently across pages.
• Validate after every template change. A trailing comma silently disables the whole block, and template edits are where that happens.
• Check hreflang reciprocity. Each locale must reference the others and itself, on indexable canonical URLs, with a valid language and region code.

4. Experience and Core Web Vitals

• Field data first. Start with the Core Web Vitals report in Search Console. Lab scores are diagnostic; field data is what gets assessed.
• LCP element identification. Find the actual largest element, then preload it, serve it in a modern format, and never lazy-load it.
• Interaction responsiveness. Long tasks from third-party scripts are the usual cause. Defer, remove, or load them on interaction.
• Layout stability. Reserve space for images, ads, and embeds; avoid injecting banners above existing content.
• Mobile parity. The mobile page must contain the same content, links, and structured data as desktop.
• Caching and compression. Long cache lifetimes on static assets and compression enabled. These are cheap wins that show up on every page.

I keep performance last on purpose. It is real work with real user benefit, but a fast page that cannot be indexed is worth nothing, while a slow page with clean HTML still ranks. If you only have one sprint, spend it on crawl, index, and render, then schedule the vitals work. The deeper version of this section is in Core Web Vitals in 2026.

Mistakes I See Most Often

• Indexable duplicates with no consolidation. Parameters, trailing slash variants, upper and lower case paths, and filter URLs all serving the same content.
• Staging directives surviving a launch. A noindex header or a disallow all robots.txt copied into production.
• Homepage canonicals on inner pages. Often introduced by a plugin default or a copied template.
• Redirecting everything retired to the homepage. It reads as a soft 404 and discards the relevance the old URL had.
• Navigation built from click handlers. The links are invisible to anything that does not execute scripts.
• Sitemaps generated from the database rather than from live status. They fill up with redirects, 404s, and noindexed URLs.
• Audit reports with 200 undifferentiated issues. Nothing gets fixed because nothing is ranked.

How I Prioritise the Findings

Not every issue deserves dev time. I sort findings into three buckets: things that block crawling, indexing, or rendering go first, because nothing else works until they are cleared. Things that split or misdirect signals, such as canonicals, duplicates, and redirect chains, go second, since they usually deliver the largest gain per hour of work. Everything else, including most performance micro-optimisations, goes third and gets scheduled rather than escalated.

The written deliverable matters as much as the audit. Each item gets a one-line problem statement, the affected URL pattern, the fix, and who owns it. A list of 200 unranked issues gets ignored; a list of eight ranked ones gets shipped.

Where This Connects to AEO and GEO

Everything above is also the foundation for AI search visibility. Retrieval systems that feed generative answers need clean, crawlable HTML with clear headings and valid structured data. If you are working on why your brand is missing from ChatGPT search, the technical layer is where that work starts, not with the markup you add last.

What to Do This Week

• Fetch robots.txt and the sitemap on the canonical host and confirm both return 200 with the right content type.
• Run URL Inspection on one URL per template and compare the declared canonical with the selected one.
• View-source three important pages and confirm the body copy and internal links are in the raw HTML.
• Export the Pages report and group the excluded reasons by template rather than by URL.
• Write the top five findings as one line each: problem, URL pattern, fix, owner. Ship those before looking for more.

That is the whole method. It is not clever, and it does not need to be. Reliable crawling, unambiguous canonicals, content in the HTML, and honest status codes account for the majority of technical gains I have ever delivered on a client site.

Frequently Asked Questions