Technical SEO is three questions in order: can a crawler fetch the page, can it render and understand the content, and does the result deserve to be shown. Most problems developers hit are the first two, which are engineering problems with definite answers, not the third, which is not.

Key takeaways

  • Crawl and render are engineering problems. They have deterministic causes and you can debug them like any other pipeline.
  • Your origin sets the floor. Time to first byte is added to every downstream metric; no front-end work removes it.
  • Structured data is not a ranking factor — it decides rich-result eligibility and entity clarity, which is a different benefit.
  • AI crawlers mostly do not execute JavaScript, so client-rendered content is at a real disadvantage for citation.
  • Citation is replacing position as the metric that matters for informational content.

Where hosting stops and SEO starts

They do not stop and start anywhere. Server response time is an infrastructure property that becomes a Core Web Vitals number, which becomes a ranking input and a crawl-budget input. The handoff people imagine between "the ops problem" and "the marketing problem" does not exist in the request path.

This is the reason the two disciplines produce such bad advice about each other. An SEO consultant tells you to improve Largest Contentful Paint without being able to say which part of the stack is responsible. An engineer optimises bundle size while a 600 ms origin delay sits untouched in front of everything. Both are looking at one end of a chain.

Crawl budget
The number of requests a search engine is willing to spend on your site in a given period. It is a function of how quickly your server responds and how much of what it finds is worth having. Slow origins get crawled less, which delays how fast new content is discovered — an infrastructure decision with a direct indexing consequence.

Stage one: can it be fetched

Before anything else, confirm the machine can get the bytes. These are the failures that silently remove pages from consideration entirely.

  • Status codes. A page that returns 200 to your browser and 403 to a crawler is invisible. Check with a crawler user-agent, not just a browser.
  • robots.txt. A single misplaced Disallow can remove a whole directory. It blocks crawling, not indexing — a blocked URL can still appear, described from third-party signals.
  • Canonical conflicts. Self-referencing canonicals on paginated or parameterised URLs routinely collapse pages into one another.
  • Redirect chains. Each hop costs crawl budget and dilutes signals. Two hops is tolerable; five is a problem.
  • Server response time. Consistently slow origins get sampled less. This is where the hosting decision reappears.
The most common one, by a distance. A staging noindex shipped to production. It is trivial to cause, invisible in a browser, and removes the site from search until someone thinks to view source. Check it first, always.

Stage two: can it be understood

Google executes JavaScript, but in a deferred second pass that can lag the initial crawl. Content that only exists after hydration is therefore indexed later and less reliably than content present in the HTML response. For most crawlers other than Google, it is not indexed at all.

This is the single most consequential architectural decision for visibility, and it is made by developers, usually without anyone framing it as an SEO decision. The practical hierarchy, best to worst:

Rendering strategies ranked by how reliably machines can read the result.
Strategy Google indexing Non-Google crawlers Best for
Static HTML Immediate Full Content, docs, marketing
Server-rendered Immediate Full Dynamic content that must be indexed
Static + hydration Immediate Shell readable Interactive content pages
Client-side only Deferred pass Usually nothing Logged-in app surfaces

Note the third column. It is why this decision matters more in 2026 than it did in 2020: the crawlers behind AI answer engines are far less capable than Googlebot, and a client-rendered page is frequently a blank document to them.

Structured data, precisely

Structured data is not a ranking factor. It determines eligibility for rich results and helps machines resolve entities and relationships. The benefit is how you are presented and how confidently you are understood, not where you are placed — a distinction most SEO content blurs.

  • Match the visible page. FAQ markup whose answers do not appear in the rendered content is a policy violation, and the usual cause of silently lost rich results.
  • Use @id to connect nodes. One Person or Organization entity, referenced across every page, resolves you as a single entity rather than many disconnected mentions.
  • Prefer JSON-LD. It is decoupled from markup, so it survives template changes that would break microdata.
  • Do not mark up what is not there. Ratings, prices and reviews that exist only in the JSON are fabrication, and are treated as such.

Valid markup and a displayed rich result are not the same claim — full breakdown, with a worked @graph example, in why schema.org compliance isn't enough.

Stage three: being cited, not just ranked

Answer engines quote sources inside a generated response, frequently with no click at all. The winning content is extractable rather than merely persuasive: direct answers, clear structure, specific figures, and a stated source. Position on a results page is no longer the only thing worth measuring.

This is the newest part of the discipline and the part with the least reliable public information. Most of what is written about it is confident guesswork. The honest version is on its own page:

On evidence. Claims here are either mechanically verifiable, documented by the search engines themselves, or explicitly marked as inference. Ranking case studies with before-and-after data publish once there are results worth showing — not before.

What to fix, in order

Effort is not evenly rewarded. This is the sequence that actually pays, from a decade of watching people start at the wrong end.

  1. Indexability. Rogue noindex, blocked directories, wrong canonicals. Nothing else matters if the page cannot be seen.
  2. Rendering. Get the content into the HTML response wherever it plausibly can be.
  3. Origin speed. Fix time to first byte at the server before touching images. See the hosting guide.
  4. Structure. Headings that describe content, internal links with real anchor text, one clear topic per URL.
  5. Structured data. Accurate, matching the page, entity-connected.
  6. Extractability. Direct answers near the question, specific numbers, stated sources — for humans and answer engines alike.
What is not on this list. Keyword density, LSI keywords, word-count targets, and the rest of the folklore that persists because it is easy to sell. None of it survives contact with how retrieval actually works.

Badri Dutta

Software engineer · 8 years technical SEO

Fifteen years building for the web and eight doing technical SEO, including small business sites taken to page one and held there through repeated algorithm updates. This section exists because the people who explain search usually cannot debug a render trace, and the people who can debug one rarely explain search.

Full background →