How to Survive the Hybrid Web

The Opening Provocation: A Number That Keeps Growing

Korner opens with a striking data point: AI agent traffic to websites increased 7,851% year-over-year. He immediately notes that this figure is almost certainly already out of date, given the pace at which the trend is accelerating. Agent traffic is growing approximately eight times faster than human traffic.

This number is not an abstraction. It represents a structural shift in who — or what — is visiting the websites that developers, agencies, and product teams build and maintain.

Korner's core argument is that most of these professionals have not yet registered the scale of this change, and that by the time they do, they will be behind.

What Is the Hybrid Web?

Korner introduces his central concept: the hybrid web. A website in 2026 does not have one type of visitor. It has three: The human customer — the person the site was designed for, browsing with eyes and intentions, comparing, clicking, and buying.

Bots — automated crawlers from search engines and AI companies that read the site to index it, train on it, or retrieve information from it.

AI agents — autonomous software systems that may visit, evaluate, and interact with a site on a human's behalf, potentially completing a transaction without the human ever loading a page.

The word "hybrid" captures the reality that websites must now work for fundamentally different types of visitors, each of which interacts with a page in a completely different way. A page designed only for human eyes — built with rich JavaScript interactions, dynamic content loading, and visual hierarchy — may be partially or entirely opaque to the machine visitors who now constitute the majority of web traffic.

The Scale of the Shift: 51% Non-Human Traffic

Korner cites the statistic that as of 2026, 51% of all traffic on the open web is non-human.

This is presented as a historic first: for the first time, the majority of visitors to any given website do not have eyes, do not have thumbs, and do not read pages the way a person would.

He is careful not to frame this as catastrophic. It is, he argues, a shift — one that demands a response but not a panic. The projection he offers is that by 2027, approximately 70% of web traffic will come from machines. The trend is accelerating, not plateauing.

How AI Is Changing Human Behaviour on the Web

Before addressing the machine audiences, Korner spends time on how AI is already transforming the behaviour of human visitors, because the two phenomena are deeply connected.

The Death of the Click Korner references the well-documented zero-click trend. On Google's AI mode, 93% of searches now end without a single click to an external website. Across Google as a whole, approximately 65% of searches result in no click. On mobile the situation is even more pronounced. He describes this as "classical gatekeeping" — search engines providing the answer directly, removing the need for the user to visit the source.

The implication is direct: organic traffic, as it has historically been understood, is declining. Sites that depend on Google search to drive top-of-funnel awareness are already seeing this play out in their analytics.

The Citation as the New Impression Korner reframes the zero-click problem through a more useful lens: the citation is the new impression. When an AI answers a question and cites a source, that citation carries brand weight even if the user never clicks through. Being cited inside an AI answer is now a form of visibility, and in some respects a more valuable one — because the AI has already endorsed the source as authoritative.

Traffic Volume Down, Intent Up One of Korner's most important reframes is around traffic quality. He cites HubSpot data showing that organic search traffic for HubSpot customers fell 27% year over year. This is a headline that sounds alarming. But he pairs it with a second data point: traffic that arrives via AI converts two to five times better than traditional Google organic traffic.

The reason is straightforward. When an AI engine answers a query, it typically collapses what used to be a multi-tab, multi-session research journey into a single conversation. By the time a user clicks through to a website, they have often already been through a shortlisting process, seen comparisons, and arrived at or near a purchase decision. The selling has happened inside the AI interface. The click, when it comes, is a closing action.

This is, Korner argues, a fundamentally different game. Less traffic, higher intent. Reporting frameworks need to account for this shift.

Parallel Discovery, Not AI Replacing Search Korner challenges the framing that AI search simply replaces Google search. He points to data showing that 95% of ChatGPT users also use Google. In practice, many users move across both channels in sequence: using an AI tool like ChatGPT to build an initial shortlist, then using Google to verify claims, then visiting a site to complete a purchase. This creates multiple touchpoints — and multiple opportunities to either appear or be absent.

Vertical Differences in AI Overview Impact Not all sectors are equally affected by AI overviews. Korner notes that B2B tech is hit hardest, with approximately 70% of searches in that category now generating an AI overview that answers the question without requiring a click. E-commerce, by contrast, sees AI overviews in only around 4% of searches. The reason is practical: AI cannot put a product in someone's home. The shopping cart remains a human-controlled mode of commerce — for now. Korner is explicit that this window for e-commerce is short and that those clients need to be preparing for agents.

Understanding Bots: Two Completely Different Species

Korner turns to the bot audience, noting that the word "bot" now does a great deal of semantic work and covers two very different types of entity.

Search Engine Crawlers Search engine crawlers — Googlebot, Bingbot — are familiar to most web professionals.

They index content for ranking purposes. The exchange is straightforward: crawl access in exchange for traffic. This bargain has existed for roughly 25 years and is well understood.

AI Crawlers AI crawlers — GPTBot, ClaudeBot, PerplexityBot — operate differently. They crawl sites to feed training data into AI models, or to perform live retrieval (fetching up-to-date content on demand for real-time answers). Critically, they generally do not send traffic back in exchange for this crawl access. The historic bargain is broken.

Korner illustrates the scale of this imbalance with a striking comparison: Googlebot: For every visitor it sends back, it crawls a site approximately 14 times.

OpenAI: For every visitor it sends back, it crawls a site approximately 1,700 times.

Claude: For every visitor it sends back, it crawls a site approximately 73,000 times.

The implications for hosting infrastructure are significant, and Korner acknowledges that this is already a live conversation in the industry about which bots to allow and which to block. Most website owners and their clients are, he notes, feeding the AI economy for free, and the vast majority have no idea this is happening.

The JavaScript Problem A critical and often overlooked issue: AI crawlers do not execute JavaScript. They fetch raw HTML once and do not retry. This is an architectural decision driven by cost. Running a headless browser capable of rendering JavaScript for billions of web pages is extremely expensive. Search engines like Google have built the infrastructure to do this — they can place a page in a rendering queue, execute JavaScript with headless Chromium, and index the rendered result. AI companies scraping at scale in real time cannot afford this and skip the rendering step entirely.

The practical consequence: any website where content loads after the initial HTML — React apps, sites built with Lovable, Replit, or other vibe-coding tools — may be entirely invisible to AI crawlers. A site can hold the number one Google ranking and simultaneously be completely dark to ChatGPT, Claude, and Perplexity.

Korner offers a simple diagnostic: right-click any page, select View Page Source, and look at the HTML. If the body contains little more than an empty <div id="root"></div>, the site is invisible to the AI web.

He is careful to note that the Lovable community and others are aware of this problem and working on solutions, but cautions that for now, builders should be very careful about deploying client websites on purely client-side JavaScript frameworks.

LLMs.txt: A Rapidly Evolving Debate Korner discusses LLMs.txt, a proposed standard (similar to robots.txt) that would give website owners a structured way to communicate with AI systems about what content they should and should not access. Yoast launched a feature to generate LLMs.txt approximately a year before this talk.

Shortly after the launch, Google's John Mueller publicly stated that no AI system was using LLMs.txt — which was accurate at the time. The space moved quickly. Cloudflare deployed LLMs.txt on their own documentation and reported a 31% reduction in tokens consumed by agents and 66% faster correct answers. Claude has since added support for LLMs.txt.

Korner's takeaway: when the implementation cost is near zero — a single toggle in a tool like Yoast — there is no reason not to deploy it. The downside risk is zero; the upside, given how fast the space is moving, could be meaningful.

Agents: The Third and Most Disruptive Audience

Korner devotes the most forward-looking section of his talk to AI agents — autonomous systems that do not merely browse the web but take actions within it.

What Agents Are Actually Doing Citing analysis of nearly a quadrillion (10^15) interactions, Korner reports: 77% of agent traffic goes to product and search pages.

Almost 3% of agents are already completing checkouts.

Agent-initiated transactions are not a future scenario. They are happening now. The challenge is that they are largely invisible in conventional analytics because agents use the same browser infrastructure as humans.

How Agents Work: A Three-Layer Stack Korner describes the technical architecture of web-capable agents: The brain — an LLM (ChatGPT, Claude, or any other AI model) that reasons, plans, and makes decisions.

The engine — a headless browser (typically Chromium, though alternatives like Playwright, Puppeteer, Perplexity Comet, and the newer Lightpanda are in use) that navigates and interacts with the web.

The translator — a tool that converts raw, messy HTML into clean Markdown that the LLM can efficiently process and reason about.

He notes that Lightpanda is a particularly interesting new entrant: it is fast, cost-effective, and comes with a built-in translator layer.

The key architectural insight is that every layer of this stack prefers structured data over rendered pages. Agents work best with clean, predictable, machine-readable content.

Why Agents Fail (and What to Do About It) Korner identifies the friction points that cause agents to abandon a site and move to a competitor: Forced account creation: If an agent cannot proceed without creating an account, it hands back to the human and the sale is lost.

Two-factor authentication and 3D Secure: Security barriers agents cannot pass.

CAPTCHAs and bot detection systems: Explicitly not something to remove — security and compliance take priority.

Cookie banners blocking content: An agent that cannot access content moves on.

Inconsistent product data: If price, availability, or specifications differ across schema markup, product feeds, and page content, the agent treats it as an error and abandons the transaction.

His warning about inconsistent data is particularly pointed: agents do not argue with messy data. They do not try to reconcile discrepancies. They move to a competitor that has cleaner data. In e-commerce, this is a silent and invisible source of lost sales.

He also notes that agents do not engage with dark patterns — manipulative design intended to confuse or trap users. They simply leave.

The Emerging Standards Landscape Korner introduces several emerging standards and protocols that will shape how agents interact with websites: The Agenti AI Foundation, hosted by the Linux Foundation and backed by Anthropic, OpenAI, Google, and Microsoft. These are companies that normally compete intensely, but they have aligned on a shared protocol for agent-to-website interaction. Korner describes it as "USB-C for agents" — a common interface so that any agent can plug into any website in a standardised way.

MCP (Model Context Protocol) server cards — a small JSON file that tells agents what a site can do. For developers comfortable with MCPs, this is a meaningful investment.

ACP (Agent Communication Protocol) and UCP — additional emerging protocol layers in this space.

Auditing Agent Readiness Korner recommends a free tool: isitagentready.com. Users enter any URL and receive a Lighthouse-style audit covering authentication flows, content accessibility, payment readiness, and other agent-facing factors. He guarantees that every site will fail the test — the question is by how much and in which ways. The report produces a list of fixable problems that can be packaged as a client service.

The Three Audiences Compared

Korner synthesises the three audiences into a comparison framework that is useful for planning.

• Primary goal: humans want an answer, comparison, or purchase; AI bots want to index, train, or cite; agents want to complete a task end-to-end.
• Arrival method: humans arrive via browser; AI bots via an HTTP fetch with no JS; agents via a headless browser such as Chromium.
• Latency tolerance: seconds for humans, moderate for AI bots, milliseconds for agents.
• What kills the session: for humans, slow load or poor UX; for AI bots, no HTML content or blocked crawlers; for agents, inconsistent data or auth barriers.
• Optimisation focus: content and UX for humans; HTML, robots.txt and schema for AI bots; schema, MCP and structured data for agents.

The latency point about agents is striking: if an agent is accessing a site via API and the response takes too long, it moves on. The threshold is in the hundreds of milliseconds, not seconds.

The Three Things That Matter: A Survival Plan

Korner offers a distilled three-part framework for surviving the hybrid web. He emphasises that these are simple to state but not simple to execute well.

1. Lead With the Answer AI engines are documented to surface direct answers 2.3 times more often than conclusions buried in text. Content structure must change. The answer or the core claim should come in the first sentence, possibly the headline itself. Long wind-ups, contextual preambles, and buried conclusions are invisible to AI systems that lift the first substantive statement they find.

2. Publish Original Data In a world where anyone can generate content with a voice prompt and ChatGPT, originality has become the primary differentiator. LLMs will not surface the same content twice. If a website is echoing its competitors — making the same claims, citing the same sources, drawing the same conclusions — it will not be cited. Sites need to be the source, not the echo. Original research, proprietary data, first-hand expertise, and genuine novel insights are what get cited. Vague, generic claims are ignored.

3. Clean, Connected Structured Data Schema markup — structured data that explicitly tells machines what a page contains, what an organisation does, what a product costs, who wrote an article — is the single most important technical investment for AI visibility. Properly implemented schema helps human users (better search result displays), helps AI bots (clearer indexing and citation), and helps agents (reliable data they can act on without guessing).

Korner's crucial observation: these three investments are not audience-specific. They serve all three audiences simultaneously. Great content gets cited by bots. Original data builds trust with agents. Clean schema gives humans better search results. This is the highest-leverage place to invest.

Yoast Tools: Practical Implementations

Korner closes with a disclosure that as a product leader at Yoast, he would be remiss not to mention relevant tooling — and then offers two specific product features.

AI Brand Insights Yoast's AI Brand Insights tool addresses the question: how is a brand being represented inside LLM responses right now? The tool: Generates five representative queries based on the brand and its description — questions that a user researching that brand or category might ask.

Sends those queries to the major LLMs (Claude, ChatGPT, and others).

Fetches the answers and runs them through Yoast's own AI analysis layer to extract brand mentions, ranking within answers, citations used, and sentiment.

Aggregates these into a single metric called the AI Visibility Index (AVI).

Korner notes that the tool often surfaces unexpected competitive intelligence: because AI bots draw from different sources than Google's crawlers, the competitors who appear in LLM responses are often different from those who rank on Google. Sentiment data can also reveal how to position or reframe a brand.

He shares an aside: the working prototype of this product was built via vibe coding in hours, then handed to real engineers and UX designers who built it into a proper SaaS product in a few months. He uses this as an illustration of how AI can accelerate product development when humans remain in the loop.

Schema Aggregation Korner describes Yoast's Schema Aggregation feature as one he is particularly proud of.

Built in collaboration with Microsoft and the founder of Schema.org, it is available for free to all Yoast SEO users (free and premium).

The feature works by treating a website's structured data not as a collection of isolated page-level schemas, but as an interconnected site-wide graph. Instead of each page having its own standalone schema block, Schema Aggregation connects the data across the entire site and consolidates it into a single output called the Schema Map.

The Schema Map is visually overwhelming to a human — a dense network diagram of all the entities, relationships, and attributes across the site. But it is precisely this richness that makes it valuable to AI. The map tells an AI system how articles relate to authors, how authors relate to the organisation, how products relate to categories, how services relate to location — the full relational structure of the site's content.

This makes it easier for AI systems to understand a website's authority and relevance, and makes the cost-benefit decision in favour of crawling and citing the site rather than moving on to a simpler source.

About the speaker Niko Korner Yoast

Niko Korner works with Yoast, focusing on SEO and how agencies can help clients navigate a web that increasingly blends traditional search with AI-driven discovery.