AI Browser Agents (Atlas, Comet) and What They Mean for Your Website

Yes, AI agents browsing your website on a real person's behalf are already a measurable part of your traffic, and it does change a few specific, practical things about how you should build and maintain a site, though not in the alarmist way most coverage suggests. ChatGPT Atlas and Perplexity Comet both launched within months of each other in 2025 and let an AI act on a page, filling a form, comparing options, completing a purchase, not just reading and summarizing it. Within nine days of Atlas's launch, more than a quarter of enterprises already had an employee using it, and Cloudflare's own network data now shows more than half of all HTML traffic on the internet is non-human. This article covers what agentic browsing actually is, how big it already is, and the specific build decisions that matter, without the hype or the panic.

If you want your site audited and fixed for how AI agents actually read and act on it, see how we do AI product integration.

What is an AI browser agent, exactly?

An AI browser agent is a browser with an AI built into its core that can take actions on a webpage, not just describe what it sees. ChatGPT Atlas, built on Chromium and launched by OpenAI on October 21, 2025, ships with a sidebar assistant that reads and summarizes the current page, plus an "agent mode" that lets the AI navigate, click, and fill forms on a user's behalf. Perplexity's Comet, which launched earlier, on July 9, 2025, works the same way: point it at a task ("find me the cheapest flight matching these dates" or "add these three items to my cart") and it drives the browser itself, checking back with the user at decision points rather than requiring a click for every step.

The distinction that matters for a site owner is the one between a browser that answers questions about a page and one that acts on the page. A search engine or a chat assistant summarizing your content is not new. An agent that fills out your contact form, adds items to a cart, or navigates your account settings, on behalf of a real customer who told it to, is the new part, and it interacts with your site the way a human would, through the same forms, buttons, and flows.

How big is agentic browsing already?

Bigger than most site owners have registered, because it does not show up in analytics the way a normal traffic spike does. Within nine days of ChatGPT Atlas launching, Cyberhaven Labs' security research found that 27.7% of enterprises already had at least one employee who had downloaded it, reaching 1.7% of corporate macOS endpoints, concentrated in technology (67% of users), pharmaceuticals (50%), and finance (40%). Atlas drew roughly 62 times more corporate downloads than Comet had reached at a comparable stage after its own launch, though Comet's downloads still jumped sixfold during Atlas's launch week, a sign the whole category was gaining attention, not just one browser winning at the other's expense.

Comet itself has kept growing. It added a Comet Enterprise tier on March 17, 2026, with admin deployment tools and named adopters including Fortune, AWS, AlixPartners, Gunderson Dettmer, and Bessemer Venture Partners, and its consumer app hit #3 on the US iOS App Store within 48 hours of going free and mobile that same month.

Zoom out from either browser individually and the trend is starker. Cloudflare's network-wide data, published July 1, 2026, found that more than 50% of HTML web traffic is now non-human, the first time bots have outnumbered people on the open web by that measure. Automated traffic overall is growing roughly eight times faster than human traffic, and agentic AI, an autonomous program browsing on behalf of an assistant like ChatGPT or Gemini, is the fastest-growing slice of that.

What is the difference between ChatGPT Atlas and Perplexity Comet?

Less than the marketing suggests. Both are Chromium-based browsers with an AI assistant built into the core experience rather than bolted on as an extension, both can summarize and answer questions about the page you are on, and both have an agentic mode that can take multi-step actions across a site on your behalf. The differences are mostly about ecosystem and rollout: Atlas is tied to OpenAI's ChatGPT distribution and reached enterprise adoption faster out of the gate, while Comet is tied to Perplexity's search-first product and has leaned harder into enterprise security integrations, including a CrowdStrike Falcon partnership that can flag or block an agent action mid-task if it detects a phishing or data-exfiltration risk.

For a site owner, the practical point is that you are not choosing between optimizing for "one AI browser," you are dealing with a category. Whatever you do to make your site legible and actionable for one agentic browser tends to help with the others, because they all read and act through the same underlying web: HTML, forms, and structured data.

Why should you care about "bots" that aren't training crawlers?

Because not all AI traffic hitting your site wants the same thing, and treating them all the same, whether by blocking everything or allowing everything, gets the decision wrong in both directions. Cloudflare's own bot-classification framework, and OpenAI's own developer documentation, both draw the same three- or four-way distinction:

• Search bots (like OpenAI's OAI-SearchBot) index your content so it can be surfaced and cited in AI answers. You generally want this on, the same reason you want a traditional search engine to crawl you.
• Agent bots (like OpenAI's ChatGPT-User, which fires specifically when a real person, inside a live session, asks the AI to visit or act on a page) represent an actual customer trying to get something done on your site. Blocking this can mean blocking a real sale or a real support request, wearing a bot's clothes.
• Training crawlers (like OpenAI's GPTBot) harvest your content to train or fine-tune models. This is the category most site owners actually want to make a deliberate decision about, separate from the other two, since it is a business tradeoff (visibility and potential future citation versus your content being used for training) rather than a customer-facing concern.

Cloudflare found that AI training-crawler requests grew from 22% of all crawler activity in spring 2025 to 52% by June 2026, and that mixed-use crawlers, ones blending search, agent action, and training in a way that is hard to distinguish from the outside, now make up more than 36% of activity. That ambiguity is exactly why a blanket "block all AI bots" or "allow all AI bots" setting is the wrong tool: you end up either turning away real customers using agent bots, or quietly opting every page into model training you never agreed to.

Does agentic browsing change how you should build your website?

In a few specific, testable ways, yes, and none of them require a rebuild. The core requirement is the same one that has always mattered for accessibility and SEO, just with a new audience: your important content and functionality need to work without depending on a human physically seeing and clicking through a specific visual layout.

• Render critical content server-side. Prices, availability, product details, and anything a customer needs to complete a task should be present in the raw HTML, not injected only after client-side JavaScript runs. Agents that do not execute every script the way a human's browser does will simply miss content that only appears post-render, the exact same failure mode that has always hurt traditional crawlers.
• Keep forms simple and standard. An agent filling out a form works by identifying standard input fields and labels. Highly custom, JavaScript-only form widgets that do not use real form elements are harder for both accessibility tools and agents to operate correctly.
• Structure your data. The same schema.org and structured-data practices that help you get cited in AI search answers also help an agent understand what it is looking at and acting on, whether that is a product, a pricing table, or a support article.
• Publish a clear robots.txt, and mean it. OpenAI's own documentation notes changes take about 24 hours to propagate, and different bots (search, agent, training) respond to different rules. A stale or overly blunt robots.txt is a common, easy-to-fix source of an agent failing to complete a legitimate task on your site.

None of this is a new discipline invented for AI agents. It is the same discipline that has always separated a well-built site from a fragile one, applied to a new class of visitor that happens to act instead of just reading.

What about CAPTCHAs and forms?

This is where "agent-friendly" gets genuinely tricky, because CAPTCHAs exist for a real reason (blocking abusive automated traffic) and a legitimate customer's AI agent looks, technically, a lot like the traffic a CAPTCHA is designed to stop. The emerging answer in the industry is not "remove your CAPTCHAs," it is verified bot identity: an approach (an IETF draft standard called Web Bot Auth is one example already in preview on major cloud platforms) that lets an agent present a cryptographic signature proving it is operating on behalf of a specific, real, opted-in user, so a site's bot protection can let it through without a visual challenge a machine cannot solve.

You do not need to implement cryptographic bot verification yourself to act on this today. The practical step is auditing where your CAPTCHAs and aggressive bot protection sit in your actual conversion path. A CAPTCHA on account creation is a reasonable, low-cost friction point. A CAPTCHA blocking every step of an already-logged-in customer's checkout is a place where you may be turning away a legitimate agent-assisted purchase without realizing it, especially as agentic commerce volume grows (see our piece on agentic commerce for how much of that volume is already real).

What is the actual security risk here?

The most concrete, documented risk with agentic browsers is prompt injection: malicious content embedded in a webpage that tricks an AI agent into taking an action its user never intended, or leaking data it should not. Security researchers at Cyberhaven Labs flagged this as the primary unresolved risk of agentic browsers when Atlas launched, and it remains the main open concern across the category. For most site owners, this is primarily a risk that sits with the person running the agent and the company that built it, not with you. But it is worth a moment's thought about whether your own pages, especially anything with user-generated content, comment sections, or embedded third-party widgets, could be a place a bad actor plants instructions aimed at hijacking someone else's agent. Treat it the same way you would treat any other user-generated-content risk: sanitize and moderate what you let strangers post on pages a legitimate agent might read on a customer's behalf.

How should you decide what to allow?

Start from intent, not from a blanket rule. Ask, for each category: does allowing this traffic help a real customer or a real prospect? Search bots, generally yes, since being findable and citable in AI answers is a discovery channel you want. Agent bots, generally yes, for the same reason you would not block a customer's browser, an agent acting on a real person's behalf is that person, functionally. Training crawlers, that is a judgment call specific to your business: some publishers now monetize this directly through pay-per-crawl arrangements rather than blocking it outright, while others opt out entirely to protect proprietary content. What you should not do is leave the decision to whatever your hosting or CDN defaults to without checking, since major platforms are actively changing their defaults (Cloudflare, for instance, is shifting new domains to block Training and Agent bots on ad-supported pages by default starting September 15, 2026), and a default set for someone else's business model may not match yours.

How to get started this quarter

Check your server logs or CDN dashboard for the specific AI user-agents (OAI-SearchBot, ChatGPT-User, GPTBot, PerplexityBot, and similar) hitting your site today, since most site owners are surprised by how much of this traffic already exists once they actually look. Then run your three or four most important pages, your pricing page, your main product or service pages, your contact or checkout flow, through a check for whether the core content and functionality survive with JavaScript disabled, which approximates what a less-capable crawler or agent sees. Fix what breaks, set a deliberate policy on search, agent, and training bots rather than an accidental default, and revisit your CAPTCHA placement on any step a real, logged-in customer might complete through an agent. None of this requires predicting exactly how big agentic browsing gets. It requires building the way good sites have always been built: content that works without depending on a specific human clicking through a specific screen.

If you would rather have this audited and fixed for you, that is what we do. We check how your site actually renders and behaves for AI browser agents and AI search, fix the gaps that are costing you visibility or blocking real customers, and set a bot policy that matches your business instead of a platform's default. Book a free consultation and we will show you what an agent sees when it visits your site today.