OpenAI’s web crawling activity has risen sharply since GPT-5 arrived in August 2025, according to an analysis by Botify and guest author Chris Long, with the company’s search-related bot now appearing more active than its training crawler in the firm’s enterprise log data. The shift suggests that OpenAI is leaning more heavily on live retrieval for some prompts, even as its overall crawl footprint still trails the biggest search engines by a wide margin.

Long, who co-founded the SEO consultancy Nectiv, examined about 7 billion OpenAI-bot log events drawn from Botify’s enterprise customer base between November 2024 and March 2026. In that dataset, OAI-SearchBot, which is used when ChatGPT searches the web, increased to roughly 3.5 times its previous level after August 2025, while GPTBot, OpenAI’s training crawler, rose by about 2.9 times. Botify says that before GPT-5 the two crawlers were running at near parity, but after the launch search activity moved ahead.

The strongest growth was concentrated in sectors where users are more likely to want current information. Healthcare sites recorded about 740% more OAI-SearchBot activity after the launch, while media and publishing saw a 702% increase. Marketplaces, software and retail also posted substantial gains, while travel showed a much smaller rise. Long and Botify say the pattern points to a split in how prompts are handled, with news and other fast-moving queries more likely to trigger live search, and health or product queries often drawing more on trained knowledge.

The dataset also showed a decline in ChatGPT-User log events, which fell 28% between December 2025 and March 2026. Because that user agent is triggered when ChatGPT fetches a page on behalf of a user, the drop may indicate fewer real-time page requests, though Botify’s team also suggested OpenAI could be relying more on stored or indexed material instead. Long did not choose between those explanations.

Even with the post-GPT-5 surge, OpenAI’s crawl volume remains far below Google’s. In Botify’s latest 30-day window, Googlebot accounted for 18.2 billion events, compared with 887 million from OpenAI’s crawlers combined, or about 4% of Google’s volume. A year earlier, OpenAI’s share was closer to 1.38%, which indicates the gap is narrowing, albeit from a very low base. Bingbot still logged far more traffic than OpenAI as well.

The findings fit with other recent reports suggesting that AI search crawling and AI training crawling are diverging. Earlier analyses from Alli AI, Hostinger and Akamai pointed to different patterns in how OpenAI’s bots are behaving across the web, while CNBC reported in August 2025 that GPT-5’s release boosted enterprise use cases and helped accelerate adoption in developer tools. Taken together, the reports suggest website owners may need to treat search-oriented AI crawlers as a distinct class of visitor rather than assuming that blocking training bots alone is enough.

Source Reference Map

Inspired by headline at: [1]

Sources by paragraph:

Source: Noah Wire Services