Your analytics dashboard says 4,000 visitors this month. How many of them were people?

For most small sites the honest answer is: fewer than you think. A growing share of every request to your server now comes from machines. Search crawlers, SEO tools, AI crawlers training models, AI assistants fetching a page because someone asked a question, link preview bots, uptime monitors, scrapers running headless Chrome on cloud servers. Some analytics tools quietly count a slice of these as visitors. Others quietly drop them. Either way, you never find out which, and you can't check the number.

I build analytics software, so I spent a lot of time looking at raw server logs next to dashboard numbers. Here is what I learned about telling people from machines, and what you can do about it on your own site.

Not all bots are the same problem

It helps to sort automated traffic into a few buckets, because each one needs different handling.

Bots that say who they are. Googlebot, Bingbot, GPTBot, ClaudeBot, CCBot, AhrefsBot, SemrushBot and friends all send an identifying user agent. These are the easy ones. The catch is that anyone can copy a user agent string. A scraper calling itself Googlebot is common, so a claimed identity should be checked against the published IP ranges or a reverse DNS lookup before you trust it.

Scripts and headless browsers. curl, python-requests, Puppeteer, Playwright. More and more of these send a perfectly normal Chrome user agent. What they rarely bother to fake is everything else a real browser sends alongside it: the full set of client hints and Accept headers, a sensible language, consistent platform details.

Data centre traffic pretending to be a laptop. A request from an AWS or Hetzner IP that claims to be Safari on a MacBook, loads the page, and never scrolls, taps or moves. Real people do browse through VPNs and cloud relays (iCloud Private Relay is a big one), so the IP alone is not enough. Combined with zero interaction, it is a strong signal.

AI assistants fetching on behalf of a person. This is the new and interesting one. When someone asks ChatGPT or Perplexity a question and it reads your page to answer, a machine made the request, but a human is behind it. That is not a visitor, but it is not noise either. It is closer to a citation, and it is worth tracking separately.

Four cheap tests, in order

You don't need machine learning or invasive fingerprinting to filter most of this. A sequence of cheap checks, from cheapest to most expensive, gets you a long way:

  1. Does it announce itself? Match the user agent against a maintained list of known bots and crawlers. Verify the big ones (Google, Bing, Apple, OpenAI) by IP range or reverse DNS. Anything claiming to be Googlebot that fails verification is a fake.
  2. Does it send real browser headers? Check that the headers a modern browser always sends are actually present and consistent with each other.
  3. Was somebody there? Did the page actually render and become visible? Was there any input at all: a scroll, a tap, a key press, a mouse move, or a couple of seconds with the tab in front? Headless browsers render pages. They don't fidget.
  4. Is it a data centre? Look the IP up against known hosting ranges. Count it as a visitor only if it also passed the interaction check.

None of these needs a cookie or a stored fingerprint, which matters if you care about privacy and want to avoid a consent banner.

The part most tools get wrong: they don't tell you

Filtering is half the job. The other half is showing your work.

A dashboard that says "88 visitors" and a dashboard that says "88 visitors, plus 2,470 requests refused: 1,320 AI crawlers, 410 search engines, 260 SEO tools, 178 scripts, 96 link previews" are reporting the same measurement. Only the second one can be checked. If the filter is too aggressive or too loose, you can see it. If a new AI crawler suddenly shows up hitting every page, you can see that too.

Storing refused traffic as simple daily counters (date, kind, name, page, count) is enough. You don't need to log every request, and it keeps the privacy story clean.

What you can do today

Even without changing tools, a few things help:

  • Compare your dashboard with your server logs for one day. Count the requests per user agent. Most people are surprised by how much of the total is GPTBot, ClaudeBot, AhrefsBot and friends.
  • Decide what AI crawlers may do. Training crawlers (GPTBot, CCBot, ClaudeBot) and on demand fetchers (ChatGPT-User, Perplexity-User) are different things. Plenty of sites block the first group in robots.txt and allow the second, because a real person asked a question there.
  • Track AI referrals as their own channel. Visitors arriving from chatgpt.com, perplexity.ai or claude.ai behave differently from search visitors, and that channel is growing fast. Lumping it into "referral" hides it.
  • Be suspicious of sudden traffic spikes with zero engagement. A jump in visitors with no change in signups, time on page or scroll depth is very often a bot that slipped through.

Why I care about this

I ran into all of this while building ConvertDash, a cookieless analytics tool that applies exactly these checks and lists every refused request by name, and ConvertNow Analytics, a free WordPress plugin that brings the same idea into wp-admin. But the ideas above work regardless of which tool you use.

The short version: your visitor count should be a number you can check, not one you have to trust. If your analytics can't tell you what it filtered out, it is worth asking why.