The Numbers Behind Your Decisions Are Probably Wrong

Over four weeks, this site appeared in Google's search results about 27,000 times, at an average position near the top of the first page. If someone handed me that on a slide, I would nod. It sounds like the site is winning. It is worthless, and I can prove it in one line: those 27,000 appearances produced two clicks.

TL;DR: A number is not a result until you know what produced it. Most of the traffic and search figures a small business looks at are measuring bots, the owner's own laptop, or nothing at all, and the good-looking ones mislead just as often as the bad-looking ones. Before you act on a number, find out what question it answers, how many events it is built on, and whether the thing doing the measuring is actually installed on the thing being measured.

What Do Search and Analytics Numbers Actually Measure?

There are two completely different families of numbers here, and they get mixed together constantly.

Search numbers come from the search engine, through a free tool called Google Search Console. They tell you how often your pages were shown in results (impressions), how often somebody clicked (clicks), and roughly where you appeared on the page (average position). These are about visibility in search, not about visits.

Analytics numbers come from a small script on your own pages that fires when a browser loads them. They tell you how many visits and pageviews you actually got, from where, on what device. These are about people arriving.

A third family sits in the middle: edge or server numbers, counted by whatever network sits in front of your site. Those count every request that reaches your domain, including automated ones.

Three sources, three different definitions of a "visit." None of them is lying. They are answering different questions, and the trouble starts when someone picks whichever answer is biggest.

The Number That Looked Like A Win

Back to the 27,000.

When I pulled it apart by query, nearly all of it came from one cluster of near-identical phrases: a garbled misspelling of my own company name, with the words in the wrong order and a stray space in the middle of the domain. One variation alone accounted for over eleven thousand of those appearances. The others were the same mangled string with "about," "team," and a business-y noun bolted on.

Nobody types that. Not once, let alone eleven thousand times in a month. Some automated crawl is running through a list of mutated brand names, and my site is the closest match Google can find for a phrase no human ever searches. So it gets shown, ranks fifth, and gets ignored, because there is no person on the other end to ignore it.

Ranking near the top for a phrase nobody searches on purpose is worth exactly nothing. The clickthrough rate was zero to two decimal places, which is the tell. Real visibility for a real phrase produces clicks. This produced an impressive-looking chart.

The rule I took from it, and the one this whole post rests on: a number is not a result until you know what produced it. Not what it measures in theory. What actually generated those specific events.

The Same Mistake, Pointed The Other Way

Here is the part that surprised me more.

Another site I run showed an average position of 1 for one valuable search phrase and 3 for another. Top of page one for terms that matter, on a site that was days old. That is the kind of number that would make you double down on a strategy.

Both were built on a single appearance. One impression each. The site was shown once, to one person, on one afternoon, and the "average position" is the arithmetic mean of a set containing one item.

I checked by hand a few days later. For that phrase, page one was three established national competitors, and my site was nowhere in the top ten. Nothing had gone wrong. Search engines briefly test-serve new sites into results to see what happens, then pull them back. The single impression was that test, not a ranking.

So the impressive number and the disappointing number were the same kind of thing: artifacts. One was inflated by bots, one was inflated by a sample size of one. In both cases the underlying reality was "nothing is happening yet," and in both cases the dashboard said something more exciting.

Any average built on a handful of events is noise, no matter how good it looks. That is not a search-engine rule. It is the reason a single five-star review does not make a restaurant the best in town.

What Was Actually Working Was Boring

There was a real signal in that second site's data. It was not the homepage, which I had spent the most time on, and it was not any of the phrases I had been hoping to rank for.

It was three narrow, specific pages. Deep in the site, each covering one tight sub-category with concrete numbers on it. They held a consistent position in the twenties and low thirties across a couple of dozen appearances each. One of them produced the site's only genuine click from a human being in four weeks.

Position 22 is not a headline. It is also the only number in the whole export that was made of enough events to mean anything, and it pointed somewhere I would not have guessed.

For a business, that pattern generalizes uncomfortably well. The pages you are proudest of are usually not the ones earning anything. The homepage, the About page, the beautifully written overview of your services - those are the ones that get all the attention internally and rarely the ones that pull anyone in. What pulls people in is the page that answers one narrow, specific, slightly boring question that somebody actually typed. You find those by reading the data honestly, and they are almost never where you expected.

Two Counters, Two Different Questions

Once I had analytics running on my sites, I had a second set of numbers to compare against. They did not agree. The counts from the network in front of the sites ran orders of magnitude above what the on-page analytics reported. Not ten percent off. Different by a factor of hundreds.

Neither one is broken. The edge counts every single request that touches the domain: search crawlers, uptime checks, security scanners, feed readers, people probing for vulnerabilities, every subdomain including internal tools that no customer ever sees. It counts requests, and requests are not people. The analytics script counts browsers that loaded a page and ran a bit of JavaScript, which is a decent approximation of a human.

So the edge number is a ceiling, not an audience. It tells you how much noise your domain absorbs. The analytics number is a floor, since it misses anyone blocking scripts. The real figure sits between them, closer to the floor.

The failure mode is not misunderstanding either number. It is picking whichever one is larger when you need to feel good, or whichever is smaller when you need to explain a bad quarter. If you have ever seen "hits" quoted in a marketing report, that is this exact move. Decide which question you are asking before you look at the answer, and use the same source every month, or your trend line is measuring your source selection rather than your business.

You Are In Your Own Data

On one particular day, the network in front of one of my sites recorded roughly 1,080 requests from a single machine. That machine was my development box, sitting on my desk, twenty feet from where I am writing this. The site had 156 real unique visitors that day. A further couple of hundred requests came from an automated scanner hunting for a well-known content-management vulnerability the site does not even run.

Add it up: my own laptop generated close to seven times the request volume of the entire actual audience.

Unfiltered, that log does not measure the business. It measures the operator. Every time I refreshed a page to check a change, every automated test run, every build that fetched the homepage to confirm it came up - all of it landed in the same bucket as a genuine visitor and got counted the same way.

This is not a small-site curiosity. It is the default condition of every small site. If you run a business with a website, a modest amount of traffic, and a team of any size, your analytics are substantially a picture of your own staff plus the internet's background radiation of bots. The office checking the new landing page thirty times on launch day is a meaningful percentage of launch day.

The fix is unglamorous and it is the single highest-value thing on this list: exclude your own network and your own machines, exclude your known automation, and filter obvious bot traffic. Everything downstream of that - conversion rates, bounce rates, "which page is performing" - is arithmetic on a contaminated sample until you do.

The Measurement That Was Not Measuring

Then there is the failure that beats all of the above, because it does not produce a wrong number. It produces a plausible one.

I keep several sites running. When I audited the analytics setup across them, three out of seven had an account configured, a property created, a dashboard entry with a name and an ID, and no tracking script on the actual live page. Two of them had been in that state since the day the analytics system was built.

Those three reported zero visitors. Zero is a perfectly reasonable number for a small site. It is also completely indistinguishable from "the measurement was never connected." Nothing in the dashboard says "not installed." It says zero, in the same font as a real zero, and I would have kept reading it as bad performance for months.

There is a related trap sitting right behind it. When I first tried to verify that tracking worked, I used an automated browser to load the page - the sensible engineering move. The analytics system correctly identified an automated browser as a bot and silently discarded the event, exactly as designed. It answered "success" and recorded nothing. So my verification reported "tracking is broken" on a system that was working perfectly, which is the same class of error in the opposite direction: a check that cannot succeed will report failure forever, and you will go fix something that was never wrong.

The honest consequence of all this, which I am putting in writing because it is the sort of detail that quietly disappears from a report: the first visit ever recorded on this site's analytics was a replayed test request I sent myself while proving the pipeline worked. It is a real row in a real database and it is not a person.

What To Do Instead

None of this requires new software. It requires a small amount of suspicion applied at the right moments.

Know which question each number answers. Impressions are not visits. Requests are not people. Sessions are not customers. Write down, once, which source you use for which question, and stop switching.

Distrust any average built on a handful of events. Ask how many things went into a rate or a position before you react to it. A stellar average over five events is a coin flip with a nice font. This is the same reasoning that makes a single bad month meaningless and a six-month trend worth a meeting.

Exclude yourself. Filter your office network, your team's machines, your own testing, and known bots before anyone builds a plan on the output. Most small businesses have never done this, and it is usually the largest single correction available.

Check that the thing measuring is actually installed on the thing being measured. Not once at setup - on a schedule. Systems get rebuilt, templates get copied, a security policy gets tightened and quietly blocks the script. A dashboard entry existing is not evidence that anything is being recorded.

Make your checks report how much they looked at. This is the general principle underneath all of it, and it applies far outside analytics. "Zero problems found" and "the check never ran" produce identical output. A backup report that says "no errors" while backing up nothing looks exactly like a healthy backup. A scanner that found no vulnerabilities because it failed to start looks exactly like a clean environment. A monitoring result is only evidence when it tells you the size of what it inspected - files copied, pages checked, records compared. If a green result cannot also tell you what it examined, treat it as unknown rather than good. That habit has caught more real problems for me than any alert ever has, and it is the same instinct behind auditing a clean recovery instead of celebrating it.

I run all of this on the same self-hosted stack this site sits on, using ordinary open-source tools, which is largely why I was able to take the numbers apart rather than accept the summary. The point is not the tooling. It is that measurement is a system like any other, with its own ways of failing quietly, and it deserves the same scrutiny you would give a backup or a firewall. Most of my automation work ends up here eventually: not building a new dashboard, but establishing whether the existing one is telling the truth.

If you are making decisions off a report and you are not certain what is actually in it, that is worth an hour on a calm afternoon rather than a discovery in the middle of a bad quarter. Get in touch and we can go through what your numbers are really counting.