How Growth Radar grades a change

Every change is tied to one number it was meant to move, such as clicks from Google to a page. When the change is recorded, Growth Radar notes where that number stood before it, so the result is never judged against a memory.

Traffic rises and falls for reasons that have nothing to do with your change: the season, a search engine update, a news story. A plain before-and-after would credit your fix with all of that.

So a changed page is compared with similar pages on the same site that did not change over the same weeks. It counts as a win when it did clearly better than those pages, and as a loss when it did clearly worse.

Search data arrives slowly and moves noisily from day to day. Most changes are graded once 28 days of data from after they went live are in; a fix for something like a broken link can be checked sooner. Until then it reads “too early”. Along the way you may see an early read showing which direction the number is moving, and it is labelled as not a result yet.

The verdict is one of four: won, lost, flat, or inconclusive. A win is checked again later to see whether it held.

Sometimes the honest answer is that nobody can tell from the data, and Growth Radar says “inconclusive” instead of guessing. The usual reasons:

• The page gets too little traffic for a real change to stand out from normal ups and downs. Small sites hit this often.
• The change touched the whole site, so there may be nothing untouched to compare it with.
• The number it targeted is one Growth Radar does not yet have a tested way to grade.

When the data is simply missing, the result is “no data”, and it is never counted as a loss. An inconclusive result is shown with its numbers, so you can judge it yourself.

A win means the changed page did clearly better than similar untouched pages over the same weeks. It is strong evidence, not proof that your change was the only cause, and a result on one site does not promise the same result on another.

Visits from known bots are left out of visitor numbers, but no filter is perfect. If something on this page does not match what you see in the app, tell us.