What a reCAPTCHA score actually means — and where to set your threshold

A Shopify merchant tuning their spam filter is adjusting a control with nine positions, of which roughly four produce distinct results. One of the nine blocks every submission the store will ever receive, customers included. The vendor help page generally says that 0.5 is a good starting point and moves on.

The number under that slider is a Google reCAPTCHA score. It is worth understanding precisely, for two reasons. Most wrong decisions merchants make about form spam begin with a wrong model of what the score measures. And the threshold debate is downstream of a fact established in our first dispatch: a solved reCAPTCHA v3 token sells for about $1.45 per thousand, so no setting on the dial makes a store unattackable. It sets a price and decides who pays it.

What the number is

Google's v3 documentation puts it plainly: reCAPTCHA "returns a score (1.0 is very likely a good interaction, 0.0 is very likely a bot)". The Cloud documentation adds the precision: the scale is quantised into eleven levels, running 0.0 to 1.0 in steps of 0.1. Until a billing account is attached to the project, a site receives only four of them: 0.1, 0.3, 0.7 and 0.9.

That is the most useful fact in this report and it is rarely repeated. An event log showing scores that are always one of four values is not broken. The store is on the free tier and Google is quantising. It also means most positions on the threshold dial are decorative, a point demonstrated below.

The score is not a probability. A 0.3 does not mean a 30 percent chance the visitor is human, and the four-value free tier is the proof: a function with four possible outputs is not estimating anything to two decimal places. Our working model, which Google does not publish and which should be attributed to us rather than to them, is that the score expresses similarity. It reports how closely an interaction resembles the traffic Google has learned to treat as legitimate on that specific site.

Google's Cloud documentation attaches reason codes to low scores. They are visible only on reCAPTCHA Enterprise, since a classic v3 verification returns a bare number and nothing else, but they indicate what the number is made of:

• AUTOMATION — the interaction matches the behaviour of an automated agent.
• UNEXPECTED_ENVIRONMENT — the event originated from an illegitimate environment.
• TOO_MUCH_TRAFFIC — traffic volume from this source is higher than normal.
• UNEXPECTED_USAGE_PATTERNS — the interaction differed significantly from expected patterns.
• LOW_CONFIDENCE_SCORE — too little traffic was received from this site to generate quality risk analysis.

Two of those codes allege automation. The other three, covering traffic volume, unexpected usage patterns and insufficient data, report unfamiliarity. A submission can be scored down for being unusual on a particular site without anything about it being robotic. That distinction is the one merchants most often miss.

Why it drifts, and why a new store's scores cannot be trusted yet

Look at that last reason code again: LOW_CONFIDENCE_SCORE. Google states that reCAPTCHA "learns by monitoring real traffic on your site", and that "scores in a staging environment and within 7 days of implementation might differ from the long-term production scores."

Note what that does and does not say. It states that the number is unreliable early on. It does not state which direction the error runs, and in our own logs it runs both ways. Some new stores score almost everything at 0.9, and the filter does nothing. Others block first customers who did nothing wrong. The result is a predictable failure mode.

The same effect explains scores that look wrong on established stores. A viral post sends unfamiliar traffic. A new market brings unfamiliar geography. A campaign lands a cohort of first-time visitors on a form. All unfamiliar, all scored down, none of them bots.

Drag the threshold

Below are twelve submissions of the kind a small Shopify store receives on an ordinary day, each carrying a score chosen to be plausible, sorted live against whatever threshold is set. Every score is one of the four values a free-tier key actually emits.

One detail matters more than it appears to. A submission scoring exactly the threshold is blocked rather than passed. Our own filter passes only when the score is strictly greater, and most implementations behave the same way. On a four-value key that boundary is hit constantly, so the rack applies the same rule.

Drag the dial and watch all three counters, along with the line above the list.

The rack makes the trade explicit. A threshold is not a quality setting on which higher is better. It selects between two kinds of failure, and the choice depends on what the form is for.

One dial, several jobs

A single store-wide threshold is the wrong shape for the problem. Google's guidance says as much, recommending that action be taken "in the context of your site" and varied by action. A blocked blog comment costs a merchant nothing. A blocked checkout costs a customer, and possibly the relationship.

The option this framing leaves out

Everything above treats the decision as binary: pass or reject. Google's own recommendation for the middle of the range is neither. It advises site owners to step up, by presenting a v2 checkbox, sending a confirmation email or holding the submission for review, and to take the action "in the background instead of blocking traffic."

That largely dissolves the dilemma this report is built around, which is why it belongs in the body rather than a footnote. A mid-scoring submission does not have to be gambled on. It can be asked one further question.

Spambuster does not do this today. The app passes or blocks, and allows a form to be switched to a v2 checkbox wholesale rather than conditionally. We regard graduated response as the right design and the single change that would most improve this category, our own part of it included. Until it exists, the threshold table above is the honest way to reason about a binary tool, and a step-up option should weigh heavily in favour of any competitor that offers one.

What scoring can't see

A score is one opinion, and Google does not publish what forms it. What we can report is what we observe. The spam reaching our customers with high scores attached tends to originate from fresh automation running on residential proxies, which is to say setups with no history for a reputation system to hold against them. Row six of the rack above is that case.

This is also where the solving market rejoins the argument. A purchased token certifies that a challenge was answered. It certifies nothing about the browser that submitted the form, and a service charging $1.45 per thousand is not going to simulate plausible mouse movement for each one. So a second and independent opinion is worth holding: what did the visitor's browser and body actually do?

Below are the signals our own detector collects, with the weights assigned to each. These are taken from our source rather than described in the abstract, because unfalsifiable claims about advanced AI detection are the reason this category is difficult to trust.

The cheapest layer in the stack

Before any of the above, there is a control that costs nothing, adds no latency and asks nothing of a customer: a field only a script can see.

Honeypots have one property worth more than it appears. They produce very few false positives. Almost no real customer completes a field they cannot see. The exception is password managers and browser autofill, which will populate an off-screen input if its name looks familiar, and that exception is the whole craft of building one. The trap field needs an unusual name and no recognisable autocomplete attribute, or the store will block precisely the customers organised enough to use a password manager.

Built correctly, it is the one layer that can run at full strictness without costing a sale, which is the reverse of the trade the threshold dial imposes.

Honeypots also have a hard ceiling. A bot written for one specific form, rather than for forms in general, will skip the trap. None of them stop the human sending guest-post pitches. This is the recurring finding of our first dispatch: every control in this category addresses one route and leaves the others open.

The setting nobody asks about

One final matter, and it is the question we would put to any vendor in this category ahead of the pricing question.

What happens when verification fails?

Every scoring system depends on a network round trip, and networks fail. When one does, the filter must choose between rejecting a submission it could not verify, which is to fail closed, and allowing it, which is to fail open.

Most vendors answer this as though it were one question. It is two, and they warrant different answers. Ours follow, in specific terms, because a vague answer is of no use to a merchant.

• If our own backend is unreachable, we allow the submission. A blip in our infrastructure must never stand between a customer and your contact form. The storefront script gives up, lets the form submit, and beacons the event into a daily report so we can see it happening.
• If Google returns a hard verification error, we block, and the customer is told why. They see "Spam protection could not be verified. Submission blocked." That is fail-closed, deliberately: a verification error usually means a broken or expired key rather than a network blip, and silently passing everything through a broken filter is how a store discovers three weeks later that it has had no protection at all.

The distinction that matters is not open against closed. It is visible against silent. The first failure is ours, is invisible to the customer, and resolves itself. The second is visible, tells the customer something is wrong, and is recoverable: they can retry, or email the store directly. What no merchant should accept is a filter that fails closed silently, rejecting real orders with no error that anyone ever sees.

Any vendor in this category should be able to say which of those two behaviours applies, and to which failure. Ours are stated above. A competitor unable to point at the equivalent paragraph in their own documentation has answered the question in a different way.