AI Detector Scores: What the Number Can and Cannot Prove

ai detector scores hero 1

AI Tools

AI Detector Scores: What the Number Can and Cannot Prove

By Earn Kit Lab TeamUpdated November 202611 min read

Freelance writer reviewing a document next to a detector score badge

☰ Table of Contents

A client forwards a screenshot with no message attached. Their content platform flagged your article at 87% AI-generated. You wrote it yourself, badly enough and slowly enough that you remember the paragraph you rewrote four times. Now you are being asked to prove a negative with a number someone else produced.

That screenshot is the whole problem with AI detector scores. The figure looks like a measurement. It is a classifier output from a model nobody will show you, trained on text that is not your text, answering a question no one can answer from a file: who typed these words. This guide covers what the score is actually computing, a twenty-minute calibration you can run on your own writing, a client reply that ends the argument without a counter-screenshot, and the contract line that prevents it from happening next time.

Why a Percentage Is Not a Verdict

Run one paragraph through three well-known detectors and you will usually get three different answers, sometimes spanning the entire range. One says 12%, one says 64%, one says 91%. All three vendors publish accuracy figures that look respectable on their own benchmarks. All three can be telling the truth about their own tests while being useless on your paragraph.

The reason is structural. Authorship is not a property of text the way word count is. Nothing in a sentence records who produced it. What a detector can observe is regularities: how predictable each word is given the ones before it, how uniform sentence lengths are, how often certain phrases recur. From those regularities it estimates a probability. The estimate is then displayed as a percentage, which makes a guess look like a measurement.

Treat the score the way you would treat a smoke alarm, not a fire inspection. It is a signal worth glancing at. It is not a finding, and it should not be the deciding evidence in a payment dispute, a rejection, or an accusation.

What the Detector Is Actually Reading

Most consumer detectors lean on two old measurements, dressed up with newer classifiers:

  • Perplexity — how surprised a language model is by each successive word. Predictable phrasing scores low. Unusual phrasing scores high. A writer with a plain, efficient style looks identical to a machine following a plain, efficient prompt.
  • Burstiness — the variance in sentence length and shape. Human drafts tend to swing between a three-word sentence and a forty-word one. Machine drafts tend to hover. But a legal clause, a product description and a lab report also hover, because the genre demands it.
  • Classifier layers — newer tools train a model to separate labelled human and machine samples. These generalize better than perplexity alone and are still bounded by whatever was in the training set. Their behaviour changes when the vendor retrains, which is why the same document can score differently in March and in November.

None of these measurements know what your article means. A translated paragraph, a paragraph written by a careful non-native English speaker, and a paragraph produced by a model all share the property of being statistically tidy. The tool cannot tell those three cases apart. It only sees the tidiness.

The Variance Problem: Same Text, Different Numbers

Before you argue with a score, know that the score moves for reasons that have nothing to do with authorship. These are the shifts documented repeatedly in public testing and in vendor changelogs. Ranges are approximate; treat them as direction, not precision.

What changed Typical effect on the score Why it happens
Vendor retrained its model Large swing, either direction New training data new decision boundary. Your text did not change.
You submitted only two paragraphs instead of eight Often double-digit swing Too little text for the statistics to stabilise. Short samples are the least reliable.
You pasted plain text instead of formatted rich text Small to moderate shift Formatting characters and heading case leak into the feature set.
The sample includes quoted material or a long blockquote Scores rise sharply Quoted text was written by someone else and is often edited for uniformity.
The text is a translation or written in a second language Scores rise, sometimes above 80% Non-native phrasing is more regular, not less human.
Detector A versus detector B on identical input Frequently 40+ points apart Different models, different thresholds, different definitions of the label.

Two detector tools showing opposite scores for the same block of text

That table is your leverage in any conversation about a flagged document. If the number can move forty points without a single word changing, it cannot be the standard of proof for whether a person wrote something.

A Twenty-Minute Calibration You Can Run Yourself

Do not run this test to prove a point to a client. Run it so you know your own false-positive baseline, which is the number you will need the next time a platform rejects your work. Collect three samples and put each through three detectors, twice: once whole, once with the first and last paragraph removed.

Sample to test What it reveals
Something you published before 2019, long before consumer language models existed Your personal false-positive rate. If genuine old writing lands at 30%, every future flag on your work should be read against that floor.
A paragraph from a government page, a statute or a standards document How the tool handles formal, uniform prose. Expect high scores. No human wrote it with a model, and the tool still cannot tell.
A block you generate on purpose, from a single prompt, unedited Your ceiling. Note how far below 100% it lands. Tools that miss their own easy case will miss yours.

Write the six numbers down. Two patterns show up almost every time: the pre-2019 human writing does not score zero, and the short samples are the wildest. Those two facts settle most disputes faster than any argument about whether a detector is “accurate”.

The number that matters mostIf your own pre-2019 work already scores 25% on a tool, then a 40% result on this year’s article tells a client almost nothing. Establish that baseline before anyone accuses you of anything, while the conversation is still calm.

What Moves a Score, and What Only Feels Like It Should

Freelance forums are full of tricks for pushing a score down. Sort them into two piles.

Things that genuinely change the number, because they change the writing:

  • Cutting the boilerplate opener. Three sentences of scene-setting before the point are the most predictable text in any article, and predictability is what the tool is measuring.
  • Varying sentence length on purpose, and letting one sentence run long because the idea needs it.
  • Replacing generic transitions with the actual relationship between the two ideas.
  • Adding one specific, checkable detail per section: a date, a version number, a quoted figure, the exact name of the thing that broke.

Things that feel like fixes and are not:

  • Deleting every em dash. It is a myth with a long life. Punctuation marks are a rounding error in the feature set.
  • Adding deliberate typos. Some tools treat error rate as a human signal, so this sometimes works. It also makes you a worse writer and gets caught by the editor, not the detector.
  • Running the text through a “humanizer”. The output is generated by a model, which means you are now submitting machine-written text to a machine detector, edited to defeat that specific detector. It breaks on the next retrain, and it changes your meaning in ways you will not notice.
  • Invisible characters or hidden text. On a content platform this is closer to misrepresentation than formatting, and it is the kind of thing that gets an account closed rather than a document approved.

Editor marking a sentence by hand with a margin note beside a checkmark screen

The honest version of the advice is dull: writing that a detector scores low tends to be writing with specifics in it. That is the same property editors pay for. Chasing the score and chasing quality happen to point in the same direction, as long as you stop before the tricks start.

When a Client Sends You a Screenshot

How you reply in the first message decides whether this is a five-minute clarification or a two-week dispute. Do three things, in order.

  1. Acknowledge without conceding. “I see the flag, thanks for sending it through.” Nothing about guilt, nothing about the tool being garbage yet.
  2. Ask what decision the number is driving. This is the important question. If the answer is “we need to be sure it is original and accurate”, you can satisfy that with evidence. If the answer is “our platform will reject it”, you need a different conversation, about the platform’s policy, not your draft.
  3. Offer evidence instead of a counter-score. A second detector saying 8% just starts a screenshot war and implies the number was ever the point.

Two lines that work in practice:

“Detector output varies a lot between tools and between model versions. I can send the draft history, the source notes and the brief so you can check the work itself. Happy to adjust any section you think reads thin.”

“If a specific detector score is a hard requirement, can we agree which tool and which threshold before I start the next piece? I can write to a house style, but I cannot guarantee a third party’s classifier output.”

Then fix the contract, not the article. A clean clause says the deliverable is judged on the agreed brief, originality and accuracy, and that third-party classifier results are not a guaranteed acceptance criterion. Freelancers in the US, Canada and the EU increasingly include a line like this, and the clients who push back hardest on it are usually the ones who were going to argue about the invoice anyway. This is a commercial point, not legal advice; if a contract is worth real money, have a lawyer read it.

The same discipline applies to how you vet the tools themselves. If you are deciding whether a detection subscription is worth paying for, run it through the kind of payback test we use in our seven-day AI tool audit, and if you are comparing vendors, use a source-first comparison rather than the marketing pages.

The Defence Pack That Beats Any Percentage

You cannot prove authorship with a number, but you can produce a record that makes the question go away. Build it as you work, not after the accusation. Fifteen minutes per project is enough.

  • Version history. Export or screenshot the revision list from your editor at delivery: timestamps, word counts, the order the sections appeared. A document that grew over nine sessions looks nothing like a paste.
  • The outline before the draft. One page, dated, in your own shorthand. This is the single strongest artefact, because it shows thinking that happened before any prose existed.
  • Source notes with links. What you read, when you read it. This doubles as your fact-check trail and protects you if a claim is challenged later.
  • The interview or briefing record. A call note, a voice memo, an email thread where the client gave you the angle. It shows where the ideas came from.
  • The rejected paragraphs. Keep the version where you tried a different structure and abandoned it. Machine output rarely has a graveyard of abandoned structures attached to it.

Laptop at night with an edit history panel visible beside the document

Bundle it once per project into a single PDF with a one-paragraph cover note. Sending a defensible record reads as professional. Sending a second detector’s score reads as defensive. The full routine, including what to record before a client handoff, is laid out in our deliverable provenance log guide.

If Your Own Writing Is What Gets Flagged

Some writers get flagged far more than others, and the pattern is not random. Careful non-native English speakers score high, because their phrasing is grammatically tidy. Technical writers score high, because documentation is uniform by design. Anyone who writes short, plain sentences in a consistent voice scores high. None of these people are doing anything wrong.

What to actually do about it:

  1. Rewrite two paragraphs in your spoken cadence. Say the sentence out loud, then type what you said. The result is usually better writing and a lower score, in that order.
  2. Add what only you know. The client’s actual complaint, the number from the dashboard, the thing that went wrong in week two. Detectors have no access to your specific knowledge, and neither does a generic model.
  3. Keep the substantive edits as evidence. Not the punctuation changes. The places where you cut a claim because it was wrong, or added a caveat because the client would have been misleading.
  4. Decide when to stop arguing. A client who treats a detector percentage as ground truth will keep doing it at invoicing time. One calm explanation, one evidence pack, and then a decision about whether the relationship is worth the overhead.

The uncomfortable conclusion after watching this argument for a couple of years: AI content detection is a useful screening signal on a platform processing millions of submissions, and a bad instrument for judging one person’s work. Treat the score as one weak piece of evidence among many, and demand the same standard from anyone who uses it against you.

FAQ

How accurate are AI detectors, really?

Accuracy figures vendors publish are measured on their own test sets, which are usually cleaner and better labelled than real submissions. Independent testing tends to show high false-positive rates on non-native English writing, translated text and formal genres. Treat any single score as a weak signal, not a verdict.

Why did the same text score differently on two detectors?

Because they are different models with different training data, different thresholds and different definitions of what counts as AI-generated. A large gap between two tools on identical text is normal, and it is the fastest way to show a client that the number is not a measurement.

Can I make my writing score lower without cheating?

Usually yes, by writing more specifically: fewer generic openers, more concrete detail, uneven sentence lengths, fewer stock transitions. That is also just better writing. Avoid humanizer tools and hidden characters; both break on the next model update and the second one can get an account closed.

What should I send a client who accuses me of using AI?

A dated outline, the version history from your editor, your source notes and a short cover note asking what decision the flag is driving. Evidence of process beats a second opinion from another detector, and it does not invite a screenshot war.

Should my contract mention AI detectors?

It is worth one clause stating that the deliverable is judged against the agreed brief, originality and accuracy, and that third-party classifier results are not a guaranteed acceptance criterion. It is a commercial protection, not legal advice, so have a professional review anything high value.

Do platforms like content mills and publishers use these scores?

Many do, often as a triage step rather than a final decision, and often without publishing the tool or threshold. If you write for one, ask which detector and which threshold before the first assignment. If they will not say, assume the flag can appear and keep your evidence pack ready.

E
Earn Kit Lab Team

We test every tool category hands-on before writing about it. Free first, always.

Leave a Comment

Your email address will not be published. Required fields are marked *