I just wrote a piece for The Monetizer that got flagged as 100% AI, and it’s not. Substack turned its own scanner on last month, which is either bad timing or exactly the point. Clearly, there are some markers that are human, but maybe technically correct human writing is still being flagged as AI.
Since July 21, readers on web and iOS can run any Substack post, note, comment, or reply longer than 100 words through Pangram and get a percentage back, with Android still coming. Some writers just are on the side of what would be technically correct, like an AI. I already wrote up how the scan works and how to switch it off. This is the other half.
What ZeroGPT Said About a Paragraph I Wrote Myself
Pangram’s own checker wants you to have an account before it scores anything, so I used ZeroGPT, which doesn’t. I ran the passages through ZeroGPT last week, I believe it was on Thursday. The first was textbook clean writing: hedged, generic, ending on an uplifting summary line, and it came back 100%.
The second was a first-person paragraph about my mornings, written in one pass. The score came up in its dashboard as 98%, and I immediately felt, “How the fuck is this 98% AI when I wrote it?” Then I started to think, “Maybe I write like AI.”
Does a Technical Writing Degree Read Like a Machine?
I do have a master’s in technical writing. Is that just AI writing? People have said that I write in a very clean, almost technical way, even when I’m trying to be entertaining or even to write in a fictional voice, not writing fictional stories. I thought, “Well, maybe I just sound like AI when I write.”
I can’t shake that theory, and the research doesn’t rule it out. By the way, if you’re new to Substack or just getting started with your writing career, be sure to check out my list of the writing rules you should know.
Pangram’s False Positive Rate: 0%, 2%, or One in 10,000
It annoys the shit out of me that sometimes technically correct writing will flag as AI, even when it’s not. Three groups have tested Pangram independently and their numbers don’t agree. A peer-reviewed Vrije Universiteit Brussel paper scored it 97.5% on fully AI-generated academic papers, zero false positives across 160 documents. University of Chicago researchers Brian Jabarian and Alex Imas ran close to 4,000 texts and found Pangram “dominates the other detectors across all thresholds.”
It makes me think, “I’m sorry, do you think that humans don’t know what a fucking em dash is? Do you think that they don’t know what proper punctuation is?” A University of Maryland experiment landed at 2% false positives. Pangram CEO Max Spero puts it closer to one in 10,000.
I get it: a lot of people are terrible writers, and I’ll be honest. I happen to be a very good writer. Two percent of a platform this size is a lot of people getting told they didn’t write their own work. In fact, when I was in fifth grade, my substitute teacher (what was her name, Mrs. Drake?) read a short story that I wrote and then told me that I should be an author. It must have been pretty fucking good because I was like 12 years old impressing a 70 year old English teacher. It was in a small town, so maybe she wasn’t the best judge, but you get it.
Chris Best, Mack Collier, and the Witch Hunt Complaint
Best is upfront about the limit: Pangram “can only detect whether AI was used to make the text, not whether great human care went into creating it, nor whether AI tools were used as a source,” he wrote. His bigger argument: AI content “pollutes the commons.”
Writers pushed back inside a week. Mack Collier said he is “not going to apologize for using AI in the creation process.” Ghostwriting coach Alice Lemee called detectors “notoriously, wildly inaccurate.” Sam Illingworth, author of Slow AI, says Substack is asking a machine to vouch for a human. I think that there’s a scale that AI is using the detectors in terms of identifying AI content, and it is not always right.
How I Rewrite a n AI Draft Before Publishing
Usually, when I use AI to help me come up with the first draft of an article, I am almost always rewriting every sentence in my own words. Sometimes I’m even rearranging the paragraphs of the sentences because I think they flow better in a more human way. It may not be as technically correct as it would be if Claude created it or if ChatGPT created it, but I rearrange things in a way that I beleive is my voice and is human.
That last part is the trick. Swapping an em dash for a colon or deleting “delve” is a workaround, not a rewrite.
Rewriting the Morning Routine Paragraph, Line by Line
Here it is on an actual paragraph; a real example. I was trying to show the difference between how AI content and human content will sound, but I probably could have cleaned up the “after” a little bit more, although “before” does sound borderline AI.
The Falsely Accused AI version (Scored 98% AI):
Building a morning routine is one of the most effective ways to boost productivity and improve overall well-being. It is important to note that a consistent routine helps regulate your body’s internal clock, leading to better sleep quality and increased energy throughout the day.
It is very technically correct, whereas the “after” reads like it was written by a new writer.
My rewritten and dumbed down human version (scored 100% human):
My mornings used to be insane. Like every other human on this planet, I would get up, check my phone for 20 minutes, scroll on Instagram, and watch countless people fall down before I even got out of bed. Then I would spend the rest of the day feeling behind. I would need to go to the gym. I would need to let the dog out because I spent too much time scrolling. Now I do three things before I do anything: I get up and stretch for five minutes, not full-on yoga, but I do some stretches. I drink some water. I usually have coffee, but right after that, I write down the one thing I actually need to get done that day, sometimes even that morning.
I guess the dog-moment is the part no model can hand you? In the end, it seems that Pangram just scores clean writing as AI and messy writing as human. Given the bias, I think every human should fill out their “How I Make This” statement before Pangram scores their work as AI and the reader makes their own judgement.
What I Would Put in “How I Make This”
Substack gives you a field for this. My “How I Make This” line states that I do use AI for organizing thoughts, for coming up with outlines, but then I write the content, by and large. Obviously, I put everything through Grammarly. I don’t want to be publishing content that’s grammatically incorrect or has spelling errors. I am using the tools that are there to make me a better writer, but the thought, the organization, the content itself is mine.
Writing that down costs nothing, and you can also scan your own drafts before publishing and report a scan you believe is wrong.
Do Not Paste Confidential Drafts Into a Detector
One thing before you go check your archive: Pangram and ZeroGPT are third-party web forms, and you’re handing them unpublished work. Don’t paste unconfirmed interview quotes, financial figures that aren’t public, or anything under NDA. A percentage on a screen isn’t worth that.
If you write on Substack, fill in your "How I make this" field this week. Not because the scan is going to out you. Because your version of the sentence is better than whatever a reader invents when the field is blank.