How AI Text Detectors Work: Perplexity and Burstiness Explained
An AI text detector does not know who wrote your text. It looks for patterns like predictable words and flat sentence rhythm. Here is what it measures, in plain English.
Educational guides on AI formatting artifacts, unicode control characters, markdown parsing idiosyncrasies, and privacy-first content sanitization.
Tired of ChatGPT inserting repetitive em-dashes into every paragraph? Discover why LLMs overuse em-dashes, exact Custom Instructions prompts to stop it, and how to normalize dashes in 1 click.
Inam Ilyas
Lead Systems Engineer
// LLM Raw Clipboard Stream
"The\u200Bfuture\uFEFFof\u00A0technology"
// Sanitized V8 String (Client-Side)
"The future of technology"
An AI text detector does not know who wrote your text. It looks for patterns like predictable words and flat sentence rhythm. Here is what it measures, in plain English.
If an AI checker says your own writing is AI, you are not alone. Learn why false positives happen, who is most affected, and what to do if your text is flagged.
There is no sure way to know, but AI writing leaves clear habits. Here are 10 simple signs, and a quick way to check text with a free AI text checker.
Ever copied text from ChatGPT, Claude, or DeepSeek into Google Docs or VS Code and noticed erratic line breaks, backspace stuttering, or regex parsing failures? We deconstruct the exact tokenization mechanics that introduce byte-level invisible artifacts into everyday AI outputs.
Learn how to strip raw Markdown tags, double bold asterisks, curly quote mismatches, and AI buzzword transitions before publishing to CMS platforms or submitting corporate documentation.
A deep technical breakdown of Unicode zero-width spaces (U+200B, ZWSP), non-breaking spaces (U+00A0), and byte order marks (U+FEFF) in web dev, database indexing, and V8 string engines.
Discover how standard PDF exports embed operating system usernames, author stamps, print queues, and Adobe XMP schema logs, and how client-side scrubbing eliminates forensic footprints.
An empirical analysis of RLHF reward models and frequency optimization biases that lead Claude and ChatGPT to generate predictable transitional structures and decorative filler phrases.
| Character Name | Codepoint | UTF-8 Hex | Common AI Source | Cleaner Action | Live Test |
|---|---|---|---|---|---|
| Zero-Width Space (ZWSP) | U+200B | E2 80 8B | ChatGPT, Claude markdown blocks | Stripped (0-byte) | |
| Byte Order Mark (BOM) | U+FEFF | EF BB BF | API responses & Copilot streams | Stripped (0-byte) | |
| Non-Breaking Space (NBSP) | U+00A0 | C2 A0 | Google Gemini line wraps | Normalized (0x20 Space) | |
| Zero-Width Non-Joiner (ZWNJ) | U+200C | E2 80 8C | Persian / Arabic AI scripts | Context-Stripped | |
| Soft Hyphen (SHY) | U+00AD | C2 AD | PDF extract & web scraping LLMs | Stripped (0-byte) | |
| Left/Right Double Quotes | U+201C / U+201D | E2 80 9C / 9D | All conversational models | Converted to straight (") |
Receive technical notifications whenever OpenAI, Anthropic, or Google roll out tokenizer updates that alter unicode representations or inject novel control sequences.
All text sanitization runs in browser memory via WebAssembly and Web Workers with absolute zero data transmission.
Specialized, 100% browser-based tools engineered for AI text sanitization, Unicode byte hygiene, and document privacy.
Strips markdown fences, asterisks, conversational courtesies, and em-dashes from ChatGPT outputs.
Purges thinking blocks, XML tags, conversational preambles, and formatting from Anthropic Claude.
Strips inline search citation markers [1][2], web grounding brackets, and formatting from Gemini.
Analyzes sentence burstiness, perplexity, and statistical markers to highlight AI-generated phrasing.
Breaks robotic syntax into varied sentence structures that sound organic and conversational.
Flags and replaces overused LLM clichés like 'delve', 'tapestry', and 'testament'.
Cleans zero-width spaces (U+200B), non-breaking spaces (U+00A0), soft hyphens, and byte-order marks.
Inspects text in real-time, highlighting hidden Unicode glyphs in red badges with hex codepoints.
Maps Cyrillic, Greek, and confusable lookalike characters disguised in text back to standard ASCII.
Calculates Flesch-Kincaid grade level, reading ease, sentence complexity, and lexical density.
Purges author identity, creation software, revision timestamps, and hidden binary stamps from PDFs.
Strips Midjourney, DALL-E, Stable Diffusion prompt parameters, EXIF tags, and C2PA credentials.
Cleans Midjourney parameter flags (--ar, --v, --s), prompt meta prefixes, and seed numbers.
Converts Markdown syntax, asterisks, hash headers, and code backticks into clean, readable plain text.