Substack launches tool to estimate AI-generated content in newsletters

What Happened

Substack has introduced a new feature that allows readers to estimate the proportion of a newsletter generated by artificial intelligence. This tool analyzes the text of a post and provides a probability metric, essentially acting as a transparency layer between creators and their audiences.

Technical Details

While Substack hasn't open-sourced the underlying model, AI detection typically relies on analyzing perplexity (how predictable the word choices are) and burstiness (the variation in sentence length and structure). Large Language Models (LLMs) tend to produce text with low perplexity and low burstiness compared to human writers. Implementing this at the platform level means Substack is likely running inference on published posts against a classifier model, surfacing the confidence score to the end user. The engineering challenge here is the well-documented false positive rate of AI text detectors, especially against non-native English speakers or highly structured technical writing.

Why It Matters

From an engineering and product perspective, this is a significant shift in how publishing platforms handle synthetic media. Until now, AI detection has largely been a third-party, bolt-on tool used primarily in academic or SEO contexts. By integrating it directly into the consumer UI, Substack is establishing content provenance as a core product metric. It acknowledges that as the marginal cost of content creation drops to zero via LLMs, the premium on verified human thought increases.

What to Watch Next

Monitor how creators adapt to this signal—whether through adversarial prompting designed to bypass detection (e.g., artificially increasing burstiness) or by leaning into AI transparency. Additionally, watch for Substack's false-positive mitigation strategies. If the classifier frequently flags legitimate human writing, the resulting trust deficit could force the platform to pivot toward cryptographic watermarking standards (like C2PA) rather than relying purely on post-hoc statistical analysis.