| name | whitespaces-dont-lie-feature-driven |
| description | Detect whether source code was written by a human or generated by an AI (ChatGPT, Copilot, etc.) using whitespace, indentation, and stylometric feature analysis. Trigger phrases: 'is this code AI generated', 'detect machine generated code', 'check if code is human written', 'analyze code origin', 'code authorship detection', 'AI code detector' |
Whitespace-Driven Machine-Generated Code Detection
This skill enables Claude to analyze source code and assess whether it was likely written by a human or generated by an LLM (ChatGPT, Copilot, Claude, etc.) by extracting and evaluating whitespace patterns, indentation consistency, stylometric features, and structural properties. The approach is based on the empirical finding that AI-generated code exhibits mechanically uniform formatting -- especially in indentation and whitespace -- while human-written code carries idiosyncratic, inconsistent formatting signatures. A feature-based detector using these signals achieves ROC-AUC 0.995 and F1 0.971 on a 600k-sample benchmark.
When to Use
- When a user pastes code and asks "Was this written by AI?" or "Is this code machine-generated?"
- When reviewing student submissions for academic integrity and needing to flag potential AI-generated code
- When performing authorship attribution on code of unknown origin
- When auditing a codebase to estimate what proportion may have been AI-assisted
- When a user wants to understand what stylistic signals distinguish human code from AI code
- When building or evaluating a code-origin detection pipeline and needing feature engineering guidance
Key Technique
The core insight is that whitespace and indentation patterns are the most discriminative features for separating human-written from AI-generated code. Human developers accumulate idiosyncratic habits: mixing tabs and spaces, varying indentation depth across blocks, leaving trailing whitespace inconsistently, inserting irregular blank-line groupings, and aligning code to personal aesthetic preferences. AI models, trained on normalized corpora and generating token-by-token, produce mechanically consistent formatting -- uniform indentation depth, predictable blank-line placement, and almost no trailing whitespace or tab/space mixing.
The paper compares two approaches. The feature-based approach extracts lightweight, interpretable features across four categories: (1) whitespace features (blank-line ratios, trailing whitespace frequency, space-to-tab ratios), (2) indentation features (indentation depth variance, consistency score, tab-vs-space preference), (3) structural features (cyclomatic complexity, nesting depth, function/block density), and (4) stylometric features (variable naming patterns, comment density, line length distribution). These feed into gradient-boosted tree classifiers (XGBoost, Random Forest) that achieve near-perfect discrimination. The embedding-based approach uses CodeBERT to encode semantic code representations, achieving slightly higher precision but less interpretability.
The practical takeaway: you do not need a neural network to detect AI code. A small set of formatting-focused features -- extractable with simple regex and counting logic -- provides excellent detection. Indentation variance and whitespace consistency alone carry most of the signal. This makes the technique fast, explainable, and deployable without GPU infrastructure.
Step-by-Step Workflow
-
Collect the code sample. Obtain the source code to analyze. Ensure it is raw (not reformatted by a linter or auto-formatter like black, prettier, or clang-format), since auto-formatting destroys the whitespace signals this technique relies on.
-
Extract whitespace features. For every line, compute:
- Leading whitespace character sequence (tabs vs. spaces vs. mixed)
- Indentation depth (number of leading whitespace characters or tab-equivalent spaces)
- Whether the line has trailing whitespace
- Whether the line is blank
Then aggregate: mean/median/stddev of indentation depth, ratio of blank lines to total lines, percentage of lines with trailing whitespace, ratio of tabs to spaces in leading whitespace.
-
Compute indentation consistency score. Measure the variance in indentation increments (the change in indentation depth between consecutive non-blank lines). Human code typically shows high variance (irregular jumps); AI code shows low variance (consistent 4-space or 2-space increments). Calculate the coefficient of variation of indentation deltas.
-
Extract structural features. Count: number of functions/methods, average function length (lines), maximum nesting depth, number of code blocks, cyclomatic complexity (branch count). AI code tends toward moderate, uniform complexity; human code varies more widely.
-
Extract stylometric features. Measure: average line length and its standard deviation, comment-to-code ratio, variable name length distribution, use of single-letter variable names, keyword diversity (unique keywords / total keywords). Human code tends to have more variable line lengths and more comments.
-
Score each feature against known distributions. Compare extracted values to typical ranges:
- AI indicators: indentation depth stddev < 1.0, zero trailing whitespace, consistent indent increment (always 4 or always 2), blank-line ratio between 0.10-0.15, line length stddev < 15
- Human indicators: indentation depth stddev > 2.0, some trailing whitespace present, mixed indent increments, tab/space mixing, irregular blank-line clustering, line length stddev > 20
-
Compute a weighted confidence score. Weight indentation/whitespace features highest (they carry ~60% of the discriminative power), structural features at ~25%, and stylometric features at ~15%. Produce a 0-1 score where values near 1.0 indicate likely AI-generated and values near 0.0 indicate likely human-written.
Concrete Examples
Example 1: Analyzing a single code snippet
User: "Can you tell if this Python code was written by a human or AI?"
def fibonacci(n):
if n <= 0:
return []
elif n == 1:
return [0]
elif n == 2:
return [0, 1]
fib_sequence = [0, 1]
for i in range(2, n):
fib_sequence.append(fib_sequence[i-1] + fib_sequence[i-2])
return fib_sequence
Approach:
- Extract whitespace features: all indentation is exactly 4 spaces, zero trailing whitespace, one blank line before
fib_sequence, consistent spacing around operators
- Compute indentation consistency: indent increments are exactly {+4, 0, 0, 0, -4, +0, +4, -4, +0, -4} -- perfectly mechanical
- Structural features: single function, low complexity, textbook structure
- Stylometric features: descriptive variable names, no comments, uniform line lengths (stddev ~12)
Output:
Assessment: LIKELY AI-GENERATED (confidence: 0.82)
Key signals:
- Indentation: perfectly consistent 4-space indent, zero variance in increment magnitude (strong AI signal)
- Whitespace: no trailing whitespace on any line (AI signal)
- Structure: textbook fibonacci implementation with no personal style markers
- Line lengths: low standard deviation (13.2) suggesting uniform generation
- Comments: zero comments (mild AI signal -- humans often add at least one)
Caveat: If this code was auto-formatted (e.g., by black), these whitespace
signals would appear AI-like regardless of true authorship. Check whether a
formatter is configured in this project.
Example 2: Comparing two implementations
User: "I have two solutions to the same problem. Which one looks human-written?"
Solution A:
def merge_sort(arr):
if len(arr) <= 1:
return arr
mid = len(arr) // 2
left = merge_sort(arr[:mid])
right = merge_sort(arr[mid:])
return merge(left, right)
def merge(left, right):
result = []
i = j = 0
while i < len(left) and j < len(right):
if left[i] <= right[j]:
result.append(left[i])
i += 1
else:
result.append(right[j])
j += 1
result.extend(left[i:])
result.extend(right[j:])
return result
Solution B:
def merge_sort(arr):
if len(arr) <= 1:
return arr
mid = len(arr)// 2
left = merge_sort(arr[:mid])
right = merge_sort(arr[mid:])
return _merge(left, right)
def _merge(l, r):
res = []
i=j=0
while i < len(l) and j<len(r):
if l[i] <= r[j]:
res.append(l[i])
i+=1
else:
res.append(r[j])
j += 1
res.extend(l[i:])
res.extend(r[j:])
return res
Output:
Solution A: LIKELY AI-GENERATED (confidence: 0.79)
- Perfect 4-space indentation throughout
- Consistent spacing around all operators
- Descriptive variable names (left, right, result)
- Uniform blank-line placement (exactly one between functions)
Solution B: LIKELY HUMAN-WRITTEN (confidence: 0.75)
- Uses tabs for indentation (personal preference signal)
- Inconsistent operator spacing: "len(arr)// 2" vs "i < len(l)" vs "i+=1" vs "j += 1"
- Abbreviated variable names (l, r, res) -- personal shorthand
- Leading underscore on _merge (style choice)
- Irregular spacing patterns across the file
The inconsistent operator spacing in Solution B is a strong human signal.
AI models almost never produce "i+=1" on one line and "j += 1" two lines later.
Example 3: Batch analysis request
User: "I need to scan a directory of student submissions for AI-generated code."
Approach:
- Generate a Python script that walks a directory, extracts the feature set per file, and outputs scores
- Focus on the highest-signal features to keep it lightweight
Output script (key excerpt):
import os, re, statistics
def extract_features(code: str) -> dict:
lines = code.split('\n')
non_blank = [l for l in lines if l.strip()]
indents = [len(l) - len(l.lstrip()) for l in non_blank]
deltas = [abs(indents[i] - indents[i-1]) for i in range(1, len(indents))]
features = {}
features['indent_stddev'] = statistics.pstdev(indents) if indents else 0
features['delta_stddev'] = statistics.pstdev(deltas) if deltas else 0
features['trailing_ws_ratio'] = sum(1 for l in lines if l != l.rstrip()) / max(len(lines), 1)
features['blank_line_ratio'] = sum(1 for l in lines if not l.strip()) / max(len(lines), 1)
features['tab_space_mix'] = any('\t' in l[:(l)-(l.lstrip())] l non_blank) \
( l[:(l)-(l.lstrip())] l non_blank)
lengths = [(l) l non_blank]
features[] = statistics.pstdev(lengths) lengths
score =
features[] < : score +=
features[] < : score +=
features[] == : score +=
features[]: score +=
features[] < : score +=
< features[] < : score +=
features[] = score
features
root, _, files os.walk():
f files:
f.endswith():
path = os.path.join(root, f)
code = (path).read()
feat = extract_features(code)
flag = feat[] > \
feat[] >
()
Best Practices
- Do: Always check whether an auto-formatter (black, prettier, gofmt, rustfmt) has been applied before drawing conclusions. Auto-formatted code will appear AI-like regardless of true origin.
- Do: Weight indentation and whitespace features most heavily -- the paper shows these carry the majority of discriminative power over structural or naming features.
- Do: Report confidence levels honestly and present the specific features that drove the assessment, not just a binary label.
- Do: Analyze multiple files from the same author when available. Consistent patterns across files strengthen the signal.
- Avoid: Claiming certainty. Even at 0.995 ROC-AUC, false positives occur. Frame results as "likely" not "definitely."
- Avoid: Applying this technique to code in languages with enforced formatting (Go with gofmt, Rust with rustfmt) -- the forced consistency eliminates whitespace signals.
- Avoid: Using only a single feature. The technique's strength comes from combining multiple independent signals across categories.
Error Handling
- Auto-formatted code: If you detect signs of auto-formatting (perfectly consistent style matching a known formatter's output), warn the user that whitespace-based detection is unreliable for this sample and suggest examining commit history or semantic features instead.
- Very short snippets (<10 lines): Too few lines produce unreliable statistics. Report that the sample is too small for confident analysis and ask for more code.
- Minified or obfuscated code: Whitespace has been intentionally removed. Report that the technique is inapplicable and suggest alternative analysis (variable naming patterns, control flow structure).
- Mixed-origin code: Some files contain both human and AI code (e.g., human skeleton with AI-filled functions). If feature distributions are bimodal, flag this possibility and suggest per-function analysis.
- Language-specific norms: Python's significant whitespace and PEP 8 conventions make some formatting uniform by community norm. Adjust thresholds accordingly -- focus on operator spacing, blank-line patterns, and trailing whitespace rather than indent depth alone.
Limitations
- Auto-formatters neutralize the technique. Any code run through black, prettier, clang-format, gofmt, or similar tools will have its whitespace signals destroyed. This is the single biggest limitation.
- AI models are improving. As LLMs are fine-tuned on more diverse code, their formatting may become less uniform. Features that discriminate today may lose power over time.
- Not effective on Go or Rust. These languages have canonical formatters (gofmt, rustfmt) that virtually all code is run through, eliminating whitespace variation.
- The approach is statistical, not forensic. It estimates likelihood based on population-level patterns. An unusually disciplined human coder may be flagged; a prompted AI told to "write messy code" may evade detection.
- Language coverage. The underlying benchmark (600k samples) covers common languages (Python, Java, C++, JavaScript). Performance on niche languages (Haskell, Kotlin, Elixir) is unvalidated.
- Single-sample reliability. The technique is most reliable when analyzing multiple files from the same source. A single short file provides weak evidence.
Reference
