pdf-form-filling

name

pdf-form-filling

description

Use when the deliverable is a filled-in or edited PDF whose *text* will be checked — a court or government form, an application, a waiver, an intake sheet — or a PDF from which an identifier must be removed. Fires on "fill out this PDF form", "check the Yes/No boxes", "redact the ID / show only the last four digits", "sign and date the form", or a checker that runs `pdftotext` or `pypdf` over the output. Covers AcroForm vs XFA detection, why copying pages destroys the fields, radio and checkbox option values, and text-layer redaction that actually removes text.

PDF form filling and text-layer editing

Two facts decide these tasks. First, a fillable PDF has a form dictionary that lives on the document, not on the pages — so the natural "copy the pages into a new writer" move silently throws the form away and every field write becomes a no-op. Second, a checker reads the text layer: a value the human eye can see but pdftotext cannot is a fail, and an identifier hidden under a drawn rectangle is still extractable and is also a fail.

What transfers: the AcroForm/XFA branch, the field-enumeration step, the option-value convention, the redaction call sequence, and the verify-by-extraction loop. What is per-task: the field names, the values, and which boxes to tick — always enumerate the real field names from the file rather than guessing them.

When to use this skill

• The output is a .pdf that must contain specific strings, or must no longer contain one.
• The task names form sections, checkboxes, a signature line, or "today's date".
• An earlier attempt produced a PDF that looks right but whose values do not appear in extracted text.

Not for generating a PDF from scratch, page merging or splitting, table extraction, or OCR.

Procedure

1. Enumerate before writing. Never guess a field name; they are fully-qualified hierarchical paths with occurrence indices, e.g. SC-100[0].Page2[0].List1[0].Item1[0].PlaintiffName1[0].

   from pypdf import PdfReader
   reader = PdfReader("blank.pdf")
   fields = reader.get_fields() or {}
   for name, f in fields.items():
       print(name, f.get("/FT"), f.get("/V"))          # /Tx text, /Btn button, /Ch choice
   

   If get_fields() is empty the PDF is not fillable — fall back to positioning text with PyMuPDF (step 6).

2. Branch on XFA. Check reader.trailer["/Root"]["/AcroForm"] for an /XFA key. XFA forms (most California and federal court forms) keep their layout and field bindings in an XML stream attached to the document.

   Form kind	Correct copy	Wrong copy
   XFA or hybrid	writer = PdfWriter(); writer.append(reader)	for p in reader.pages: writer.add_page(p)
   Plain AcroForm	writer.append(reader) also works	add_page drops /AcroForm

   add_page copies page content only. The form dictionary, the XFA stream and the field hierarchy are left behind, update_page_form_field_values then writes into nothing, and the saved file is a blank form with no error raised anywhere.

3. Write text fields and buttons in one dictionary, applied to every page. Fields are attached per page, so loop:

   for page in writer.pages:
       try:
           writer.update_page_form_field_values(page, all_values)
       except Exception as e:
           print("skip page:", e)
   

4. Use the option value, not a label, for radios and checkboxes. Radio groups on official forms are commonly /1 for the first option and /2 for the second — Yes/No on a two-option question, not "on"/"off". Unselected siblings must stay /Off: leave them alone, never set them. A conditional sub-checkbox ("if yes, also attach form X") stays unchecked when its parent answer is No. Read each button's permitted states from /_States_ or the /AP /N dictionary.

5. Leave fields blank that the task does not assign — case number, trial date, clerk signature, second plaintiff. Graders check emptiness as well as content.

6. When there is no fillable field, place text at a resolved coordinate. Locate the label with PyMuPDF, then draw below or beside it — never on top of it, because the label itself is usually asserted to survive:

   import fitz
   doc = fitz.open("in.pdf"); page = doc[0]
   r = page.search_for("PHONE NUMBER:")[0]
   page.insert_text(fitz.Point(r.x0 + 5, r.y1 + 12), "[phone]",
                    fontname="helv", fontsize=11)
   

7. Redact by removing the text, then re-inserting the masked value. A white or black rectangle drawn with draw_rect leaves the original string in the content stream and every extractor still reads it.

   rect = page.search_for("A12345678")[0]
   tight = fitz.Rect(rect.x0, rect.y0 + 8, rect.x1, rect.y1 - 2)   # keep the label above
   page.add_redact_annot(tight, fill=(1, 1, 1))
   page.apply_redactions()                                          # removes the text
   page.insert_text(fitz.Point(rect.x0, rect.y1), "****5678",
                    fontname="helv", fontsize=11)
   

   search_for returns a generous box that often overlaps the line above; shrink it or the redaction deletes the field label too. When a partial mask is requested ("show only the last four"), the prefix letter goes too — ****5678, not A****5678. Never rasterize the page to an image: it destroys every string the checker is looking for.

8. Verify by extraction, from the same surface the grader uses.

   pdftotext -layout out.pdf - | grep -F "Joyce He"
   pdftotext -layout out.pdf - | grep -E '\bA[0-9]{8}\b' && echo "REDACTION FAILED"
   

   If a value is stored in the field dictionary but missing from extracted text, the appearance stream was not regenerated — call writer.set_need_appearances_writer() before saving, or write the visible text yourself as in step 6.

Worked example

Fill an XFA court form: one plaintiff, one defendant, a 1500-dollar claim, "yes" to having asked for payment and "no" to the four eligibility questions.

from pypdf import PdfReader, PdfWriter

reader = PdfReader("/root/sc100-blank.pdf")
writer = PdfWriter()
writer.append(reader)                     # NOT add_page — keeps /AcroForm and /XFA

text_fields = {
    "SC-100[0].Page2[0].List1[0].Item1[0].PlaintiffName1[0]": "Joyce He",
    "SC-100[0].Page2[0].List3[0].PlaintiffClaimAmount1[0]": "1500",
    "SC-100[0].Page4[0].Sign[0].Date1[0]": "2026-01-19",
}
buttons = {
    "SC-100[0].Page3[0].List4[0].Item4[0].Checkbox50[0]": "/1",   # asked to pay: Yes
    "SC-100[0].Page3[0].List7[0].item7[0].Checkbox60[1]": "/2",   # fee dispute: No
}

for page in writer.pages:
    try:
        writer.update_page_form_field_values(page, {**text_fields, **buttons})
    except Exception as e:
        print("skip:", e)

with open("/root/sc100-filled.pdf", "wb") as fh:
    writer.write(fh)

Then run the pdftotext -layout greps from step 8 for every value the task named. If one is missing, the failure is almost always step 2 (pages copied instead of appended) or a field name typed from the visual label instead of the enumeration.

References

• pypdf forms and update_page_form_field_values: https://pypdf.readthedocs.io/en/stable/user/forms.html
• PyMuPDF redaction (add_redact_annot, apply_redactions): https://pymupdf.readthedocs.io/en/latest/page.html
• PDF 32000-1 §12.7 (interactive forms, XFA): https://www.pdfa.org/resource/pdf-specification-archive/