endnote-library-rescue

name

endnote-library-rescue

description

Run an end-to-end recovery workflow for messy or unstable EndNote libraries on macOS: diagnose bad desktop behavior, audit a copied `.enl`, repair DOI-backed metadata, rescue non-DOI records conservatively, rebuild fragile helper indexes when bulk writes fail, refresh stale `sdb.eni` UI caches, and safely replace the original library while preserving attachments. Use when the goal is full-library recovery rather than manual GUI cleanup.

EndNote Library End-to-End Repair

Use this skill when the task is to recover an EndNote library at the library level rather than fixing citations manually in the EndNote GUI.

Primary Reference

Treat this as the main operating document:

• notes/endnote_end_to_end_repair_guide.md

Workflow

1. Stabilize the task:
   • confirm the exact .enl path
   • confirm whether EndNote is open
   • confirm whether the original must remain untouched
   • confirm whether attachments in .Data/ matter
2. Work on a copied .enl, not the original.
3. Close EndNote before database work.
4. Run the audit first:
   • scripts/01_audit_endnote.sh
5. Validate structural health:
   • inspect quick_check
   • inspect integrity_check(refs_ord)
   • suspect helper-index damage when GUI behavior is much worse than library size suggests
6. Apply deterministic local cleanup:
   • malformed year normalization
   • removal of obvious field pollution
7. Repair DOI-backed records in batches:
   • scripts/02_sync_endnote_doi.py
8. Rescue non-DOI leftovers only by exact title match:
   • scripts/03_rescue_non_doi_by_title.py
9. Re-run the audit after each major pass.
10. If EndNote UI still shows old values, compare the repaired .enl with My Library.Data/sdb/sdb.eni on the same record id and refresh the sidecar cache if needed:

• scripts/04_refresh_endnote_ui_cache.sh

11. If the repaired copy is meant to replace the original, replace safely:

• if the working copy only repaired the .enl, usually keep the original .Data/ for attachments
• refresh sdb.eni after replacement if GUI values are stale

12. Validate in EndNote desktop with record and attachment spot checks.

Safety Rules

• Never start on the original .enl.
• Treat .enl and .Data/ as a pair when planning production replacement.
• Keep single-threaded SQLite writes.
• Parallelize metadata fetches only.
• If bulk writes fail with apparent corruption, inspect refs_ord integrity and rebuild refs_ord_keys_index.
• If EndNote still displays stale metadata after .enl repair, refresh My Library.Data/sdb/sdb.eni from the repaired .enl while EndNote is closed.
• Skip uncertain title matches instead of guessing.

Operational Notes

• EndNote .enl files are SQLite, but writes require EndNote-specific collations and functions.
• The main hidden instability may be the helper sort index, not the core refs table.
• EndNote UI may read stale data from My Library.Data/sdb/sdb.eni; repairing the .enl alone may not update what the GUI displays.
• Modern records may legitimately lack classic issue or page data; do not assume every blank field is wrong.
• If attachments matter and only the .enl was repaired, the original .Data/ folder is often the correct attachment store to preserve.

Batch Commands

Audit:

zsh scripts/01_audit_endnote.sh "My EndNote Library copy.enl" audit_run

DOI sync:

ENDNOTE_FIX_MODE=full_doi \
ENDNOTE_FIX_MIN_ID=1 \
ENDNOTE_FIX_LIMIT=150 \
ENDNOTE_FIX_FETCH_WORKERS=4 \
ENDNOTE_FIX_JITTER_MIN=0.3 \
ENDNOTE_FIX_JITTER_MAX=1.1 \
ENDNOTE_REBUILD_REFS_ORD_INDEX=1 \
python3 scripts/02_sync_endnote_doi.py "My EndNote Library copy.enl" audit_batch

Non-DOI rescue:

python3 scripts/03_rescue_non_doi_by_title.py \
  "My EndNote Library copy.enl" \
  "audit_final_fullsync/fix_harder_without_doi.csv" \
  audit_non_doi_rescue

Refresh UI cache after repair:

zsh scripts/04_refresh_endnote_ui_cache.sh \
  "My EndNote Library copy.enl" \
  "My EndNote Library copy.Data"

Replacement Guidance

When promoting a repaired copy to production:

• close EndNote first
• back up the original .enl and .Data/
• if only the .enl was repaired, pair it with the original .Data/
• refresh sdb.eni from the repaired .enl
• reopen EndNote and verify a few repaired records plus a few attachments

Do not assume the working repair copy's .Data/ folder is the right one unless that folder was intentionally part of the repair process.

Read When Needed

• End-to-end repair guide:
  • notes/endnote_end_to_end_repair_guide.md
• AI-oriented method notebook:
  • notes/endnote_library_rescue_for_ai.md
• Researcher collaboration notebook:
  • notes/endnote_library_rescue_for_researchers.md
• Troubleshooting:
  • notes/endnote_rescue_troubleshooting.md
• Key heuristics:
  • notes/endnote_rescue_secret_sauce.md