open-science-skills

Scaffold or audit an entire research project repository organized around its source library. Use whenever the user is starting, structuring, organizing, or reviewing a whole project — "set up a research repo", "how should I structure/organize this project", "initialize my sources folder", "new paper or literature-review project", "audit my repo structure", "is my sources folder set up right", "check my project layout". Builds the full tree from the sources spine outward — sources/{og,md,unprocessed}, references.bib, a PDF→Markdown convert script (OpenDataLoader PDF), a process-source intake command, CLAUDE.md/AGENTS.md, .gitignore, .venv — plus the analysis, manuscript, and review folders; or audits an existing repo and reports what is present, partial, or missing. NOT for intaking or converting a single PDF (use process-source) or building a publication replication package (use replication-package).

LLM token logprobs and calibration: per-decision confidence, ECE, Brier, reliability diagrams, low-confidence triage.

LLM council/panel voting: multi-model coders, consensus rules, inter-rater agreement (kappa, alpha), correlated-error diagnostics.

Compare OCR systems before a bulk run: candidate set, stratified ground truth, CER/WER, normalization, per-language and per-stratum accuracy.

Fact-check a manuscript's claims against the cited sources themselves: locate each source's knowledge-base Markdown file and verify the in-text claim is actually supported. Runs a pre-flight gate that refuses unless a per-source Markdown knowledge base exists and is clean (PDFs converted via process-source); then runs citation-check; then audits claim support, overclaiming, direction, scope, and misattribution.

Audit citation existence and fabrication risk, in-text/reference parity, DOIs, claim support, and style.

Typeset a working paper or journal submission in house-style LaTeX from any draft — Markdown, Word (.docx), TeX, ODT, RTF, or HTML. Convert with pandoc, wrap in an EB Garamond template, build the PDF with latexmk, and prepare for a specific journal (spacing, page limit, anonymization, disclosures, citation style). Use for "format/typeset/convert my paper to LaTeX", "make a working paper", "prepare this for submission to <journal>".

Clean and reshape Qualtrics conjoint exports to analysis-ready long format.

Diagnose conjoint design integrity, estimation choices, and validity.

Design cross-national survey experiments: power, equivalence, localization.

Audit manuscript and replication package against FAIR open-science principles.

Build falsifiable causal hypotheses: DAGs, FPCI, equivalence testing.

Draft a senior peer-review report on a social-science manuscript.

Design and diagnose list experiments (item count technique).

Build or audit a literature review: evidence map, gaps, synthesis plan.

Check methods reporting against CONSORT, JARS, DA-RT standards.

Draft or audit scientific introductions: argument logic, framing, multi-experiment coherence.

Cross-model adversarial pre-submission audit. Claude and Codex independently apply the paper-review-lite specification, then each cross-checks the other's findings; surviving issues are annotated by confidence.

Pre-submission audit: argument, numerics, refs, writing, figures, replication.

Clean post-OCR text: correction, QA, multilingual handling, provenance.

Write pre-analysis plans: PAP structure, registry, analysis strategy.

Run the standalone presubmit CLI: adversarial 30+ stage peer-review pipeline.

Scaffold or audit a social-science replication package at a target directory. Generates folder structure, README, master.R, figure/table crosswalk, codebook template, LICENSE placeholder, .gitignore, and pre-release checklist. Adapted from Yusaku Horiuchi's replication-package-guide with FAIR-principle integration; platform-neutral (Harvard Dataverse, OSF, Zenodo, GitHub releases, institutional archives).

LLM-based text classification: codebook, validation, agreement statistics.

Structural topic modeling: STM spec, topic count, coherence-exclusivity.

VLM-based OCR pipeline: model selection, prompts, architecture, evaluation.

Design and format publication-quality figures: chart choice, color, scales, legends, captions, reproducibility.

Audit figures, tables, captions, cross-references, and statistical notes.

Generate 3–5 conceptually distinct approaches to a task before implementing. Labels each by creativity dimension: Novel, Surprising, Diverse, or Conventional. Holds for user selection before writing any code. Based on Creative Preference Optimization (Ismayilzada et al., 2025) — brainstorm-then-select for maximizing novelty, surprise, and diversity in outputs.

Delegate creative divergence to Codex (GPT-5.4). Codex generates 3-5 conceptually distinct approaches before any implementation; Claude presents them for selection, then has Codex implement the chosen one. Cross-model brainstorm-then-select.

Before implementing, generate 3-5 conceptually distinct approaches labeled by creativity dimension (Novel, Surprising, Diverse, Conventional), then hold for selection. Brainstorm-then-select to resist defaulting to the most obvious solution.

Design and format publication-quality tables: column order, row grouping, notes, precision, reproducibility.

Design conjoint experiments: attributes, power, AMCE/AMIE estimation.

Design survey instruments: questions, scales, flow, social desirability.