check-docs

name	check-docs
description	Check documentation consistency against code changes. Audits supported ops table, RFC links, API docs, compiler docs, runtime docs, profiling, and user/developer guides for staleness or drift.

Check Documentation Consistency

You are auditing the torch-spyre documentation for consistency with the current state of the codebase. Run through each section below systematically. Report every issue found with the file path, what is wrong, and a suggested fix.

0. Preflight: fetch the latest upstream refs (read-only)

The audit has to read against the latest code, but it must not change the user's branch. Fetch the upstream refs only — do not merge, rebase, pull, reset, or check anything out. When you compare docs to code below, read from upstream/main (for example git show upstream/main:path/to/file.py) rather than relying on the local checkout, which may be behind.

The canonical repo is https://github.com/torch-spyre/torch-spyre.git. In a fork-based workflow the remote pointing at it is typically upstream, with origin being the user's own fork. In a direct clone of the canonical repo it is origin. The steps below assume upstream; substitute the correct name if git remote -v says otherwise.

Steps:

1. Run git remote -v and confirm which remote points at torch-spyre/torch-spyre. If it is not upstream, swap in the correct name in everything that follows.
2. Run git fetch upstream --prune. This only updates the remote-tracking refs; it does not touch the working tree or the current branch.
3. For "what landed on main since I last synced", run git log --oneline $(git merge-base HEAD upstream/main)..upstream/main and skim the subjects so the audit knows what is new.
4. Throughout the audit, read against upstream/main whenever the local checkout might be stale. Examples:
   • git show upstream/main:torch_spyre/_inductor/customops.py
   • git diff upstream/main -- docs/source/
5. Do not run git merge, git rebase, git pull, git reset, git checkout, git stash, or anything else that mutates the user's branch or working tree. An audit is read-only.

Terminology guardrails

A few terms keep getting muddled in doc passes. Reviewers have flagged these directly, so check for them in any prose, table, caption, or SVG label:

• DMA and DCI (Data Conversion Information) describe transfers between host memory and LPDDR5 only — the PCIe / DMA-engine path. A DCI is the host-side descriptor (loop ranges, host strides, device strides, plus dtype info) that drives one of these transfers; in code it is the DataConversionInfo struct built by generate_dci() in spyre_mem.cpp and consumed by copyAsync. The dma_sizes and dma_strides fields on SpyreTensorLayout feed the DCI.
• Do not call LPDDR5 ↔ LX scratchpad transfers DMA. Those are driven by load/store instructions emitted by the compiler for the on-core load/store units. Use wording like "the compiler stages tiles into the scratchpad" or "load/store instructions move tiles between LPDDR5 and the LX scratchpad" — never "DMA" or "DCI" for this hop.

Audience

Documentation serves two personas:

• Users — data scientists and ML engineers running models on Spyre via torch.compile. They care about: installation, quickstart, supported ops, profiling, debugging, and examples.
• Developers — engineers contributing to torch-spyre. They care about: compiler architecture, Inductor integration, adding operations, tensor layouts, work division, runtime internals, and RFCs.

Check that content is appropriate for its target audience and does not leak internal implementation details into user-facing pages.

1. Supported Operations Table

File: docs/source/user_guide/supported_operations.md

The table has four key columns: Operation, Eager, Compiled, Execution, and Notes. Each must be verified independently.

1a. Compiled support (torch.compile path)

Cross-reference the "Compiled" column against the actual op registrations in:

• torch_spyre/_inductor/customops.py (custom ops)
• torch_spyre/_inductor/decompositions.py (decompositions)
• torch_spyre/_inductor/lowering.py (lowerings)
• torch_spyre/ops/fallbacks.py (fallback registrations)

Use tests/inductor/test_inductor_ops.py as the primary reference for what is truly tested in the compiled path:

• Every op tested there should appear in the table with Compiled = Y.
• Check for ops in test_inductor_ops.py that are missing from the table.
• Check for ops in the table that have no corresponding test.

1b. Eager support (direct tensor ops without torch.compile)

Eager support comes from three sources:

• torch_spyre/ops/eager.py — manually registered ops via @torch.library.register_kernel (e.g., mm, silu, mish).
• torch_spyre/ops/fallbacks.py — CPU-fallback registrations via @register_fallback and register_fallback_default (e.g., arange, embedding, cumsum, tril/triu, isin).
• torch_spyre/_inductor/decompositions.py — five decompositions also dispatch eagerly: rms_norm, layer_norm, softplus, linear, and scaled_dot_product_attention.

To verify the Eager column:

• Read eager.py and fallbacks.py for the explicit registrations.
• Confirm that any op tested with run_eager=False in tests/inductor/test_inductor_ops.py does NOT have Eager = Y; those are compiled-only.
• Ops registered through register_fallback should appear under the "CPU Fallback" section of the table with Execution = "CPU fallback".

1c. View ops

View ops (reshape, transpose, permute, squeeze, unsqueeze, clone, contiguous, expand, narrow, select) require special attention:

• View ops with partial support should be noted (e.g., squeeze/unsqueeze may trigger internal recompile for certain shapes).
• View ops that are planned but not yet implemented (e.g., expand, narrow) should be listed with empty Eager/Compiled cells and a "Planned" note. Check for TODOs in test_inductor_ops.py.
• Most view ops are compiled-only — verify against run_eager=False.

1d. Execution column

• Ops in torch_spyre/ops/fallbacks.py (registered via @register_fallback) run on CPU — mark as "CPU fallback".
• All other ops run on Spyre — mark as "Spyre".

1e. General checks

• Check for ops that exist in code but are missing from the table.
• Check for ops listed in the table that no longer exist in code.

2. RFC Links and References

File: docs/source/rfcs/index.md

• All RFC links must point to https://github.com/torch-spyre/rfcs (external repo), NOT to torch-spyre/torch-spyre/RFCs/ (old location, now deleted).
• Grep the entire docs/ tree for any remaining references to the old RFC paths: RFCs/, torch-spyre/torch-spyre/blob/main/RFCs.
• Check that every RFC listed in the index table actually exists at the linked URL (verify the path structure: <number>-<Name>/<number>-<Name>RFC.md).
• Check for new RFCs in https://github.com/torch-spyre/rfcs that are not yet listed in the index.

3. Compiler Documentation

Files: docs/source/compiler/*.md

• architecture.md — Verify the compilation pipeline stages match the actual code flow in torch_spyre/_inductor/__init__.py and spyre_kernel.py.
• inductor_frontend.md — Check that extension points (PrePass, PostPass, SchedulerPass) match what is registered in torch_spyre/_inductor/passes.py and torch_spyre/_inductor/pass_utils.py.
• backend.md — Verify DeepTools invocation paths match torch_spyre/_inductor/dsc.py.
• adding_operations.md — Confirm the three patterns (direct mapping, decomposition, custom op) still match the current code patterns. Check that example ops cited still exist.
• work_division_planning.md — Verify op_dim_splits representation and dimension labels match torch_spyre/_inductor/core_division.py.
• work_division_codegen.md — Check code generation patterns match torch_spyre/_inductor/codegen/compute_ops.py and data_ops.py.

4. Runtime Documentation

File: docs/source/runtime/overview.md

• Verify device registration flow matches torch_spyre/__init__.py.
• Check allocator description matches torch_spyre/csrc/spyre_mem.cpp.
• Verify tensor implementation details match torch_spyre/csrc/spyre_tensor_impl.cpp.
• Check for new runtime features (e.g., streams in torch_spyre/streams.py, torch_spyre/csrc/spyre_stream.cpp) that are not yet documented.

5. API Reference

Files: docs/source/api/torch_spyre.rst, torch_spyre/__init__.py, torch_spyre/streams.py

The API reference is manually maintained because torch_spyre cannot be imported during the Sphinx build (C++ extensions and Spyre hardware are required). This means the API docs will drift unless explicitly checked.

When adding or updating API entries, follow the PyTorch torch.cuda API documentation style — use .. function::, .. class::, .. method::, and .. attribute:: directives with full signatures, typed parameters, return types, and code examples where usage is non-obvious.

5a. Device Management API

Cross-reference the "Device Management" table in torch_spyre.rst against the functions exposed by make_spyre_module() in torch_spyre/__init__.py:

• Check every mod.xxx = lambda: impl.xxx() line in make_spyre_module().
• If a new function was added to _SpyreImpl and wired into the module, it must be added to the table in torch_spyre.rst.
• If a function was removed or renamed, update or remove the corresponding row.

5b. Streams API

Cross-reference the "Streams" tables in torch_spyre.rst against torch_spyre/streams.py:

• Check the __all__ list in streams.py — every exported name must appear in the docs.
• Check Stream class methods and properties — compare against the Stream members table.
• If new methods or properties were added to Stream, add them.
• Verify the function signatures (parameter names, types, defaults) match the code.

5c. Random Number Generation

• Check manual_seed and manual_seed_all signatures in _SpyreImpl still match the docs.

5d. Constants and Environment Variables

• Verify torch_spyre.constants.DEVICE_NAME value matches the docs.
• Cross-reference the environment variables table against:
  • torch_spyre/_inductor/logging_utils.py (Spyre Inductor logging vars)
  • torch_spyre/__init__.py (runtime vars like TORCH_SPYRE_DEBUG)
  • CLAUDE.md env var table
• If new env vars were added anywhere in the codebase, add them to the API docs table.

5e. New Public Modules

Check if any new public modules have been added that are not yet in the API docs:

• torch_spyre/ops/ — eager ops and fallbacks
• torch_spyre/device/ — device interface
• torch_spyre/execution/ — async compile, kernel runner
• torch_spyre/memory/ — memory management

For each, determine if it exposes public API that users would call directly. If so, add a section to torch_spyre.rst.

6. User Guide

Files: docs/source/user_guide/*.md

• running_models.md — Verify torch.compile usage examples are correct. Check that environment variables (SENCORES, etc.) match torch_spyre/constants and actual behavior.
• tensors_and_layouts.md — Verify SpyreTensorLayout fields match torch_spyre/csrc/spyre_tensor_impl.h. Check RFC links point to external repo.
• profiling.md — Check that profiling instructions match any new profiling infrastructure (e.g., logging utilities in torch_spyre/_inductor/logging_utils.py).
• debugging.md — Verify environment variables and compiler artifact paths are still accurate.
• examples.md — Check that referenced example scripts exist in examples/.

7. Getting Started

Files: docs/source/getting_started/*.md

• installation.md — Verify Python and PyTorch version requirements match pyproject.toml and requirements/run.txt.
• quickstart.md — Verify code examples actually work with current API.

8. Contributing Guide

File: docs/source/contributing/guidelines.md

• Verify development workflow instructions are current.
• Check that linting tools listed match .pre-commit-config.yaml.
• Verify test commands match pytest.ini configuration.

9. Inductor Integration Changes

Check for drift between docs and code in these areas:

• New passes added to torch_spyre/_inductor/passes.py or temp_passes.py that are not documented.
• New codegen patterns in torch_spyre/_inductor/codegen/superdsc.py.
• Changes to torch_spyre/_inductor/wrapper.py (host code generation).
• New modules like torch_spyre/_inductor/views.py, torch_spyre/_inductor/multi_dim_reduction_pass.py, torch_spyre/_inductor/op_spec.py that may need documentation.

10. Sensitive Content Audit

Scan all documentation files for:

• Internal Slack channel names (e.g., #aiu-inductor, #torch-spyre).
• Internal URLs (e.g., *.ibm.com, internal wikis, Jira links).
• Employee names or emails that should not be public.
• Proprietary tool names or internal codenames not meant for public docs.
• References to internal build systems or infrastructure.

Flag any findings and suggest replacements.

11. Cross-Reference and Link Integrity

• Check all relative links between docs pages resolve correctly.
• Check all external links (GitHub, PyTorch docs, Python docs) are valid.
• Verify image references in _static/images/ — every referenced image exists, every image file is referenced somewhere.
• Check intersphinx_mapping in conf.py points to valid inventory URLs.

12. Knowledge Graph Explorer

Files: docs/source/_ext/extract_graph.py, docs/source/_ext/knowledge_graph_ext.py, docs/source/_static/js/knowledge_graph.js, docs/source/explorer/index.md

The Knowledge Graph Explorer is an interactive Cytoscape.js visualization auto-generated from the codebase at Sphinx build time. It has four views (Operations, Compiler Passes, Architecture, Configuration). The extraction script uses AST parsing — no runtime imports needed.

12a. Extraction coverage

Run the extraction script standalone and verify it picks up new registrations:

python3 docs/source/_ext/extract_graph.py torch_spyre

Check that the output counts reflect recent code changes:

• New ops: If a new @register_spyre_decomposition, @register_spyre_lowering, @torch.library.custom_op, register_fallback_default, or register_torch_compile_kernel was added, it should appear in the node count. If not, the extractor may need updating (e.g., a new registration pattern or a new source file path).
• New passes: If a new Custom*Passes class or pass function was added to torch_spyre/_inductor/passes.py, verify it appears in the graph.
• New modules: Any new .py file under torch_spyre/ should appear as a module node with correct import edges.
• New environment variables: If a new os.environ.get() or os.getenv() call was added, it should appear in the Configuration view.

12b. Source file paths in extractors

The build_graph() function in extract_graph.py has a hardcoded list of file paths for op-level extractors:

• torch_spyre/_inductor/decompositions.py
• torch_spyre/_inductor/lowering.py
• torch_spyre/_inductor/customops.py
• torch_spyre/ops/fallbacks.py
• torch_spyre/ops/eager.py
• torch_spyre/_inductor/passes.py

If any of these files were renamed, split, or if new registration files were created (e.g., a second lowering file), verify the extractor list is updated. Also check the class extraction file list and the config extraction target.

12c. JS view definitions

If a new category of nodes is added to the extraction script (new type value), verify that:

1. docs/source/_static/js/knowledge_graph.js has a corresponding entry in TYPE_META with a color and label.
2. The new type is included in the appropriate view's types array (or a new view is added if it does not fit existing ones).
3. Any new edge relationship value has a matching entry in the view's relationships array and a style rule in getStyles().

12d. Build integration

Verify the Sphinx extension still works:

• docs/source/conf.py must have sys.path.insert(0, os.path.abspath("_ext")) and "knowledge_graph_ext" in the extensions list.
• docs/source/index.rst must have explorer/index in a toctree.
• .gitignore must include docs/source/_static/js/graph.json (generated artifact, not committed).
• Cytoscape.js loads from a pinned jsDelivr CDN in explorer/index.md, not a vendored file. To bump the version, update both the cytoscape@<version> URL and the integrity SRI hash in the same <script> tag. Compute the hash with openssl dgst -sha384 -binary cytoscape.min.js | openssl base64 -A.

12e. Common maintenance scenarios

Scenario	Action
New op added with existing decorator pattern	No code change needed — extractor picks it up automatically
New registration pattern introduced	Add a new extractor function in extract_graph.py and wire it into build_graph()
Source file renamed or split	Update the file path in the extractors list in build_graph()
New node type category	Add to TYPE_META in JS, add to or create a view in VIEWS, add edge style if needed
Graph not rendering after code change	Run extract_graph.py standalone to check for AST parse errors
Sphinx build fails on knowledge_graph_ext	Check that extract_graph.py can import cleanly (no runtime deps)

13. Build Verification

• Run python -m sphinx docs/source docs/build/html -W --keep-going and report any warnings or errors.
• Verify suppress_warnings in conf.py only suppresses intentional warnings (e.g., mocked autodoc), not real issues.

Output Format

For each issue found, report:

### [SECTION] File: path/to/file.md

**Issue:** Description of the problem.
**Evidence:** What the docs say vs what the code shows.
**Fix:** Suggested correction.
**Severity:** critical | warning | info

At the end, provide a summary table:

Section	Issues	Critical	Warnings	Info
...	...	...	...	...