run-solution-analyzer

name	run-solution-analyzer
description	Run the Solutions Analyzer scripts: mapper, documentation generator, ASIM browser. Use when: running solution analyzer, generating CSVs, generating docs, refreshing caches, force-refresh, invalidating caches, generating ASIM browser, running mapper, running collect_table_info, running collect_asim_fields.

Run Solution Analyzer

This skill covers running the Solutions Analyzer locally against this clone of Azure-Sentinel. Each script writes CSVs or docs to a directory you choose; the scripts themselves are agnostic to where that directory lives.

If your workflow includes publishing the generated CSVs/docs to additional repositories, GitHub Pages, or Kusto, those targets are user-specific and should be recorded in agent memory (workspace or user scope) rather than in this skill.

Scripts Overview

Script	Purpose	Invocation
map_solutions_connectors_tables.py	Mapper: generates all CSV data	Run directly
generate_connector_docs.py	Doc generator: markdown + HTML pages, static + interactive indexes	Run directly
generate_asim_browser.py	ASIM browser: interactive HTML schema browser	Run directly
upload_to_kusto.py	Upload CSVs to Azure Data Explorer (Kusto) cluster	Run directly
collect_table_info.py	Fetch table reference data from Microsoft docs	Auto-invoked by mapper (--force-refresh=tables)
collect_asim_fields.py	Fetch ASIM field/schema definitions from Microsoft docs	Auto-invoked by mapper (--force-refresh=asim)
generate_interactive_docs.py	Generate index.html and HTML entity pages	Auto-invoked by generate_connector_docs.py

All scripts run from Tools/Solutions Analyzer.

Default Output Location

By default the mapper writes CSVs into Tools/Solutions Analyzer/ in the current branch. The doc generator and ASIM browser require an explicit --output-dir so they never pollute the source tree.

Sync the Working Branch With origin/master

Always do this before any analyzer run. A working branch that has drifted from master makes the mapper analyze a stale source tree — every connector merged to master after the divergence point can appear as a new "added" connector in subsequent diffs, even though no mapper logic changed.

git fetch origin master
git merge origin/master --no-edit

• Use git merge, not git rebase, so subsequent git log --first-parent origin/master analysis (used by compare_runs.py and friends) still reflects merge dates correctly.
• Resolve any conflicts before running the mapper.
• Do this even for incremental/docs-only runs: docs link to GitHub paths that may have moved.

After merging, verify the working tree is clean of unintended changes:

git status --short | Select-Object -First 20

Cache Invalidation Model

The mapper's analysis cache is keyed by (analysis_type, file_path) and stores the source file mtime. When a source file is modified (including via git merge), its mtime changes and the entry is automatically re-analyzed on the next run. Source-content changes do NOT require --force-refresh.

--force-refresh exists for two distinct reasons:

1. Analyzer logic changed (mapper code modified) — invalidate cached results for the affected analysis types so they get recomputed from current source. This is the local-only case.
2. Network-fetched data is stale — re-fetch tables / marketplace / ASIM schema fields from external sources. This is the network case and is reserved for explicit full refresh.

Type	Triggers network fetch?	When to refresh
parsers	No	Mapper parser-analysis logic changed
solutions	No	Mapper solution-analysis logic changed
standalone	No	Mapper standalone-content logic changed
asim	Yes — re-runs collect_asim_fields.py against MS docs	ASIM analyzer logic changed AND network refresh wanted
marketplace	Yes — Azure Marketplace API	Marketplace data is stale
tables	Yes — re-runs collect_table_info.py against MS docs	Table reference info is stale

Note: asim is hybrid — it invalidates the local ASIM-parser analysis cache and re-runs collect_asim_fields.py. There is currently no flag to do only the local half.

After a git merge from master

Default: just run the mapper with no flags. Mtime-based invalidation re-analyzes every file that changed during the merge:

python map_solutions_connectors_tables.py

Network-fetched CSVs (tables_reference.csv, asim_fields.csv, asim_entity_fields.csv, asim_logical_types.csv, asim_vendors_products.csv, marketplace cache) will be reused from cache. That's expected — they shouldn't drift just because master moved forward.

Full Mapper Run (Invalidating All Caches)

Use only when explicitly asked to do a full/clean run, when "invalidate all caches" is requested, or when analyzer logic was modified across multiple analysis types.

cd "Tools/Solutions Analyzer"
python map_solutions_connectors_tables.py --force-refresh=all

• --force-refresh=all clears all 6 cache types (asim, parsers, solutions, standalone, marketplace, tables) AND re-runs collect_asim_fields.py and collect_table_info.py against the network.
• --force-refresh=all-offline clears parsers, solutions, standalone, AND asim (which still re-runs collect_asim_fields.py despite the name; only marketplace/tables are skipped).
• Multiple types: --force-refresh=parsers,solutions,standalone (purely local, no network calls).
• Run with isBackground: false and timeout: 0.
• Do NOT truncate/filter output (no Select-Object piping).

Known issue (as of 2026-04-28): collect_asim_fields.py fails with HTTP 404 because the Microsoft Sentinel docs were migrated out of MicrosoftDocs/azure-docs. The mapper logs a warning and proceeds, leaving asim_fields.csv, asim_entity_fields.csv, asim_logical_types.csv, and asim_vendors_products.csv empty/stale. Until the script is updated to fetch from the new source, avoid --force-refresh=all and --force-refresh=asim unless you accept stale ASIM schema CSVs.

Generate Documentation

python generate_connector_docs.py --output-dir <DOCS_DIR> [--skip-input-generation] [--html-output-dir <HTML_DIR>] [--html-docs-path <REL_PATH>] [--html-index-url <URL>]

• --output-dir — mandatory, never omit.
• --skip-input-generation — skip the embedded mapper invocation when CSVs are already fresh.
• --html-output-dir, --html-docs-path, --html-index-url — control where index.html and the interactive entity pages are written and how they link back to the index. These are typically only set when publishing to a static-site / GitHub Pages location; defaults are sensible for a local preview.
• Run with isBackground: false and timeout: 0. Do NOT truncate output.

Generate ASIM Browser

python generate_asim_browser.py --output-dir <DIR> [--docs-base-path <REL_PATH>] [--index-url <URL>] [--link-extension <EXT>]

--docs-base-path and --index-url make sense only when the browser is deployed alongside an interactive docs index. For a standalone local preview, --output-dir alone is enough.

When to Re-Run the Mapper Before Docs

Run mapper before generating docs if:

• The mapper script itself was modified.
• Any override YAML file in the overrides/ folder was modified.
• You specifically need to refresh the CSV data.
• You are explicitly asked to run the mapper.

Otherwise, just run the doc generator with --skip-input-generation.

Caching and Logging

• Cache: Tools/Solutions Analyzer/.cache/
• Logs: Tools/Solutions Analyzer/.logs/
• Log file: Tools/Solutions Analyzer/.logs/map_solutions_connectors_tables.log

Watching Mapper Progress While It Runs

The mapper writes timestamped progress lines ([ N.Ns] ...) to its own log file in real time. To see live status during a run, tail that file directly:

Get-Content "Tools/Solutions Analyzer/.logs/map_solutions_connectors_tables.log" -Tail 12

Do NOT rely on ... | Tee-Object -FilePath <log> | Select-Object -Last N for live progress. PowerShell pipelines buffer until the upstream command finishes, so the Tee'd file stays empty until the entire run completes (including the post-processing/CSV-write phase, which can be many minutes after the per-parser progress lines were emitted). The mapper's own .logs/map_solutions_connectors_tables.log is unbuffered and updates within seconds.

Upload to Kusto

python upload_to_kusto.py -c <cluster-url> -d <database> --solution-analyzer --source-dir ./

• --solution-analyzer uploads the standard set of CSVs defined in SOLUTION_ANALYZER_FILES.
• --source-dir ./ uses local CSVs instead of downloading from GitHub.
• Without --source-dir, downloads CSVs from the public Azure-Sentinel GitHub repo.
• --dry-run shows what would be uploaded without making changes.
• Requires Azure CLI authentication (az login).

The cluster URL and database name are environment-specific and not part of this skill.

User-Specific Publishing Pipelines

If you maintain a publishing pipeline (e.g. CSV output worktree, separate docs repository, GitHub Pages site, specific Kusto cluster), record the concrete paths, URLs, and the order of copy/commit/push steps in agent memory — for example, in /memories/repo/ for a workspace-scoped note or in /memories/ for a cross-workspace one. The agent should consult that memory to resolve any placeholders above before running.