ast-grep-search

# ast-grep Search Expert knowledge for using `ast-grep` as a powerful AST-based code search and refactoring tool with structural pattern matching across 20+ programming languages. ## Core Expertise **ast-grep Advantages** - AST-based matching (more precise than text-based tools) - Extremely fast (written in Rust, multi-core processing) - Supports 20+ languages via tree-sitter - Pattern code syntax (write code to match code) - Built-in rewriting capabilities - Interactive mode for safe transformations - Language server protocol support - YAML-based rule configuration for custom linting **Supported Languages** JavaScript, TypeScript, Python, Java, Go, Rust, C++, C, C#, Ruby, PHP, Swift, Kotlin, Scala, HTML, CSS, YAML, JSON, and more. ## Basic Usage ### Simple Pattern Search ```bash # Basic pattern matching ast-grep -p 'console.log($MSG)' --lang js ast-grep -p 'function $NAME($$$ARGS) { $$$ }' --lang js ast-grep -p 'def $FUNC($$$): $$$' --lang py # Search in specific files/directories ast-grep -p 'import $PKG' src/ ast-grep -p 'class $NAME:' tests/ --lang py ``` ### Pattern Syntax **Meta Variables (Wildcards)** - `$VAR` - Match any single AST node (named node) - `$$VAR` - Match any single unnamed node - `$$$ARGS` - Match zero or more nodes (e.g., function arguments) - `$_` - Match single node without capturing **Naming Rules** - Must start with `$` - Followed by uppercase letters, underscores, or digits - Valid: `$MATCH`, `$META_VAR`, `$VAR1`, `$_`, `$_123` - Invalid: `$invalid`, `$Svalue`, `$KEBAB-CASE` ### Language Selection ```bash # Specify language explicitly ast-grep -p 'pattern' --lang js ast-grep -p 'pattern' --lang py ast-grep -p 'pattern' --lang rs ast-grep -p 'pattern' --lang go # Common language codes # js/ts/jsx/tsx - JavaScript/TypeScript # py - Python # rs - Rust # go - Go # java - Java # cpp/c - C++/C # rb - Ruby # php - PHP ``` ## Advanced Pattern Matching ### Capturing and Reusing Variables ```bash # Match same variable used twice ast-grep -p '$A == $A' --lang js # Matches: x == x (not x == y) # Multiple captures with same name must match identically ast-grep -p 'if ($COND) { $$$ } else if ($COND) { $$$ }' --lang js # Use underscore prefix to allow different matches ast-grep -p '$_VAR == $_VAR' --lang js # Matches: x == y ``` ### Multi-node Matching ```bash # Match function calls with any number of arguments ast-grep -p 'console.log($$$)' --lang js # Matches: console.log(), console.log(x), console.log(x, y, z) # Match function definitions with any parameters ast-grep -p 'function $NAME($$$PARAMS) { $$$BODY }' --lang js # Match try-catch blocks ast-grep -p 'try { $$$ } catch ($ERR) { $$$ }' --lang js ``` ### Nested Patterns ```bash # Find nested function calls ast-grep -p 'React.useState($$$)' --lang jsx # Find method chains ast-grep -p '$OBJ.$METHOD1().$METHOD2()' --lang js # Find specific imports ast-grep -p "import { $$$IMPORTS } from '$PKG'" --lang js ``` ## Code Search and Rewrite ### Search and Replace ```bash # Basic rewrite ast-grep -p 'var $VAR = $VAL' -r 'let $VAR = $VAL' --lang js # Update function syntax ast-grep -p 'function($$$ARGS) { $$$BODY }' \ -r '($$$ARGS) => { $$$BODY }' --lang js # Replace deprecated API calls ast-grep -p 'oldAPI($$$ARGS)' -r 'newAPI($$$ARGS)' --lang py ``` ### Interactive Mode ```bash # Review changes before applying ast-grep -p 'var $V = $X' -r 'let $V = $X' -i --lang js # Update all automatically (use with caution) ast-grep -p 'console.log($$$)' -r '// removed log' -U --lang js ``` ### Dry Run and Preview ```bash # Show what would be changed without modifying files ast-grep -p 'pattern' -r 'replacement' --lang js # (Default behavior - shows matches and proposed changes) # Apply changes to all files ast-grep -p 'pattern' -r 'replacement' -U --lang js ``` ## Command-Line Options ### Main Commands **ast-grep run** - One-time search or rewrite (default) ```bash ast-grep run -p 'pattern' [PATHS] ast-grep -p 'pattern' -r 'rewrite' -l js src/ ``` **ast-grep scan** - Scan using YAML configuration ```bash ast-grep scan # Use default sgconfig.yml ast-grep scan -c sgconfig.yml # Specific config file ast-grep scan -r rule-name # Run specific rule ast-grep scan --filter 'console' # Filter rules by pattern ast-grep scan --inline-rules 'rule.yml' # Inline rule file ``` **ast-grep test** - Test ast-grep rules ```bash ast-grep test # Run all tests ast-grep test -c custom-config.yml # Test with custom config ast-grep test --snapshot-dir snapshots/ # Specify snapshot directory ``` **ast-grep new** - Create new project/rules/tests ```bash ast-grep new project my-linter # Initialize new project ast-grep new rule no-console-log # Create new rule template ast-grep new test test-suite # Create new test template ast-grep new util common-patterns # Create utility rule ``` **ast-grep lsp** - Language server for editor integration ```bash ast-grep lsp # Start language server ``` ### Common Flags | Flag | Purpose | Example | |------|---------|---------| | `-p, --pattern` | Search pattern | `ast-grep -p 'console.log($MSG)'` | | `-r, --rewrite` | Replacement pattern | `ast-grep -p 'var $V' -r 'let $V'` | | `-l, --lang` | Target language | `ast-grep -p 'pattern' -l js` | | `-i, --interactive` | Interactive mode | `ast-grep -p 'old' -r 'new' -i` | | `-U, --update-all` | Auto-apply all changes | `ast-grep -p 'old' -r 'new' -U` | | `--json` | JSON output | `ast-grep -p 'pattern' --json` | | `-A, -B, -C` | Context lines | `ast-grep -p 'pattern' -A 3` | | `--color` | Color output | `ast-grep -p 'pattern' --color always` | | `--heading` | Group by file | `ast-grep -p 'pattern' --heading` | | `--debug-query` | Debug pattern parsing | `ast-grep -p 'pattern' --debug-query` | ### Output Formats ```bash # Default: colorized, grouped by file ast-grep -p 'pattern' # JSON output (for tooling integration) ast-grep -p 'pattern' --json # Pretty JSON ast-grep -p 'pattern' --json=pretty # Stream JSON (one result per line) ast-grep -p 'pattern' --json=stream # Compact JSON ast-grep -p 'pattern' --json=compact ``` ## YAML Rule Configuration ### Rule File Structure A complete ast-grep rule file contains these sections: ```yaml # Minimal rule (required fields only) id: rule-identifier language: JavaScript rule: pattern: console.log($$$) --- # Complete rule with all fields id: no-await-in-promise-all language: TypeScript severity: error message: Avoid await inside Promise.all note: | Using await inside Promise.all defeats the purpose of parallel execution. Extract async operations before Promise.all. url: https://docs.example.com/no-await-promise-all # Finding rule: pattern: Promise.all($ARGS) has: pattern: await $_ stopBy: end constraints: ARGS: regex: '^\[.*\]$' utils: is-promise: pattern: Promise.$METHOD($$$) # Patching transform: VAR: substring: source: $ARG startChar: 0 endChar: -1 fix: | const results = await Promise.all($ARGS) # Linting labels: - label: problematic await source: await $_ # Globbing files: - '**/*.ts' - '**/*.tsx' ignores: - '**/*.test.ts' - '**/node_modules/**' # Metadata metadata: category: async tags: [performance, best-practices] ``` ### Rule Types **Atomic Rules** - Match individual AST nodes ```yaml # Pattern matching rule: pattern: console.log($$$) # Kind matching (node type) rule: kind: function_declaration has: field: name regex: '^test_' # Regex matching rule: regex: 'TODO|FIXME|XXX' ``` **Relational Rules** - Match node relationships ```yaml # has: parent contains child rule: pattern: Promise.all($ARGS) has: pattern: await $_ stopBy: end # Stop at nearest enclosing function # inside: child appears within parent rule: pattern: await $_ inside: pattern: Promise.all($$$) # follows: node appears after another rule: pattern: $A follows: pattern: $B # precedes: node appears before another rule: pattern: $A precedes: pattern: $B ``` **Composite Rules** - Combine multiple rules ```yaml # all: AND logic - all rules must match rule: all: - pattern: function $NAME($$$) { $$$ } - not: has: pattern: return $$$ - inside: kind: class_declaration # any: OR logic - any rule can match rule: any: - pattern: var $VAR = $$$ - pattern: let $VAR = $$$ # not: negation rule: pattern: function $NAME($$$) { $$$ } not: has: pattern: return $$$ # matches: reference utility rules rule: pattern: $CALL($$$) matches: is-console-method ``` **Utility Rules** - Reusable patterns ```yaml # In rule file utils: is-console-method: kind: call_expression has: field: function pattern: console.$METHOD is-async-function: any: - pattern: async function $NAME($$$) { $$$ } - pattern: async ($$$) => $$$ has-side-effect: any: - matches: is-console-method - pattern: $OBJ.$MUTATE($$$) - pattern: $VAR = $$$ # Using utility rules rule: pattern: $EXPR matches: has-side-effect ``` ### Constraints and Transformations **Constraints** - Filter meta-variables by conditions ```yaml rule: pattern: if ($COND) { $$$ } constraints: COND: # Regex constraint regex: '^true$|^false$' COND: # Kind constraint kind: binary_expression COND: # Pattern constraint pattern: $A == $B ``` **Transformations** - Manipulate captured variables ```yaml transform: # String replacement NEW_NAME: replace: source: $OLD_NAME replace: 'Test' by: 'Spec' # Substring extraction TRIMMED: substring: source: $TEXT startChar: 1 endChar: -1 # Convert to uppercase