codebase-analysis

name	codebase-analysis
description	Analyze the codebase structure and count lines of code by category using cloc. Use when generating code statistics, calculating test-to-application ratios, documentation-to-code ratios, or identifying generated code percentages.

Codebase Analysis Instructions

This document provides step-by-step instructions for analyzing the ArduPilot Methodic Configurator codebase structure and counting lines of code across different categories.

Prerequisites

• Ensure cloc (Count Lines of Code) tool is installed:

  which cloc
  

  If not installed, install it via your package manager (e.g., sudo apt install cloc on Ubuntu).

Code Categories

The codebase is organized into the following categories:

1. Test Code

• Location: tests/ directory
• Purpose: Unit tests, integration tests, and test assets
• Command: cloc tests/

2. Main Application Code

• Location: ardupilot_methodic_configurator/ directory
• Purpose: Core application logic, GUI, backends, and business logic
• Command: cloc ardupilot_methodic_configurator/

3. Documentation Files

• Location: Root directory and subdirectories

• Purpose: User manuals, guides, architecture documentation, and README files

• File Types: Markdown files (*.md)

• Commands:

  # Find all markdown files
  find . -name "*.md" -type f
  
  # Count lines in documentation
  find . -name "*.md" -type f | xargs cloc
  

4. Configuration Files

• Location: Root directory and subdirectories

• Purpose: Project configuration, build settings, CI/CD configuration, and metadata

• File Types: JSON files (*.json)

• Commands:

  # Find all JSON files
  find . -name "*.json" -type f
  
  # Count lines in configuration files
  find . -name "*.json" -type f | xargs cloc
  

5. Generated Code

• Location: Within ardupilot_methodic_configurator/ directory and scripts/ directory

• Purpose: Auto-generated files that should not be manually edited

• Files (as documented in ARCHITECTURE.md):

  • data_model_fc_ids.py (generated by update_flight_controller_ids.py)
  • configuration_steps_strings.py (generated by update_configuration_steps_translation.py)
  • vehicle_components.py (generated by update_vehicle_components_translation.py)
  • scripts/generate_codebase_pie_chart.py (generated by AI assistant for codebase analysis)

• Command:

  cloc ardupilot_methodic_configurator/data_model_fc_ids.py \
       ardupilot_methodic_configurator/configuration_steps_strings.py \
       ardupilot_methodic_configurator/vehicle_components.py \
       scripts/generate_codebase_pie_chart.py
  

6. Utility Scripts

• Locations:

  • Root directory Python scripts
  • scripts/ directory

• Purpose: Build scripts, maintenance tools, development utilities

• Commands:

  # Find all Python scripts in root directory
  find . -maxdepth 1 -name "*.py" -type f
  
  # Count lines in root directory scripts
  cloc ./unix2dos.py ./create_pot_file.py ./insert_missing_translations.py \
       ./mavproxy_param.py ./copy_magfit_pdef_to_template_dirs.py ./setup.py \
       ./update_flight_controller_ids.py ./test_dpi_scaling.py ./mavproxy_ftp.py \
       ./extract_missing_translations.py ./post_install.py ./param_reorder.py \
       ./param_zip.py ./find_exclusive_parameter_names.py ./get_server_pem_cert.py \
       ./update_vehicle_components_translation.py ./update_vehicle_templates.py \
       ./param_filter.py ./create_mo_files.py ./copy_param_files.py \
       ./update_configuration_steps_translation.py ./merge_pot_file.py \
       ./mavftp.py ./dos2unix.py
  
  # Count lines in scripts directory
  cloc scripts/
  

Complete Analysis Workflow

Step 1: Identify Generated Files

First, check ARCHITECTURE.md to identify which files are auto-generated:

grep -n "autogenerated" ARCHITECTURE.md

Step 2: Count Lines by Category

1. Test Code:

   cloc tests/
   

2. Main Application Code:

   cloc ardupilot_methodic_configurator/
   

3. Documentation Files:

   find . -name "*.md" -type f | xargs cloc
   

4. Configuration Files:

   find . -name "*.json" -type f | xargs cloc
   

5. Generated Code (subset of main application and scripts):

   cloc ardupilot_methodic_configurator/data_model_fc_ids.py \
        ardupilot_methodic_configurator/configuration_steps_strings.py \
        ardupilot_methodic_configurator/vehicle_components.py \
        scripts/generate_codebase_pie_chart.py
   

6. Root Directory Scripts:

   find . -maxdepth 1 -name "*.py" -type f | xargs cloc
   

7. Scripts Directory:

   cloc scripts/
   

Step 3: Calculate Summary Metrics

Extract the "code" column from each cloc output to calculate:

• Total Test Lines: Python code lines from tests/
• Total Application Lines: Python code lines from ardupilot_methodic_configurator/
• Total Documentation Lines: Markdown lines from *.md files
• Total Configuration Lines: JSON lines from *.json files
• Generated Code Lines: Python code lines from generated files
• Utility Script Lines: Python code lines from root + scripts/ directories
• Test-to-Application Ratio: Test lines / Application lines
• Generated Code Percentage: (Generated lines / Application lines) × 100
• Documentation-to-Code Ratio: Documentation lines / (Application lines + Test lines)

Expected Output Format

Present results in this structure:

### Tests Directory:
- **Total code lines: X** (Python only)
- Additional assets: Y lines of XML, Z lines of HTML, etc.

### Main Application (ardupilot_methodic_configurator/):
- **Total Python code lines: X**
- Additional files: Y lines of XML, Z lines of JSON, etc.

### Documentation Files (*.md):
- **Total documentation lines: X** (across Y files)
- Key files: README.md (X lines), USERMANUAL.md (Y lines), etc.

### Configuration Files (*.json):
- **Total configuration lines: X** (across Y files)
- Key files: pyproject.toml equivalent configs, CI/CD configs, etc.

### Generated Code (within the application):
- **Total generated Python code lines: X**
- **Percentage of application code: Y%**
- Files: list of generated files

### Scripts (root + scripts/ directories):
- **Root directory scripts: X Python code lines** (N files)
- **Scripts directory: Y Python code lines + Z other lines** (N files)
- **Total script lines: X**

### Key Insights:
1. **Tests**: X lines - test coverage assessment
2. **Core Application**: X lines (Python only)
3. **Documentation**: X lines (comprehensive user/developer docs)
4. **Configuration**: X lines (project setup and CI/CD)
5. **Generated Code**: X lines (~Y% of application code)
6. **Utility Scripts**: X lines for build/maintenance tasks
7. **Total Python Code**: ~X lines across all categories
8. **Test-to-Application Ratio**: X:1
9. **Documentation-to-Code Ratio**: X:1

Notes

• Focus on Python code lines for the main metrics
• Generated code should not be manually edited and should be counted separately from hand-written code
• Documentation quality is assessed by documentation-to-code ratio
• Configuration files include project setup, CI/CD, and metadata files
• The project structure follows clean architecture principles
• Test coverage and generated code percentages are key quality indicators
• XML files in the application directory are likely parameter definitions
• PO files are translation files for internationalization
• The generate_codebase_pie_chart.py script is itself a generated file and should be excluded from utility script counts

Maintenance

This analysis should be run:

• Before major releases
• After significant code refactoring
• When evaluating code quality metrics
• For project health assessments

Update this document if:

• New generated files are added (check ARCHITECTURE.md)
• Directory structure changes significantly
• New categories of code are introduced

Generating Visualization

A Python script is available to automatically generate a pie chart visualization of the codebase structure:

python scripts/generate_codebase_pie_chart.py

This script will:

• Create a pie chart showing the distribution of code across all categories
• Save the chart as both PNG (high resolution) and SVG (scalable) formats in the images/ directory
• Display detailed analysis with key metrics and quality indicators
• Show the chart interactively (if display is available)

Requirements: matplotlib and numpy packages must be installed:

pip install matplotlib numpy

Output files:

• images/codebase_structure_pie_chart.png - High-resolution PNG for documentation
• images/codebase_structure_pie_chart.svg - Scalable SVG for presentations