pytest-ml-tester

name	pytest-ml-tester
description	ML-specific testing skill using pytest with fixtures for data, models, and predictions.
allowed-tools	["Read","Write","Bash","Glob","Grep"]
graph	{"domains":["domain:data-science","domain:software-engineering"],"specializations":["specialization:data-science-ml"],"skillAreas":["skill-area:model-validation-testing","skill-area:data-quality-testing","skill-area:unit-testing"],"topics":["topic:trunk-based-development"],"roles":["role:ml-engineer","role:data-scientist","role:qa-engineer"],"workflows":["workflow:ml-model-lifecycle","workflow:pull-request-lifecycle"]}

pytest-ml-tester

Overview

ML-specific testing skill using pytest with specialized fixtures for data validation, model loading, prediction testing, and ML pipeline verification.

Capabilities

• Data validation fixtures
• Model loading fixtures
• Prediction testing utilities
• Performance regression tests
• Integration test helpers
• Coverage reporting for ML code
• Property-based testing with Hypothesis
• Parameterized test generation

Target Processes

• ML System Integration Testing
• Model Evaluation and Validation Framework
• Data Collection and Validation Pipeline

Tools and Libraries

• pytest
• pytest-cov
• hypothesis
• great-expectations (optional)

Input Schema

{
  "type": "object",
  "required": ["action"],
  "properties": {
    "action": {
      "type": "string",
      "enum": ["run", "generate", "coverage", "fixtures"],
      "description": "Testing action to perform"
    },
    "testConfig": {
      "type": "object",
      "properties": {
        "testPath": { "type": "string" },
        "markers":         
            
            
            
      
    
     
       
       
            
            
                
      
    
     
       
       
         
           
              
        
            
            
            
      
    
     
       
       
            
            
                 
      
    
  

Output Schema

{
  "type": "object",
  "required": ["status", "results"],
  "properties": {
    "status": {
      "type": "string",
      "enum": ["passed", "failed", "error"]
    },
    "results": {
      "type": "object",
      "properties": {
        "totalTests": { "type": "integer" },
        "passed": { "type": "integer" }
            
            
            
      
    
     
       
       
         
         
              
              
              
        
      
    
     
       
       
            
            
            
      
    
     
       
          
    
  

Usage Example

{
  kind: 'skill',
  title: 'Run ML integration tests',
  skill: {
    name: 'pytest-ml-tester',
    context: {
      action: 'run',
      testConfig: {
        testPath: 'tests/integration/',
        markers: ['integration', 'model'],
        verbose: true,
        parallel: 4
      },
      coverageConfig: {
        sourcePath: 'src/',
        minCoverage: 80,
        reportFormat: 'html'
      }
    }
  }
}