eval-generator-gen-pages

name	eval-generator-gen-pages
argument-hint	[path to your .tsx generative page file] [optional: BRD file or OneDrive URL]
description	Generate a three-layer eval suite (presence checks + unit tests + security analysis) for Power Apps Generative Pages in Model-Driven Apps. Generative pages are single-file React/TypeScript components using Fluent UI V9 and the dataApi prop for Dataverse operations — NOT Code Apps, NOT PCF components. Code may be generated by the App Agent in make.powerapps.com or by AI code generation tools (Claude Code, GitHub Copilot CLI) using the /genpage skill. Always produces: evals/manifest.json, evals/presence/<id>.check.ts, evals/unit/<feature>.test.tsx, evals/runner/run-evals.ts, evals/runner/presence-runner.ts, evals/runner/security-runner.ts, and evals/dashboard/index.html. Triggers: 'eval my generative page', 'generate evals for gen page', 'review my genpage code', 'test my generative page', 'check my model driven app page', 'eval generator gen pages'.

Eval Generator — Power Apps Generative Pages

Purpose

Answer: "Is this Generative Page correctly structured, does the logic work, and are there security risks?"

Three eval layers are always generated:

1. Presence — static checks that assert code artifacts exist and follow Generative Page conventions
2. Unit — Vitest tests for business logic using a mocked dataApi
3. Security — static analysis across 14 security categories specific to gen page patterns

All three are mandatory.

---

Generative Page Stack — Critical Assumptions

Generative Pages are single-file React/TypeScript components embedded in Model-Driven Apps. They are NOT Code Apps, NOT PCF components. The architecture is fundamentally different from both.

Concern	Technology
Runtime	React 17 + TypeScript, compiled and hosted by Power Apps
UI components	Fluent UI V9 — @fluentui/react-components
Data access	dataApi prop — passed into the component by the Power Apps host
Data operations	dataApi.queryTable(), dataApi.retrieveRow(), dataApi.createRow(), dataApi.updateRow(), dataApi.deleteRow(), dataApi.getChoices()
Data source	Dataverse only (up to 6 tables per page)
Input params	props.recordId, props.entityName, props.data (optional, configured in page definition)
File structure	Single .tsx file — export default GeneratedComponent
Type generation	pac model genpage generate-types → local *.d.ts files
Deployment	pac model genpage upload via PAC CLI
Generated by	App Agent (make.powerapps.com) or AI code generation tools (Claude Code / GitHub Copilot CLI with /genpage skill)
Package manager	npm (for local development and eval harness)
Test runner	Vitest + @testing-library/react

⚠️ NEVER look for: power.config.json, src/generated/services/, @microsoft/power-apps, @microsoft/powerapps-component-framework, context.webAPI.*, context.parameters, initialize(), PowerProvider. These are Code App or PCF patterns — they do not exist in Generative Pages.

⚠️ ALWAYS look for: props.dataApi or dataApi usage, @fluentui/react-components imports, export default component with React TSX, Dataverse operation patterns via dataApi.*.

---

Step 0 — Snapshot Previous Results

Run this first, before collecting inputs or writing any files.

1. Check if evals/results/latest.json exists relative to the project directory.
2. If it does exist:
   • Read it.
   • Count files matching evals/results/snapshots/iter-*.json to determine N (next iteration = count + 1).
   • Write a copy to evals/results/snapshots/iter-<N>-<ISO-timestamp>.json.
   • Update evals/results/snapshots/index.json (create if absent, append if present).
3. If it does not exist — skip silently. This is the first invocation.

---

Step 0b — Collect Inputs

Ask the user (via m_ask_user for structured choices) for:

1. Generative Page file or directory — one of:

   • Path to a single .tsx file (the gen page)
   • Path to a directory containing one or more .tsx gen page files
   • Current working directory (default if already in the right place)

2. Requirements document — OPTIONAL. One of:

   • Local file path (.md, .txt, .docx)
   • OneDrive / SharePoint URL
   • Paste / describe in chat
   • None / Skip — features will be derived from code review (Step 1b)

3. Eval output mode (optional):

   • scaffold+write — write all eval files into an evals/ directory (default ✅)
   • describe-only — print what would be generated, no file writes

⚠️ Ask ALL questions in a SINGLE m_ask_user call before doing any file work.

Determine project directory: If user provided a single file, PROJECT_DIR = the file's parent directory. If a directory was given, PROJECT_DIR = that directory. All evals/ output goes into PROJECT_DIR/evals/.

---

Step 1 — Read Requirements Document (if provided)

If no requirements document was provided, skip to Step 1b.

Local .md or .txt

Use the view tool.

Local .docx

Use the docx skill. Call m_get_skill('docx') first.

OneDrive / SharePoint URL

Use m365_download_file (resolve file ID with m365_search_files first).

---

Step 1b — Code-Review Fallback (No BRD)

Run this step when no requirements document was provided, AND on every re-run.

Read all .tsx files in the project directory (non-recursively first, then src/ if present).

For each gen page file, use view and grep to identify:

Signal	Derived Feature
dataApi.queryTable('tableName', ...)	One feature per distinct tableName — data list/grid
dataApi.retrieveRow('tableName', ...)	Detail/record view feature
dataApi.createRow('tableName', ...)	Create/form feature
dataApi.updateRow('tableName', ...)	Edit/update feature
dataApi.deleteRow('tableName', ...)	Delete/remove feature
dataApi.getChoices(...)	Choice field rendering feature
props.recordId or props.entityName	Input parameter handling feature
Named React sections / major UI blocks	UI composition features (dashboards, cards, grids)
useState / useEffect clusters	State management features
Localization / translation dictionary	Localization feature
loadMoreRows / pagination	Pagination feature
Filter/search inputs	Search/filter feature
Chart / visualization components	Data visualization feature

Synthesize a feature entry per logical unit with id, title, description, acceptance_criteria, priority, and tables (Dataverse table names involved).

Set manifest.generatedFrom = "code-review".

Re-runs: Repeat Step 1b to pick up new components or new dataApi calls. Match features by id — preserve presence_status for any existing id.

---

Step 2 — Parse Requirements → Feature Manifest

Extract a structured feature list from the requirements document (Step 1) or use the code-derived list from Step 1b.

For each feature, capture:

• id — slugified identifier (e.g. account-list, contact-detail, opportunity-create)
• title — short label
• description — what the feature does
• acceptance_criteria — array of verifiable statements
• priority — high / medium / low (default medium)
• tables — Dataverse table logical names involved (e.g. ["account", "contact"])
• operations — dataApi operations expected: ["queryTable", "retrieveRow", "createRow"]

Track in SQL:

CREATE TABLE IF NOT EXISTS features (
  id TEXT PRIMARY KEY,
  title TEXT,
  description TEXT,
  acceptance_criteria TEXT,
  priority TEXT DEFAULT 'medium',
  tables TEXT,
  operations TEXT,
  presence_status TEXT DEFAULT 'pending',
  mapped_file TEXT
);

Write evals/manifest.json:

{
  "generated": "<ISO timestamp>",
  "project": "<page file name or directory basename>",
  "generatedFrom": "brd | code-review",
  "features": [
    {
      "id": "account-list",
      "title": "Account List",
      "description": "Display a paginated list of Account records using dataApi.queryTable",
      "acceptance_criteria": [
        "Calls dataApi.queryTable('account', { select: [...], pageSize: N })",
        "Shows loading state while data fetches",
        "Shows empty state when no records returned",
        "Handles dataApi errors with a user-friendly message"
      ],
      "priority": "high",
      "tables": ["account"],
      "operations" 
            
    
  

---

Step 3 — Audit the Generative Page File(s)

3a. Verify Generative Page structure

Use glob and view to confirm:

• At least one .tsx file exists with export default — the gen page component
• The file imports from @fluentui/react-components (Fluent UI V9)
• The component uses props.dataApi or destructures dataApi from props
• The component does NOT import from @microsoft/power-apps or @microsoft/powerapps-component-framework

If @microsoft/power-apps or power.config.json is found → stop and warn:

"This appears to be a Code App, not a Generative Page. Use the eval-generator-code-app skill instead."

If ComponentFramework or context.webAPI is found → stop and warn:

"This appears to be a PCF component. This skill only supports Power Apps Generative Pages."

3b. Map the gen page structure

Use grep on the .tsx file(s) to identify:

Pattern	Role
export default function |export default const	Root component (required)
import { .* } from '@fluentui/react-components'	Fluent UI V9 components in use
props\.dataApi|const { dataApi }	dataApi access pattern
dataApi\.queryTable\s*\(	Table query operations
dataApi\.retrieveRow\s*\(	Single record retrieval
dataApi\.createRow\s*\(	Record creation
dataApi\.updateRow\s*\(	Record update
dataApi\.deleteRow\s*\(	Record deletion
dataApi\.getChoices\s*\(	Choice column lookup
props\.recordId|props\.entityName|props\.data	Input parameter handling
hasMoreRows|loadMoreRows	Pagination
useState.*[Ll]oading|isLoading	Loading state
useState.*[Ee]rror|setError	Error state
\.length === 0|rows\.length === 0	Empty state
try\s*{	Error handling
useEffect\s*\(	Side effects / data fetching triggers

Record matched line numbers in the SQL features table (mapped_file, operations columns).

3c. Flag structural issues

A feature is Not Implemented if its expected dataApi.* call is not found. Mark presence_status = 'not_found'.

Flag these as warnings (not blockers) if found:

• dataApi.queryTable called without a select array → performance risk (fetches all columns)
• dataApi.* calls outside a try/catch block → unhandled async errors
• No loading state detected → poor UX
• No empty state detected → poor UX

---

Step 4 — Generate Presence Checks

Create evals/presence/<feature-id>.check.ts for each feature.

Presence check template

// evals/presence/<feature-id>.check.ts
// AUTO-GENERATED by eval-generator-gen-pages
// Feature: <feature title>
// Description: <feature description>

import { readFileSync, existsSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { globSync } from 'glob';

// Path-anchored to this file's location — works correctly from any working directory
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const PROJECT_DIR = join(__dirname, '..', '..');  // evals/presence → evals → project dir

interface PresenceResult {
  featureId: string;
  checks: Array<{ name: string; passed: boolean; detail?: string }>;
}

export function check(): PresenceResult {
  const checks: PresenceResult[] = [];

  
   tsxFiles = [
    ...(),
    ...(),
  ];

   (tsxFiles. === ) {
    checks.({ : , : ,
      :  });
     { : , checks };
  }

   allSource = tsxFiles.( (f, )).();

  
   dataApiUsed = .(allSource);
  checks.({
    : ,
    : dataApiUsed,
    : dataApiUsed ?  : ,
  });

  
   queryFound = .(allSource);
  checks.({
    : ,
    : queryFound,
    : queryFound ?  :  +  + ,
  });

  
   selectFound = .(allSource);
  checks.({
    : ,
    : selectFound,
    : selectFound ?  : ,
  });

  
  
   tryCatchFound = .(allSource.(, ));
  checks.({
    : ,
    : tryCatchFound,
    : tryCatchFound ?  : ,
  });

  
   loadingState = .(allSource);
  checks.({
    : ,
    : loadingState,
    : loadingState ?  : ,
  });

  
   errorState = .(allSource);
  checks.({
    : ,
    : errorState,
    : errorState ?  : ,
  });

  
   emptyState = .(allSource);
  checks.({
    : ,
    : emptyState,
    : emptyState ?  : ,
  });

  
   fluentV9 = .(allSource);
  checks.({
    : ,
    : fluentV9,
    : fluentV9 ?  : ,
  });

   { : , checks };
}

Generative Page-specific presence patterns

Feature type	Checks to generate
Data list / grid	dataApi.queryTable('tableName', {...}) called AND select: specified AND loading state AND empty state AND try/catch
Detail / single record	dataApi.retrieveRow('tableName', { id: ..., select: [...] }) AND input param (props.recordId) handled
Create form	dataApi.createRow('tableName', {...}) AND form validation AND submit disabled during save
Edit form	dataApi.updateRow('tableName', rowId, {...}) AND rowId sourced from props.recordId or state
Delete action	dataApi.deleteRow('tableName', rowId) AND confirmation UI (dialog/button) AND error handling
Choice fields	dataApi.getChoices('tableName-columnName') AND choices rendered as dropdown or display label
Input params	props.recordId, props.entityName, or props.data destructured AND used in useEffect or initialization
Pagination	hasMoreRows checked AND loadMoreRows() callable AND loading state for next-page fetch
Localization	Translation dictionary or useEffect for language detection AND date/number locale formatting
Search / filter	Filter state connected to dataApi.queryTable filter option AND no raw user input in filter string
Data visualization	Chart component found AND data mapped from dataApi.queryTable result
Error disclosure	No e.message or String(e) set directly into user-visible state

⚠️ Never generate presence checks that grep for context.webAPI, initialize(), PowerProvider, src/generated/, or result.value — these are Code App / PCF patterns and will never appear in a Generative Page.

---

Step 5 — Generate Vitest Unit Tests

⛔ This step is MANDATORY on every invocation. Write real unit tests grounded in actual source code. Use it.todo() stubs only when a feature has no testable logic. Never skip this step — even on re-runs.

Re-run behaviour

If evals/unit/ already exists, merge — add tests for new features, preserve passing tests for existing ones. Never delete existing passing test files.

File layout

evals/unit/
  helpers/
    setup.ts          → @testing-library/jest-dom matchers, global mocks
    mocks.ts          → mockDataApi factory (vi.fn() for all 6 methods)
    factories.ts      → createMock<TableName>Row() helpers for Dataverse rows
  <Feature>.test.tsx  → one file per logical feature group

evals/unit/helpers/setup.ts

// evals/unit/helpers/setup.ts
import '@testing-library/jest-dom';
import { vi } from 'vitest';

// Suppress FluentUI console warnings in tests
vi.spyOn(console, 'warn').mockImplementation(() => {});

evals/unit/helpers/mocks.ts

// evals/unit/helpers/mocks.ts
import { vi } from 'vitest';

// mockDataApi — the dataApi prop passed by Power Apps host, fully mocked
export function createMockDataApi() {
  return {
    queryTable: vi.fn().mockResolvedValue({ rows: [], hasMoreRows: false }),
    retrieveRow: vi.fn().mockResolvedValue({}),
    createRow: vi.fn().mockResolvedValue('new-row-id'),
    updateRow: vi.fn().mockResolvedValue(undefined),
    deleteRow: vi.fn().mockResolvedValue(undefined),
    getChoices: vi.fn().mockResolvedValue([]),
  };
}

evals/unit/helpers/factories.ts

⚠️ CRITICAL: Before writing this file, read the actual gen page source to find what fields each dataApi.queryTable result is used for. Factory fields must match the actual column names the component reads from rows. Never invent field names.

// evals/unit/helpers/factories.ts
// Grounded in actual source: fields match what the component reads from dataApi results

// Example — replace with actual fields from source
export function createMockAccountRow(overrides: Partial<Record<string, unknown>> = {}) {
  return {
    accountid: 'acc-001',
    name: 'Contoso Ltd',
    emailaddress1: '[e-mail]',
    telephone1: '555-0100',
    statecode: 0,
    ...overrides,
  };
}

Unit test template — per feature

// evals/unit/<feature>.test.tsx
// Feature: <feature title>
// Tests grounded in actual source: <gen page file name>

import { describe, it, expect, vi, beforeEach } from 'vitest';
import { render, screen, waitFor, fireEvent } from '@testing-library/react';
import { createMockDataApi } from './helpers/mocks';
import { createMockAccountRow } from './helpers/factories';

// Import the default export from the gen page file
// Adjust path relative to evals/unit/ — typically ../../PageName.tsx
import GeneratedComponent from '../../<PageName>.tsx';

describe('<feature-id>', () => {
  let dataApi: ReturnType<typeof createMockDataApi>;

  beforeEach(() => {
    dataApi = createMockDataApi();
  });

  it('<feature-id>: renders loading state while data fetches', async () => {
    // Delay resolution so we can catch the loading state
    dataApi.queryTable.mockReturnValue(new Promise(() => {}));
    render(<GeneratedComponent dataApi={dataApi} />);
    
    
     loadingIndicator =
      screen.() ??
      screen.() ??
      screen.();
    (loadingIndicator).();
  });

  (,  () => {
     rows = [(), ({ : , :  })];
    dataApi..({ rows, :  });
    ();
     ( {
      (screen.()).();
      (screen.()).();
    });
  });

  (,  () => {
    dataApi..({ : [], :  });
    ();
     ( {
      
      (screen.()).();
    });
  });

  (,  () => {
    dataApi..( ());
    ();
     ( {
      
      (screen.()).();
    });
  });

  (,  () => {
    dataApi..({ : [], :  });
    ();
     ( {
      (dataApi.).(
        ,   
        expect.({ : expect.([]) })
      );
    });
  });
});

it() naming convention — CRITICAL for dashboard traceability

Every it() description MUST begin with the exact feature id from manifest.json, followed by a colon:

it('account-list: renders loading state while data fetches', ...)    // ✅ correct
it('account list: renders loading state', ...)                        // ❌ wrong — space not hyphen
it('renders loading state', ...)                                      // ❌ wrong — no feature ID prefix

The runner extracts the feature ID using /([\w-]+):\s*(.+)$/ and uses it to map test results to dashboard rows. A mismatch causes tests to pass but show "⏭ No tests" per feature in the dashboard.

Post-generation verification (MANDATORY)

After writing all test files, grep each test file for each manifest feature ID to confirm traceability:

For each feature.id in manifest.json:
  grep -c "it\('" + feature.id + ":" in evals/unit/**/*.test.tsx
  → If count === 0: traceability broken — fix the it() names before proceeding

---

Step 5b — Eval Package Setup

The gen page project may not have a package.json with test dependencies. Check:

1. Does PROJECT_DIR/package.json exist?
2. If yes, does it have vitest, @testing-library/react, jsdom in dev dependencies?
3. If no to either: create or patch package.json with the minimal eval harness.

Minimal package.json for evals (create only if needed)

{
  "name": "<project-basename>-evals",
  "private": true,
  "type": "module",
  "scripts": {
    "eval": "tsx evals/runner/run-evals.ts",
    "eval:presence": "tsx evals/runner/presence-runner.ts",
    "eval:security": "tsx evals/runner/security-runner.ts",
    "eval:unit": "vitest run --config evals/vitest.config.ts"
  },
  "devDependencies": {
    "@testing-library/jest-dom": "^6.0.0",
    "@testing-library/react": "^16.0.0",
    "@fluentui/react-components": "^9.0.0",
    "jsdom": "^25.0.0",
    "react": 
     
     
     
     
     
     
  

If package.json already exists, append only the missing dev dependencies and scripts — never overwrite existing ones.

After writing package.json, run npm install to install dependencies.

evals/vitest.config.ts

// evals/vitest.config.ts
import { defineConfig } from 'vitest/config';
import { resolve, dirname } from 'path';
import { fileURLToPath } from 'url';

const __dirname = dirname(fileURLToPath(import.meta.url));
const PROJECT_ROOT = resolve(__dirname, '..');   // evals/ → project root

export default defineConfig({
  root: PROJECT_ROOT,   // CRITICAL: anchors all paths to project root, not evals/
  test: {
    environment: 'jsdom',
    globals: true,
    setupFiles: [resolve(__dirname, 'unit/helpers/setup.ts')],
    include: ['evals/unit/**/*.test.{ts,tsx}'],
    reporters: ['json'],
    outputFile: resolve(__dirname, 'results/unit-results.json'),
  },
  resolve: {
    alias: {
      // Allow tests to import from the project root without relative ../../
      '@': PROJECT_ROOT,
    },
  },
});

⚠️ root: PROJECT_ROOT is mandatory. Without it, Vitest defaults root to evals/, causing all setupFiles and include paths to resolve from evals/evals/... — a path that doesn't exist.

---

Step 6 — Generate Security Runner

Write evals/runner/security-runner.ts:

#!/usr/bin/env tsx
// evals/runner/security-runner.ts
// Static security analysis for Power Apps Generative Pages

import { readFileSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { globSync } from 'glob';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const ROOT = join(__dirname, '..', '..');   // evals/runner → evals → project root

type Severity = 'critical' | 'high' | 'medium' | 'low';

interface SecurityFinding {
  id: string; category: string; severity: Severity;
  file: string; line: number; snippet: string; detail: string;
}

interface SecurityResult {
  : ;
  : ; : ; : ;
  : [];
}

 : [] = [];

 srcFiles = (, { :  });

 () {
   ( file  srcFiles) {
     : ;
     { content = (file, ); }  { ; }
     lines = content.();
    lines.( {
       (pattern.(line) && !(exclude && exclude.(line))) {
        findings.({ id, category, severity,
          : file.( + , ).( + , ),
          : i + , : line.().(, ), detail });
      }
    });
  }
}


(, , ,
  ,
  );


(, , ,
  ,
  );





(, , ,
  ,
  ,
  );



(, , ,
  ,
  ,
  );


(, , ,
  ,
  );


(, , ,
  ,
  ,
  );


(, , ,
  ,
  );



(, , ,
  ,
  );



(, , ,
  ,
  );


(, , ,
  ,
  );


(, , ,
  ,
  );

(, , ,
  ,
  );

(, , ,
  ,
  );


(, , ,
  ,
  ,
  );

(, , ,
  ,
  );


(, , ,
  ,
  );

(, , ,
  ,
  );


(, , ,
  ,
  );

  = [
  ,,,,,,
  ,,,,,,
  ,,
];

 hitIds =  (findings.( f.));
 :  = {
  : ,
  : .( !hitIds.(c)).,
  : hitIds.,
  : ,
  findings,
};
process..(.(result));

---

Step 7 — Generate the Eval Runners

evals/runner/presence-runner.ts

#!/usr/bin/env tsx
// evals/runner/presence-runner.ts

import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { pathToFileURL } from 'url';
import { globSync } from 'glob';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const PRESENCE_DIR = join(__dirname, '..', 'presence');

interface LayerResult {
  layer: 'presence';
  passed: number; failed: number; skipped: number;
  details: Array<{ featureId: string; name: string; status: 'pass' | 'fail' | 'skip'; detail?: string }>;
}

const checkFiles = globSync(`${PRESENCE_DIR}/*.check.ts`).sort();
 : [] = [];
 passed = , failed = ;

 ( file  checkFiles) {
   mod =  ((file).);
   result = mod.();
   ( c  result.) {
     status = c. ?  : ;
     (c.) passed++;  failed++;
    details.({ : result., : c., status, : c. });
  }
}

 :  = { : , passed, failed, : , details };
process..(.(output));

evals/runner/run-evals.ts

⚠️ CRITICAL: Copy the code below EXACTLY. Do not simplify or restructure.

#!/usr/bin/env tsx
// evals/runner/run-evals.ts
// Orchestrates all three eval layers and bakes results into the dashboard.

import { execSync, execFileSync } from 'child_process';
import { readFileSync, writeFileSync, mkdirSync, existsSync, copyFileSync } from 'fs';
import { join, dirname, resolve } from 'path';
import { fileURLToPath, pathToFileURL } from 'url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const ROOT = join(__dirname, '..', '..');  // evals/runner → evals → project root
const RESULTS_DIR = join(ROOT, 'evals', 'results');
mkdirSync(RESULTS_DIR, { recursive: true });

function run(cmd: string, label: string): string {
  try {
    return execSync(cmd, { : , : , :  });
  }  (: ) {
    
     e. || e.?.[] || ;
  }
}

.();
 presOut = (, );

.();
 unitRawOut = (, );

.();
 secOut = (, );


 :  = { : , : , : , : , : [] };
 { presLayer = .(presOut); }  { .(); }


 :  = { : , : , : , : , : [] };
 {
   vitestJson = .(unitRawOut);
   : [] = [];
   p = , f = , s = ;
  
   ( suite  vitestJson. || []) {
     ( t  suite. || []) {
       m = t.?.();
       featureId = m ? m[] : ;
       name = t. || t.;
       status = t. ===  ?  : t. ===  ?  : ;
       detail = t.?.().(, ) || ;
      details.({ featureId, name, status, detail });
       (status === ) p++;   (status === ) s++;  f++;
    }
  }
  unitLayer = { : , : p, : f, : s, details };
}  { .(); }


 :  = { : , : , : , : , : [] };
 { secLayer = .(secOut); }  { .(); }


 :  = { : [] };
 manifestPath = (, , );
 { manifest = .((manifestPath, )); }  {}


  = (, );
 : [] = [];
 {
   idx = .(((, ), ));
  snapshots = idx.( {
     {  .(((, s.), )); }  {  ; }
  }).();
}  {}

 timestamp =  ().();
 evalResults = {
  timestamp,
  : manifest. || ,
  : snapshots. + ,
  : [presLayer, unitLayer, secLayer],
  snapshots,
  : {
    : manifest..,
    : manifest. || [],
    : manifest. || [],
    : secLayer. || [],
  },
};


((, ), .(evalResults, , ));


 dashPath = (, , , );
 ((dashPath)) {
   html = (dashPath, );
   dataBlock = ;
  html = html.(, dataBlock);
  (dashPath, html);
  .();
}


 totalFailed = presLayer. + unitLayer.;
.();
.();
.();
 (totalFailed > ) { .(); process.(); }
.();

---

Step 8 — Generate the Dashboard

Step 8a — Read template

Read the dashboard template from:

path: ~/.copilot/m-skills/eval-generator-gen-pages/dashboard-template.html

Expand ~ to the user's actual home directory. Use PowerShell if needed: (Resolve-Path ~).Path.

Step 8b — Adapt security categories for gen pages

The dashboard template's CATEGORIES array in renderSecurity() must be replaced with the gen-page-specific categories:

var CATEGORIES=[
  {id:'GP-SEC-001',label:'OData Injection',          severity:'critical', what:'filter strings built from user input — injection risk for Dataverse queries'},
  {id:'GP-SEC-002',label:'XSS',                      severity:'critical', what:'dangerouslySetInnerHTML or .innerHTML= — prefer Fluent UI components which escape automatically'},
  {id:'GP-SEC-003',label:'Hardcoded Secret',          severity:'critical', what:'API keys, passwords, or tokens assigned as string literals in source'},
  {id:'GP-SEC-004',label:'PII in Logs',               severity:'high',     what:'Sensitive fields (email, userId, DisplayName) written to the browser console'},
  {id:'GP-SEC-005',label:'Error Disclosure',          severity:'high',     what:'Raw e.message or error.message surfaced directly to UI state'},
  {id:'GP-SEC-006',label:'Unsafe fetch() Call',       severity:'high',     what:},
  {:,:,   :,     :},
  {:,:,    :,   :},
  {:,:,  :,   :},
  {:,:,   :,      :},
  {:,:,    :, :},
  {:,:,:,    :},
  {:,:,  :,     :},
  {:,:,  :,   :},
];

Step 8c — Make exactly three substitutions in the template

1. Replace every occurrence of PROJECT_NAME with the gen page file/directory name

2. Replace ITERATION_NUMBER with the current iteration number

3. Replace the FEATURES placeholder:

     /* GENERATED — replace with actual features: { id, title, priority, note? } */
   

   with the actual feature array from the manifest:

   { id: 'account-list', title: 'Account List', priority: 'high' },
   { id: 'contact-detail', title: 'Contact Detail', priority: 'high' },
   

4. Replace the CATEGORIES array in renderSecurity() with the gen-page categories above.

Step 8d — Write evals/dashboard/index.html

Step 8e — Self-verify (THREE checks, all mandatory)

After writing, verify all three:

1. Grep for fetch( — if found: delete and repeat from 8a
2. Grep for FEATURES = [ → confirm next non-whitespace line is NOT /* GENERATED — if placeholder present: delete and repeat
3. Grep for PROJECT_NAME — if still present: delete and repeat

---

Step 9 — Package.json Script

Ensure package.json has these scripts (add if missing, never overwrite existing):

{
  "scripts": {
    "eval": "tsx evals/runner/run-evals.ts",
    "eval:presence": "tsx evals/runner/presence-runner.ts",
    "eval:security": "tsx evals/runner/security-runner.ts",
    "eval:unit": "vitest run --config evals/vitest.config.ts"
  }
}

---

Step 10 — Run Evals

Run:

npm run eval

from PROJECT_DIR. Capture output. If failures are reported, summarise them for the user with specific file and line references.

After a successful run, tell the user to open evals/dashboard/index.html in their browser.

---

Re-run Guard

When evals/ already exists:

Artifact	Rule
manifest.json	Regenerated (may have new features from code changes)
presence/*.check.ts	Regenerated for all features
unit/helpers/	Preserved — never overwritten
unit/*.test.tsx	Merged — add tests for new features, preserve passing tests
runner/*.ts	Regenerated
dashboard/index.html	Regenerated (FEATURES + CATEGORIES updated)
results/latest.json	Overwritten after snapshot

⚠️ Unit tests are ALWAYS merged, never skipped on re-runs. Any interpretation of "merge" as "skip unit test generation" is incorrect.

---

Mandatory Checklist (verify before delivering)

• evals/manifest.json written with correct feature IDs
• One .check.ts per feature in evals/presence/
• evals/unit/helpers/setup.ts, mocks.ts, factories.ts written
• Factory fields grounded in actual source column names (never invented)
• One .test.tsx per feature group in evals/unit/
• Every it() name starts with exact manifest.json feature ID + colon
• Post-generation grep confirms traceability for all feature IDs
• evals/runner/security-runner.ts written with GP-SEC-001 through GP-SEC-014
• evals/runner/presence-runner.ts and run-evals.ts written
• evals/vitest.config.ts has root: PROJECT_ROOT set explicitly
• evals/dashboard/index.html written, FEATURES substituted, CATEGORIES substituted
• Dashboard self-verify: no fetch(, no placeholder, no PROJECT_NAME
• npm run eval runs successfully (or failures reported to user)
• Dashboard baked with real results

---

Quality Rules

1. Never invent dataApi call patterns — grep the actual source to confirm what tables and operations are used
2. Never reference Code App patterns — no power.config.json, src/generated/, @microsoft/power-apps, result.value, result.success, initialize()
3. Never reference PCF patterns — no context.webAPI, context.parameters, ComponentFramework
4. Factory fields must come from source — read the .tsx file to see what fields the component accesses from rows before writing factories
5. Feature IDs are immutable — once assigned in manifest.json, every downstream artifact must use them verbatim
6. Error handling in every dataApi call — always generate presence checks that verify try/catch exists
7. select: is always best practice — flag absence of select in queryTable as GP-SEC-014

---

Error Handling

Situation	Action
No .tsx file found	Ask user to confirm the path; offer to search recursively
@microsoft/power-apps found	Stop: redirect to eval-generator-code-app skill
ComponentFramework found	Stop: this is a PCF component, wrong skill
npm install fails	Show error; user may need to resolve peer deps manually
npm run eval fails (unit)	Report specific test names and failure messages
Dashboard FEATURES still shows placeholder after write	Delete and re-write; never deliver empty dashboard
Feature ID mismatch detected in post-generation grep	Fix it() names before running evals
evals/ already exists	Merge (never skip unit tests), snapshot first