feature-testing

name	feature-testing
description	Guides manual Playwright testing and conversion into Drupal FunctionalJavascript tests for AI module features, including fixture and provider-recording workflows.

Feature Testing Skill for the Drupal AI Module

This skill guides you through testing AI module features: first manually with Playwright, then converting the manual test into a PHPUnit FunctionalJavascript test.

Environment

If they are using DDEV, please use ddev prefix on all drush and composer commands. This also means that npm and npx commands should be run from the host machine, not inside the DDEV container. For example, use ddev drush en ai_test -y to enable the module, and npx playwright test to run Playwright tests from the host.

Prerequisites

Before starting, collect the following from the user:

1. Local site URL (e.g., https://superagent.ddev.site)
2. Admin username and password for the local Drupal site
3. Whether a real AI provider is configured — if yes, we can record real responses; if not, we craft the YML fixtures manually
4. Playwright CLI must be installed — verify with npx playwright --version or playwright --version. If missing, install with npm init playwright@latest.

Phase 1: Manual Testing with Playwright

Step 1: Enable the ai_test module

The ai_test module lives at @web/modules/contrib/ai/tests/modules/ai_test/. It must be enabled on the local site so it can record AI requests and responses as they happen.

drush en ai_test -y

Step 2: Enable request recording (if a real AI provider is configured)

If the user has a real AI provider set up (e.g., OpenAI, Anthropic), enable recording so every AI call gets captured as an ai_mock_provider_result entity:

1. Navigate to /admin/config/ai/providers/ai-test
2. Check "Catch results" — this enables the LogMockRequests event subscriber which listens to PreGenerateResponseEvent and PostGenerateResponseEvent and saves every request/response pair as a content entity
3. Optionally check "Catch processing time" to record how long each call took (stored as sleep_time in milliseconds)
4. Save the form

When recording is enabled, every non-echoai AI call made on the site will be stored as an ai_mock_provider_result entity with fields:

• label — defaults to "Unnamed", can be renamed
• request — YAML-serialized request object
• response — YAML-serialized response object
• sleep_time — processing time in milliseconds
• operation_type — e.g., "chat", "embeddings", "text_to_image"
• mock_enabled — boolean flag for playback
• tags — tags from the original request

Step 3: Write a Playwright test script

Create a Playwright test that exercises the feature being tested. This test will:

• Log in to the Drupal site
• Navigate to the relevant page
• Interact with the feature (fill forms, click buttons, wait for AJAX)
• Assert the expected outcome
• Take screenshots at each step

Example Playwright test structure:

const { test, expect } = require('@playwright/test');

test.describe('Feature Name', () => {
  test('should do the thing being tested', async ({ page }) => {
    // Login
    await page.goto('SITE_URL/user/login');
    await page.fill('#edit-name', 'USERNAME');
    await page.fill('#edit-pass', 'PASSWORD');
    await page.click('#edit-submit');
    await expect(page).toHaveURL(/\/user\/\d+/);

    // Navigate to feature
    await page.goto('SITE_URL/path/to/feature');
    await page.screenshot({ path: 'screenshots/01-initial.png' });

    // Interact with the feature
    await page.fill('#edit-field-name', 'test value');
    await page.screenshot({ path: 'screenshots/02-filled-form.png' });

    // Trigger the AI action
    await page.click();
    
     page.( resp.().());
    
     page.();
     page.({ :  });

    
     resultValue =  page.();
    (resultValue).();
     page.({ :  });
  });
});

Step 4: Run the Playwright test

Run the Playwright test at least once to verify the manual flow works:

npx playwright test path/to/test.spec.js --headed

Use --headed so you can visually confirm the test is doing the right thing. If recording is enabled (Step 2), this run also captures the AI request/response pairs.

Step 5: Export recorded AI responses

After the Playwright test passes with a real AI provider:

1. Navigate to /admin/config/ai/providers/ai-test/collection to see all recorded ai_mock_provider_result entities
2. Find the relevant recorded requests
3. Click the "Export to test" operation on each entity — this downloads a YML file via the ExportForTesting controller
4. Alternatively, use the entity edit form to copy the request/response YAML

The exported YML file has this structure:

request:
  messages:
  - role: user
    text: "The prompt that was sent to the AI"
    images: []
    tools: null
    tool_id: null
  debug_data: {}
  chat_tools: null
  chat_structured_json_schema: {}
  chat_strict_schema: false
response:
  normalized:
    role: assistant
    text: "The AI response text"
    images: []
    tools: null
    tool_id: null
  rawOutput: []
  metadata: {}
  tokenUsage:
    input: null
    output: null
    total: null
    reasoning: null
    cached: null

Step 6: If no real AI provider is available

If there is no real AI provider configured, you must craft the YML fixture manually. Use the structure above and:

1. Determine what prompt/request the feature will generate by reading the module code
2. Create a plausible AI response that the test can assert against
3. The request must match exactly what the code sends (the EchoProvider matches via Json::encode($request['request']) === Json::encode($array))
4. Save the file as TestClassName.yml

Step 7: Place the YML fixture file

Place the exported/crafted YML file at:

your_module/tests/resources/ai_test/requests/{operation_type}/TestClassName.yml

Where {operation_type} is one of: chat, embeddings, text_to_image, text_to_speech, speech_to_text, moderation, image_classification, image_to_image.

The EchoProvider scans all enabled modules for files in tests/resources/ai_test/requests/{operation_type}/ and matches incoming requests against them during test playback.

Phase 2: Writing the FunctionalJavascript Test

FunctionalJavascript test essentials (from Drupal docs)

FunctionalJavascript tests use a real browser (via WebDriver/ChromeDriver) to test JavaScript-dependent behavior. Key points:

• Tests extend WebDriverTestBase (or in our case, BaseClassFunctionalJavascriptTests)
• Each test method gets a fresh Drupal installation with only the specified modules enabled
• Use $this->getSession()->getPage() to interact with the page (Mink API)
• Use $this->assertSession() for assertions
• Call $this->assertSession()->assertWaitOnAjaxRequest() after triggering AJAX operations
• Use $page->fillField('field_name', 'value') to fill form fields
• Use $page->pressButton('Save') or $this->click('.css-selector') to click elements
• Use $this->assertSession()->fieldValueEquals('field_name', 'expected') to assert field values
• Use $this->assertSession()->pageTextContains('text') to assert page content
• Use $this->assertSession()->waitForElementVisible('css', '.selector') to wait for elements
• Tests should be annotated with @group module_name

Step 1: Create the test class

All FunctionalJavascript tests for the AI module MUST extend BaseClassFunctionalJavascriptTests:

@web/modules/contrib/ai/tests/src/FunctionalJavascriptTests/BaseClassFunctionalJavascriptTests.php

This base class provides:

• $defaultTheme = 'stark' — uses the Stark theme
• $strictConfigSchema = FALSE — disables strict config schema checking
• takeScreenshot($filename) — captures the current page state as a PNG screenshot, organized by module and test class

Step 2: Write the test class

Place the test at the appropriate path under your module's tests/src/FunctionalJavascriptTests/ directory.

Example test structure (based on the real AutoCompleteTagsTaxonomyTest):

<?php

namespace Drupal\Tests\your_module\FunctionalJavascriptTests;

use Drupal\Tests\ai\FunctionalJavascriptTests\BaseClassFunctionalJavascriptTests;

/**
 * Tests the feature description.
 *
 * @group your_module
 */
class YourFeatureTest extends BaseClassFunctionalJavascriptTests {

  /**
   * {@inheritdoc}
   */
  protected static $modules = [
    'ai',
    'ai_test',  // REQUIRED — provides the EchoProvider mock
    // Add other modules your feature depends on.
    'node',
    'user',
  ];

  /**
   * {@inheritdoc}
   */
  protected $screenshotModuleName = 'your_module';

  /**
   * {@inheritdoc}
   */
  protected function setUp(): void {
    parent::setUp();
    // Create content types, fields, vocabularies, config entities, etc.
    // Load test configuration from YAML files if needed:
    // $data = Yaml::parseFile(__DIR__ . '/../../config/test_config.yml');
  }

  
   {
    
     = ->([
      ,
      ,
      // Add required permissions.
    ]);
    ->();

    
    ->();
    ->();

    
     = ->()->();

    
    ->(, );
    ->();

    
    ->();
    ->();

    
    ->()->();
    ->();

    
    ->()->(, );
    ->();
  }

}

Step 3: Enable video recording

When running the test via PHPUnit with ChromeDriver, enable video recording by adding browser capabilities to the test. You can do this by overriding $minkDefaultDriverArgs or setting environment variables for your WebDriver configuration.

For DDEV or local setups, configure the MINK_DRIVER_ARGS_WEBDRIVER environment variable to include video recording capabilities. Example:

export MINK_DRIVER_ARGS_WEBDRIVER='["chrome", {"browserName":"chrome","goog:chromeOptions":{"args":["--disable-gpu","--headless","--window-size=1920,1080"]}}, "http://localhost:9515"]'

For video recording specifically, use a Selenium Grid with video recording enabled, or configure your CI to capture the browser session.

Step 4: Take screenshots at every step

Every meaningful step in the test MUST call $this->takeScreenshot() with a descriptive filename. This creates a visual record of the test progression:

$this->takeScreenshot('01_initial_page');
// ... do something ...
$this->takeScreenshot('02_after_action');
// ... do something else ...
$this->takeScreenshot('03_final_state');

Screenshots are saved to sites/default/files/simpletest/screenshots/{module_name}/{TestClassName}/.

Step 5: Run the FunctionalJavascript test

# From the Drupal root directory
php core/scripts/run-tests.sh --class 'Drupal\Tests\your_module\FunctionalJavascriptTests\YourFeatureTest'

# Or with PHPUnit directly
./vendor/bin/phpunit -c core modules/contrib/your_module/tests/src/FunctionalJavascriptTests/YourFeatureTest.php

How request matching works at test time

During the FunctionalJavascript test:

1. The ai_test module is enabled (listed in $modules)
2. The EchoProvider (provider id: echoai) is automatically available
3. When the feature triggers an AI call using the echoai provider, the EchoProvider::getMatchingRequest() method:
   • Loads any ai_mock_provider_result entities from the database where mock_enabled = TRUE
   • Scans all enabled modules for YML files in tests/resources/ai_test/requests/{operation_type}/
   • Compares the incoming request (JSON-encoded) against each stored request
   • Returns the matching response if found
4. If no match is found, the EchoProvider returns a generic echo response

Test configuration files

If your test requires specific Drupal configuration (automators, form displays, field settings), store the config YAML files under:

your_module/tests/config/test_name/config_entity_id.yml

Load them in setUp():

$config_path = __DIR__ . '/../../config/test_name/';
$data = Yaml::parseFile($config_path . 'your_config.yml');
\Drupal::entityTypeManager()
  ->getStorage('entity_type')
  ->create($data)
  ->save();

Workflow Summary

1. Enable ai_test module on local site
2. Enable "Catch results" at /admin/config/ai/providers/ai-test (if real provider available)
3. Write and run a Playwright test against the local site (at least once)
4. Export captured request/response YML fixtures (or craft them manually)
5. Place YML fixtures at tests/resources/ai_test/requests/{operation_type}/TestName.yml
6. Write the FunctionalJavascript test extending BaseClassFunctionalJavascriptTests
7. Include ai_test in the test's $modules array
8. Take screenshots at every step with $this->takeScreenshot()
9. Run the PHPUnit test to verify it passes with the mock provider