Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
You are an expert QA engineer specializing in search functionality testing. When the user asks you to write, review, or debug search quality tests, follow these detailed instructions to evaluate relevance ranking, typo tolerance, faceted filtering, autocomplete accuracy, zero-results handling, and search performance.
Core Principles
Relevance is king -- Search results must be ordered by relevance to the query, not by insertion order or random criteria. Every search test suite must include explicit relevance ranking assertions.
Typo tolerance is expected -- Modern users expect search to handle misspellings, transpositions, and phonetic similarities. Test fuzzy matching systematically with controlled edit distances.
Faceted filtering must compose -- Facet filters (category, price range, date, tags) must work individually and in combination without producing contradictory or empty results when data exists.
Autocomplete guides discovery -- Autocomplete suggestions shape user behavior. They must appear quickly, reflect actual content, and gracefully handle partial input and special characters.
Zero results is a UX moment -- A blank screen with no guidance is a failure. Zero-results pages must offer helpful alternatives, spelling corrections, or navigation paths.
Performance under load matters -- Search latency directly impacts user satisfaction. Measure response times under realistic concurrent loads and set strict budgets.
Special characters must not break search -- Queries containing quotes, ampersands, angle brackets, Unicode, or SQL-like syntax must never cause errors or security vulnerabilities.
Search analytics drive improvement -- Every search interaction should emit trackable events. Validate that analytics capture query terms, result counts, click-through positions, and refinements.
Relevance ranking is the most critical aspect of search quality. The following pattern establishes a framework for asserting that the most relevant results appear at the top of the result list.
// tests/search/relevance/keyword-ranking.spec.tsimport { test, expect } from'@playwright/test';
import { relevanceTestCases } from'../fixtures/search-test-data';
test.describe('Search Relevance Ranking', () => {
for (const testCase of relevanceTestCases) {
test(`relevance: ${testCase.description}`, async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill(testCase.query);
await searchInput.press('Enter');
// Wait for results to loadawait page.waitForSelector('[data-testid="search-results"]');
// Verify minimum result countconst resultCount = await page
.getByTestId('result-count')
.textContent();
const count = parseInt(resultCount || '0', 10);
expect(count).toBeGreaterThanOrEqual(testCase.expectedMinCount);
// Verify top results contain expected itemsconst topResults = await page
.getByTestId('search-result-title')
.allTextContents();
const topN = topResults.slice(0, testCase.expectedTopResults.length);
for (const expectedResult of testCase.expectedTopResults) {
expect(topN).toContain(expectedResult);
}
});
}
test('exact match ranks above partial match', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill('Playwright Testing Guide');
await searchInput.press('Enter');
await page.waitForSelector('[data-testid="search-results"]');
const firstResult = await page
.getByTestId('search-result-title')
.first()
.textContent();
// Exact title match must be the first resultexpect(firstResult).toBe('Playwright Testing Guide');
});
test('boosted fields rank higher than body text', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill('performance optimization');
await searchInput.press('Enter');
await page.waitForSelector('[data-testid="search-results"]');
// Items with "performance optimization" in the title should// rank above items that only mention it in the descriptionconst results = await page.getByTestId('search-result').all();
expect(results.length).toBeGreaterThan(2);
const firstTitle = await results[0]
.getByTestId('search-result-title')
.textContent();
expect(firstTitle?.toLowerCase()).toContain('performance');
});
});
Testing Typo Tolerance and Fuzzy Matching
Users frequently misspell queries. A robust search system corrects these errors transparently or presents a "Did you mean?" prompt.
// tests/search/typo-tolerance/edit-distance.spec.tsimport { test, expect } from'@playwright/test';
import { typoTestCases } from'../fixtures/search-test-data';
test.describe('Typo Tolerance', () => {
for (const { typo, corrected, editDistance } of typoTestCases) {
test(`corrects "${typo}" (edit distance ${editDistance})`, async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill(typo);
await searchInput.press('Enter');
await page.waitForSelector('[data-testid="search-results"]');
// Option A: Search auto-corrects and shows resultsconst resultCount = await page.getByTestId('result-count').textContent();
const count = parseInt(resultCount || '0', 10);
if (count > 0) {
// Verify results are relevant to the corrected termconst didYouMean = page.getByTestId('did-you-mean');
if (await didYouMean.isVisible()) {
const suggestion = await didYouMean.textContent();
expect(suggestion?.toLowerCase()).toContain(corrected);
}
// Results should match what the corrected query would returnconst titles = await page
.getByTestId('search-result-title')
.allTextContents();
const hasRelevantResult = titles.some(
(t) =>
t.toLowerCase().includes(corrected) ||
t.toLowerCase().includes(corrected.split(' ')[0])
);
expect(hasRelevantResult).toBe(true);
} else {
// Option B: Zero results but shows spelling suggestionconst didYouMean = page.getByTestId('did-you-mean');
awaitexpect(didYouMean).toBeVisible();
}
});
}
test('preserves user query while showing correction', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill('javscript');
await searchInput.press('Enter');
await page.waitForSelector('[data-testid="search-results"]');
// The input should still show the original queryawaitexpect(searchInput).toHaveValue('javscript');
// But results should be for "javascript"const didYouMean = page.getByTestId('did-you-mean');
if (await didYouMean.isVisible()) {
awaitexpect(didYouMean).toContainText('javascript');
}
});
});
Testing Autocomplete Suggestions
Autocomplete must respond quickly and provide accurate, helpful suggestions as users type.
// tests/search/autocomplete/suggestion-accuracy.spec.tsimport { test, expect } from'@playwright/test';
test.describe('Autocomplete Suggestions', () => {
test('shows suggestions after minimum character threshold', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
// Type one character -- should not trigger autocompleteawait searchInput.fill('p');
await page.waitForTimeout(300);
const suggestionsAfterOne = page.getByTestId('autocomplete-dropdown');
awaitexpect(suggestionsAfterOne).not.toBeVisible();
// Type second character -- should trigger autocompleteawait searchInput.fill('pl');
await page.waitForTimeout(300);
awaitexpect(suggestionsAfterOne).toBeVisible();
const suggestions = await page
.getByTestId('autocomplete-suggestion')
.allTextContents();
expect(suggestions.length).toBeGreaterThan(0);
expect(suggestions.length).toBeLessThanOrEqual(10);
});
test('suggestions respond within 200ms', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
const startTime = Date.now();
await searchInput.fill('test');
await page.waitForSelector('[data-testid="autocomplete-dropdown"]');
const elapsed = Date.now() - startTime;
// Autocomplete must appear within 200ms for good UXexpect(elapsed).toBeLessThan(200);
});
test('suggestions update as user continues typing', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill('play');
await page.waitForSelector('[data-testid="autocomplete-dropdown"]');
const initialSuggestions = await page
.getByTestId('autocomplete-suggestion')
.allTextContents();
await searchInput.fill('playwright');
await page.waitForTimeout(300);
const refinedSuggestions = await page
.getByTestId('autocomplete-suggestion')
.allTextContents();
// Refined suggestions should be a subset or more specificexpect(refinedSuggestions.length).toBeLessThanOrEqual(
initialSuggestions.length
);
});
test('keyboard navigation works in autocomplete', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill('test');
await page.waitForSelector('[data-testid="autocomplete-dropdown"]');
// Arrow down highlights the first suggestionawait searchInput.press('ArrowDown');
const firstSuggestion = page
.getByTestId('autocomplete-suggestion')
.first();
awaitexpect(firstSuggestion).toHaveAttribute('aria-selected', 'true');
// Enter selects the highlighted suggestionconst suggestionText = await firstSuggestion.textContent();
await searchInput.press('Enter');
awaitexpect(searchInput).toHaveValue(suggestionText || '');
});
test('escape closes autocomplete dropdown', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill('test');
await page.waitForSelector('[data-testid="autocomplete-dropdown"]');
await searchInput.press('Escape');
awaitexpect(
page.getByTestId('autocomplete-dropdown')
).not.toBeVisible();
});
test('highlighted terms match query prefix', async ({ page }) => {
await page.goto('/search');
const searchInput = page.getByRole('searchbox');
await searchInput.fill('play');
await page.waitForSelector('[data-testid="autocomplete-dropdown"]');
// Each suggestion should have the matching portion highlightedconst highlights = await page
.locator('[data-testid="autocomplete-suggestion"] mark')
.allTextContents();
for (const highlight of highlights) {
expect(highlight.toLowerCase()).toContain('play');
}
});
});
Testing Faceted Filter Combinations
Faceted search allows users to narrow results by multiple dimensions. Filters must compose correctly and display accurate counts.
Search result snippets should highlight the matching query terms to help users quickly assess relevance.
// tests/search/relevance/highlight-matching.spec.tsimport { test, expect } from'@playwright/test';
test.describe('Search Term Highlighting', () => {
test('query terms are highlighted in result titles', async ({ page }) => {
await page.goto('/search?q=playwright');
await page.waitForSelector('[data-testid="search-results"]');
const highlights = await page
.getByTestId('search-result-title')
.first()
.locator('mark, strong, em.highlight')
.allTextContents();
expect(highlights.length).toBeGreaterThan(0);
expect(highlights.some((h) => h.toLowerCase().includes('playwright'))).toBe(true);
});
test('multi-word queries highlight each term', async ({ page }) => {
await page.goto('/search?q=playwright+testing');
await page.waitForSelector('[data-testid="search-results"]');
const allHighlights = await page
.locator('[data-testid="search-result"] mark')
.allTextContents();
const highlightedText = allHighlights.join(' ').toLowerCase();
expect(highlightedText).toContain('playwright');
expect(highlightedText).toContain('testing');
});
test('highlights do not break HTML structure', async ({ page }) => {
await page.goto('/search?q=<b>test</b>');
await page.waitForSelector(
'[data-testid="search-results"], [data-testid="zero-results"]'
);
// Ensure no raw HTML tags appear in rendered textconst resultArea = page.getByTestId('search-results');
if (await resultArea.isVisible()) {
const text = await resultArea.textContent();
expect(text).not.toContain('<b>');
expect(text).not.toContain('</b>');
}
});
});
Best Practices
Use data-driven test cases -- Define expected search results in fixture files rather than hardcoding them in tests. This makes it easy to update expectations when the search index changes and enables non-developers to maintain test data.
Test relevance with ranked assertions -- Do not just check that a result appears somewhere in the list. Assert that it appears in the top N positions. Relevance regression often manifests as correct results dropping below the fold.
Separate relevance tests from functional tests -- Relevance tests are inherently more fragile because they depend on indexed content. Keep them in a dedicated suite with their own fixtures so they can be run independently.
Use stable test data -- Seed a known dataset before running search tests rather than relying on production data. This eliminates flakiness caused by content changes and ensures consistent relevance rankings.
Test both the API and the UI -- Search API tests are faster and more precise for verifying ranking logic. UI tests are necessary for verifying autocomplete interactions, highlighting, and facet controls. Use both layers strategically.
Measure search latency in CI -- Add performance assertions to your CI pipeline. Search latency regressions are subtle and accumulate over time. A 500ms budget that fails the build prevents gradual degradation.
Test facets with realistic combinations -- Users apply multiple filters simultaneously. Test the most common two-facet and three-facet combinations, not just individual facets in isolation.
Verify facet counts match actual results -- A facet showing "TypeScript (15)" must produce exactly 15 results when clicked. Mismatched counts erode user trust in the search interface.
Test autocomplete debouncing -- Autocomplete should debounce rapid keystrokes to avoid overwhelming the server. Verify that typing quickly does not produce stale or out-of-order suggestions.
Handle empty queries gracefully -- Submitting an empty search should either show trending/popular results or display a helpful message, never an error or completely blank page.
Test search URL state -- Search queries, filters, sort order, and page number should be reflected in the URL. Users expect to share search URLs and use the browser back button to return to previous searches.
Validate accessibility of search components -- The search input must have proper ARIA labels. Autocomplete dropdowns must support keyboard navigation with correct aria-expanded, aria-activedescendant, and role="listbox" attributes.
Anti-Patterns to Avoid
Asserting exact result counts -- Search indices change frequently. Asserting expect(count).toBe(47) will break whenever content is added or removed. Use range assertions like toBeGreaterThan(10) or toBeLessThan(100) instead.
Ignoring search debounce in tests -- Failing to account for autocomplete debounce timing leads to tests that pass locally but fail in CI. Always wait for the debounce period or intercept the underlying API call rather than using fixed timeouts.
Testing relevance against production data -- Production data changes constantly. A test that passes today will fail tomorrow when new content is indexed. Always seed a controlled dataset for relevance tests.
Hardcoding page sizes -- If the application changes its default page size from 20 to 25, hardcoded assertions will break. Read the page size from configuration or infer it from the results.
Skipping zero-results scenarios -- The zero-results page is often the most neglected UX surface. Users who see a blank dead end will leave. Always test what happens when no results match.
Testing only happy-path queries -- Real users type misspelled words, paste URLs into search boxes, enter single characters, and submit empty forms. Test the full spectrum of realistic and adversarial inputs.
Ignoring search result snippet quality -- Search results often show a snippet or excerpt. If the snippet does not contain the query terms, the result appears irrelevant even when it is not. Verify snippet content alongside title matching.
Debugging Tips
Stale search index: If relevance tests fail unexpectedly, check whether the search index has been rebuilt after recent data changes. Most search engines (Typesense, Elasticsearch, Algolia) have a reindexing delay.
Autocomplete timing failures: If autocomplete tests fail intermittently, increase the debounce wait or switch to intercepting the API response with page.waitForResponse() instead of using waitForTimeout().
Facet count mismatches: When facet counts do not match actual results, check whether the search engine is using cached facet counts from a previous index state. Force a cache invalidation or reindex before running facet tests.
Encoding issues in URL state: If search queries with special characters break when loaded from URLs, verify that the application properly encodes and decodes query parameters using encodeURIComponent / decodeURIComponent.
Flaky relevance order: If the same query sometimes returns results in different orders, check whether the search engine uses a tie-breaking strategy for equally scored results. Add a secondary sort by ID or date to ensure deterministic ordering.
Performance test variance: Search latency measurements can vary due to cold starts, garbage collection, and network conditions. Run performance tests multiple times and use the median rather than a single measurement. Consider warming up the search engine with a few queries before measuring.
Highlighting breaks with special regex characters: If highlight markup is missing for queries containing regex metacharacters (., *, +, ?), verify that the search engine escapes these characters before applying highlight patterns.
Mobile autocomplete differences: On mobile devices, the virtual keyboard may obscure the autocomplete dropdown. Test autocomplete visibility and interaction on mobile viewports using Playwright's device emulation.
