xiaohongshu-search-full

# Xiaohongshu — Search Notes (Full Fields) > keyword + filters → note list with full metadata (title, body, images, video URL, tags, stats) + detail enrichment ## Language All process output to user (progress updates, process notifications) follows the user's language. ## Objective Search Xiaohongshu notes by keyword, apply page filter options, and extract the maximum available fields from both the search results list and individual note detail pages. ## Prerequisites - Browser opened to `https://www.xiaohongshu.com/search_result/?keyword={keyword}` - User is logged in (avatar or username visible in the left sidebar) ## Phase 0: Collect User Inputs **First action**: check whether the search keyword is already present in the user's message. **Keyword is present** (e.g., "搜索 BrowserAct", "search for AI tools", "/xiaohongshu-search-full blockchain") → use it directly, skip to Pre-execution Checks. **Keyword is NOT present** → STOP. Do NOT output "ready" or proceed with any execution. Ask the user for the keyword: - If the **AskUserQuestion** tool is available → call it immediately with: - Question 1 (required): "请问您想在小红书上搜索什么关键词？" - Question 2 (optional): pages needed (default: first page only) - Question 3 (optional): filters — sort order, note type, publish time range - If AskUserQuestion is NOT available → output the following text and wait for the user's reply before doing anything else: > "请问您想搜索什么关键词？（如需指定页数或筛选条件，也可一并告知）" Do NOT guess, infer, or assume any keyword. Wait for the user's explicit reply. ## Pre-execution Checks ### 1. Tool Readiness If browser-act has been confirmed available in the current session → skip this step. Invoke `browser-act` via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry. ### 2. Login Verification If login status for Xiaohongshu has been confirmed in the current session → skip this step. Otherwise: open `https://www.xiaohongshu.com` and observe the left sidebar: - User avatar or "Me" entry visible → logged in, continue execution - "Login" button visible → not logged in, inform the user that login is required, use `remote-assist` to let the user scan the QR code User refuses or cannot log in → terminate execution. ## Capability Components > This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the `scripts/` directory, invoked via `python scripts/xxx.py {params} | browser-act --session <name> eval --stdin`. Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read `scripts/*.py` source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution. ### DOM: extract search feeds list (preferred method) Navigate to the search page, then read the rendered note list directly from Vue state. This is the **preferred method** because it reflects exactly what the page renders — including exact-match notes that the search API omits due to semantic expansion. 1. `navigate https://www.xiaohongshu.com/search_result/?keyword={keyword}` 2. `wait stable` 3. (optional) Apply filters — see **AI Workflow: apply filters** below 4. `python scripts/extract-search-feeds.py | browser-act --session <name> eval --stdin` — retrieves all page results. **Do NOT pass `--keyword` for a full keyword search** (title + body); see **AI Workflow: keyword search (title + body match)** below. Pass `--keyword {keyword}` only when title-only pre-filtering is explicitly required. Parameters: - `--keyword {keyword}` (optional): plain text keyword — when passed, `items` contains only title-matched notes and `keyword_count` is populated. Omit to get all rendered notes (required for body-text matching). Output example: ```json { "total_count": 40, "keyword_count": 3, "has_more": true, "items": [ { "id": "6a422f88000000001603cdd3", "xsec_token": "ABbCLyAhrP9qgixS_rCW...", "note_url": "https://www.xiaohongshu.com/explore/6a422f88000000001603cdd3?xsec_token=ABbCLy...%3D&xsec_source=pc_search", "type": "video", // "normal" = image-text, "video" = video note "title": "note title text here", "publish_date": "06-29", // human-readable date string (not exact timestamp) "cover_url": "http://sns-webpic-qc.xhscdn.com/...", "liked_count": "3", "collected_count": "7", "comment_count": "0", "shared_count": "0", "author_nickname": "author nickname here", "author_id": "670caaaa000000001d0239d1", "author_avatar": "https://sns-avatar-qc.xhscdn.com/..." } ] } ``` - `total_count`: all notes rendered by the page (includes semantically expanded results) - `keyword_count`: notes whose **title** fuzzy-matches the keyword (Levenshtein edit distance ≤ 20% of keyword length, min 1 edit) - `items`: the fuzzy-filtered list (only notes whose title fuzzy-matches the keyword) - `keyword_count: null` means no `--keyword` was passed and `items` = all notes Fuzzy matching details: `--keyword` uses Levenshtein edit distance with threshold `max(1, floor(kw.length * 0.2))` per sliding window over each word in the title. This catches typos, capitalization variants (e.g., `linfox` → `LinkFox`, `linxfox`), and minor spelling differences within the threshold. Note: `has_more` indicates whether more pages exist. `body text (desc)`, `topics (tagList)`, `video stream URL`, and `exact publish timestamp (ms)` are NOT in this response — use the detail component below to obtain them. When `--keyword` is passed, matching is title-only (fuzzy); for title + body matching, omit `--keyword` and follow **AI Workflow: keyword search (title + body match)** below. Error handling: if `error: true`, verify `wait stable` completed and the page is not a login gate. Reload and retry once. ### AI Workflow: keyword search (title + body match) Run this workflow whenever a keyword is provided. It finds all notes where the keyword appears in either the **title** OR the **body text (desc)**. **Step 1 — Extract all page results** (no `--keyword` flag): ```bash python scripts/extract-search-feeds.py | browser-act --session <name> eval --stdin ``` → `all_items` list (`total_count` items, no pre-filtering) **Step 2 — Title filter** (in-memory, no extra requests): - `title_matches` = items where `keyword` fuzzy-matches `title` (use same Levenshtein threshold as `--keyword`: `max(1, floor(len(keyword) * 0.2))` edits per word, case-insensitive, exact substring first) - `candidates` = remaining items (ALL items not matched by title fuzzy match) - Inform user: "Found {len(title_matches)} title match(es). Checking body text of {len(candidates)} candidates..." - **CRITICAL: When `title_matches == 0`, do NOT stop. Zero title matches means body-text checking is MORE important, not less — the keyword may appear only in the body. Proceed to Step 3 unconditionally unless `total_count == 0`.** **Step 3 — Body-text check** (detail page per candidate, skip only if `candidates` is empty): For each item in `candidates`: 1. `browser-act --session <name> navigate https://www.xiaohongshu.com/explore/{id}?xsec_token={xsec_token}&xsec_source=pc_search` 2. `browser-act --session <name> wait stable` 3. `python scripts/extract-note-detail.py {id} | browser-act --session <name> eval --stdin` 4. If `keyword` fuzzy-matches (case-insensitive, same Levenshtein threshold) in `desc` → add to `body_matches` 5. Wait 2–3 seconds before next request. **Step 4 — Report**: - `final_results` = `title_matches` + `body_matches` (in original page order, deduplicated by `id`) - Inform user: "{len(title_matches)} title match(es) + {len(body_matches)} body-only match(es) = {total} notes containing '{keyword}' (from {total_count} page results)" - Clearly label each result: `match_type: "title"` or `match_type: "body"` **Edge cases**: - `title_matches == 0 AND body_matches == 0` → inform user: no notes found containing `{keyword}` in title or body text; suggest checking login status or trying a different keyword - `total_count == 0` → inform user: no results at all; suggest checking login or trying a different keyword ### Network Capture: search notes list (fallback method) Use as fallback only when the DOM feeds extraction above fails. Navigate to the search page and read results from the `so.xiaohongshu.com` API response captured in browser traffic. **Important limitation**: the search API applies semantic expansion — for niche or brand-specific keywords, exact-match notes may be absent or pushed to later pages in the API response even when they appear prominently in the rendered page. Always prefer the DOM feeds method above. 1. `navigate https://www.xiaohongshu.com/search_result/?keyword={keyword}` 2. `wait stable` 3. (optional) Apply filters — see **AI Workflow: apply filters** below 4. `network requests --type xhr,fetch --filter so.xiaohongshu` 5. Identify the request with URL containing `/api/sns/web/v2/search/notes` 6. `network request <id>` 7. Parse `response_body` JSON — field paths: `data.items[n].id`, `data.items[n].xsec_token`, `data.items[n].note_card.display_title`, `data.items[n].note_card.interact_info.*` Note: `data.has_more` indicates whether more pages exist. Error handling: If no request matching `/api/sns/web/v2/search/notes` is found, check that `wait stable` completed and the page is a search result page (not a login gate). Reload the page and retry once. ### DOM: extract note detail (body, topics, video URL, exact timestamp) Navigate to the note detail page and extract enriched fields from the Vue SSR state: 1. `navigate https://www.xiaohongshu.com/explore/{note_id}?xsec_token={xsec_token}&xsec_source=pc_search` 2. `wait stable` 3. `python scripts/extract-note-detail.py {note_id} | browser-act --session <name> eval --stdin` Parameters: - `{note_id}`: note ID from the search list result (`id` field) - `{xsec_token}`: security token from the search list result (`xsec_token` field) Output example: ```json { "noteId": "694692cc000000001d039ea3", "title": "note title here", "desc": "note body text here", // full body text "type": "video", // "normal" or "video" "time": 1766232780000, // exact publish timestamp (milliseconds) "ipLocation": "Fujian", // IP location, null if unavailable "userId": "59e9658411be10340721cd79", "nickname": "author nickname here", "avatar": "https://sns-avatar-qc.xhscdn.com/avatar/...", "likedCount": "0", "collectedCount": "0", "commentCount": "0", "shareCount": "0", "tagList": [ {"name": "topic name", "type": "topic", "id": "123"} // topics/hashtags ], "imageList": [ {"url": "http://sns-webpic-qc.xhscdn.com/...", "width": 720, "height": 960} ], "videoUrl": "http://sns-video-v6.xhscdn.com/stream/1/110/259/..." // null for image-text notes } ``` Error handling: if `error: true` is returned, verify the page URL is a note detail page, `wait stable` has completed, and `note_id` matches the URL. If `noteDetailMap` is empty after wait, try `wait --selector ".note-content" --state attached --timeout 15000` then re-run the extraction. ### AI Workflow: apply filters (before search capture) Run this workflow **before** step 4 of the search capture component when filters are needed. Open the filter panel and select desired options, then wait for the filtered search request to fire. 1. `state` — locate the "Filter" button (labeled with filter icon) in the top-right area of the search content area → `click <index>` 2. Wait for filter panel to appear (visible on the right side of the page) 3. For **sort order** — `state` locate the desired sort tag in the "Sort By" row → `click <index>` 4. For **note type** — `state` locate the desired type tag in the "Note Type" row → `click <index>` 5. For **publish time** — `state` locate the desired range tag in the "Publish Time" row → `click <index>` 6. For **search scope** — `state` locate the desired scope tag in the "Search Scope" row → `click <index>` 7. For **location distance** — `state` locate the desired distance tag in the "Location Distance" row → `click <index>` 8. `wait stable` 9. Proceed with `network requests --type xhr,fetch --filter so.xiaohongshu` to capture the filtered results Filters are sent in the POST body `filters` array. Each filter entry has shape `{"type": "{filter_id}", "tags": ["{selected_value}"]}`. See Enum Parameters for all verified values. ### Composite: search + enrich (search list → detail page loop) Use when full fields including body text, topics, video URL, and exact timestamp are needed for **all** notes regardless of keyword filtering. Also used as a sub-step in **AI Workflow: keyword search (title + body match)** for the body-text candidate checking phase. 1. Navigate to search page and extract feeds via DOM (preferred): `python scripts/extract-search-feeds.py | browser-act --session <name> eval --stdin` 2. Parse `items` array from result to get note list with `id` and `xsec_token` 3. For each note: a. `navigate https://www.xiaohongshu.com/explore/{id}?xsec_token={xsec_token}&xsec_source=pc_search` b. `wait stable` c. `python scripts/extract-note-detail.py {id} | browser-act --session <name> eval --stdin` d. Merge detail fields with feeds list fields (cover from feeds is higher resolution than detail page cover) 4. Save merged result per note Recommended batch delay: 2–3 seconds between detail page requests to avoid anti-scraping triggers. ## Enum Parameters Note: Some API parameter values below are platform-internal Chinese strings — they must be sent to the server exactly as shown (the server rejects English substitutes). All surrounding documentation text is English per Skill standards. [AI] sort_type — `filters` array entry `{"type": "sort_type", "tags": ["{value}"]}`. Verified values from filter API response: | UI Label | API Value (send as-is in `tags`) | |---|---| | General (default) | `general` | | Latest | `time_descending` | | Most Liked | `popularity_descending` | | Most Commented | `comment_descending` | | Most Collected | `collect_descending` | [AI] filter_note_type — `filters` array entry `{"type": "filter_note_type", "tags": ["{value}"]}`. API values are platform-internal Chinese strings (non-substitutable): | UI Label | API Value | |---|---|