-
clawemail create can auto-add a profile with the same name as the prefix (for example newsletter) to ~/.config/mail-cli/config.json.
-
A new sub-mailbox can pass mail-cli auth, folder listing, and send tests while still silently failing to receive outside newsletter mail if external communication is not enabled in Claw mailbox policy.
-
Check mail-cli clawemail info --uid <mailbox@claw.163.com> --json early. If outside newsletters are not arriving, verify commLevel: 2, extReceiveType: 1, and extSendType: 1 before debugging signup forms.
-
For newsletter confirmations, useful first-pass keywords are: confirm, confirmation, verify, verification, activate, subscription, welcome, and sender/platform names like Substack, Rundown, Ben, Latent, TLDR, Neuron, Interconnects, and Ethan.
-
Browser automation often cannot complete newsletter signup on sites protected by Vercel Security Checkpoint or Cloudflare. A local browser-harness / real Chrome session may succeed where the standard browser tool fails.
-
On Substack flows, the common successful pattern is: enter email -> click Subscribe -> choose the free plan -> skip the recommendations page. The recommendations step is optional and can be skipped.
-
Some Substack newsletters do not show a normal free-plan selector. Instead they show a pledge screen such as Pledge your support .... In that case explicitly choose No pledge, then continue with the Continue without pledging path.
-
On some Substack pages, clicking Subscribe may not produce a clear confirmation state in the DOM. Treat the mailbox as the source of truth rather than the webpage.
-
Some Substack pages can immediately return Something went wrong even with a valid email. Treat that as a failed submission and retry later or with a different browser session instead of assuming it worked.
-
Beehiiv pages may get stuck on a transient Subscribing state without showing a clean success page. Again, treat the mailbox as the source of truth.
-
If browser-harness leaves many completed signup pages open, use list_tabs(include_chrome=False) to inspect tabs and cdp("Target.closeTarget", targetId=<id>) to close finished confirmation/signup tabs in bulk.
-
Some web sources are better handled as scheduled page digests instead of email subscriptions. For example, Hugging Face Daily/Weekly Papers pages are suitable for periodic browser-based digest jobs rather than mailbox onboarding.
-
For recurring newsletter automation on this machine, the canonical wiki target is the _global vault under /Users/leiyuxuan/Documents/PyCharm_Project/wiki/_global rather than a standalone ~/wiki tree. Digest notes belong in wiki/queries/ inside that vault, and raw email/article snapshots belong in _global/raw/articles/.
-
Hermes cron jobs do not run unless the Hermes gateway is running. Always verify hermes gateway status and hermes cron status before assuming a scheduled newsletter job will fire.
-
The digest pipeline that worked in practice was: newsletter_digest_context.py to list recent inbox items, newsletter_extract.py <message-id> to parse Beehiiv/Substack-style email bodies into structured sections, then write bilingual digest output into the _global vault.
-
When a scheduled monitor or digest job should leave an audit trail, prefer deliver=local and explicitly instruct the cron prompt to overwrite a known markdown file inside the wiki rather than relying on deliver=origin.
-
Do not classify an old confirmation email as still pending if a later welcome/completion email from the same newsletter/sender is present; the later welcome email is usually the source-of-truth signal that the confirmation link was already completed.
-
When matching an old confirmation to a later welcome/completion email, do not rely only on the confirmation subject naming the newsletter. Generic subjects like Please confirm your subscription may omit the brand; match by sender display name/address/domain and nearby timestamps against strict welcome subjects such as Welcome to <newsletter>. If they align, treat the old confirmation as superseded and redact the tokenized URL.
-
When an automated monitor initially flags a pending confirmation but the same sender/newsletter has a later strict welcome/completion message, manually correct the status note to 0 pending, redact any previously listed confirmation URLs, and clean up any erroneous appended log entries from the same run so the latest audit trail does not contain stale “pending” results.
-
When writing status/audit notes, include full tokenized confirmation URLs only for messages that still need manual action. For superseded confirmation emails, state that a link was detected but intentionally omit/redact the URL to avoid leaking active Substack/Beehiiv tokens into wiki/log files. Treat the entire URL as sensitive: Substack/Beehiiv tokens may live in the path rather than the query string, so do not preserve path components when redacting superseded links.
-
Apply the same redaction discipline to tool/stdout summaries, not only to final wiki/log files. If an aggregate scanner reads superseded confirmation bodies, do not print(json.dumps(...)) with raw extracted links; redact or omit link fields before printing tool output, because tool results enter the conversation transcript and may be delivered or stored even when the written status file is clean. Full pending-action confirmation URLs may be printed only when manual action is genuinely required.
-
After writing or rewriting a confirmation-monitor status/log note, verify that no superseded tokenized confirmation URL leaked into the wiki. Use search_files over the target vault for high-risk domains/fragments such as mg-d1, email.mg-d1.substack.com, or explicit Substack/Beehiiv URL/token regexes. Avoid broad word co-occurrence leak regexes such as confirm.*substack over rendered status files: they can falsely match safe rows that contain a confirmation subject plus a sender address/domain, even when no URL/token is present. If any actual high-risk URL/domain/token match appears in newly written status/log content, rewrite those entries to avoid even naming the high-risk tracking domain; use a generic phrase such as confirmation domain and tokenized URL redacted, then verify again. Full pending-action confirmation URLs may be included only when manual action is still required. For routine verification, first check the just-written status file and the latest log entry directly; a broad https?:// search over the whole wiki/log can return many unrelated historical links and should not be treated as a token leak unless it matches the high-risk confirmation domains/regexes. When writing regex leak checks, require URL/high-risk context for generic confirmation words: patterns like confirm[a-zA-Z0-9_./?=&%-]{20,} can falsely match safe filenames such as newsletter-confirmation-monitor-latest.md or normal text. Prefer checks like https?://\S*(confirm|verify|activate|validate|token)\S*, [?&](token|code|signature|redirect|email)=..., plus explicit high-risk domains. Do not flag bare platform/source words such as Beehiiv or Substack in a scan-scope sentence as token leaks; require a URL, token parameter, or known high-risk tracking/confirmation host such as mg-d1 / email.mg-d1.substack.com. When rendering superseded confirmation rows, avoid concatenating multiple redaction phrases from separate classification fields. Only say link detected if the body/anchors were actually read and a confirmation URL was observed; if classification used headers/search results only, use wording like Confirmation URL, if present, was intentionally not extracted or written because no manual action is needed.
-
Avoid treating generic newsletter footer/action links as pending confirmations. Links labeled or containing upgrade subscription, upgrade your subscription, manage subscriptions, unsubscribe, preferences, or generic subscribe CTAs in normal newsletter posts are content/management links, not confirmation links. Require explicit confirmation/verification/activation wording in the subject, near-top body text, or link text/URL (confirm, verify, activate, validate) before marking a message as pending action.
-
Avoid treating tracking pixels or image beacon URLs as confirmation links, even if they contain query parameters like token=. Substack welcome/completion emails may include substackcdn.com/.../p.gif?token=... tracking pixel URLs; these are not user-action links and should be ignored. Confirmation-link extraction should prefer explicit <a> link text/button labels such as Confirm subscription, Verify email, or action URLs from clickable anchors, and should filter image/pixel URLs (.gif, p.gif, substackcdn.com/o/...) before classifying a message as pending.
-
Avoid treating normal newsletter article/content links as pending confirmations just because they contain words like verification, verified, or activate (for example an article link titled “photo ID and selfie verification” or model text like “activate per token”). A pending confirmation requires subscription/email/account-action context: subject or near-top body says please confirm, confirm your subscription/email, verify your email, activate your subscription/account, complete your subscription, or a link itself is clearly labeled as a confirmation/verification action. If a scan initially writes a false-positive pending status from content-only links, immediately rewrite the status to 0 pending, remove sensitive/non-action links, and replace the same-run log entry so the audit trail does not preserve stale pending results.
-
Avoid treating any occurrence of the word welcome inside a newsletter body as a completed subscription signal; many normal newsletter posts contain generic greetings. Prefer strict welcome/completion evidence: subject starts with Welcome, You're subscribed, Subscription confirmed, or body contains explicit phrases like thanks for subscribing, successfully subscribed, or subscription confirmed near the top.
-
When writing confirmation-monitor status notes, list as "matching messages" only strict confirmation/verification/activation candidates and strict welcome/completion signals. Keyword hits from normal newsletter posts (for example subscription in a footer, sender names like Substack, or platform/newsletter content) are recall candidates only; do not include them in the status note unless they meet the strict confirmation/welcome criteria.
-
If an old confirmation candidate is classified as superseded by a later strict welcome/completion message, include both messages in the status note: the confirmation row should say it is superseded and redact/omit the tokenized URL, and the later welcome/completion row should be listed separately as the source-of-truth evidence. Do not hide the welcome row merely because it was referenced by the superseded confirmation row.
-
In practice, recent inbox triage works best as: mail list --fid INBOX --order date --desc --limit N --json -> keyword searches for recall -> read body --json on candidates -> parse HTML anchors -> classify as pending confirmation vs newsletter content only vs welcome/post-subscription. Do not assume bare mail list --fid INBOX --limit N --json returns newest messages; it may return older mailbox order unless --order date --desc is supplied.
-
mail search does not accept --order on the observed mail-cli version. Use --order date --desc only with mail list; for keyword searches, call mail search --fid INBOX --keyword "..." --limit N --json without ordering flags, then sort/filter returned headers in code if needed.
-
When parsing mail-cli read body --json, do not assume data.html is a string. It can be a nested object like {"id":"1","contentType":"text/html","content":"..."}; extract data.html.content before feeding HTML to BeautifulSoup/HTMLParser. If the parser appears to see the whole JSON/header instead of the email body, inspect the JSON shape and handle this nested content field.
-
Avoid shell pipelines like mail-cli ... | python3 -c ... when parsing email bodies: the security scanner may block piping untrusted email content into an interpreter. Use execute_code with Python subprocess.run([...], capture_output=True, text=True) and json.loads(stdout) instead, then parse HTML/text in-process.
-
When building aggregate execute_code scans across mail list plus many mail search commands, avoid brittle positional checks such as cmd[5] == "list" to infer command type; the mail-cli --profile newsletter mail list ... shape puts list at a different index than expected and can crash cron runs. Store an explicit label alongside each argv list, or infer by searching for the subcommand token robustly before parsing results.
-
When sorting mail-cli results by date inside Python, wrap email.utils.parsedate_to_datetime() in try/except and fall back to datetime.fromisoformat() plus common formats such as %Y-%m-%d %H:%M:%S. The observed mail-cli date shape 2026-04-23 19:34:38 can raise ValueError in parsedate_to_datetime() and crash the whole cron scan if not guarded.
-
When an aggregate execute_code scanner keeps internal datetime objects for sorting, remove them or convert them to strings before json.dumps; otherwise the scan can finish successfully but crash at final serialization with TypeError: Object of type datetime is not JSON serializable, wasting the cron run.
-
When extracting snippets from HTMLParser output, normalize parser text defensively with something like ' '.join(' '.join(text_chunks).split()); do not call .split() on the parser's raw list of text chunks. A list/string mixup can cause a body parse error after links were already extracted, leaving confusing partial results.
-
When extracting email-body snippets from HTML with Python HTMLParser, explicitly ignore <style>, <script>, and ideally <head> content; Substack/Beehiiv bodies often contain large CSS blocks before the useful content, and naive text extraction produces noisy @media ... snippets.
-
Some Substack pages can immediately return Something went wrong even with a valid email. Treat that as a failed submission and retry later or with a different browser session instead of assuming it worked.
-
Beehiiv pages may get stuck on a transient Subscribing state without showing a clean success page. Again, treat the mailbox as the source of truth.
-
If browser-harness leaves many completed signup pages open, use list_tabs(include_chrome=False) to inspect tabs and cdp("Target.closeTarget", targetId=<id>) to close finished confirmation/signup tabs in bulk.
-
Some web sources are better handled as scheduled page digests instead of email subscriptions. For example, Hugging Face Daily/Weekly Papers pages are suitable for periodic browser-based digest jobs rather than mailbox onboarding.
-
For Beehiiv/Substack newsletter ingestion, raw mail-cli read body --json output is often too noisy for high-quality wiki summaries. A better local pattern is to run a dedicated HTML-to-structure extractor script first, then summarize from its sectioned output. On this machine a working implementation was saved at ~/.hermes/scripts/newsletter_extract.py.
-
If python ~/.hermes/scripts/newsletter_extract.py <message-id> fails with ModuleNotFoundError: No module named 'bs4', do not block the cron job: immediately fall back to mail-cli --profile newsletter read body --id <message-id> --json, parse with json_parse/json.loads(..., strict=False) if control characters break strict JSON, and explicitly note that extraction quality is partial/noisy. Install beautifulsoup4 in the exact Hermes cron/runtime venv, not just the default shell Python: /Users/leiyuxuan/.hermes/hermes-agent/venv/bin/python -m pip install beautifulsoup4. Verify with /Users/leiyuxuan/.hermes/hermes-agent/venv/bin/python ~/.hermes/scripts/newsletter_extract.py <message-id>.
-
If newsletter_extract.py <message-id> fails once because the underlying mail-cli --profile newsletter read body ... --json exits nonzero (for example return code 6 / transient Ajax or mailbox read failure), retry the same extractor command once before falling back or recording extraction failure. In practice a DeepSeek/AINews message failed on the first extraction attempt and succeeded immediately on retry; do not preserve a failed raw snapshot if the retry succeeds.
-
When turning newsletter ingestion into a scheduled Hermes cron job, the gateway must be running or jobs will never fire. Check with hermes cron status and hermes gateway status. If the gateway is down, start/install it before debugging the job itself.
-
If the gateway was down past a job's grace window, Hermes may fast-forward missed runs to the next scheduled time instead of replaying them. That means a newly fixed cron pipeline should be validated with a fresh one-shot selftest or a manual cron run, not by waiting for an old missed run.
-
A reliable cron-chain validation pattern is: create a one-shot local-delivery cron job that only writes a tiny file into the wiki and appends one log entry. If that succeeds, gateway + cron + model execution + file write is confirmed healthy; only then debug the heavier digest workflow.
-
For scheduled confirmation monitors, keep the scan bounded and staged: first run mail list --fid INBOX --limit N --json plus a small set of keyword searches, then read bodies only for actual confirmation/welcome candidates. A single large execute_code pass that searches many keywords and reads many noisy HTML bodies can exceed the cron/tool timeout; if that happens, fall back to targeted mail-cli terminal calls and write the status note directly.
-
Do not trust a custom aggregate parser's 0 results conclusion until it is sanity-checked against direct mail-cli keyword searches for the strict terms (confirm, welcome, verification, activate, subscription). In practice, a broad execute_code scan can silently over-filter or mishandle mail-cli JSON shapes and return no candidates even when targeted mail search shows strict matches. Use direct terminal searches as the source of truth before writing a "no relevant messages" status.
-
Do not apply a hard recency cutoff to keyword-search recall before strict confirmation/welcome matching. A scheduled monitor once used a 30-day cutoff inside an aggregate parser and returned 0 results, while direct mail search --keyword confirm/welcome found an older Please confirm your subscription immediately followed by Welcome to .... First collect direct keyword hits, classify strict candidates and superseding welcome/completion messages, then decide whether the final report should mention older superseded evidence.
-
If execute_code repeatedly times out or returns suspect empty results while trying to read/parse bodies, do not keep retrying the same full parser. Use the structured headers from direct mail list/search calls as the primary evidence, especially for the common pattern Please confirm your subscription followed shortly by a strict Welcome to <newsletter> from the same sender; classify that old confirmation as superseded, redact tokenized links, and finish the wiki status/log update without body extraction.
-
When appending log.md from a cron job, avoid terminal commands that run python -c/inline scripts: the terminal security scanner may block them awaiting approval, which cron cannot provide. Use execute_code with raw pathlib.Path.read_text()/write_text() for small file appends, or use top-level read_file + write_file, and verify with read_file/search_files. Do not use hermes_tools.read_file() inside execute_code as the source for a full-file rewrite: it can include display line-number prefixes (123|) and corrupt markdown logs unless stripped first.
-
When writing/overwriting a *-latest.md status file from a scheduled monitor, a write_file warning about a sibling subagent modifying the same file can happen if cron runs overlap or a manual run recently touched the file. For intentional latest-file overwrites, do not stop solely on that warning; immediately verify the final file contents with read_file, verify the new log.md entry with search_files, and check for token leaks before finalizing.
-
For scheduled confirmation monitors whose prompt says to suppress delivery when nothing is new, still complete the durable side effects first: overwrite the latest status file, append the wiki log.md entry, verify direct keyword searches/status/log/token leaks, and only then return exactly [SILENT] if there are 0 pending manual-action confirmations and no new actionable finding. Do not skip the status/log update just because the final user-facing response is silent.
-
When writing status/audit notes into the _global wiki, append log.md entries using the wiki log schema (## [YYYY-MM-DD] query | <title> followed by bullets) rather than only a free-form bullet, so later grep/index tooling can parse the operation.
-
When verifying a newly appended log.md entry, do not rely only on search_files with a broad monitor title such as Newsletter confirmation monitor: it may return older matches first and hide the latest append behind the result limit. Read the tail of log.md with read_file(offset=<near total_lines>) or search for a unique scan-time/result phrase to confirm the latest entry.
-
After appending _global/log.md, verify the markdown structure as well as content: each new entry should be separated from the previous entry by a blank line before the ## [YYYY-MM-DD] query | ... heading. If a direct append accidentally concatenates headings onto the prior bullet list, patch the log immediately to insert the missing blank line, then re-read the tail and re-run token-leak checks.
-
When appending _global/log.md via Path.read_text() / Path.write_text(), do not compute the separator from old.endswith('\n\n') and then concatenate old.rstrip('\n') + sep: if the file already ends with blank lines, this can drop the separator and concatenate the new ## heading onto the previous bullet. Prefer old.rstrip('\n') + '\n\n' + entry whenever old is non-empty, then verify the tail.
-
For confirmation-monitor cron logs, avoid leaving audit lines that say verification is still pending after verification has completed. Either append the log after verification is done, or patch the same latest log entry from pending to the final verified result, then re-read the tail so the audit trail does not preserve stale uncertainty.
-
When verifying just-written status/log files with search_files, prefer passing the exact file path in path for each target instead of relying on brace-style file_glob filters such as {newsletter-confirmation-monitor-latest.md,log.md} at the vault root. The file-glob filter may match unintended same-named files under raw/legacy directories or otherwise broaden the search, producing false positives unrelated to the current status/log update.
-
For combined daily AI digest cron jobs, produce exactly one root query file at _global/wiki/queries/daily-newsletter-digest-YYYY-MM-DD.md; put all source snapshots/raw captures under _global/raw/articles/. If a prompt references _global/wiki/workflows/bilingual-daily-digest-template.md but that file is absent, check _global/wiki/strategies/bilingual-daily-digest-template.md before proceeding; in practice the template lived under wiki/strategies/.
-
For Chinese AI source add-ons in the daily digest, use only the explicitly allowed sources. 量子位 (https://www.qbitai.com/) is a WordPress-style page whose homepage static HTML exposes recent article links and enough article text for compact source snapshots. 机器之心 should be retried every run at /, /articles, and /rss; in practice /rss returned 数据服务 | 机器之心, /articles returned a JS/static shell, and exposed PRO/SOTA links returned login/app shells with no readable body. If still unreadable, explicitly say so and invent no 机器之心 items.
-
When writing raw Chinese source snapshots for the digest, keep them compact and provenance-focused: list fetched URLs, extraction method, notable qbitai article titles/URLs/snippets, and the exact 机器之心 endpoints retried. Compute sha256 over the snapshot body below the raw frontmatter.
-
After running python ~/.hermes/scripts/fetch_cn_ai_sources.py YYYY-MM-DD, explicitly verify the raw Chinese snapshot frontmatter includes a valid sha256:. If the script wrote the snapshot without sha256, add it immediately by hashing only the body after the closing frontmatter delimiter. Do not assume helper scripts enforce the wiki raw-source schema. Do not trust an in-process helper's "sha added" status alone: re-read the file with read_file or a separate verification pass. When patching frontmatter programmatically, parse the first YAML block robustly with a regex such as ^---\n(.*?)\n---\n(.*)$ and compute the hash over group 2; avoid brittle split('---\n', 2) helpers that can silently fail or be misverified in cron.
-
Apply the same raw-source sha256 verification to newly written newsletter raw snapshots, not only the Chinese-source snapshot. In cron, helper scripts or wrapper code may compute the hash before adding final newlines/frontmatter and leave stale hashes. Do not compute sha256 only from an in-memory body string before rendering the final markdown: an extra blank line after the closing frontmatter delimiter means the regex body (group 2) begins with \n, producing a different hash. After all raw snapshots are written, run a separate verification pass that re-reads each file, parses frontmatter with ^---\n(.*?)\n---\n(.*)$, hashes the exact captured group 2 bytes/text, updates sha256: if needed, and verifies all new raw files before writing the final digest/log entry.
-
For reruns of an existing daily digest date, overwrite/refresh the same wiki/queries/daily-newsletter-digest-YYYY-MM-DD.md file and verify there is still exactly one matching query file. Do not add duplicate index.md entries; append a new log entry only if the rerun materially rewrote or reverified sources.
-
When adding combined daily digest entries to _global/index.md, match the existing heading exactly: in practice the section is ### queries/ (with trailing slash), not ### queries. If automation searches for ### queries\n, it may create a duplicate heading at the bottom. After updating, verify with read_file/search_files that the new digest appears once under ### queries/ and that no stray ### queries section was introduced.
When running the scheduled daily AI newsletter digest with AI HOT and Feishu output:
