Analyse a project's source code and generate a simply-annotate database file with LLM-created annotations categorised by tag: `defun` for functions and large syntactical blocks, `line` for smaller sub-blocks. File-wide narrative lives in a sibling `.org` overview file, not in an overlay. Use this skill when the user asks to annotate a project, generate code descriptions, or create a simply-annotate database for a codebase. Trigger on phrases like "annotate this project", "generate annotations", "describe this codebase", "create annotations for", or "annotate the source".
Analyse a project's source code and generate a simply-annotate database file with LLM-created annotations categorised by tag: `defun` for functions and large syntactical blocks, `line` for smaller sub-blocks. File-wide narrative lives in a sibling `.org` overview file, not in an overlay. Use this skill when the user asks to annotate a project, generate code descriptions, or create a simply-annotate database for a codebase. Trigger on phrases like "annotate this project", "generate annotations", "describe this codebase", "create annotations for", or "annotate the source".
Annotate Project Skill
You are generating a simply-annotate database file that describes a
project's source code using two tag categories — defun (functions, classes,
types, and other major syntactical blocks) and line (smaller sub-blocks worth
calling out). When the user opens files in Emacs with simply-annotate-mode,
they see these as tagged annotations they can filter and step through.
Important: simply-annotate's annotation model is tag-based, not
level-based. The legacy level field (file/defun/line) no longer
exists as a first-class concept — what used to be a level is now a tag on a
thread-format annotation. File-wide overview annotations (overlays spanning
the whole buffer) are not produced; that narrative lives exclusively in
the sibling directory overview .org file created in Step 9.
Step 1: Discover Source Files
If the user provides a specific directory or glob argument (e.g.
/annotate-project src/ or /annotate-project java/cuis-server), scan only
that scope. Otherwise scan the project root.
Use the Glob tool to find files matching common source extensions:
**/*.el
**/*.py
**/*.js
**/*.ts
**/*.tsx
**/*.jsx
**/*.rs**/*.go**/*.java**/*.c**/*.cpp**/*.h
**/*.rb**/*.lua**/*.sh**/*.clj**/*.hs**/*.proto
Directories to skip
Respect .gitignore and always skip these directories/patterns:
After discovering files, count them and report the total to the user
before doing any reading or annotation. Include:
The total number of source files found.
A breakdown by immediate subdirectory (top 10 by count).
Any directories that look auto-generated (containing generated, gensrc,
gen, proto/generated, pb.go, .pb.h, etc.) — flag these and ask
whether to include them.
If the file count exceeds 100 files, warn the user:
Found N source files. Annotating all of them will take a while.
Would you like to proceed, or narrow the scope to a specific subdirectory?
Wait for confirmation before continuing. If the user narrows scope, re-discover
within that scope.
If the count is ≤ 100, proceed automatically.
Step 2: Determine the Database Path
Annotations are stored in a project-local database file by default. This
keeps each project's annotations self-contained, makes them shareable via
version control if desired, and avoids one giant global database growing
unbounded.
Resolve the database path in this order:
Project-local (default). Find the project root by walking up from the
current working directory looking for one of: .git, .hg, .svn,
package.json, Cargo.toml, pyproject.toml, go.mod, pom.xml,
build.gradle, build.gradle.kts, Makefile. Use that directory. If
nothing is found, use the current working directory itself.
Global fallback. Only use ~/.emacs.d/simply-annotations.el if the
user explicitly asks for the global database (e.g. "use the global db",
"save to ~/.emacs.d/...").
Use the resolved path consistently in all subsequent steps (backup,
generate, validate). Refer to it below as $DB_PATH.
Step 3: Write the Elisp Helper Script
Since the annotations target Emacs, use a native elisp batch script to
convert line numbers to Emacs point positions. This guarantees byte offsets
match what Emacs sees — no manual conversion needed.
The script produces annotations in the current thread-alist format used by
simply-annotate:
It also accepts an optional project root third argument. When supplied,
the DB keys are rewritten to project-relative paths (src/foo.ts rather than
/home/user/project/src/foo.ts) — this is required for project-local
databases, because simply-annotate's simply-annotate--file-key returns a
project-relative path when looking up annotations, and a DB keyed by absolute
paths will silently find nothing.
Write the following script verbatim to /tmp/gen_annotations.el at the
start of the session. Do not modify it — it is tested and known to work.
;;; gen_annotations.el --- Convert line-based annotation JSON to simply-annotate (thread-tag format) -*- lexical-binding: t; -*-
;; Usage: emacs --batch -l /tmp/gen_annotations.el /tmp/annotations.json $DB_PATH [PROJECT_ROOT]
;;
;; When PROJECT_ROOT is supplied, keys are rewritten to project-relative paths.
;; This is REQUIRED for project-local databases.
;;
;; Input JSON shape (per file):
;; { "/abs/path/file.ts": [
;; {"sl": 8, "el": 52, "text": "...", "tag": "defun"},
;; {"sl": 32, "el": 51, "text": "...", "tag": "line"}
;; ] }
(require 'json)
(require 'cl-lib)
(defvar gen-annotations--counter 0)
(defun gen-annotations--uid (prefix)
"Return a stable unique id with PREFIX."
(setq gen-annotations--counter (1+ gen-annotations--counter))
(format "%s-%s-%06d"
prefix
(format-time-string "%s")
gen-annotations--counter))
(defun gen-annotations--make-thread (text tag)
"Wrap TEXT into a minimal thread alist tagged with TAG."
(let ((now (format-time-string "%Y-%m-%dT%H:%M:%S%z")))
`((id . ,(gen-annotations--uid "thread"))
(created . ,now)
(status . "open")
(priority . "normal")
(tags . (,tag))
(comments . (((id . ,(gen-annotations--uid "comment"))
(parent-id . nil)
(author . "Claude")
(timestamp . ,now)
(text . ,text)
(type . "comment")))))))
(defun gen-annotations--line-to-point (buf line-number)
"In BUF, return the point at the beginning of LINE-NUMBER (1-based)."
(with-current-buffer buf
(save-excursion
(goto-char (point-min))
(if (and (> line-number 0)
(zerop (forward-line (1- line-number))))
(point)
(point-max)))))
(defun gen-annotations--end-of-line-region (buf line-number)
"In BUF, return the point at end of LINE-NUMBER (1-based)."
(with-current-buffer buf
(save-excursion
(goto-char (point-min))
(if (and (> line-number 0)
(zerop (forward-line (1- line-number))))
(line-end-position)
(point-max)))))
(defun gen-annotations--convert-file (filepath annotations)
"Convert ANNOTATIONS for FILEPATH from line numbers to point positions.
Each input annotation has keys sl, el, text, tag."
(if (not (file-exists-p filepath))
(progn
(message "WARNING: File not found, skipping: %s" filepath)
nil)
(let ((buf (generate-new-buffer " *gen-ann-temp*"))
result)
(unwind-protect
(progn
(with-current-buffer buf
(insert-file-contents filepath))
(dolist (ann annotations)
(let* ((sl (alist-get 'sl ann))
(el (alist-get 'el ann))
(text (alist-get 'text ann))
(tag (alist-get 'tag ann))
(start (gen-annotations--line-to-point buf sl))
(end (if (= el -1)
(with-current-buffer buf (point-max))
(gen-annotations--end-of-line-region buf el)))
(thread (gen-annotations--make-thread text tag)))
(push `((start . ,start)
(end . ,end)
(text . ,thread))
result))))
(kill-buffer buf))
;; Sort by start position, ascending.
(sort (nreverse result)
(lambda (a b) (< (alist-get 'start a) (alist-get 'start b)))))))
(defun gen-annotations--load-existing (output-path)
"Load existing database from OUTPUT-PATH if it exists."
(when (file-exists-p output-path)
(with-temp-buffer
(insert-file-contents output-path)
(let ((content (string-trim (buffer-string))))
(unless (string-empty-p content)
(condition-case err
(car (read-from-string content))
(error
(message "WARNING: Could not parse existing database: %s"
(error-message-string err))
nil)))))))
(defun gen-annotations--db-key (filepath project-root)
"Return the DB key for FILEPATH, made project-relative when PROJECT-ROOT is set."
(if (and project-root
(stringp project-root)
(> (length project-root) 0)
(string-prefix-p (file-name-as-directory project-root) filepath))
(substring filepath (length (file-name-as-directory project-root)))
filepath))
(defun gen-annotations--main ()
"Main entry point for batch annotation generation."
(let* ((args (last command-line-args 3))
(json-path (nth 0 args))
(output-path (and (nth 1 args) (expand-file-name (nth 1 args))))
(project-root (and (nth 2 args)
(file-name-as-directory
(expand-file-name (nth 2 args))))))
(unless (and json-path output-path)
(error "Usage: emacs --batch -l gen_annotations.el /tmp/annotations.json output.el [project-root]"))
(let* ((json-object-type 'alist)
(json-array-type 'list)
(json-key-type 'symbol)
(annotations (with-temp-buffer
(insert-file-contents json-path)
(json-read)))
(existing-db (gen-annotations--load-existing output-path))
(new-db existing-db)
(file-count 0))
(dolist (entry annotations)
(let* ((filepath (symbol-name (car entry)))
(db-key (gen-annotations--db-key filepath project-root))
(file-annotations (cdr entry))
;; Read from absolute filepath for line-to-point conversion;
;; only the DB key is made relative.
(converted (gen-annotations--convert-file filepath file-annotations)))
(when converted
(cl-incf file-count)
(if (assoc db-key new-db #'string=)
(setcdr (assoc db-key new-db #'string=) converted)
(push (cons db-key converted) new-db)))))
(setq new-db (sort new-db (lambda (a b) (string< (car a) (car b)))))
(with-temp-file output-path
(insert ";;; Simply Annotate Database\n")
(insert ";;; This file is auto-generated. Do not edit manually.\n\n")
(let ((print-level nil)
(print-length nil))
(prin1 new-db (current-buffer)))
(insert "\n"))
(message "Generated annotations for %d files -> %s" file-count output-path))))
(gen-annotations--main)
Step 4: Read and Analyse Each File
For each source file, read the contents and produce annotations under two
tags: defun and line. Record annotations using line numbers (sl/el)
— the helper script will convert to byte offsets.
Do not produce file-wide overlay annotations. In the previous model there
was a file level covering the whole buffer (sl: 1, el: -1). That overlay
is no longer part of the new format — a single annotation spanning the whole
file overlaps every other annotation and confuses tag filtering. The file's
high-level narrative belongs in the sibling .org overview file created in
Step 9, where it can be read as prose.
JavaScript/TypeScript: function, class, arrow function assignments,
export default, React components
Rust: fn, impl, struct, enum, trait, mod
Go: func, type, interface
C/C++: function definitions, struct/class/enum definitions
Ruby: def, class, module
Java: method definitions, class definitions
Shell: function definitions
Proto: message, enum, service, rpc definitions
The annotation sl and el should span the full block (from the first line of
the definition to the closing delimiter line). Write a description covering:
what it does, parameters/return values (briefly), key side effects, and how it
relates to the rest of the file. 1-3 sentences.
Skip defun-tagged annotations for trivial files (< 30 lines, or pure
boilerplate like single-exception classes, simple data holders, or marker
interfaces).
Tag: line
Annotations for smaller but meaningful sub-blocks within defun-tagged regions:
Significant let/let* binding blocks
Important conditional branches (if/cond/pcase/match/switch)
Non-obvious expressions that benefit from explanation
Do NOT annotate every line — focus on blocks where a reader would benefit from
context. Aim for roughly 3-8 line-tagged annotations per defun-tagged
block, fewer for simple functions. Skip line annotations entirely for
simple functions.
Step 5: Produce Annotation JSON
Collect all annotations into a single JSON object. Use absolute paths as
the keys (the helper script will rewrite them to project-relative paths using
the project-root argument in Step 7):
Note there is no file-wide entry (sl: 1, el: -1) — that narrative goes
into the .org overview file in Step 9.
Write this JSON to /tmp/annotations.json.
Step 6: Backup the Existing Database
Before generating, always back up the current database if it exists.
Use the $DB_PATH resolved in Step 2:
cp"$DB_PATH""$DB_PATH.bak" 2>/dev/null || true
This ensures the user can recover if the generation or merge corrupts the file.
If something goes wrong, restore with: cp "$DB_PATH.bak" "$DB_PATH".
Step 7: Generate the Database
Run the elisp helper script (written in Step 3) against $DB_PATH. Pass
the project root as a third argument so keys are written as project-relative
paths — without this, simply-annotate will fail to find the annotations when
the user opens a file:
Where $PROJECT_ROOT is the directory resolved in Step 2 (the one containing
.simply-annotations.el). Omit the project-root argument only when using
the global database under ~/.emacs.d/, which keys by absolute path.
The script handles everything:
Merging: files already in the database are replaced; files not in the
new JSON are preserved untouched.
Key rewriting: when PROJECT_ROOT is supplied, absolute paths inside
that root become project-relative (e.g. src/index.ts).
Thread format: each annotation's text is a full thread alist with
tags, matching the current simply-annotate on-disk format.
Format: output is written with prin1, matching
simply-annotate--save-database format — a raw alist readable by
read-from-string. No (setq ...) wrapper.
Step 8: Validate the Database
After generating, always run the validation script to confirm the database
is well-formed and won't break simply-annotate-mode. Write the following
verbatim to /tmp/validate_annotations.el and run it against $DB_PATH.
;;; validate_annotations.el --- Validate a simply-annotate thread-tag database -*- lexical-binding: t; -*-
;; Usage: emacs --batch -l /tmp/validate_annotations.el $DB_PATH
(require 'cl-lib)
(defun validate-annotations--thread-p (obj)
"Return non-nil if OBJ looks like a simply-annotate thread alist."
(and (listp obj)
(listp (car-safe obj))
(assq 'id obj)
(assq 'comments obj)))
(defun validate-annotations--main ()
"Validate the simply-annotate database file."
(let* ((db-path (expand-file-name (car (last command-line-args))))
(errors 0)
(warnings 0)
(total-annotations 0)
(tag-counts (make-hash-table :test 'equal))
db)
(unless (file-exists-p db-path)
(message "FAIL: Database file does not exist: %s" db-path)
(kill-emacs 1))
;; Step 1: Can we read it?
(condition-case err
(with-temp-buffer
(insert-file-contents db-path)
(setq db (car (read-from-string (string-trim (buffer-string))))))
(error
(message "FAIL: Cannot parse database: %s" (error-message-string err))
(kill-emacs 1)))
;; Step 2: Is it a list?
(unless (listp db)
(message "FAIL: Database is not a list (got %s)" (type-of db))
(kill-emacs 1))
;; Step 3: Validate each entry
(dolist (entry db)
(let ((filepath (car entry))
(annotations (cdr entry)))
(unless (stringp filepath)
(message "ERROR: Entry key is not a string: %S" filepath)
(cl-incf errors))
(unless (listp annotations)
(message "ERROR: Annotations for %s is not a list" filepath)
(cl-incf errors))
;; File-existence check: project-relative keys won't exist as absolute
;; paths, so skip the check for non-absolute keys.
(when (and (stringp filepath)
(file-name-absolute-p filepath)
(not (file-exists-p filepath)))
(message "WARNING: File does not exist: %s" filepath)
(cl-incf warnings))
(dolist (ann annotations)
(cl-incf total-annotations)
(let ((start (alist-get 'start ann))
(end (alist-get 'end ann))
(text (alist-get 'text ann)))
(unless (and start end text)
(message "ERROR: %s: annotation missing required fields: %S" filepath ann)
(cl-incf errors))
(when (and start (not (and (integerp start) (> start 0))))
(message "ERROR: %s: invalid start: %S" filepath start)
(cl-incf errors))
(when (and end (not (and (integerp end) (> end 0))))
(message "ERROR: %s: invalid end: %S" filepath end)
(cl-incf errors))
(when (and start end (integerp start) (integerp end) (> start end))
(message "ERROR: %s: start (%d) > end (%d)" filepath start end)
(cl-incf errors))
;; text must be a string OR a thread alist.
(unless (or (stringp text)
(validate-annotations--thread-p text))
(message "ERROR: %s: text is neither a string nor a thread alist: %S"
filepath text)
(cl-incf errors))
;; Accumulate tag counts for reporting.
(when (validate-annotations--thread-p text)
(dolist (tag (alist-get 'tags text))
(puthash tag (1+ (gethash tag tag-counts 0)) tag-counts)))))))
;; Report
(message "Validation complete: %d files, %d annotations, %d errors, %d warnings"
(length db) total-annotations errors warnings)
(when (> (hash-table-count tag-counts) 0)
(message "Tag breakdown:")
(maphash (lambda (tag count) (message " %s: %d" tag count)) tag-counts))
(if (> errors 0)
(progn
(message "FAIL: Database has %d errors. Restore backup with:" errors)
(message " cp %s.bak %s" db-path db-path)
(kill-emacs 1))
(message "OK: Database is valid.")
(kill-emacs 0))))
(validate-annotations--main)
If validation fails, immediately restore the backup:
cp"$DB_PATH.bak""$DB_PATH"
Then investigate the /tmp/annotations.json for the bad data and re-run
generation.
Step 9: Generate Directory Overview Org File
For the annotated directory, create an org file alongside it with the same
name as the directory. For example, annotating /project/java/client creates
/project/java/client.org.
The org file should contain:
#+TITLE: <directory-name> — Directory Overview
#+AUTO_TANGLE: nil
* Purpose
<2-4 sentence overview of what this directory/module does and its role in the
wider project.>
* Structure
<Brief description of how the directory is organised — subdirectories, key
groupings of files, and the relationships between them.>
** <subdirectory-or-grouping-1>
<1-2 sentences on what this group of files does.>
** <subdirectory-or-grouping-2>
...
* Key Files
| File | Description |
|------+-------------|
| file1.java | One-line summary |
| file2.java | One-line summary |
| ... | ... |
List ALL source files in the directory scope, not just the "important" ones.
Group by subdirectory if the directory has subdirectories.
* Important Notes
<Bullet points covering anything a newcomer should know: key patterns used,
auto-generated files to ignore, non-obvious conventions, gotchas, entry points,
important interfaces, etc. Skip this section if there is nothing noteworthy.>
Rules for the org file
One org file per annotated directory scope — if the user annotates
java/client, create java/client.org. If they annotate java/cuis-server,
create java/cuis-server.org.
The file sits alongside (as a sibling of) the directory, not inside it.
Keep it concise — this is a quick-reference map, not exhaustive documentation.
The Key Files table and Structure section are the primary home for
file-wide narrative (since there is no file-level overlay annotation). Each
file's one-line summary here is what a file-level annotation used to
provide.
If the directory has auto-generated subdirectories (e.g. gensrc/), note
them in Important Notes.
Step 10: Report
After writing, report:
Number of files annotated
Total annotations generated (broken down by tag — defun and line)
The database file path (clearly state whether it is project-local or global)
Validation result (pass/fail)
The directory overview org file path
Remind the user to open files in Emacs and enable simply-annotate-mode
(or use simply-annotate-show-all for the global overview)
Guidelines
Be concise: defun-tagged descriptions 1-3 sentences; line-tagged
1 sentence. File-wide overview prose lives in the .org overview file
(2-4 sentences per file in the Key Files table).
Be descriptive, not prescriptive: Describe what the code does, not what
it should do. This is documentation, not code review.
Use plain language: Avoid jargon where possible. The goal is to help
someone unfamiliar with the codebase orient themselves.
Process files in parallel: Use the Agent tool to analyse multiple files
simultaneously when practical — but keep batches reasonable (10-20 files per
agent).
Handle large files: For files over 1000 lines, you may need to read them
in chunks. Still produce annotations for the full file.
Respect the user's scope: If they say "just the src/ directory" or "only
Python files", honour that constraint.
Don't annotate the world: The goal is useful coverage of code the user
cares about, not exhaustive annotation of every file in the repo.
