marmite-development

# Marmite Development Guide This skill is for developers contributing to the marmite codebase. For building sites with marmite, use the `marmite` skill instead. - Repository: https://github.com/rochacbruno/marmite - Site: https://marmite.blog - License: AGPL-3.0-or-later - Language: Rust (edition 2021) ## Tools to call during development ### Show urls ``` marmite input_folder --show-urls ``` ``` { "archives": [ "/archive-2027.html", "/archive.html" ], "authors": [ "/author-jao.html", "/authors.html" ], "feeds": [ "/index.rss", "/internet.rss", "/tag-python.rss", "/author-jao.rss", "/archive-2026.rss" ], "file_mappings": [], "misc": [ "/index.html" ], "pages": [ "/about.html", "/pages.html" ], "pagination": [ "/pages-1.html", "/tag-dev-1.html", "/tag-python-1.html", "/author-jao-1.html", "/archive-2026-1.html", "/index-1.html", ], "posts": [ "/code-django.html", "/code-python.html", ], "redirects": [], "series": [], "streams": [ "/code.html", "/streams.html" ], "summary": { "archives": 1, "authors": 1, "feeds": 5, "file_mappings": 0, "meta": { "absolute_urls": false, "url": "" }, "misc": 1, "pages": 1, "pagination": 1, "posts": 2, "redirects": 0, "series": 0, "streams": 2, "tags": 1, }, "tags": [ "/tag-python.html", "/tags.html" ] } ``` This tool allows the agent to: - compare urls before and after a refactor - check generated content slug - count number of contents ### init site ``` marmite input_folder --init-site ``` Initialize a new site structure from scaffolding. ### New post ``` marmite input_folder --new "title" -t tag ``` Create a new post. ### More Use `marmite --help` to check what else is available. ## Code Quality Requirements ### Formatting and Linting Every change must pass formatting and linting before it can be committed. ```bash # Format code (always run first) mask fmt # Check formatting + clippy lints mask check # Pedantic clippy (run before opening a PR) mask pedantic ``` `mask check` runs `cargo fmt -- --check` followed by `cargo clippy`. Both must pass clean. `mask pedantic` adds `-W clippy::pedantic` for stricter analysis - fix pedantic issues only after confirming with the project maintainer. Do not suppress clippy warnings with `#[allow(...)]` unless there is a justified reason. If a suppression is needed, use the narrowest possible scope (on the item, not the module). ### Safe Rust Practices - **No `unwrap()` in non-test code.** Use `Result` and `Option` with proper error propagation (`?` operator), `unwrap_or`, `unwrap_or_else`, `unwrap_or_default`, or pattern matching. Reserve `unwrap()` for test code and cases where the value is guaranteed at compile time (e.g., embedded assets loaded via `rust_embed`). - **No `panic!` in library code.** Return `Result` or `Option` instead. Use `error!` from the `log` crate to report problems, then return an error or a sensible default. - **No unsafe code** unless absolutely necessary and approved by the maintainer. - **Handle all `Result` types.** Do not silently discard errors with `let _ = ...` unless the discard is intentional and documented. - **Prefer `&str` over `String`** in function parameters when ownership is not needed. - **Use `Arc` for shared ownership** across threads. The project uses `Arc<PathBuf>` and `Arc<Cli>` for passing shared state. - **Avoid `.clone()` when a borrow suffices.** Cloning is acceptable when the borrow checker requires it or when the data needs to outlive the current scope, but do not clone as a reflex. - **Use `LazyLock` for static initialization** instead of `lazy_static!` in new code. The project is transitioning to `std::sync::LazyLock`. ### Error Handling Patterns The project uses the `log` crate for user-facing messages: ```rust use log::{info, warn, error, debug, trace}; // User-facing progress info!("Processing {} files", count); // Recoverable problems the user should know about warn!("Skipping file with invalid frontmatter: {}", path.display()); // Fatal or important errors error!("Failed to write output: {}", err); // Developer debugging (visible with -vv) debug!("Resolved slug: {}", slug); // Detailed tracing (visible with -vvv) trace!("Template context: {:?}", context); ``` Functions that can fail should return `Result<T, E>` or `Option<T>`. Use `process::exit(1)` only in `main.rs` for top-level CLI errors. ### Naming Conventions - `snake_case` for functions, methods, variables, and modules - `PascalCase` for types, structs, enums, and traits - `SCREAMING_SNAKE_CASE` for constants and static variables - CLI flags: use `#[arg(long)]` with descriptive names (e.g., `--skip-image-resize`, `--init-templates`). Clap automatically converts underscores to hyphens. - Config fields: use `snake_case` matching the YAML key name. Always add `#[serde(default)]` with a default function. ## Testing Every change needs tests. The project has two testing layers: ### Unit Tests (`src/tests/`) Unit tests call marmite code directly. Each module has a corresponding test file. ``` src/tests/ content.rs # Content parsing, slug generation, streams shortcodes.rs # Shortcode regex matching and expansion parser.rs # Markdown to HTML conversion tera_functions.rs # Custom Tera template functions tera_filter.rs # Custom Tera filters feed.rs # RSS feed generation gallery.rs # Image gallery processing site.rs # Site generation logic templates.rs # Template initialization embedded.rs # Embedded asset loading server.rs # HTTP server image_resize.rs # Image resizing image_provider.rs # Image download theme_manager.rs # Theme installation ``` Unit test pattern: ```rust use super::*; use crate::config::Marmite; #[test] fn test_descriptive_name_of_what_is_tested() { // Setup let config = Marmite::default(); // Act let result = function_under_test(&config); // Assert assert_eq!(result, expected_value); } ``` Run unit tests: ```bash mask test_unit # or a specific test cargo test --bin marmite test_name ``` ### Integration Tests (`tests/`) Integration tests run marmite as a subprocess via `std::process::Command` and assert on the output files. They test end-to-end behavior. ``` tests/ basic_functionality.rs # CLI help, version, minimal site generation content_generation.rs # Content processing end-to-end features.rs # Feature-specific integration tests streams.rs # Stream content type wikilinks_integration.rs # Wikilink resolution highlight.rs # Syntax highlighting image_resize_integration.rs # Image resizing pipeline themes.rs # Theme loading and application ``` Integration test pattern: ```rust use std::fs; use std::process::Command; use tempfile::TempDir; #[test] fn test_feature_name_end_to_end() { let temp_dir = TempDir::new().unwrap(); let input_dir = temp_dir.path().join("input"); let output_dir = temp_dir.path().join("output"); fs::create_dir_all(input_dir.join("content")).unwrap(); fs::write( input_dir.join("marmite.yaml"), "name: Test Site\ntagline: Test", ).unwrap(); fs::write( input_dir.join("content").join("test.md"), "# Test Page\n\nContent here.", ).unwrap(); let output = Command::new("cargo") .args(["run", "--quiet", "--", input_dir.to_str().unwrap(), output_dir.to_str().unwrap()]) .output() .expect("Failed to execute marmite"); assert!(output.status.success()); // Assert on generated files let html = fs::read_to_string(output_dir.join("test.html")).unwrap(); assert!(html.contains("Test Page")); } ``` Run integration tests: ```bash mask test_integration # or a specific file cargo test --test basic_functionality ``` Run all tests: ```bash mask test ``` ### What to Test - **New features:** Both unit tests for the logic and integration tests for CLI/end-to-end behavior. - **Bug fixes:** A test that reproduces the bug and verifies the fix. The test should fail without the fix applied. - **Config options:** Test default values, custom values, and edge cases. - **CLI flags:** Integration test that passes the flag and checks the result. ## Avoiding Breaking Changes Marmite is used by people who depend on its current behavior. Follow these principles: - **Prefer feature flags over behavior changes.** If a change alters existing behavior, add a config option to opt in. Make the current behavior the default so existing sites are unaffected. - **New config fields must have sensible defaults.** Use `#[serde(default = "default_function")]` so that existing `marmite.yaml` files without the new field continue to work. - **Do not remove or rename existing config fields.** If a field needs to be superseded, deprecate it (log a warning when it is used) and support both the old and new field. - **Do not remove or rename existing CLI flags.** Use `#[arg(hide = true)]` to hide deprecated flags from `--help` while still accepting them. - **Do not change the default output structure** (file names, directory layout) without a migration path or a feature flag. - **Template variable changes must be backward-compatible.** Adding new variables is fine; removing or renaming existing ones breaks custom templates. ## Documentation Requirements Every new feature must be documented in multiple places. The marmite website is generated from `example/content/`, so documentation and the site are the same thing. ### Blog post for new features Create a new markdown file in `example/content/` documenting the feature. Follow existing patterns: - Use a dated filename: `YYYY-MM-DD-feature-name.md` - Include frontmatter with title, description, and relevant tags - Write a practical guide showing how to use the feature, with examples - If the feature adds CLI flags, also update `example/content/2024-11-26-marmite-command-line-interface.md` ### LLM documentation (`example/ai/llms.txt`) Add an entry to `example/ai/llms.txt` under the appropriate section for any significant change: new CLI flag, new config option, new content capability, new template variable, or new shortcode. This file is served at `marmite.blog/llms.txt` and is used by AI agents to understand marmite's capabilities. ### Agent skill references (`.agents/skills/marmite/references/`) Update the corresponding reference file when the feature changes: | What changed | File to update | |-------------|---------------| | Config options | `references/config-reference.md` | | CLI flags | `references/cli-reference.md` | | Frontmatter fields | `references/frontmatter.md` | | Template variables or functions | `references/tera-templates.md` | | Shortcodes | `references/shortcodes.md` | | Content organization (streams, series, fragments) | `references/content-organization.md` | | Deployment or hosting | `references/deployment-guide.md` | | Comment systems | `references/comment-system.md` | These files are compiled into the binary via `rust_embed` and installed with `--skill-install`. They must stay accurate. When adding a new feature, add to the top level SKILL.md ### Release notes Every new feature and bug fix must be added to the current draft release notes. Find the latest release notes file with `stream: draft` in frontmatter under `example/content/release-notes/`. The filename pattern is `marmite-X-Y-Z-release-notes.md`. - For new features, add a section describing what the feature does and how to use it. - For bug fixes, describe what was broken and how it is fixed. - If no draft release notes file exists, create one following the same pattern, using the next expected version number. ## Template Changes Marmite has two sets of templates that must be kept in sync: ### Main theme (`example/templates/` and `example/static/`) This is the default theme used by marmite when no custom theme is set. Changes to templates, CSS, JavaScript, or static assets go here first. - `example/templates/` - Tera HTML templates (`base.html`, `content.html`, `list.html`, `group.html`) - `example/static/` - CSS, JavaScript, fonts, colorschemes ### Alternative theme - `theme_template` (`example/theme_template/`) This is the scaffolding theme used when users run `marmite <folder> --start-theme <name>`. It provides a starting point for new custom themes. It has its own copies of templates and static assets: - `example/theme_template/templates/` - Template files - `example/theme_template/static/` - Static assets (CSS, JS, fonts, colorschemes)