developing-content-embeds

# Developing content embeds An embed is a box in a text note that shows content other than text: another note, an attachment of the note, or a source kept in the note itself. The box (title row, size, caption, fullscreen, toolbar, menu) is shared; what it shows is a **content component** chosen by the kind of content. Two components show what a fully featured embed looks like: - **Canvas drawing** — `apps/client/src/widgets/type_widgets/canvas/CanvasDrawing.tsx`. Excalidraw scene in an attachment (`CANVAS_ATTACHMENT_MIME`). A picture while not edited, Excalidraw while Editable is on. Its own tools in the embed toolbar (`CanvasEmbedTools.tsx`). - **Mermaid diagram** — `apps/client/src/widgets/type_widgets/mermaid/MermaidDiagram.tsx`. A *source embed*: the source sits inline as a code block, or in an attachment (`MERMAID_ATTACHMENT_MIME`); one component serves both. Source, split or preview view; zoom and syntax-help tools. **The rule both follow: the embed is built from the note type's own component, not beside it.** `CanvasDrawing` renders `CanvasEditor` from the canvas note (`canvas/Canvas.tsx`); `MermaidDiagram` renders `SplitLayout` from `helpers/SplitEditor.tsx` and `SvgPreview` / `SvgZoomControls` / `useSvgRender()` / `useSvgZoom()` from `helpers/SvgSplitEditor.tsx`, the same parts the Mermaid note uses. A first version of the Mermaid embed reimplemented its panes, viewer and error box and drifted at once (a raw error box instead of the note's error card, no divider). When the note type's component is bound to a note (labels, `noteContext`, `EditableCode`), **extract the note-independent part into an exported component in the same module** and have the note type use it too — `SplitEditor` → `SplitLayout` + `useMountedPanes()` is the worked example. Only the parts that are about storage stay separate (the embed's source pane is `CodeEmbedEditor` bound to a `ContentEditor`, the note's is `EditableCode` bound to the note). ## The storage forms What the text note saves, and what the editor model holds (`contentEmbed` element): | Form | Saved HTML | Model attributes | |---|---|---| | Included note | `<figure class="include-note" data-note-id="…" [data-block="…"]>` | `noteId`, `block` | | Attachment | `<figure class="include-note" data-attachment-id="…">` | `attachmentId` | | Inline source | `<pre data-box-size=… [data-display-mode=…] data-hide-title data-editable><code class="language-mermaid">…</code></pre>` | `language`, `source` | | Video | `<section class="link-embed" data-url=… data-embed-type="youtube" … [data-position=…]>` | the link preview fields: `url`, `embedType`, `title`, `image`, … | | Later kind | `<section data-trilium-embed="<kind>" …>fallback</section>` | `unknownContentEmbed`: `embedAttributes`, `embedContent` | All three carry `boxSize`, `hideTitle`, `editable`, `displayMode` (`data-box-size`, `data-hide-title`, `data-editable`, `data-display-mode`) and the resized size as `--include-note-width` / `--include-note-height` in `style`. The figure keeps an optional `<figcaption>`. **Captions of an embed whose element is not a `<figure>`** (`_defineCaptionedEmbedConverters()` in `content_embed.ts`). While such an embed has a caption, the note saves its element in a `<figure class="captioned-content-embed">` (`CAPTIONED_CONTENT_EMBED_CLASS`), followed by the `<figcaption>`, and without a caption it saves the bare element again. Every kind gets this without code of its own: a `low` data downcast listener wraps whatever element the kind's converter bound, a data mapper hook puts the caption after that element, and an upcast at `high` converts the element and turns the `<figcaption>` into the `caption` of the embed (`isEmbedCaptionToggleable()`), or into text after content that takes none. The figure is bound to the embed as well, view to model only, so the mapper counts it as one node; the embed still maps to its element, where the attribute converters (width, position) write. A kind opts in through `hasCaptionSupport()`; only inline source embeds stay out. Read-only views replace the figure with the box and move the caption into it (`replaceWithEmbedBox()` in `content_renderer_text.ts`); share pages move `style`, `data-box-size` and `data-position` from the element to the figure (`moveBoxToCaptionedFigure()`), which the share CSS places; Markdown export keeps the figure as raw HTML. **An embed whose caption has the selection looks selected**, as one whose player or editor has the focus (which selects it): `markEmbedOfSelectedCaption()` adds `ck-widget_selected` after `Widget` marks the selection and takes it away on `cleanSelection`. `Widget` itself marks only widgets inside the selection. - **Inline source** (`inline_source.ts`, `ContentEmbedInlineSource`) keeps the document readable elsewhere: the note exports to Markdown as a fenced `mermaid` block, other apps render it, and undo in the note undoes source edits through the shared undo history (see _Undo history_ below). `setSource()` writes outside the undo stack and does **not** redraw the embed, which would remount its editor. The plugin hands out one `InlineEmbedSource` per embed element, which keys its history. It is converted only for languages in `contentEmbed.inlineLanguages` (from `SOURCE_EMBED_TYPES` in `text/config.ts`); a code block of another language stays a code block. - **Video** (`video_embed.ts`, `ContentEmbedVideo`) is the _Embed_ display mode of a link preview whose URL has a video (`getLinkPreviewVideoId()` in commons). It keeps the saved form of the link preview, so released versions still play it, and adds `data-box-size`, `data-hide-title`, `data-position` and `--include-note-width`. Its sizes are Medium and Full (`VIDEO_EMBED_BOX_SIZES`); `readVideoEmbedBox()` reads a video saved before as Full and untitled, and `createVideoEmbed()` makes a new one Medium at `getVideoMediumWidth()` (20em high), which the box size command sets again. Height follows width (`isHeightResizable` is per element). A size change only restyles the figure: the `attribute:boxSize` converter skips `reloadEmbeddedContent()` for a video, since a new player stops the playback. A caption is saved in the captioned figure (above). The video toolbar also shows while the caret is in the caption (`findRelatedVideo()`), and its Copy address (URL) and Display act on the video through `ChangeLinkDisplayCommand._getTarget()`: the selected preview, else, unless the caret is on a link, the video of the caption (`getCaptionedEmbed()`). `getSelectedLinkWidget()` itself reads only the selected element, as the link balloon shares the command and Ctrl+K in a caption must edit the caption's own link. `rebuildLinkWidget()` copies the caption into a preview that keeps its shape; a Card, Icon & text or Text link has no caption, so `keepCaptionText()` moves its content into a paragraph after the video. The shape is the link preview field `aspectRatio` (`data-aspect-ratio="9/16"`), which the server reads from oEmbed for the video's `/shorts/` address (the watch address of a Short answers 16:9). `getVideoAspectRatio()` validates it; every box that draws a player (the editing figure, `applyVideoEmbeds()`, `renderLinkEmbedHtml()`, the share renderer's `setVideoBox()`) writes it as `--video-aspect-ratio`, which the player's `aspect-ratio` and the 70vh cap of Full read. Only Full is capped; Medium keeps its saved width. `setVideoBox()` also writes `data-box-size` on the shared section, Full for a video saved before box sizes. A preview without one is 16:9. A new link preview field goes into `LINK_PREVIEW_DATA_ATTRIBUTES` and `readLinkPreviewData()`, both `LinkEmbedMetadata` types (commons and the editor's `augmentation.ts`) and the client's `fetchMetadata()`; `META_KEYS`, the schemas and the converters follow the map. It renders through `contentEmbed.loadVideo()` (`loadVideoEmbed()`, whose title row opens the embed menu only in the editor) and has a toolbar of its own (`videoEmbed` in `content_embed_toolbar.ts`): Copy address (URL), the link preview's Display dropdown and ⋮. The caption is toggled from ⋮ only. Its ⋮ menu (`getVideoItems()`, picked by the `url` of `ContentEmbedState`) puts _Open externally_ (`openUrlExternally()` in `services/open.ts`) above the embed's commands and _Delete_ below. `ReadOnlyText`'s `applyVideoEmbeds()` swaps the section for the same box. The box has no backdrop (`DIRECT_CONTENT_TYPES`), so one click plays and selects. The facade takes the focus on click and `playVideo()` hands it to the player once loaded: Chromium blurs the editor when a focused element is removed. A click into a frame fires no focus event in the page, so `fireFrameFocusEvents()` in `content_embed.ts` fires `focus` at a frame in an embed that a window `blur` leaves focused, and `blur` on the next window `focus`; the embed's own `focus` listener and `FocusTracker` do the rest. A click from one frame into another fires no window event either, so after a window `blur` one shared interval (`watchFrameFocus()`, 200 ms, for every editor's check in `frameFocusChecks`) reads `document.activeElement` until no frame has the focus or the window has it back. These synthetic events neither bubble nor fire `focusin`/`focusout`, so the box's `useFocusWithin()`, which draws the `.include-note-body.active` outline, also listens to `focus`/`blur` in the capture phase. `createLinkPreview()` in `link_embed_commands.ts` is the one place that picks `linkMention`, `linkEmbed` or a video embed for a display mode. The YouTube video prompt (`youtube_video.ts`) inserts the player only once `fetchMetadata()` has returned, with no link meanwhile: its `onInsert()` rejects on unresolved metadata, and the host's prompt keeps Insert disabled while it waits and shows an error toast when it rejects. - **Legacy blocks** — a `<pre>` without `data-box-size` predates the embed (the old Mermaid plugin). `getModeOfSavedView()` loads it editable, in the view of its `data-trilium-display-mode` or split, at Medium. A saved embed keeps exactly what it has. - **Later kind** (`CONTENT_EMBED_KIND_ATTRIBUTE` in `packages/commons/src/lib/content_embeds.ts`) is the form for any new kind that is neither a note/attachment reference nor a code block, so that an older version keeps it instead of dropping it on save, without a sync version bump. `UnknownContentEmbed` (`unknown_embed.ts`) keeps the section raw through `registerRawContentMatcher()`, renders it as an `.include-note` box that the host fills through `contentEmbed.loadUnknown()` (`ContentEmbedError`, the box of a missing note, with the `bx-error` icon), and writes it back with its `class` and `data-*` attributes (anything else, `style` included, is dropped) and its content, sanitized by the host's `contentEmbed.sanitizeHtml`. Its upcast runs at `high`, before General HTML Support. A new kind must follow the contract older versions rely on: - Settings go in `data-*` attributes on the section, never in `style` (the import sanitizer allows `--include-note-*` only on `figure` and `pre`). - A note reference is `data-note-id` on the section itself, which `findIncludeNoteLinks()` turns into the `includeNoteLink` backlink. - An attachment reference is `data-attachment-id` on any element (or an `api/attachments/<id>/image` URL), which `checkImageAttachments()` keeps from erasure and rewrites when the attachment is copied. - The content is a static fallback (a link, a picture, the source in a `<pre>`): old read-only views, share pages and Markdown export show it. Read-only views never show the fallback either: `emptyUnknownEmbeds()` (`content_renderer_text.ts`) empties each section into an `.include-note` box, keeping only the kind, `data-box-size` and the block ID. `ReadOnlyText` then mounts the same error box (`loadUnknownEmbed()`), and `postProcessRichContent()` leaves the box empty, as it leaves a missing note's. Share pages replace the section, and an include of a missing or unshared note or attachment, with `<div class="include-note-unavailable">` and a generic message (`replaceWithUnavailable()` in core's `share/content_renderer.ts`), as the page is public. The version that adds a kind registers its converter above `high`. If that converter needs the section's children, narrow `ENVELOPE` in `unknown_embed.ts` to leave the kind out: the raw matcher removes the children before any converter runs. - **Attachment** suits large or binary content and content with its own save cycle (Excalidraw). The attachment's content type comes from `getFileContentType()` in `packages/commons/src/lib/notes.ts` (`canvasDrawing`, `mermaidDiagram`), which `getRenderedContent()` dispatches on. ## How a box gets its content 1. **Editor plugin** (`packages/ckeditor5/src/plugins/content_embed/`). `content_embed.ts` owns the model, the converters, the commands (`contentEmbedBoxSize`, `toggleContentEmbedTitle`, `toggleContentEmbedCaption`, `toggleContentEmbedEditable`, `contentEmbedDisplayMode`, `convertEmbedToLink`, `deleteContentEmbed`), the box-size rules and observables such as `isSelectedEmbedInline`. `deleteContentEmbed` serves any kind: `getDeleteEmbedGroup()` in `menus/link_context_menu.ts` ends the ⋮ menu of a note, an attachment, a video and an extended code block with it. Other inline sources (Mermaid) do not offer it. `content_embed_toolbar.ts` lists the toolbar items; `content_embed_tools.ts` turns the content's tools into buttons; `content_embed_box_size_dropdown.ts` the size menu. The editing view renders an `.include-note-wrapper` and asks the **host** to fill it. 2. **Host contract** (`ContentEmbedHost` in `content_embed.ts`, read from the editor config as `contentEmbed`): `loadNote()`, `loadAttachment()`, `loadInline(source, wrapper, boxSize, { isFocused })`, `loadUnknown()`, `focusAttachment()`, `openMenu()`, `getTools()`, `enter()` and `getAttachmentHref()`. `EditableText.tsx` implements them as `contentEmbedHandlers`, which `CKEditor.tsx` hands to `buildConfig()` through a getter. `sanitizeHtml()` is set in `buildConfig()` itself. The plugin never imports client code. 3. **Box** (`text/utils.ts`). `loadEmbeddedAttachment()` / `loadInlineEmbed()` render the content with `content_renderer`, wrap it in `ContentEmbed` (`text/ContentEmbed.tsx`: title row, fullscreen, exit-fullscreen overlay) and `mountEmbedBox()`. Content that left the page while it loaded is disposed; `watchContentEmbeds()` disposes boxes removed later. 4. **Content** (`services/content_renderer.ts`). `renderCanvasDrawing()` and `renderSourceEmbed()` mount the component through `mountInteractiveWidget()` when `options.interactive`, with `getContentEditor()` as its `ContentEditor`; otherwise a static rendering (the drawing's picture). Lazy modules are listed in `CONTENT_RENDERERS.file` so read-only pages and the share theme never load the editors. ## The content component contract A content component receives the content and an optional `ContentEditor` (`canEdit()`, `getUnsavedContent()`, `scheduleSave()`, `release()`; no editor = read-only). Source embeds receive `SourceEmbedProps` (`content`, `editor`) whichever storage the source has: `InlineSourceEmbed` and `AttachmentSourceEmbed` in `text/SourceEmbed.tsx` adapt the two. - **`useEditableEmbed(rootRef, { editor, note, tools, focusTarget })`** (`text/editable_embed.tsx`) returns `canEdit` (an editor that can save, no `#readOnly`, database writable) and `isEditing` (`canEdit` and the embed's Editable toggle on). It registers `tools` while `canEdit` and moves the focus to `focusTarget` when the box takes it. Nested embeds stay read-only (`NestedEmbedContext`). - **Toolbar tools** — a `ContentEmbedToolProvider` (`text/content_embed_tools.ts`): `getTools()`, `execute(id)`, `subscribe()`, plus - `hasEditableFlag` — shows the Editable toggle; the embed carries `data-editable="true"`. - `hasDisplayModes` — shows Source/Split/Preview; the embed carries `data-display-mode`. - `hasPositions` — gives an inline source the Position submenu of the ⋮ menu and the toolbar's position dropdown; the embed carries `data-position`. A note, attachment or video embed has the submenu without it (`hasEmbedPositions()`), and the dropdown only with it. - `hiddenToolbarItems` — moves box size, title, convert-to-link or Editable from the toolbar to the ⋮ menu (an extended code block hides box size and Editable). - `boxSizeIcon` — an SVG the box size dropdown shows in place of the size name, which moves to its tooltip (`Box size: %0`). Code and Markdown embeds set it even while read-only (`hasEditableFlag: false`), so every one of them shows the icon; code embeds through `useContentEmbedMenus()`. - `getMenuItems()` — items of the ⋮ menu after Editable, in a group of their own