3 The block model & protocol

In Taliesin the document is a list of blocks, each of which carries its identity and its origin. Our post.tmd is three of them, the # Cooling coffee heading, the paragraph, and the Python cell, plus a fourth the cell produces: the plot, a derived block keyed to the cell.

3.1 The block model

Every top-level element is a block. Each block carries three data attributes that the rest of the system keys off:

• data-block-id is a content hash (with a positional tiebreak for duplicates). Because it is content-derived, an unchanged block keeps its id across edits and is never touched.
• data-sourcepos is the startLine:startCol-endLine:endCol range the block came from. Click-to-source uses it in both directions.
• data-source-file is present only on blocks pulled in from an {{< include >}}d file, so cross-file source mapping still works.

Source mapping, incremental re-render, and live-state preservation all build on this model. A code cell’s output is itself a block, keyed to the cell’s id, so when an upstream cell re-runs, only the affected output blocks update.

The exact field shapes (Block, Cell, RenderedDoc, and friends) live in crates/core/src/render/model.rs, one small file with a doc comment per field.

3.1.1 How the id is computed

make_id (render/mod.rs) is small and deterministic, so two renders of the same source always agree and the diff has a stable anchor:

1. Take the block’s raw Markdown source, trimmed of leading and trailing whitespace.
2. Hash it with 64-bit FNV-1a (offset basis 0xcbf29ce484222325, prime 0x100000001b3), the same hash the freeze cache uses for cell keys.
3. Format the digest as 16 hex digits, keep the first 12, and prefix b-. So an id looks like b-3af19c0b7e21.
4. If a byte-identical block already produced that base id, append a positional -N suffix (the first duplicate becomes b-…-1, the next b-…-2, and so on), via dedup_with_suffix. This is the only place position enters the id, and only to break exact-content ties, so ids stay unique within a document.

Position stays out of the hash so that inserting a paragraph at the top of a document leaves every following block’s id unchanged, and the diff sees one insert rather than “everything moved”. The -N tiebreak lets the diff treat ids as unique (the basis for the O(n log n) shortcut below).

A generated block has no source text of its own (the synthesized title header, the References block, the footnotes section), so it skips the hash entirely and takes a fixed id with an empty data-sourcepos.

3.2 The diff (diff.rs)

diff_blocks(old, new) turns two block lists into the minimal set of DOM ops the client applies. Because ids are content hashes, an unchanged block has the same id in both lists, so the algorithm is an LCS over the id sequences:

1. The LCS finds the matched id pairs: the stable anchors that didn’t move.
2. The runs of blocks between consecutive anchors are the changed regions. emit_gap pairs them positionally: each pair → an Update (replace the old block’s element with the new HTML); a surplus of old blocks → Remove; a surplus of new blocks → Insert after the preceding block’s id (or at the start when there is none).
3. A matched anchor is normally left untouched. When its HTML does differ, anchor_op chooses between two outcomes by comparing the bodies with the source-position attributes masked out:
   • if only data-sourcepos moved (an edit above shifted the block’s line numbers, but its content is byte-identical), it emits a SetMeta: patch the positions in place, with no re-render, so the live element keeps its DOM state;
   • if the body itself changed, it emits an Update. This is the path a derived block whose id isn’t a pure content hash (a code cell’s output block, keyed to the cell id) takes when an upstream cell re-runs.

3.2.1 How SetMeta vs Update is decided

The “with the source-position attributes masked out” step compares the two htmls with every data-sourcepos attribute value blanked and everything else required to match byte for byte. A data-source-file that changed is therefore a real difference, and takes Update.

A leaf block carries one position, on its own element, so the common case is a cheap string comparison with the first value blanked. A ::: div carries its inner blocks’ positions too, and a line shift above it moves all of them, so its SetMeta also carries inner: the new value of every position inside it, in document order, which the client writes onto the div’s own descendants. Those values are read through the tag walker, so prose that merely quotes data-sourcepos="…" inside a <code> span is content and must match. A div that does not close is re-rendered instead, because the browser has moved the blocks after it inside it and its descendants are no longer its own html’s.

The whole decision is mechanical once the anchors are known (Figure 3.1):

flowchart TD
  lists["old block ids vs new block ids"] --> lcs["LCS over the ids"]
  lcs --> anchor{"matched anchor?"}
  anchor -->|"HTML identical"| keep["leave untouched"]
  anchor -->|"only sourcepos moved"| sm["SetMeta"]
  anchor -->|"body changed (cell output)"| u1["Update"]
  lcs --> gap{"gap between anchors"}
  gap -->|"old / new pair"| u2["Update"]
  gap -->|"surplus old"| rem["Remove"]
  gap -->|"surplus new"| ins["Insert after prev id"]

3.2.2 Why the diff is O(n log n)

The textbook LCS is an O(m·n) dynamic-programming table, and on a save it would be reallocated on every keystroke: tens of megabytes once a document reaches a few thousand blocks. Taliesin never builds it.

The shortcut is that block ids are unique within a document. A common subsequence is then just a set of shared ids in the same relative order, so the LCS collapses to a Longest Increasing Subsequence: map each old block to its position in the new list, and the longest still-increasing run is the set of anchors that didn’t move. LIS is solved in O(n log n) time and O(n) space by patience sorting, which is what lcs_pairs does. The cost of a diff is therefore dominated by the handful of blocks that changed, not by the size of the document.

An unchanged block is never touched, so the browser keeps its scroll position and any live DOM state inside it (a running Three.js canvas, a {js} cell) survives a save (goal #2).

Change tau and the diff emits one op, an Update for the plot’s block, while the heading and the paragraph, their content hashes unchanged, never move.

A real warm edit also shows the cost in bytes. Typing one new paragraph above an already-open collapse callout, benchmarked by tools/live-edit-bench, emits 55 operations: 54 of them (every shifted paragraph, heading, figure and code cell below the edit, and the open ::: {.callout-note collapse="true"} div with the positions of its inner blocks) are SetMeta patches like the one above, so anything live in those blocks keeps running and the callout stays open. Together with the one insert for the new paragraph itself they weigh about 3.2 KB, and each block’s cost depends only on how many positions it carries. (See Choosing Taliesin for the full measurement.)

3.2.3 Removes before inserts (the ordering rule)

diff_blocks produces the ops anchor by anchor in document order, then applies a stable sort that hoists every Remove ahead of every Insert. The client applies the ops in the order it receives them, so its correctness depends on this ordering.

The reason is the no-Move case above. A block that moves forward is emitted as a Remove at its old slot plus an Insert of the same id at its new slot. If the Insert ran first, the new element (carrying that id) would already be in the DOM when the Remove ran, and Remove’s id-based lookup would find and delete the freshly inserted element instead of the stale one. Hoisting the Remove makes the old element gone before the new one arrives, so the lookup is unambiguous. Removes are positionally independent (each names a concrete target_id), so reordering them is safe; the sort is stable, which keeps the Inserts in document order so their after_id chains still resolve.

3.3 The websocket protocol

On connect, the server sends a full_render; on each save it diffs the new block list against the old one (an LCS over block ids) and broadcasts only the differences. The exception is raw HTML the page does not hold as one element of its own: a comment and a lone closing tag emit no element, and a wrapper’s unclosed opening line swallows the blocks after it. A save whose ops aim at, anchor on or bring in such a block is sent as a full_render instead (needs_remount in diff.rs).

These message shapes are defined once, in protocol.rs, and shared by the dev server and the browser client. serve_site/ and web-client/ are the two ends of this contract, so they can’t drift apart from each other. The update / insert / remove / set_meta messages are just the serialized BlockOp enum (the variant name lowercased into the type field); its four Rust variants and their fields are declared in crates/core/src/diff.rs. For the exact JSON shape of every message and when each is sent, read crates/server/src/protocol.rs; for how the client applies one, web-client/client.js.

Two orderings matter to the live view. On a rebuild, each code cell the edit changed goes out as an update of the block it replaces (and its old output as a remove) before any cell runs: the edit changed the cell’s id, and its running badge and live output are addressed to the new one. And a cell-state of running sent again for a cell that is already running empties its live output, which is how the server redraws it after clear_output or an in-place display update, which append / replace_last cannot express.

3.3.1 Applying the ops (the client)

The browser side (web-client/client.js) is thin: each op is one small DOM mutation, and the diff above keeps that set small.

• update finds the target by id (a child of #tali-root first, since ops name top-level blocks) and replaceWiths the new HTML; insert builds a fragment and places it after after_id (or prepends it when there is none); remove drops the element; set_meta patches only data-sourcepos / data-source-file on the existing element (and the inner positions of a ::: div on its descendants) and does not re-render. An op whose target or anchor is not in the page reloads it rather than guess, and a <script> an op brings in is re-created so it runs, as it would have on load.
• Every mutation runs inside keepScroll, which pins window.scrollY across the change (instantly, never animated), and is followed by afterChange, which rebuilds the TOC, re-runs the code enhancers, and re-scans for cell errors. set_meta skips afterChange (nothing rendered).
• The first paint is server-rendered into #tali-root, so the first full_render after connect is identical and is skipped, which avoids a flash and a needless {js} re-init. A reconnect re-mounts normally, and a dropped socket retries on a short loop.
• Because block ids are unique, an insert first drops any element already carrying the incoming id. The server emits all removes before any inserts, so this is normally a no-op; it only matters if a reorder (a remove + insert of the same id) ever arrived out of order.

3.4 Inverse search & forward search

Click-to-source (goal #1) uses only the two source attributes:

• Inverse search (preview → editor). Ctrl-clicking (Cmd-clicking on Mac) a block reads its data-sourcepos (+ data-source-file) and opens the exact source line: a vscode://file/<path>:<line>:<col> deep link by default. The source positions start as comrak’s per-node ranges; on the way to becoming blocks they are translated through the include source map (includes.rs records a LineOrigin for every expanded line → its origin file + line), so a block that came from an {{< include >}}d partial carries that file in data-source-file and the original line, which lets click-to-source work across files.
• Forward search (editor → preview). The client also speaks a postMessage protocol as the integration surface for an embedded editor host. The in-repo VS Code companion is that host: the client posts tali-goto up to it (which reveals the source), and accepts tali-cursor {file, line, reveal}. That last one maps a source line back to a block: highlightAtLine picks the smallest block whose data-sourcepos covers the line (else the nearest block starting before it) and outlines it with .tali-hl, scrolling it into view only when reveal is set. So an editor can host the preview and add cursor sync without forking the client.

Both directions only navigate and highlight: the .tmd file stays the single editing surface, and nothing in the preview or the editor-client integration ever writes back to it.