1 Architecture

How a save flows from the file watcher to a repainted block, and where each crate sits.

Saving post.tmd updates the browser in tens of milliseconds (the measurements), with no page reload and no cold start, and the scroll position holds. That works because all of the intelligence lives in the Rust core behind a stable websocket protocol, and the browser preview is a thin client (Figure 1.1).

flowchart LR
  BR["Browser preview<br/>(vanilla JS)"]
  subgraph Server["taliesin server"]
    direction TB
    WS["Websocket<br/>+ file watcher"]
    subgraph Core["taliesin-core"]
      direction TB
      P["Parser<br/>comrak + sourcepos"] --> M["Block model<br/>+ diff"] --> R["Render<br/>page / site / book"]
    end
    K["Warm Jupyter kernel"]
  end
  F[(".tmd files")]
  BR <-->|JSON protocol| WS
  WS --> P
  R -->|changed blocks| WS
  F -.->|watch| WS
  M -->|code cells| K --> M

Figure 1.1: Thin clients talk to the server over a websocket; all rendering lives in the Rust core.

Component	Crate / path	Responsibility
Core	`crates/core`	Parsing (comrak + sourcepos), the block model, HTML rendering
Server	`crates/server`	Websocket dev server, file watcher, Jupyter kernel pool
Web client	`web-client/`	Browser preview client (vanilla JS)

The protocol is open, so an embedded editor client is one more thin client and needs no rewrite. The in-repo VS Code companion (covered in extending Taliesin) is such a client.

1.1 How a save flows

Change tau from 10 to 12 in post.tmd and save. Only the changed blocks reach the preview: the heading and the paragraph stay put, and only the plot the cell redraws crosses the wire.

The order is easy to get wrong: a save is re-render, then execute, then diff, not re-render then diff. The server re-runs the render pipeline (render/) to get a fresh block list, then hands that list to the executor (exec.rs), which runs the changed code cells (plus everything downstream) and splices each cell’s output back into the list as a derived output block. Only then does diff.rs compare the resulting list against the live one, so the diff never sees the bare render output. ({js} cells are the exception: they run in the browser, never through exec.rs, so a {js} block crosses the wire as-is. See Interactive cells.)

The same save as an interaction over time, module by module, is Figure 1.2:

sequenceDiagram
  autonumber
  participant FS as .tmd file
  participant SV as serve_site/mod.rs (watcher)
  participant RN as render/ (core)
  participant EX as exec + kernel
  participant DF as diff.rs
  participant CL as client.js
  FS->>SV: changed (notify)
  SV->>RN: re-render (once its events stop)
  RN->>EX: run first changed cell + downstream
  EX-->>RN: output blocks
  RN-->>SV: new block list
  SV->>DF: new blocks vs live blocks
  DF-->>SV: BlockOps (changed only)
  SV->>CL: ws update / insert / remove
  CL->>CL: swap blocks in place (scroll + live state kept)

Figure 1.2: A save, module by module: only changed blocks (and re-run cells) cross the wire.

The round trip between render/ and exec + kernel in that diagram is the execution step, not a render loop: the block list diff.rs finally sees is the post-execution one.

One server handles a project and a single document alike (a document resolves to the project it belongs to, or becomes a project of itself): navigation between pages is an ordinary full page load (no SPA), and warm kernels are pooled. serve_site/exec_pool.rs keeps the six most-recently-built pages’ executors alive (MAX_WARM_PAGES) in a deterministic LRU and drops the rest, so revisiting a recent chapter reuses its kernel while an unbounded map cannot grow one 80 to 150 MB kernel per page visited; an evicted page pays a cold start on its next edit, and nothing else. The multi-page subsystem (page discovery, chrome, cross-page references) lives a layer up, in crates/core/src/site/, whose mod.rs header is the entry point to read.

1.1.1 The watcher

spawn_watcher runs notify on its own thread (notify is synchronous) and watches the project’s directory. Events are pumped through a channel so that thread owns the watcher and can register watches for directories created after startup: the recursive-watch model added an inotify descriptor per directory, including node_modules and .git, which a large project uses to exhaust max_user_watches and kill hot reload.

The watcher skips _freeze: the executor writes its own cache there on every run, and without the skip each write would kick a redundant rebuild. It skips every .-prefixed tree too, as discovery and the build do: a project’s own .venv is thousands of directories. The skip list is matched against the path relative to the project root, never the whole path: the watcher deals in absolute event paths, and a project can live under a directory someone else called _site or .git. Scanning the whole path vetoed every event in such a project, which stopped hot reload silently while each page still served 200. Every other event reaches the rebuild, which decides by what the pages read (below), not by a file’s extension. A save’s events are gathered until they stop for 15 ms: every editor’s save finishes its events within about a millisecond, and the fixed 80 ms wait this replaced was most of every save. A directory that arrives by a rename or a move gets watches of its own, like a created one: notify drops a moved directory’s watch.

1.1.2 What a save rebuilds

A save reaches only the pages a tab is watching. A page depends on every file its last render read or looked for: its source, each {{< include >}} it tried, each .bib it loaded (the project’s shared one too), each image it measured or checked, and each page it links to. The read sites record those paths themselves (crates/core/src/reads.rs), so no second reader has to agree with them, and a file looked for and not found counts: creating it rebuilds the page. every_corpus_doc_records_the_files_its_render_reads holds the recorded set to the includes, bibliographies and images of every corpus page.

A save is judged by what it changed, not by how the editor wrote it. Other pages render a page’s front matter and its leading # H1 (a listing card, a nav label, a book’s drawer and chapter numbers), so a save that moves either re-discovers the project, and so does a page added, renamed or deleted; a save that moves neither, whether written in place or renamed over the old file, rebuilds the pages that read it. After a re-discovery, a tab whose chrome (navbar, drawer, pager, footer) moved reloads, and every other tab takes the change as block ops and keeps its live state. A _site.yml change reloads every tab.

1.1.3 Panic isolation

The render → execute → diff pipeline above runs inside a spawned task per page, so a panic anywhere along it would otherwise silently kill hot reload for the rest of the session. build_page_guarded wraps each build in catch_unwind (via AssertUnwindSafe over the awaited future, so the panic is caught rather than propagated): on a caught panic it pulls a human message off the payload (panic_msg downcasts the Box<dyn Any> to &str/String), logs it, and broadcasts a protocol::error so every connected client shows an error overlay. The worker survives, so the next good save re-renders and clears the overlay (the errored flag makes that recovery a full re-mount even when the diff is empty). Because the state lives behind a parking_lot::Mutex (not std::sync::Mutex), a panic while the lock is held releases it cleanly rather than poisoning it.

1.2 Binding, and the three guards

The server binds 127.0.0.1 and nothing else: there is no flag that puts it on a network. That is why the three guards in serve/security.rs are all about a local peer or a page in the author’s own browser rather than a remote one.

ws_origin_ok gates the websocket upgrade on Origin against Host. A page on another site must not be able to open the control channel and send restart_kernel, which would destroy the warm kernel. A loopback Origin is trusted (a second dev server, the editor companion); anything else that is not same-origin gets a 403.

with_host_guard applies host_allowed to every HTTP response: the Host header must name a loopback address. This is the DNS-rebinding defence, and it cannot be folded into the origin check, because under a rebind both headers are the attacker’s domain and compare equal. A request with no Host is allowed: only a browser can mount a rebind, and a browser always sends one.

The same middleware reads the request’s Fetch Metadata. A request from a page on another site, or on another port of this one (Sec-Fetch-Site: cross-site or same-site), is served only when it is a navigation: a link, a prefetch, or the editor companion’s webview <iframe>, which is cross-site because the webview is its own origin. Anything else gets a 403, because a browser runs a <script src> from any origin and /search-index.js hands the including page every page’s text, drafts included. A request with no Fetch Metadata (curl, the takeover’s identity probe, the websocket upgrade) is unaffected.

Reading a draft on another device is a build and a static file server, not a flag on the preview.

1.3 The codebase map

Figure 1.3 shows the dependencies between modules: the server orchestrates and owns the runtime, the core does all the rendering and never touches the network, and the browser client only mounts blocks and applies ops.

flowchart TB
  subgraph client["web-client/ · browser (vanilla JS)"]
    CL["client.js<br/>mount · apply ops · click-to-source"]
  end

  subgraph server["crates/server · Taliesin (binary)"]
    direction TB
    MAIN["main.rs · CLI"]
    SITE["serve_site/<br/>THE dev server"]
    SRV["serve/<br/>shared HTTP + guards"]
    BLD["build.rs"]
    LSP["lsp*.rs<br/>offline, kernel-free"]
    PR["protocol.rs<br/>shared ws contract"]
    EX["exec.rs · executor"]
    KR["kernel.rs<br/>warm ZMQ kernel"]
    MAIN --> SITE
    MAIN --> BLD
    MAIN --> LSP
    SITE --> SRV
    SITE --> EX
    SITE --> PR
    EX --> KR
  end

  subgraph core["crates/core · taliesin-core (library)"]
    direction TB
    RN["render/<br/>parse → block model → HTML"]
    DF["diff.rs"]
    SM["site/<br/>multi-page projects"]
    INC["includes.rs"]
    FM["frontmatter.rs"]
    CI["cite/"]
    MA["math.rs"]
    HL["highlight.rs"]
    RN --> INC
    RN --> FM
    RN --> CI
    RN --> MA
    RN --> HL
    SM --> RN
  end

  CL <-->|ws JSON| SITE
  SITE --> RN
  SITE --> SM
  SITE --> DF
  BLD --> RN
  LSP --> RN
  EX -.->|"output blocks<br/>(data flow, not a call)"| DF

Figure 1.3: Module dependencies across the three layers. A solid arrow is ‘uses / calls’; the dashed exec→diff arrow is data flow (exec produces the blocks diff consumes), not a call.

The data those modules pass around is small (Figure 1.4): a RenderedDoc is a list of Blocks, an executable block carries a Cell, and diff.rs turns two block lists into a stream of BlockOps.

classDiagram
  class RenderedDoc {
    Option~String~ title
    bool toc
    Vec~Block~ blocks
  }
  class Block {
    String id
    String sourcepos
    Option~String~ source_file
    String html
    Option~Cell~ cell
  }
  class Cell {
    String lang
    String code
    bool echo
    bool include
  }
  class BlockOp {
    <<enumeration>>
    Update
    Insert
    Remove
    SetMeta
  }
  RenderedDoc "1" *-- "many" Block : blocks
  Block "1" o-- "0..1" Cell : cell
  Block ..> BlockOp : diff emits

Figure 1.4: The core data model (key fields shown): what flows from the parser to the client.

The file-by-file detail is not restated here: each file’s own header already states it, and a table of module responsibilities would be a second copy that goes stale. Start at crates/core/src/render/mod.rs for the pipeline, crates/core/src/site/mod.rs for multi-page projects, and crates/server/src/serve_site/mod.rs for the dev server. Each opens with a header describing what it owns.