Architecture

currentoverviewarchitectureoverviewinvariants

Top-level architecture linking data ingest, storage, API surfaces, the application UI, and the public static wiki.

Architecture

Lorewright is a Python + TypeScript monorepo with two user-facing frontends and a source-backed engineering wiki.

The monorepo layout is a deliberate choice over separate repositories. All three publishing surfaces — the application UI, the public wiki, and the FastAPI API — share the same data pipeline, schema definitions, and generated artifacts. A multi-repo setup would require cross-repo versioning and artifact publishing, adding coordination cost without meaningful isolation benefits. The trade-off is a larger checkout and a broader CI surface, which is acceptable at the current project scale.

Ownership Map

  • Data Pipeline owns ingest and generation. It is the only subsystem that writes to data/5e-database.sqlite and emits files under progressions/.
  • SQLite Schema defines storage conventions and merged view targets. Two databases serve distinct roles: 5e-database.sqlite holds ingested SRD/homebrew source data, while lorewright-app.sqlite holds runtime state (characters, TL;DR datasets).
  • Server exposes operational APIs, including character, spell, and wiki routes. It is a thin orchestration layer — business logic lives in engine/, and the server delegates to engine functions.
  • Engine holds runtime and orchestration modules. It is the only Python package with a CLI entry point (engine/app.py), and that CLI now spans ingest, progression generation, GRPO combat finalization/enrichment, TL;DR build/validate/sync, character management, and stateless level-up.
  • web/ is the React application UI for progression browsing and character level-up flows. It consumes the FastAPI API and mirrors Python Pydantic schemas as TypeScript types in web/src/types.ts.
  • site/ is the static public wiki frontend deployed to Vercel. It has no runtime dependency on Python — it reads wiki/*.md at build time and emits static HTML.
  • DM Orchestration documents the DM assistant integration path.

Core Runtime Flow

  1. poetry run lorewright db-refresh updates SQLite from SRD + canonical homebrew.
  2. poetry run lorewright generate-progressions emits JSON artifacts and manifest entries. The same engine namespace also exposes the GRPO and TL;DR dataset jobs through engine/app.py.
  3. server/__init__.py serves progression, spells, homebrew, review, rebuild, SRD level-up, and persisted character routes.
  4. web/ consumes those APIs for the application workflow, while site/ builds the public wiki from markdown.
  5. server/wiki.py separately exposes the engineering-facing wiki API for metadata, health, and source inspection.

Primary wiki API implementation reference: server/wiki.py:L1-L260.

This pipeline is strictly unidirectional: source JSON flows into SQLite, SQLite flows into generated artifacts, and the API reads artifacts at runtime. There is no feedback loop where the API writes back to SQLite or modifies source files. This makes the system easy to reason about and safe to rebuild from scratch at any time — db-refresh followed by generate-progressions reproduces the full artifact set deterministically.

Frontend Split

  • web/ is the application frontend: it talks to FastAPI for progression data, level-up preview/apply flows, and persisted characters.
  • site/ is the documentation frontend: it statically compiles wiki markdown for public browsing on Vercel.
  • The static site and FastAPI wiki API overlap on page content, but not on behavior. The site is a static reader; the FastAPI surface adds API-only health checks, commit/source views, TOC payloads, and staleness metadata.

The two-frontend design reflects two fundamentally different deployment models. The application UI requires a running Python backend (FastAPI + SQLite), so it is not suitable for static hosting. The wiki is pure read-only content with no backend dependency, making static export the natural choice. Combining them into a single frontend would force the wiki into the application deployment lifecycle, adding unnecessary coupling and making the wiki unavailable when the backend is down.

Design Rationale

Pydantic as the schema backbone. All data boundaries — API request/response shapes, pipeline intermediate formats, progression artifacts — are defined as Pydantic v2 models with extra="forbid". This strict validation catches shape mismatches at the boundary rather than deep inside business logic. The TypeScript frontend mirrors these shapes manually in web/src/types.ts; there is no code generation step. The manual mirroring is a known maintenance cost, but it avoids introducing a build-time schema compiler and keeps the frontend build simple.

Immutable character snapshots. The level-up flow (preview → decide → apply) never mutates existing state. Each successful apply creates a new snapshot row in lorewright-app.sqlite. This design was chosen to support undo/history without tombstone logic, and to make the apply operation idempotent — replaying the same apply request produces the same snapshot. The trade-off is storage growth, which is negligible for character data.

CLI and API parity. The CLI commands (db-refresh, generate-progressions) and the API rebuild endpoints (/api/rebuild/*) call the same engine functions. This ensures that a developer running locally and the API responding to a rebuild request produce identical results. The server module (server/init.py) imports directly from engine.ingest rather than shelling out to the CLI.

Assumptions & Constraints

  • Single-writer SQLite: both databases assume a single writer at a time. The API server is the only writer to lorewright-app.sqlite, and pipeline commands are the only writers to 5e-database.sqlite. Concurrent writes would require WAL mode or an external lock, neither of which is currently configured.
  • 2014-only for v1 characters: the saved-character level-up flow explicitly guards against 2024 edition data. This constraint is intentional — the 2024 SRD level-up rules are not yet fully modeled, and allowing partial 2024 support would create confusing edge cases.
  • Generated artifacts are disposable: everything under progressions/ can be deleted and regenerated from source data. The manifest and JSON files are not hand-edited and should never be committed with manual changes.
  • Flat wiki namespace: wiki slugs are derived from filenames (wiki/{slug}.md), so all pages share a single namespace. Name collisions are prevented by convention, not by tooling.

Conceptual Model

The architecture has three layers with strictly downward dependencies:

  1. Presentation layer (web/, site/, FastAPI routes in server/): consumes data, never produces it. The two frontends and the API are all read-only consumers of the artifacts and databases below.
  2. Engine layer (engine/): owns all business logic, schemas, and pipeline operations. This is the only layer that transforms data — ingesting source JSON, generating progressions, computing level-up previews, orchestrating DM sessions.
  3. Data layer (srd/, homebrew/, data/, progressions/): static source files, runtime databases, and generated artifacts. This layer has no code — it is pure data operated on by the engine.

The key boundary is between presentation and engine: the server never contains business logic, and the engine never imports from server/ or web/. This separation means the engine can be tested, run via CLI, and reasoned about independently of any HTTP concerns.

Key Invariants

  • SQLite remains the canonical runtime data store.
  • Character history is immutable: preview and apply operate on snapshots, and successful applies persist a new snapshot.
  • Canonical homebrew is authored once and projected into both editions.
  • Wiki pages must cite source files and remain linked into the graph.
  • Rebuild API behavior mirrors CLI pipeline behavior.
  • Static wiki publishing is documentation-only; operational wiki diagnostics stay in the FastAPI API.

Related References