# CreatorStudio — Architecture & System Documentation This repo holds the **CreatorStudio suite** — two apps that make up one product: - the **CreatorStudio authoring app** (`web/creator/`, **Angular 20 + PrimeNG**) where operators *design* announcements — organised into **Projects** (playlists, triggers, voices, lexicon), and - the **CreatorStudio Engine** (`src/`, Node/TS) — an always-on backend that *executes* that config live, turning transit trip data into **spoken passenger announcements** and **LED sign content** for an entire fleet — plus the Angular **dashboard** (`web/`) to operate, monitor, and audit it. > The original React/Vite authoring app in `audiocreator/` has been migrated to the Angular Creator > at `web/creator/` and is **retired/vendored** (no longer served — see §5). The **database source of > truth** is `db/migrations/` (plain SQL, applied by `npm run db:migrate`); its old Supabase backend > (`audiocreator/supabase/`) was removed in the Azure migration. All three are served together behind a single **portal** (`scripts/dashboard-server.ts`): `/` landing · `/creator/` authoring app · `/monitor/` dashboard (see §5). The engine is the primary subject of this document; the authoring app is covered where it drives the engine (config, trigger parity) and in §5. This document is the single source of architectural truth for the project. Diagrams are [Mermaid](https://mermaid.js.org) and render on GitHub and in the portal's **Documentation Center** (`/docs` → the consolidated handbook), which is where every document is published — the dashboard itself no longer carries a documentation panel. --- ## About this document — who should read what | You are a… | Start with | Then read | | --- | --- | --- | | **Stakeholder / exec** | §1 What it does · §3 Capabilities | §14 Gaps · §15 Roadmap | | **Product manager** | §1 · §3 Capabilities · §7 Interfaces | §14 Gaps · §15 Roadmap · §16 Glossary | | **Designer** | §1 · §3 Capabilities · §6.4 LED/sign rendering · §5 Frontend | §7 Interfaces | | **Backend developer** | §4 Backend architecture · §6 Software design · §7 Interfaces | §8 Runtime · §10 Performance · §11 Scalability | | **Frontend developer** | §5 Frontend · §7 Interfaces (HTTP + payloads) | §6.4 rendering | | **DevOps / SRE** | §9 Deployment · §10 Performance · §11 Scalability · §12 Security | §13 Testing | --- ## 1. What it does (system function) Modern transit vehicles must announce stops to passengers **audibly** (speakers) and **visually** (interior + exterior LED signs), in multiple languages, triggered automatically by where the bus is and what it is doing. Authoring that content (playlists, trigger rules, voices) is done in the **CreatorStudio** web app. **This engine is the runtime** that executes it live across the fleet. In one sentence: **it subscribes to each vehicle's live trip feed, decides what to say and show, synthesizes the audio, renders the sign graphics, and publishes them back to the vehicle — for hundreds of buses at once — while keeping a tamper-evident record of everything that played.** ```mermaid flowchart LR FEED["Live trip data
(position, stops, doors…)"] --> ENGINE AUTHOR["CreatorStudio app
(playlists · triggers · voices)
in-repo · /creator/"] -. "publishes config" .-> ENGINE ENGINE["CreatorStudio Engine"] --> AUDIO["🔊 Spoken announcements"] ENGINE --> INT["▦ Interior LED sign"] ENGINE --> EXT["▦ Exterior destination signs"] ENGINE --> PROOF["🧾 Proof-of-play audit"] ENGINE -. "telemetry" .-> DASH["📡 Dashboard
/monitor/"] ``` The authoring app and the dashboard are both in this repo (`web/creator/` and `web/`) and served by the portal alongside the engine; the retired React original in `audiocreator/` still deploys standalone to Lovable. **Value delivered** - **Automated, consistent announcements** — no driver action; every stop announced the same way. - **Multilingual** — the same trigger speaks in the configured language with the right stop-name variant. - **Multi-surface** — audio, interior scrolling sign, and front/side/rear destination signs stay in sync. - **Authored, not coded** — operators change wording/voices/rules in CreatorStudio and **publish live**; the engine hot-swaps in ~1 s with no redeploy. - **Fleet-scale & cost-aware** — identical phrases are synthesized once and reused across every bus. - **Auditable** — a durable *proof-of-play* log records what was played/shown, where and when, for compliance and dispute resolution. --- ## 2. Tech stack | Layer | Technology | | --- | --- | | **Backend (engine)** | Node.js ≥ 20 (dev on 24), TypeScript 5.9 (ESM / NodeNext), run via `tsx` (dev) or `tsc`→`node` (prod) | | **Authoring app** (`web/creator/`) | **Angular 20 + PrimeNG 20**, themed with the Figma `@primeuix/themes` preset (`web/src/theme/figma-preset`, dark default); signals, hash routing, lazy chunks. Built under portal base `/creator/`. (Migrated from the React/Vite app in `audiocreator/`, which remains as standalone/Lovable source.) | | **Dashboard** (`web/`) | **Angular 20** (standalone components, signals, hash routing, lazy chunks), `@angular/build:application` (esbuild); Leaflet, Mermaid, PrimeNG + the same Figma preset. Built under portal base `/monitor/` | | **Portal** | `scripts/dashboard-server.ts` — one no-framework HTTP server mounts both Angular front-ends + the data endpoints on one origin (`:8080`); one shared **Figma/PrimeNG** design system across both (see §5) | | **Transport / integration** | **MQTT 3.1.1** — TCP `1883`, WebSocket `9001`. The primary integration surface. | | **Broker** | `aedes` in-process (dev, `scripts/dev-broker.ts`); Mosquitto (prod) | | **Text-to-speech** | **Azure AI Speech** (active) · Acapela Cloud · ElevenLabs — pluggable behind one interface (engine); authoring previews via the portal's `/tts-preview` | | **Config source** | **CreatorStudio → Azure Database for PostgreSQL** (live, `pg_notify`/LISTEN) with an on-disk JSON fallback | | **Cache & storage** | In-memory LRU → filesystem → Azure Blob Storage (MP3 cache); append-only JSON-lines (audit) | | **Sign protocols** | **Vendor-independent** — one render pipeline, per-vendor encoders: generic 1-bpp bitmap (JSON) + **Mobitec FF** (driver board 1463-L, graphic font `w`) + **Hanover HCPS/SuperX** (graphic picture frames) | **Backend runtime deps** (`package.json`): `mqtt`, `pg`, `dotenv`, `leaflet`, `marked`, `mermaid`, `jszip`, `qrcode-generator` (the last four serve the portal: docs rendering, GTFS/zip import and the player QR code). **Dev/tooling**: `typescript`, `tsx`, `aedes`, `ws`, `@types/ws`, `@types/node`, `concurrently`, `cross-env`. Codebase: **75 backend modules** (`src/**/*.ts`), **74 engine test files (671 tests)**, plus the Angular dashboard **and** Creator under `web/` (**72 spec files, 877 tests**), and the retired/vendored React CreatorStudio app under `audiocreator/` (its own `package.json` + Vitest suite). Full breakdown and coverage: [`TEST-REPORT.md`](TEST-REPORT.md). --- ## 3. Capabilities (feature map) | Capability | What it does | Where | | --- | --- | --- | | **Trigger engine** | 33-type CreatorStudio trigger set; edge-detected from live PIS-PT signals (stop flow, doors, journey lifecycle, distance/time thresholds, geofence, speed, occupancy, exit-side) | `pis/ptEngine.ts` | | **Prerequisite gates** | Any trigger can require door-open/closed, stop-button, or a velocity window before it fires; authored per trigger under **Prerequisites** in the Creator | `pis/ptEngine.ts` | | **Playlist rendering** | static/dynamic text, pauses, pre-recorded audio clips → one MP3 + transcript | `engine/playlistRenderer.ts` | | **Multilingual** | Resolves `name_Multilanguage` stop/destination variants per element's language | `engine/variableResolver.ts` | | **Per-element voice / volume** | Elements may override voice; volume applied where the supplier declares prosody-volume support (its capability descriptor — Azure/Google/Polly), dropped with a log line elsewhere. Voice **pitch is fixed at natural** and is not authorable — a shifted pitch produces audibly broken speech that no log or payload reveals | `playlistRenderer.ts`, `tts/provider.ts` | | **Priority scheduling** | One announcement per vehicle at a time; higher priority interrupts / queues / drops | `engine/scheduler.ts` | | **Interior LED sign** | Renders the spoken line to a 16×144 amber dot-matrix bitmap (scrolls if wide) | `engine/ledRender.ts` | | **Exterior destination signs** | Front/side/rear headsigns (route + destination), shrink-to-fit + scroll | `engine/exteriorSign.ts` | | **Matrix LED templates** | Luminator MatrixRenderer contract: Display → Cycles → Layout → Text/Image/Rectangle; expressions over `globalState`; trigger face overrides; Ultima vehicle roster | `domain/ledTemplate.ts`, `engine/matrix/*`, Creator `/led-signs` | | **Pre-programmed destinations** | The codes a driver keys in: a stored list resolves `pis/0/destination.number` (optionally scoped by `lineCode`) into sign text + spoken destination, with a per-list priority against the live feed; optionally republished retained on `pis/0/list/destinations` | `engine/destinationList.ts`, `shared/ledDestinations.ts`, Creator `/destinations` | | **Certified coverage report** | A signed, self-contained proof-of-play report (summary, per-route breakdown, integrity hash, CSV/HTML) served at `/proof/report`, with GTFS-derived *expected* stop coverage to show what was **missed**, not just what played | `engine/proofReport.ts`, `gtfs/expectedCoverage.ts` | | **GTFS feed** | Imported schedule data behind the portal (`/gtfs/*`): routes, trips, stop names for the simulator, the lexicon import and expected-coverage matching | `gtfs/feed.ts`, `scripts/dashboard-server.ts` | | **FNT fonts** | Real Luminator `.FNT` / `.FON` fonts + font ladder for layout text | `shared/fnt.ts`, `shared/fnt-fonts.generated.ts` | | **Mobitec FF output** | Mono graphic-font `w` frames **and** RGB COLORTEXT / colour-bitmap frames (03090) on separate topics | `engine/ff/ffEncoder.ts`, `engine/ff/ffRgbEncoder.ts` | | **Hanover output** | The same bitmaps as HCPS/SuperX `{\pic}` graphic frames on parallel `…/hanover` topics — sign-vendor independence from one render pass | `engine/hanover/hanoverEncoder.ts` | | **TTS resilience** | Retry + backoff + circuit breaker + fallback audio around the provider; per-supplier latency/error/volume counters; open breaker → render through the configured failover supplier | `tts/resilient.ts`, `tts/index.ts` | | **Fleet-wide cache** | Identical phrase synthesized once; memory → FS → Azure Blob | `tts/cache.ts` | | **Predictive pre-render** | Warms the cache for upcoming stops so live announcements are instant | `engine/prerender.ts` | | **AI authoring assistant** | Describe the announcements in plain language → a reviewable plan of playlists + triggers (§7.3b). Runs against a **local** model through the portal (no cloud, no per-call cost) — or any OpenAI-compatible endpoint — with an offline keyword draft as the fallback. The model proposes; a client-side validator rebuilds every field against the real catalogue, and nothing lands until the author applies it — as one undo | `web/creator/app/lib/assistant.ts`, `scripts/lib/assistant-ai.ts`, `src/shared/assistantPrompt.ts`, Creator `/assistant` | | **Live config sync** | Publish in CreatorStudio → engine hot-swaps via Postgres `pg_notify`/LISTEN (~1 s); vehicles poll the portal API on a retained MQTT nudge | `config/pgConfigSource.ts`, `config/httpConfigSource.ts` | | **Proof of Play** | Durable audit of audio + interior + exterior events with GPS + time; table/map/replay/analytics | `engine/history.ts`, `web/…/proof` | | **Vehicle players** (§5.2) | The client side — what actually makes a sound on the bus. Headless **Go** and **Python** clients (x86-64 + ARM64, Docker) and a zero-install **browser** player; each plays one vehicle and acks `engine/played` when the clip finishes | `clients/go`, `clients/python`, `public/player.html` | --- ## 4. Backend architecture ### 4.1 System context ```mermaid flowchart LR subgraph fleet["Fleet"] BRIDGE["GTFS-RT → PIS-PT bridge
(per tenant)"] end subgraph veh["On the vehicle — the client side (§5.2)"] PLAYER["Vehicle player
Go · Python · browser
→ loudspeaker"] SIGNS["Sign controller
RS-485 · Mobitec FF / Hanover HCPS"] end AUTH["CreatorStudio app
authors playlists + LED signs + triggers
in-repo · portal /creator/"] SB[("Azure PostgreSQL
config")] BLOB[("Azure Blob
MP3 cache")] BROKER[["MQTT Broker"]] ENGINE["CreatorStudio Engine
(Node/TS)"] DASH["Dashboard (Angular)
portal /monitor/"] TTS["Azure AI Speech (REST)"] BRIDGE -- "{tenant}/{vehicleId}/pis/0/#" --> BROKER BROKER -- "PIS-PT" --> ENGINE AUTH -- "Publish to engine (portal /api/publish)" --> SB SB -- "pg_notify: engine_config_changed" --> ENGINE ENGINE -- "synth (HTTPS)" --> TTS ENGINE <-- "MP3 cache" --> BLOB ENGINE -- "tts · display · display/ff" --> BROKER BROKER -- "…/pis/0/tts (ADT audio)" --> PLAYER BROKER -- "…/display/ff (sign frames)" --> SIGNS PLAYER -- "engine/played" --> BROKER BROKER -- "played-ack" --> ENGINE ENGINE -- "metrics · health" --> BROKER DASH <-- "ws:// MQTT + HTTP" --> BROKER ``` The engine and dashboard **never call each other directly** — MQTT topics + a small HTTP surface are the entire contract (see §7). The vehicle is no different: a player is just another MQTT client, so anything that can subscribe to a topic and decode an MP3 can be the speaker. Note the loop is **closed**. The player publishes `engine/played` once a clip has *finished*, so the proof-of-play trail records what a passenger actually heard — not merely what the engine dispatched. That distinction is what an accessibility (ADA) audit asks for. ### 4.2 Module dependency map ```mermaid flowchart TB index["index.ts (composition root)"] config["config.ts · ConfigStore"] cfgsrc["config/pgConfigSource.ts
· httpConfigSource.ts"] bus["mqtt/client.ts · MqttBus"] ctx["pis/ptContext.ts"] eng["pis/ptEngine.ts"] cust["engine/customTriggers.ts
(fact snapshot + expression)"] pipe["pipeline.ts"] sched["engine/scheduler.ts"] rend["engine/playlistRenderer.ts"] vars["engine/variableResolver.ts"] vol["engine/volume.ts · lexicon.ts"] adt["engine/adtPayload.ts"] hist["engine/history.ts"] sink["engine/proofSink.ts
(durable mirror)"] pre["engine/prerender.ts"] led["engine/ledRender.ts · exteriorSign.ts"] matrix["engine/matrix/*
globalState · publish · layoutRenderer"] dest["engine/destinationList.ts"] ff["engine/ff/ffEncoder.ts · ffRgbEncoder.ts"] ttsf["tts/index.ts (factory)"] res["tts/resilient.ts"] prov["tts/provider.ts"] cache["tts/cache.ts"] fleet["engine/fleet.ts"] shared["shared/payloads.ts"] sled["shared/led*.ts
expression · cycle · layout · destinations
shared with the browser"] index --> config & cfgsrc & bus & ctx & eng & cust & pipe & sched & pre & hist & sink & led & matrix & dest & ff & ttsf & cache & fleet config --> cfgsrc pipe --> rend & adt & led & ff & vol rend --> vars & prov & cache matrix --> sled & ff dest --> sled ttsf --> res --> prov led & ff & adt --> shared ``` ### 4.3 Module responsibilities | Module | Responsibility | Key exports | | --- | --- | --- | | `index.ts` | Composition root: wires MQTT → context → engine → scheduler → pipeline. Runtime hardening: ingest gating, cached enabled-trigger conditions, render backpressure (`acquire`/`release` counting semaphore, `MAX_CONCURRENT_RENDERS`), context eviction, render-scope gate, exterior-sign publishing, fleet directory, `engine/health` + `engine/metrics`. | `main()` | | `config.ts` | `loadEnv()`→`EnvConfig`; `ConfigStore` — remote-first (Postgres/API) with file fallback, hot-reload, `version()` bump on reload. | `loadEnv`, `ConfigStore` | | `config/pgConfigSource.ts` | Reads the published `engine_config` row over `pg` and **LISTEN**s for `engine_config_changed` on a dedicated connection — self-healing (capped 30 s backoff, refetch-on-reconnect). Enabled by `DATABASE_URL`. | `PgConfigSource` | | `config/httpConfigSource.ts` | The vehicle variant (no DB reachability): `GET {CONFIG_API_URL}/api/engine-config`, refreshed when the portal's retained MQTT config notify (`creatorstudio/{tenant}/{fleet}/config/updated`) arrives. | `HttpConfigSource` | | `db/pool.ts` | The shared `pg` connection pool (lazy, `sslmode` honoured) used by the config source, proof sink and portal API. | `getPool` | | `mqtt/client.ts` | Broker connection; subscribe inbound + control; publish JSON or **raw Buffers** (FF frames); topic templating; never rejects on disconnect (daemon). | `MqttBus` | | `pis/ptContext.ts` | Parses topics, aggregates retained PIS-PT per vehicle, derives normalized `JourneyState` + extras (location, journeyRef, ML name maps). | `VehicleContext`, `deriveState`, `parseTopic` | | `pis/ptEngine.ts` | Edge-detects trigger events on signal change; evaluates threshold conditions **and universal prerequisite gates**. | `PtTriggerEngine` | | `domain/announcement.ts` | Authoring model (mirror of the UI): playlists, elements, triggers, conditions, voices, volume, lexicon, priority. | `AnnouncementConfig`, … | | `pipeline.ts` | Per-event orchestration: resolve trigger→playlist(s) → render → ADT → publish; **priority resolution**; per-element runs; metrics + **proof records**. | `renderAndPublish`, `resolveTriggerPriority` | | `engine/scheduler.ts` | Per-vehicle **priority scheduler**: play / interrupt / queue / drop, priority-ordered queue with overflow. An interrupt **aborts the superseded render** (`AbortSignal`) so it never double-plays; `forget()` releases a vehicle's slot on eviction. | `AnnouncementScheduler` | | `engine/fleet.ts` | Builds the compact per-vehicle **fleet directory** (route · destination · current stop · stops-left · phase, in-service first) from in-memory contexts; published retained on `engine/fleet` so the dashboard's picker needn't subscribe to the whole fleet. | `buildFleet` | | `engine/playlistRenderer.ts` | Playlist → MP3 + transcript + timing; splits runs by voice/volume; **caches + retries pre-recorded clips**. | `renderPlaylist` | | `engine/variableResolver.ts` | 20+ dynamic variables; language-aware name variants. | `resolveVariables` | | `engine/adtPayload.ts` | Builds the **ADT 4.x** audio message ([AsyncAPI spec](https://adt.transhub.io/4.x/asyncapi/)): MP3/OPUS, integer speakers 0–100. | `buildAdtPayload` | | `engine/history.ts` | Durable **proof-of-play** audit — append-only JSON-lines, bounded/auto-trimmed, **async serialized writes** (never blocks the event loop) with `flush()` on shutdown. | `HistoryLog`, `historyPath` | | `engine/prerender.ts` | Predictive pre-rendering — warms the cache for the next N stops, deduped, throttled. | `Prerenderer` | | `engine/led-font.ts` · `ledRender.ts` | Embedded 8-wide × 8-tall bitmap font (letters + composable diacritics) doubled to 16 rows; text → 1-bpp bitmap (center/scroll). | `renderLed`, `fontToJSON` | | `engine/ledEncoder.ts` · `exteriorSign.ts` | Interior payload encoder; exterior front/side/rear composition (route + shrink-to-fit destination). | `encodeDisplay`, `buildExteriorPayload` | | `engine/customTriggers.ts` | The custom (fact-based) trigger runtime: snapshots the live facts from a `VehicleContext`, evaluates the authored boolean expression + geofences, and keeps per-vehicle edge/cooldown state (`becomes-true` vs `while-true`). | `CustomTriggerEngine`, `snapshotFacts` | | `engine/matrix/*` | The MatrixRenderer pipeline: `globalState.ts` (facts + geofences + destination fields the template expressions read), `publish.ts` (render every roster sign from its bound Display → JSON payload + FF/Hanover frames per the sign's protocol, signature-deduped), `layoutRenderer.ts` (elements → pixels, mono + colour), `png.ts` (image decode). | `buildGlobalState`, `buildLedSigns`, `renderLayout` | | `engine/destinationList.ts` | Runtime half of the **pre-programmed destination lists** — turns the driver's code (`pis/0/destination.number`, or its override twin) into destination text for speech and signs, honouring the list's priority against the live feed. | `applyDestination`, `selectedDestination` | | `engine/volume.ts` · `lexicon.ts` | Volume adaptation rules — combined conditions over time-of-day, weekdays, route, stop and geofence (all must hold; most specific match wins; legacy single-condition rules still honoured) — and the pronunciation lexicon (whole-word respelling, IPA ``, language scoping). | `resolveVolume`, `applyLexicon` | | `engine/proofSink.ts` | Optional **durable mirror** of every proof record to Postgres (`proof_of_play`, batched multi-row INSERT via the shared pool; opt-in `PROOF_DB_ENABLED=true`), failure-isolated — the local JSON-lines log stays the primary. | `createProofSink` | | `engine/proofReport.ts` | The certified coverage report served at `/proof/report`: summary + per-route breakdown, GTFS *expected* vs confirmed stops, integrity hash, CSV and self-contained HTML. | `buildProofReport`, `reportCsv`, `reportHtml` | | `gtfs/feed.ts` · `expectedCoverage.ts` | Lazy cached reader over a GTFS `.zip` (routes / trips / stops for the simulator and lexicon import) and the trip-matching that turns a journey into *expected* stop coverage. | `GtfsFeed`, `expectedCoverage` | | `shared/led*.ts` | The LED core shared **verbatim with the browser** (`@shared/*`): expression evaluation, cycle resolution, layout rendering, text rules/abbreviation, via + main-stops facts, destinations, symbols, route colours — so Creator preview, dashboard and engine rasterise identically. | `resolveActiveLayoutId`, `renderLayout`, `GlobalState` | | `engine/ff/ffEncoder.ts` | Mobitec 1463-L FF frames via **graphic font `w`** (5-dot columns); checksum + addressing; hardware scroll (`0xA5`/`0xD5`); `decodeFf` for tests. | `encodeFf`, `decodeFf` | | `engine/ff/ffRgbEncoder.ts` | The colour half of the same protocol (03090 §7.6): `COLORTEXT`, 16-entry RGB444 colour table, colour bitmap — RGB panels get real frames, not a mono approximation. | `encodeFfRgb` | | `engine/hanover/hanoverEncoder.ts` | **Hanover** LED destination signs — same bitmaps, different vendor: HCPS frame (`STX·cmd·addr·body·ETX` + two ASCII-hex checksum chars) carrying a **SuperX** `{\pic}` vertical-raster graphic; Hilde text + infohub MQTT-bridge JSON too; `decodeHanover` for tests. Checksum + packing verified byte-exact against the vendor docs in `Hanover/`. | `encodeHanoverGraphic`, `encodeHanoverText`, `decodeHanover` | | `tts/provider.ts` · `index.ts` · `resilient.ts` | Provider contract; factory (`TTS_PROVIDER`); resilience wrapper (retry/breaker/fallback, same cache keys). | `TtsProvider`, `createTtsProvider` | | `tts/azure.ts` · `acapela.ts` · `elevenlabs.ts` | REST clients; Azure builds SSML (rate/pitch/volume prosody, ``). | `AzureSpeechClient`, … | | `tts/cache.ts` | Layered MP3 cache — bounded LRU → FS → Azure Blob Storage (optional, container SAS URL); **single-flight `getOrSynth`** (concurrent callers for the same key join one synth) and **FS eviction** (`CACHE_MAX_FILES`, default 20000 — oldest MP3s pruned by mtime); `stats()`. | `TtsCache` | | `shared/payloads.ts` | **Single source of truth** for wire contracts, imported by engine **and** Angular via `@shared/*`. | `ADTAudioPayload`, `DisplayPayload`, `ExteriorPayload`, `MetricsPayload`, `HealthPayload` | ### 4.4 External services | Service | Endpoint / protocol | Used by | Auth | | --- | --- | --- | --- | | **MQTT broker** | MQTT 3.1.1 over `mqtt://` / `ws://` | `MqttBus` | user/pass (optional) | | **Azure AI Speech** | `POST …/cognitiveservices/v1` (SSML) | `AzureSpeechClient` | `Ocp-Apim-Subscription-Key` | | **Azure Database for PostgreSQL (config)** | `pg` — SQL reads + `LISTEN engine_config_changed` (`engine_config`) | `PgConfigSource` (back office; vehicles use the portal API via `HttpConfigSource`) | `DATABASE_URL` (`sslmode=require`) | | **Azure Blob Storage (cache)** | raw HTTPS against a container SAS URL, objects `tts-cache/.mp3` | `TtsCache` | `CACHE_BLOB_SAS_URL` (SAS) | | **Acapela / ElevenLabs** | REST | respective clients | account key/token | > **Corporate TLS:** all Node processes run with `--use-system-ca` so HTTPS trusts the OS > certificate store (required behind TLS-inspecting proxies). --- ## 5. Client architecture (`web/` · `clients/`) Everything downstream of the broker: the operator-facing apps (§5.1) and the vehicle players that actually make a sound (§5.2). Both are ordinary MQTT clients — neither has a private API into the engine. ### 5.1 Dashboard & Creator (`web/`) Angular 20 standalone-component app: **signals** for state, **hash routing**, **lazy-loaded** routes (every panel is its own chunk; the cockpit and its Leaflet map are the heavy ones), esbuild bundle. It imports the engine's payload contracts directly from `src/shared/payloads.ts` via the `@shared/*` alias — producer and consumer share one type source — and the LED core from `src/shared/led*.ts`. Six menu entries, each a route: **Monitor** · **Performance & Health** · **Proof of Play** · **History** · **Play on a device** · **Config**. Documentation is not one of them — it lives in the Documentation Center, reached from the suite rail. | Area | Module | Purpose | | --- | --- | --- | | **MQTT store** | `mqtt.service.ts` (root) | WebSocket connection; routes every engine topic into signals; selection + auto-play | | **UI state** | `ui-state.service.ts` (root) | Panel state persisted across navigation **and** sessions (localStorage) | | **Monitor (cockpit)** | `panels/monitor-cockpit.component.ts` | The merged operator view: heading-up map hero, fleet/vehicle feed rail, and a collapsible drawer holding the source/inject controls, the raw PIS-PT table and the live **Displays** (interior + exterior LED, matrix signs, TFT). Monitor / Live / Map / LED Signs were separate pages once; they are one screen now, and those routes redirect here | | **Proof of Play** | `panels/proof.component.ts` | Groups (journey/route/destination/date/vehicle) · Table · Map · Replay; CSV/JSON export; mini LED previews | | **History** | `panels/history.component.ts` | The announcement audit trail (every announcement across restarts) with filters and CSV export | | **Diagnostics** | `panels/diagnostics.component.ts` | Performance and Health under one menu, switched by a segmented control (`performance.component` + `health.component`) | | **Map** | `panels/map.component.ts` (lazy) | Live event map (dark theme): every trigger/announcement plotted where + when, numbered markers coloured by trigger, rich popups compressing the monitor's outputs (audio + interior/exterior sign text, surfaces, timing), a synced time-ordered timeline, current-position marker, route shape/stops, and a legend | | **Play on a device / Config** | `panels/device-player.component.ts`, `panels/config.component.ts` | QR hand-off of a vehicle to a phone; the live triggers→playlists view of what the engine is executing | | **Sign rendering** | `led-render.ts`, `led-layout-render.ts`, `led-mini/interior-sign/exterior-sign`, `led-exterior-live.component.ts` | Canvas dot-matrix and full matrix layouts using the **same font and the same shared LED core** as the engine (`/led-font.json`, `@shared/led*`) | **Unified portal server** (`scripts/dashboard-server.ts`) — a tiny no-framework server that serves **both** front-ends and the runtime data endpoints from one origin (no CORS/CDN), so the two apps appear as one product behind a landing-page menu: | Mount | Served | Build-time base | | --- | --- | --- | | `/` | Redirects to the Engine monitor (`/monitor/#/monitor`) — the default screen | — | | `/portal` | Landing page (`public/portal.html`) — the menu | — | | `/monitor/` | Angular engine dashboard (`web/dist/web`) | `baseHref /monitor/` (prod config) | | `/creator/` | Angular CreatorStudio authoring app (`web/dist/creator`) | `baseHref /creator/` (`creator` project) | | `/docs` | **Documentation Center** (`public/docs.html`) — the one entry point for every document | — | | `/documentation.html` · `/documentation.md` · `/docs/src/*` · `/docs/media/*` | The consolidated handbook (assembled **live** from the repo's Markdown on each request), its raw sources and screenshots | — | | `/stakeholder`, `/presentation`, `/architecture-overview`, `/leaflet`, `/one-pager`, `/software-team` | The decks and leave-behinds under `docs/showcase/` | — | | `/player` (+ `/player/v//`) | The zero-install browser vehicle player (`public/player.html`) + its bridge (`/player/stream`, `/player/ack`, `/player/qr.svg`, `/player/hosts`, `/player/live`) | — | | `/config`, `/history`, `/proof/report`, `/led-font.json`, `/architecture.md`, `/tts-*`, `/gtfs/*` | Shared runtime/data endpoints (§7.2) | root | Both apps are **Angular projects** in the `web/` workspace, mounted purely at **build time** via each project's `baseHref` — so the server just maps a URL prefix to a `dist` dir with SPA fallback, no per-framework logic. Their API fetches are root-absolute and the dashboard's broker WebSocket targets `…:9001`, so both keep working under their prefixes; the Creator talks only to the portal's same-origin `/api` (projects, config publish, GTFS expand). Old `/#/…`, `/ng/…`, and the migration's temporary `/creator-ng/…` links redirect to their new homes. In dev, `ng serve` (`npm run stack:dev`) runs on `:4200` and proxies data endpoints to `:8080`. The retired **React** CreatorStudio (`audiocreator/`) is still runnable standalone (`npm run cs:dev` on `:8090`, Lovable deploy) but is no longer served by the portal. **Suite navigation** — both apps render a persistent 64px left **rail** (app switcher: brand → portal, **Creator** `/creator/`, **Engine** `/monitor/`, **Player** `/player`, and the **Documentation Center** `/docs` set apart at the foot as reference rather than an app; the current app is highlighted). The Angular Creator reuses the dashboard's `web/src/app/suite-rail.component.ts` directly. Both share the **Figma/PrimeNG** design system (§2), so the suite reads as one product. Documentation deliberately lives *outside* either app's own menu: one page publishes the handbook, the decks and the sources, and both apps link to it. ### 5.2 Vehicle players — the client side (`clients/`, `/player`) The engine renders audio and publishes it; **something on the vehicle has to play it.** Three interchangeable clients do that, all speaking the same contract, so a fleet can mix them. | Client | Source | Runs on | Use it for | | --- | --- | --- | --- | | **Go** | `clients/go` | x86-64 · ARM64, headless | The onboard unit. One static binary, no runtime to install. | | **Python** | `clients/python` | x86-64 · ARM64, headless | A Pi, or a box that already has Python; easiest to adapt in the field. | | **Browser** | `public/player.html` → `/player` | Any phone, tablet, laptop | Demos, spot-checks, a driver's phone. No install at all. | **The contract — identical in all three:** ``` subscribe {tenant}/{vehicleId}/pis/0/tts ADT 4.x — audio[0].content is base64 MP3 (or WAV) publish engine/played {traceId, tenant, vehicleId, playedAt, status} ``` Four behaviours matter and are deliberate: - **The ack is sent when the clip *finishes*, not when it arrives.** That is what turns a dispatch log into proof of play (§1) — the difference an accessibility audit cares about. - **Clips are queued and played strictly one at a time.** An announcement must never talk over the next one. (The dashboard's own auto-play is a *monitor*, not a vehicle, and does not queue.) - **Every received clip is cached on the vehicle** under the payload's `clipKey` (a content hash the engine stamps on every message; repeats of an announcement share a key). A payload may then arrive **key-only** — `audio[0]` with no `content` — and the player replays the cached bytes: a few hundred bytes over the link instead of a few hundred kilobytes. Go/Python keep the cache on disk (`CLIP_CACHE_DIR`, LRU-bounded); the browser player uses IndexedDB. The **closed-corpus pre-push** (`PREPUSH_ENABLED=true`, §15) fills this cache ahead of need: cache-only clip deliveries arrive on `…/tts/cache` (`kind: "clip-cache"`) — stored, never played, never acked. - **`expiryDateTime` is enforced.** A clip past its expiry is dropped and acked **`expired`**, never played — a late "next stop Central" minutes after Central is worse than silence, and the proof trail must record the drop rather than a false play. **How audio reaches the loudspeaker** (headless clients): ``` player → ffmpeg (decode MP3/WAV, apply speakers.INTERNAL volume) → aplay → ALSA → /dev/snd/pcmC0D0p → sound card → amplifier → loudspeaker ``` `ffplay` is deliberately **not** used, although it looks like the obvious choice: it needs SDL and is absent from Alpine's `ffmpeg` package, so a client that reaches for it silently falls back to a backend that cannot decode MP3 at all. **Deployment.** Both headless clients ship as **multi-arch** Docker images (`linux/amd64` + `linux/arm64`; the Go image cross-compiles rather than emulating, and its dependencies are vendored so the build is hermetic). A container has no sound card, so it needs `--device /dev/snd` — and the entrypoint joins the *host's* audio group by numeric gid, because it is **18 on Alpine but 29 on Debian/Raspberry Pi OS**, and the kernel checks the number, not the name. Details: `clients/README.md`. **Not handled.** The exterior loudspeaker. The payload carries `speakers.EXTERNAL`, but driving it on a real vehicle also means raising a digital output to enable that amplifier while the clip plays — unit-specific wiring (§7.1, on-vehicle integration), not implemented in these clients. --- ## 6. Software design ### 6.1 Core patterns - **MQTT topics as the API.** The engine, dashboard, bridge, and vehicle units are fully decoupled; anything that speaks the topic contract can join. Enables simulation, multi-consumer, and language-agnostic integration. - **Multi-topic aggregation → edge-detected events.** PIS-PT spreads trip state across many retained topics; `VehicleContext` merges them into one `JourneyState`, and `PtTriggerEngine` fires events only on meaningful *transitions* (debounced), not on every message. - **Pluggable TTS + resilience decorator.** `TtsProvider` is one interface; `ResilientTtsProvider` wraps any provider with retry/backoff/circuit-breaker/fallback while preserving cache keys — and counts each supplier's synths/latency/characters for the per-provider health surface. While the primary's breaker is open, the switchable provider renders through a **configured failover supplier** (`config.ttsFallbackProvider` / `TTS_FALLBACK_PROVIDER`) — a vendor outage speaks in another vendor's voice instead of the fallback beep. The failover decision happens at `pin()` time, before the cache key is computed, so key and audio always belong to the same supplier. - **Layered, content-addressed cache.** Cache key = hash of the exact synthesis inputs, so an identical phrase is generated **once for the whole fleet** and reused across memory → disk → Azure Blob. Pre-recorded clips share the same cache keyed by URL. - **Priority scheduler.** One announcement plays per vehicle at a time; the scheduler enforces interrupt / queue / drop from authored (or tier-default) priority — safety announcements never wait behind ambient ones. Interrupting a still-rendering announcement **aborts it** (`AbortSignal`) so its audio is never published on top of the higher-priority one. - **Single source of truth for wire types.** `shared/payloads.ts` is imported by both the engine and the Angular app; a contract change is a compile error on both sides. - **Deterministic sign rendering.** The browser rasterizes signs with the *same font* the engine ships (`/led-font.json`), so previews and proof-of-play replays match the hardware exactly. ### 6.2 Configuration flow CreatorStudio authors into a **`projects` row** (the source of truth for what the Creator lists); **Publish to engine** maps it (a no-op — the app already exports the engine's `AnnouncementConfig` shape) and upserts it to Postgres **`engine_config`** via the portal's `POST /api/publish` — one transaction with a monotonic `config_versions` row. A DB trigger fires `pg_notify('engine_config_changed')`; the engine's `PgConfigSource` **reads that row and LISTENs**, hot-swapping the fleet config in ~1 s (vehicles get the same effect from `HttpConfigSource` + the retained MQTT notify). The on-disk `CONFIG_PATH` is the offline fallback and is itself hot-reloaded. The direction matters operationally: `projects` is the **input**, `engine_config` the **output**, and the file the **fallback**. Reads are remote-first, so a config written to the file is invisible while the live source is enabled, and a config written to `engine_config` does not change what the Creator lists — it is overwritten by the next publish. **To change what an author sees, write the project.** An example of using that direction deliberately: the worked-example playlist library (`config/announcement-config.sample.json`) can be merged into any of the three targets *without its triggers* — `scripts/lib/example-playlists.mjs` appends the playlists and asserts `triggers` / `customTriggers` are byte-identical before writing. Since a `Playlist` carries no trigger field and the engine only ever walks trigger → playlist (§6.3), an unreferenced playlist is inert: authors get a library to copy from and the fleet says exactly what it said before. ### 6.3 Trigger → announcement decisioning `ptEngine` fires an event only when (a) a state transition occurred, (b) the debounce elapsed, (c) the trigger's **threshold condition** matches (distance/time/geofence/speed/occupancy), and (d) its **universal prerequisite gates** hold (door/stop-button/velocity). `pipeline` then resolves interior/exterior/legacy playlists + sequences + repetitions, renders per-element (splitting TTS runs by voice/volume), applies volume adaptation, and routes to speakers via the ADT payload. ### 6.4 LED / sign rendering (for designers) - **Interior:** 16 rows × 144 columns amber dot-matrix. Text centered when it fits, marked `scroll` when wider; the sign/dashboard pans a 144-wide window. - **Exterior:** front (192) / side (160) / rear (48) px wide, 24 tall. A fixed route number (≤ 25 % of width, never scrolls) + destination that **shrinks to fit**, scrolling only as a last resort. - **FF/Mobitec:** the same bitmap re-encoded as graphic-font-`w` column bytes (each byte = a 5-pixel vertical slice) inside an addressed 1463-L frame for RS-485 hardware. - **Hanover:** the same bitmap again as a SuperX `{\pic}` vertical raster inside an addressed HCPS frame — sign-vendor independence: Luminator and Hanover panels show identical pixels from one render pass. --- ## 7. Interfaces & APIs ### 7.1 MQTT topic contract (primary integration surface) | Topic | Dir | Payload | Producer → Consumer | | --- | --- | --- | --- | | `{tenant}/{vehicleId}/pis/0/` | in | PIS-PT JSON (see below) | bridge/simulator → engine, dashboard | | `{tenant}/{vehicleId}/pis/0/tts` | out | **ADT 4.x** audio (base64 MP3 + integer speakers + transcript) | engine → **vehicle players** (§5.2), dashboard | | `engine/played` | in | `{traceId, tenant, vehicleId, dispatchedAt, playedAt, status}` — sent when a clip **finishes** | **vehicle player → engine** (proof of play) | | `…/pis/0/display` · `…/display/exterior` | out | 1-bpp bitmap JSON (interior · front/side/rear) | engine → sign controller, dashboard | | `…/pis/0/display/ff` · `…/display/exterior/ff` | out | **raw Mobitec FF frames** (font `w`; addr 10 · 1/2/3). Each frame also mirrors on its address subtopic — `…/ff/{address}` — so a controller subscribes to its own bus address | engine → RS-485 gateway / sign controller | | `…/pis/0/display/hanover` · `…/display/exterior/hanover` | out | **raw Hanover HCPS/SuperX frames** — the same bitmaps for Hanover signs, in parallel with FF (`DISPLAY_HANOVER_ENABLED`, on by default). Same per-address fan-out: `…/hanover/{address}` | engine → RS-485 gateway / sign controller | | `…/pis/0/display/signs` · `…/display/signs/ff` · `…/display/signs/hanover` | out | **matrix template signs** — one frame per rostered vehicle face (`LedSignsPayload` JSON; FF mono/RGB or Hanover HCPS per the sign's `protocol`), signature-deduped. Gated by `DISPLAY_TEMPLATE_MODE`. The raw-frame topics also fan out per bus address — `…/signs/ff/{address}` · `…/signs/hanover/{address}` carry only that address's frames, so a controller subscribes to its own address instead of filtering the aggregate | engine → sign controller, dashboard | | `…/pis/0/list/destinations` | out | The active **pre-programmed destination list** (retained, `{number, name, lineNumber}` rows) so a driver console can offer the codes. Off unless `DESTINATION_LIST_PUBLISH=true` — a real PIS system may own this topic | engine → driver console | | `engine/metrics` | out | per-announcement timing + `lat`/`lon` | engine → dashboard | | `engine/health` | out | throughput/health (vehicles, served, renderScope, msg/s, events/s, loop lag, RSS, cache hit-rate, in-flight, queued, dropped, interrupted, synthCoalesced) | engine → dashboard / ops | | `engine/fleet` | out | fleet directory (retained): per-vehicle route · destination · current stop · stops-left · phase | engine → dashboard (vehicle picker) | | `engine/bridge` | out | bridge health (retained): up/local, mode, tenant, bridged vehicles, throughput, dedup rate | bridge → dashboard | | `engine/control/select` | ctl | `{"vehicleKeys":[…]}` (retained) — the served set (render scope) | dashboard → engine | | `engine/control/tenant` | ctl | `{"tenant":"…"}` (retained) — narrows the bridge's discovery to one tenant | dashboard → bridge | **Inbound PIS-PT sub-topics consumed:** `journey`, `destination`, `destination/override`, `list/stops`, `linkprogress`, `stopinfo`, `journeystate`, `sensors/door`, `sensors/stop_button`, `vehicle/gnss_location`, `vehicle/exit_sides`, `connections`, `passenger_load`, `alarm_activation`, `shape`. ### 7.2 HTTP API (dashboard-server) | Route | Serves | | --- | --- | | `GET /` + assets | portal landing, then the two SPAs under `/monitor/` and `/creator/`; unknown paths inside a mount → its `index.html` (hash routing) | | `GET /led-font.json` | shared LED font (browser rasterizes identically to the engine) | | `GET /architecture.md` | this document (raw) | | `GET /documentation.md` · `/documentation.html` | the consolidated handbook, **assembled live** from the source Markdown on every request (`scripts/lib/docs.ts`); `/docs` is its browsable home and `/docs/src/*` serves the whitelisted sources | | `GET/POST /config` | live config (remote-first, file fallback); `POST` imports a CreatorStudio export | | `GET /history` | proof-of-play. Filters: `tenant`, `vehicle`, `type` (audio\|exterior), `route`, `destination`, `journey`, `trig`, `q`, `since`, `until`, `limit`. `format=csv` for CSV. `groupBy=journey\|route\|destination\|date\|hour\|vehicle` for server-side analytics aggregation. | | `GET /proof/report` | the **certified coverage report** over that trail — scope by `tenant`/`route`/`vehicle`/`from`/`to`, `expected=1` to cross-reference the GTFS schedule (what was *never attempted*), `format=json\|csv\|html` (the HTML is self-contained and carries an integrity hash) | | `GET /tts-providers` · `/tts-voices?provider=` · `POST /tts-preview` | supplier catalogue (which are configured, their default voice), a supplier's voice list, and a one-line audition through the engine's real client | | `POST /gtfs/import` · `GET /gtfs/{status,values,routes,trips,trip}` | the GTFS feed behind the simulator, lexicon import and expected coverage (upload capped; a missing feed answers `{feed:null}` rather than failing) | | `GET/PUT/DELETE /api/projects` · `GET /api/engine-config` · `GET /api/config-versions` · `POST /api/publish` · `POST /api/gtfs/expand` | the **Creator API** (`scripts/lib/creator-api.ts`): projects CRUD, the published config row, version history, the transactional publish (upsert + `config_versions` + MQTT notify) and AI abbreviation expansion | | `GET /player` (+ `/player/v//`, `/player/stream`, `/player/hosts`, `/player/qr.svg`, `/player/live`, `POST /player/ack`) | the zero-install browser vehicle player and its server bridge (SSE audio stream, LAN hosts, QR hand-off, played-ack relay, and the recently-announcing vehicles the setup screen prefills from). QR hand-offs use the `/player/v/…` path form — query strings have been seen stripped between scan and phone | **Auth.** Every **mutating or cost-bearing** route (`POST /config`, `/tts-preview`, `/gtfs/import`, `/player/ack`, and the Creator API writes `PUT/DELETE /api/projects`, `POST /api/publish`, `POST /api/gtfs/expand`) is gated by `authorized()`: with `DASHBOARD_TOKEN` set it requires `Authorization: Bearer …` compared in constant time; unset (the lab default) leaves the portal open. Read routes are unauthenticated by design — the dashboard is an ops tool on a trusted network. ### 7.3 Config source (`engine_config` in Postgres) One row per **(tenant, fleet)**: `tenant`, `fleet`, `config` (jsonb — the `AnnouncementConfig`), `schema_version`, `published_project_id` (which project produced it; FK → `projects`, engine ignores it), `updated_at`. Keyed by a composite `(tenant, fleet)` primary key. Writes go through the portal's `POST /api/publish` (`scripts/lib/creator-api.ts` — one transaction: the `engine_config` upsert plus a monotonic `config_versions` row); a DB trigger fires `pg_notify('engine_config_changed')` on every insert/update. The engine reads with `pg` and LISTENs on that channel (`PgConfigSource`) — and when `CONFIG_DB_TENANT_VALUE` is set, filters to its own `(tenant, fleet)` row (empty = single-tenant). Vehicles, which cannot reach the database, use `HttpConfigSource` instead: `GET {CONFIG_API_URL}/api/engine-config`, re-fetched when the portal's retained MQTT notify (`creatorstudio/{tenant}/{fleet}/config/updated`) arrives. The whole schema lives in `db/migrations/` (baseline `0001_baseline.sql`, applied by `npm run db:migrate`), so the DB is reproducible from empty on any stock Postgres with no external dependency. ### 7.3a Projects & multitenancy (authoring side) Authoring is organised into **Projects** (`public.projects`: `id, tenant, fleet, name, description, config jsonb, timestamps`) — named self-contained configs, **one published per (tenant, fleet)**. Writes go through the portal's **`PUT` / `DELETE /api/projects`** (bearer-gated when `DASHBOARD_TOKEN` is set); reads are `GET /api/projects?tenant=&fleet=`. Every query is parameterized and **tenant-scoped in the portal API** — tenant is an explicit parameter today; deriving it from an authenticated session (Entra ID sign-in, verified `published_by`) is the documented follow-up (§12). The Creator's `TenantService` resolves the active tenant (`?tenant=` → `window.__AUDIO_SUITE_TENANT__` → last used → default) and scopes every project read/write, so tenants are fully separated. Engine payload/MQTT/trigger contracts are **unchanged** by any of this. ### 7.3b AI authoring assistant (local-first) The Creator's `/assistant` screen turns a plain-language description ("announce the next stop inside the bus, route + destination outside") into a **plan** of playlists and triggers. It asks two providers **in order**, so the feature degrades instead of disappearing: | Order | Provider | Where | Needs | | --- | --- | --- | --- | | 1 | **Portal** `POST /ai/assistant` | `scripts/lib/assistant-ai.ts`, same origin as the Creator | Any OpenAI-compatible chat-completions endpoint — a **local** Ollama / LM Studio / llama.cpp / vLLM, or a hosted one. `ASSISTANT_AI_URL`/`_MODEL`/`_KEY`; unset = local Ollama | | 2 | **Offline draft** | the browser, `draftPlanLocally` | nothing — keyword rules over the same trigger catalogue, labelled *Offline draft* | So an on-prem install needs **no cloud backend and no AI gateway**: the portal already runs on the operator's machine and already holds the credentials, the Creator is served from the same origin (no CORS), and no announcement text leaves the site. The turn records which provider answered and the screen shows it, because a 7B local model and a hosted one write visibly different announcements. The server half takes the same request and returns the plan at the top level. The *client* sends the catalogue of what may be built — trigger types, engine `{variables}`, the custom-trigger fact list, the project's existing playlists and geofences — because the Angular domain model is the single source of truth for it. The prompt itself lives **once**, in `src/shared/assistantPrompt.ts` — the portal imports it directly, so there is no carried copy to drift. Replies are parsed leniently — local models routinely wrap JSON in ``` fences or chat around it. The model's answer is **never** trusted as config. `web/creator/app/lib/assistant.ts` rebuilds every field against the real model — unknown trigger type, a type the engine can never fire, a fact the PIS feed doesn't carry, an unresolvable `{variable}` (which would make the engine drop the whole announcement), an audio URL outside the preset list, an empty rule that would fire on every update — each is dropped and reported rather than applied. Nothing reaches the config until the author ticks it on the review card, and the whole plan lands as **one undo entry** (`ConfigService.applyAssistantPlan`). When the gateway is unreachable the same screen answers from a local keyword draft, marked *Offline draft*, through the identical validate → review → apply path. ### 7.4 Payload contracts Defined once in `src/shared/payloads.ts`: `ADTAudioPayload`, `ADTAudioItem`, `DisplayPayload`, `ExteriorPayload`/`ExteriorFace`, `MetricsPayload`, `HealthPayload`. The ADT audio message conforms to the transHub **ADT 4.x** AsyncAPI spec. --- ## 8. Runtime sequence — signal to loudspeaker ```mermaid sequenceDiagram participant B as Broker participant I as index.ts participant Ctx as VehicleContext participant E as PtTriggerEngine participant S as Scheduler participant P as pipeline participant R as renderPlaylist participant T as TTS participant C as Cache participant H as History participant V as Vehicle player (§5.2) B->>I: …/pis/0/linkprogress I->>Ctx: update(sub, payload) I->>E: ingest(ctx) (state-changing topics only) E-->>I: [approaching-stop] (condition + prerequisite gates pass) I->>S: offer(event, priority) S->>P: play (or interrupt/queue/drop) P->>R: renderPlaylist (per-element voice/volume) R->>C: get(cacheKey) alt cache miss R->>T: synthesize (SSML, HTTPS) T-->>R: MP3 R->>C: put(cacheKey) end R-->>P: audio + transcript + timing P->>B: publish …/tts (ADT) · …/display · …/display/ff P->>B: publish engine/metrics P->>H: append proof record (dispatched) B->>V: …/pis/0/tts (ADT audio) V->>V: decode → ALSA → loudspeaker (queued, never overlapping) V->>B: engine/played (AFTER the clip finishes) B->>H: played-ack → the record becomes proof of PLAY, not just dispatch ``` The last three steps are the client side (§5.2), and they are what make the record admissible: up to `publish …/tts` the engine only knows what it *sent*. --- ## 9. Deployment & processes ```mermaid flowchart LR subgraph host["Back office — host / container"] E["engine — node dist/src/index.js"] D["dashboard-server :8080"] end subgraph unit["Each vehicle — onboard unit (x86 or ARM)"] P["player (Go or Python)
docker · --device /dev/snd"] end BR[["MQTT broker :1883 tcp · :9001 ws"]] BROWSER["Browser — dashboard · /player"] SB[("Azure PostgreSQL
+ Azure Blob")] E <--> BR E <--> SB D <--> SB BROWSER <--> BR BROWSER <--> D P <--> BR ``` - **Engine** — single long-running process (per-vehicle state in memory), one replica per inbound stream. `Dockerfile` builds it; runs with `--use-system-ca`. Waits for the broker (never crashes on a missing broker). - **Dashboard server** — static file + data server (ops/monitoring); also serves the browser player. - **Vehicle player** (§5.2) — **one per vehicle**, on the onboard unit, not in the back office. Go (single static binary) or Python; both as multi-arch images for **x86-64 and ARM64** (`docker buildx build --platform linux/amd64,linux/arm64`). Needs `--device /dev/snd` to reach the speaker, and only outbound MQTT — no inbound ports, so it sits behind a vehicle's NAT unchanged. `restart: unless-stopped` and it reconnects on its own when the link drops. - **Broker** — Mosquitto (prod) / `aedes` (dev); WebSocket listener required for the browser (dashboard **and** the `/player` page). - **Dev stacks** (`concurrently`): `stack:ui` (broker+engine+dashboard), `stack:live` (adds the upstream bridge), `stack:dev` (broker+bridge+engine+dashboard+`ng serve` live-reload). Demos: `proof:demo`, `ff:demo`, `led:demo`, `multi:demo`, `bridge:test`, `swedish`, `simulate`, `config:test`, `tts:test`. - **Fully local backend** (stock Postgres): `db:start` / `db:stop` start/remove a plain `postgres:16` docker container; `db:migrate` applies `db/migrations/` (the whole schema builds from empty — no cloud dependency); **`stack:local`** runs broker+bridge+engine+dashboard together with **every process** pointed at the local Postgres — it sets `DATABASE_URL=postgres://postgres:postgres@127.0.0.1:5432/creatorstudio` (and runs the migrations first), so a hosted database in `.env` is neither read nor written while it runs. Use `stack:dev` to keep the engine on the `.env` target. **`stack:vm`** is the deployed variant: same four processes, everything read from `.env` (the Azure pipeline runs `db:migrate` before starting it). --- ## 10. Performance **Latency budget (per announcement)** — trigger fire → resolve → render (TTS) → publish. Measured on a live local run (Azure Speech, real journey): | Path | Latency | Notes | | --- | --- | --- | | **Cache hit** (phrase seen before) | **~2 ms** total, `ttsMs=0` | Dominant case fleet-wide | | **Cache miss** (first synth of a unique phrase) | **~1.0 s** (`ttsMs≈1015`) | One-time per unique phrase, then cached | | MP3 size | ~16–30 KB per announcement | base64 in the ADT payload | **Why it stays fast at scale** - **Content-addressed dedup** — a phrase like "Now arriving at Central Station" is synthesized once and reused by every bus on that route (observed: miss → `cache 0/1`, then `cache 1/1` hits). - **Predictive pre-render** warms the cache for upcoming stops, so most live announcements are hits. - **Ingest gating** — trigger detection runs only on state-changing topics; high-frequency GNSS and the large static `shape` don't trigger work unless a speed/geofence trigger is enabled. - **Render backpressure** — a global `acquire`/`release` counting semaphore bounds concurrent TTS renders (`MAX_CONCURRENT_RENDERS`); duplicate concurrent synths for the same phrase are collapsed by the cache's single-flight `getOrSynth` (surfaced as `synthCoalesced` in health). - **Bounded memory** — LRU MP3 cache (`TTS_MEM_CACHE_MAX`) + on-disk FS cap (`CACHE_MAX_FILES`, oldest pruned by mtime) + idle `VehicleContext` eviction (`CONTEXT_TTL_MS`) keep RSS flat under fleet load. - Health telemetry (`engine/health`) exposes msg/s, events/s, event-loop lag, RSS, cache hit-rate, in-flight, dropped, interrupted for live observation. --- ## 11. Scalability - **Multi-vehicle, opt-in by default.** A single engine can serve **many vehicles concurrently**: per-vehicle contexts + trigger snapshots + scheduler slots, a global render semaphore (`MAX_CONCURRENT_RENDERS`) bounding total TTS concurrency, and per-vehicle serialization (one announcement per vehicle). Two render scopes: - **`RENDER_SCOPE=selection` (default, production-safe)** — the engine renders only the vehicles the dashboard opts in via `engine/control/select` (the *served set*). It serves nothing until an operator picks vehicles; an empty set = silent. Multi-select in the dashboard builds this set. - **`RENDER_SCOPE=all` (load-test only)** — serves every **fully-tracked** forwarded vehicle at once, ignoring the selection (discovery-only vehicles are still skipped). Only the exact literal `all` enables it; the dashboard surfaces a visible **"ALL mode"** warning when the engine reports this scope in its health beat. Verified with `npm run multi:demo -- N` (load-test/`all`) and `npm run multi:demo -- N select` (served-set/`selection`) — 10 selected of 15 driven, decoys stay silent, render concurrency bounded, fair (no starvation), **zero cross-talk** even when every phrase is a unique synth. - **Per-vehicle state is in memory** → the natural horizontal unit is **one engine replica per inbound stream / shard**. - **Horizontal sharding** — split the fleet by MQTT topic filter (e.g. per tenant or vehicle-id hash), one replica per shard; or externalize per-vehicle state to Redis to scale statelessly. - **Selection control channel** — the dashboard can scope rendering to specific vehicles (`engine/control/select`) to bound TTS cost during operation/debugging. - **Cost scales with unique phrases, not vehicles** — the shared cache plus **single-flight** `getOrSynth` (concurrent identical synths collapse to one) mean fleet growth adds near-zero TTS cost for repeated announcements. - **Broker** is the throughput backbone; Mosquitto handles fleet-scale fan-in/out. Phase-0 hardening (bounded caches, eviction, ingest gating, backpressure, health metrics) lets one instance degrade gracefully; the full multi-vehicle / ~2000-bus plan is in [`SCALING.md`](SCALING.md). - **Lean upstream bridge.** The bridge (`scripts/bridge.ts`) mirrors the live PIS feed to the local broker and is kept deliberately quiet: it **follows the selection** (full `#` only for served vehicles), keeps fleet **discovery minimal** (route+destination+phase only — `list/stops` and other heavy/high-rate topics are *not* bridged fleet-wide), **tenant-scopes** discovery (once a tenant is selected via `engine/control/tenant` it narrows from `+/+/…` to `{tenant}/+/…`, so it stops pulling the whole broker), and **dedups** byte-identical re-publishes so retained-value churn doesn't ripple through trigger detection and the dashboard. It publishes `engine/bridge` health (forwarded/s, deduped/s, bridged vehicles, upstream subs) which the dashboard surfaces. Verified with `npm run bridge:test`. --- ## 12. Security & operational concerns - **Secrets** live in a gitignored `.env` (Azure key, upstream broker creds, `DATABASE_URL`, `CACHE_BLOB_SAS_URL`). **No key of any kind ships in a browser bundle** — the Creator talks same-origin to the portal, and every credential stays server-side. - **Database access**: the Postgres server is **private** (VM allow-list / VNet, `sslmode=require`); browsers never reach it. All reads and writes go through the portal API, where every query is **parameterized** and tenant-scoped (tenant is an explicit request parameter today — deriving it from an authenticated session is the follow-up below). - **Publish identity** — the Creator has no login; it forwards a host-shell-supplied token (`SessionService`) as the bearer when there is one, and `config_versions.published_by_verified` defaults **false** — the row is honestly marked unverified. The follow-up is **Entra ID sign-in** with a server-verified `published_by`; the seams (SessionService bearer, the `published_by_verified` column, the tenant columns) are already in place. - **Portal endpoints** — the mutating / cost-bearing HTTP routes (config import, TTS preview, GTFS upload, played-ack, and the Creator API writes: project save/delete, publish, GTFS expand) are behind a constant-time `Authorization: Bearer` check when `DASHBOARD_TOKEN` is set; unset means an open lab portal, which is the default and must not be the deployed one. - **TLS inspection** — `--use-system-ca` trusts the corporate root so HTTPS works behind proxies. - **Outstanding:** the Azure Speech key was shared in plaintext during development and **should be rotated** in the Azure portal. - **Proof-of-play** provides tamper-evident evidence (append-only log) for compliance/disputes. --- ## 13. Testing & quality - **581 engine tests across 67 files** (`node --test`, `tsx`) at **96.5 % line coverage**, covering: trigger detection & prerequisite gates, custom fact triggers & geofence geometry, priority resolution & scheduling, the full matrix-LED pipeline (FNT parsing, layout render, cycles, colour rules, mono + RGB FF frames, browser↔engine parity), pre-programmed destination lists, project CRUD/migration/validation, ADT payload conformance, multilingual variable resolution, per-element voice/volume, SSML, TTS resilience & the three supplier HTTP clients, pre-recorded audio caching/retry, config env permutations + Postgres LISTEN reconnect, GTFS parsing & expected coverage, proof-of-play correlation/report/sink — plus two **real-broker** tests (`aedes`): a MqttBus round-trip and a full `startEngine()` boot. - **758 front-end tests across 67 files** (Vitest + jsdom via `@angular/build:unit-test`): every dashboard and Creator component rendered under TestBed, every service, and all pure logic. - Exact per-suite counts, coverage and the full case catalog: [`TEST-REPORT.md`](TEST-REPORT.md). - **Trigger-parity guard** (`src/domain/triggerParity.ts`) — a strict, bidirectional contract between the engine's trigger set and the CreatorStudio manifest (same types both ways, matching `local` flags, every MQTT trigger wired to an emitted event). Enforced by the `manifestSync` contract test AND `npm run trigger:parity` (which also accepts `--file`/`--url` to check a live/ freshly-exported manifest). This is what catches `service-disruption`-style drift automatically. - **Typecheck** (`tsc --noEmit`) gates the shared contract across engine + Angular. - **End-to-end demos** double as living verification: `proof:demo`, `ff:demo`, `led:demo`, `swedish`. - **CI** (`azure-pipelines.yml`) runs typecheck + tests + the trigger-parity guard + the dashboard build on every PR/push to `main`, blocking merges on drift or a broken build. --- ## 14. Gaps & known limitations | Area | Status | | --- | --- | | **Config coverage** | Everything CreatorStudio authors for live MQTT triggers is honored (voices, per-element voice/volume/pitch, all conditions, prerequisite gates, priority, volume adaptation, lexicon, dynamic variables). | | **`bus-type` trigger** | Declared but **not fired** — PIS-PT carries no vehicle formation/type signal. Needs an upstream source. The Creator marks it **unavailable** (no condition editor, no fresh enable, an Issues error if a legacy config enables it), so nobody can author against it unknowingly. | | **`requiresDirection` gate** | Non-blocking — no forward/reverse signal in the feed. | | **`stationary-display` / `volume-calibration`** | Intentionally not engine-driven (local kiosk/technician). Their arrivals variables (`{arrivingBuses}` …) were removed from the Creator's picker — with no real-time arrivals source they resolved empty and dropped the announcement; a variable-parity test now keeps picker and resolver in lockstep. | | **`{serviceExpiry}` variable** | Empty — no expiry field in the line-status feed. | | **`dialect` / `parentPlaylistId`** | Not used — `voiceId` encodes locale; multilingual handled via in-playlist name variants. | | **FF hardware scroll** | Implemented for the **interior** sign (`0xA5`/`0xD5`, four settings bytes per 03090 R5 §7.3.3 — R7's two-byte Example 3 is an error R5 corrected). Not yet verified on hardware: §5.4.1.1 says `0xD2` is ignored under horizontal scroll, so the per-band x-reset a multi-band graphic-font-w bitmap relies on needs a bench test. **Exterior** signs still send static frames — their destination scrolls inside a sub-region, which needs a second text field in the frame. | | **MP3 concatenation** | Binary byte-wise join (plays fine everywhere); ffmpeg needed for sample-accurate gapless. | | **Per-element volume on plain-text suppliers** | Every supplier declares a capability descriptor (`{ ssml, prosodyVolume, phoneme, offline }`, `tts/provider.ts`). Volume is honoured where `prosodyVolume` is declared (Azure/Google/Polly — SSML prosody or an API gain); on the ~10 plain-text suppliers the renderer drops it predictably — runs stay merged, a log line names the drop, the playlist editor warns while authoring and the Voices screen shows per-supplier feature badges. (Voice **pitch** is fixed at natural on every provider — see §3.) | | **Audio ducking on interrupt** | Interrupt hard-stops the current clip (no cross-fade) — the engine doesn't own the speaker. | | **Exterior loudspeaker (players)** | The players (§5.2) drive `speakers.INTERNAL` only. The exterior speaker also needs a **digital output raised to enable its amplifier** while the clip plays — unit-specific wiring, not implemented. | | **Player transport security** | The broker's WebSocket listener is plain `ws://` with `allow_anonymous`. Fine on a lab LAN; before a player runs on a real vehicle it needs `wss://` + credentials, or any device on the network can subscribe to a vehicle's audio — or publish **fake played-acks** into the ADA audit trail. | | **Multi-instance state** | Per-vehicle state is in-memory; horizontal scale needs sharding or Redis (see §11). | --- ## 15. Roadmap / TODO **Near-term** 1. **Rotate the Azure Speech key** (security — outstanding). 2. **FF hardware scroll on a bench sign** — confirm multi-band x behaviour under horizontal scroll, then extend it to the exterior destination sub-region. 3. **Pre-recorded audio provenance** — optional pixel/audio snapshots in proof-of-play for legal-grade evidence. **Mid-term** 4. **Two-way / upstream drive** — publish engine output to real vehicle units (needs a dry-run mode + ops sign-off). 5. **`bus-type` + stationary arrivals** — implement once the fleet feed exposes formation/type and real-time arrivals. 6. **Per-tenant retention** — separate/rotating proof-of-play logs per tenant for audit isolation. **Resilience — graceful degradation on the vehicle (TTS market-research Finding 2 / Scenario E "hybrid")** > The engine *central-renders-and-streams*: the content-addressed cache lives in the back office. Two layers now sit on the vehicle: the **on-vehicle clip cache** (§5.2 — every payload carries a content-hash `clipKey`, every received clip is kept on the device, a **key-only** payload replays cached bytes, `expiryDateTime` is enforced) and the **closed-corpus pre-push** (`PREPUSH_ENABLED=true`, `src/engine/prepush.ts`): the moment a journey's stop list is known, the engine renders the journey's whole announcement corpus — every remaining stop for stop-varying triggers, once for journey-constant ones, volatile free-text excluded — and delivers the clips to `{tts topic}/cache`, with the pushed `clipKey` byte-identical to what the live trigger later publishes. A pre-pushed journey therefore needs only tiny key-only messages at announcement time; the corpus is ~95 % closed (stop names, safety, service phrases). > > **What the pre-push is NOT: an offline guarantee.** In a central-render deployment the trigger *decision* is made in the back office from PIS-PT that crosses the WAN — when the link is fully down the engine never learns the bus reached the stop, so no message (full or key-only) is sent and the cached clips sit unused. The pre-push buys zero announcement latency and survival of a *degraded* link (a few hundred bytes gets through where hundreds of kilobytes time out); it does not survive a *dead* one. **The shipped fully-offline path is the on-vehicle engine deployment** ([`ON-VEHICLE.md`](ON-VEHICLE.md), `docker-compose.vehicle.yml`): PIS-PT originates on the bus's own network, so triggers never cross the WAN, and the engine's persistent cache covers the audio. Keeping a *thin* player fully offline instead would need local trigger evaluation in the players (a pre-pushed stop→clipKey manifest + the player watching local stop progress, with careful engine-vs-player arbitration) — that belongs with the gated item below, not with this pre-push. 7. **Embedded fallback engine in the player** — last-resort local synth (eSpeak NG or Piper) for free-text when offline, mirroring the server-side provider clients. Completes Scenario E. *Note F4:* Piper is now GPLv3 (legal review before shipping in the appliance); eSpeak is the intelligibility floor but robotic — degraded fallback only, never primary. Local trigger evaluation (playing the pre-pushed corpus with no link at all) belongs with this item. > **Gating for item 7:** on-vehicle deployment first needs the pre-production security blockers closed — `wss://` + broker credentials/ACLs (today's `ws://` + `allow_anonymous` lets any device subscribe to a vehicle's audio *or publish fake played-acks into the ADA audit trail*, §14) — which are Luminator Suite-integration scope. **Scale** 8. **Horizontal sharding / Redis-backed state** for the 2000-bus target ([`SCALING.md`](SCALING.md)). 9. **ffmpeg gapless concatenation** for sample-accurate multi-segment audio. --- ## 16. Glossary | Term | Meaning | | --- | --- | | **PIS-PT** | Passenger Information System — Public Transport. The inbound MQTT trip-data protocol (multi-topic, retained). | | **ADT 4.x** | The transHub audio message spec published on `…/tts` (base64 audio + speaker volumes). | | **Tenant** | An operator/agency namespace (e.g. `baltimore-md-mta`); the first topic segment. | | **Journey** | One scheduled run of a vehicle; identified by `vehicleJourneyRef` (used to group proof-of-play). | | **Trigger** | A rule that fires an announcement/sign on a PIS-PT condition (33 types). | | **Project** | A named, self-contained `AnnouncementConfig`; exactly one is published per (tenant, fleet). | | **Tenant** | Isolation boundary; projects and `engine_config` are scoped by (tenant, fleet). | | **Playlist** | An ordered list of elements (text, dynamic text, pause, audio) rendered to one announcement. | | **FF / Mobitec 1463-L** | The LED sign driver-board serial protocol; **font `w`** is its graphic (bitmap) font. | | **Hanover HCPS / SuperX** | Hanover Displays' sign protocol: HCPS frames (`STX…ETX` + two ASCII-hex checksum chars) carrying SuperX messages; `{\pic}` embeds a bitmap as a vertical raster. Specs in `Hanover/`. | | **Proof of Play** | The durable audit record of what was played/shown, where and when. | | **Selection** | The set of vehicles the engine currently renders for (dashboard-controlled). |