From ebf11889a3e56d359fe8687f2f2eb9d763d0bfd7 Mon Sep 17 00:00:00 2001 From: enzotar Date: Fri, 27 Feb 2026 11:15:54 -0800 Subject: [PATCH] docs: comprehensive documentation update MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - DREAMSTACK.md: rewritten with accurate counts (48 examples, 14 components, 136 tests), full CLI reference, architecture diagrams, quick start guide, comparison table, and phased roadmap - IMPLEMENTATION_PLAN.md: rewritten with all 10 phases showing accurate completion status, current capabilities, and next steps - BITSTREAM_INTEGRATION.md: updated test count (82 → 136) - USE_CASES.md and STREAM_COMPOSITION.md: already current, unchanged --- BITSTREAM_INTEGRATION.md | 2 +- DREAMSTACK.md | 812 +++++++++++++++++++-------------------- IMPLEMENTATION_PLAN.md | 423 +++++++------------- 3 files changed, 545 insertions(+), 692 deletions(-) diff --git a/BITSTREAM_INTEGRATION.md b/BITSTREAM_INTEGRATION.md index eeebefb..4bcfead 100644 --- a/BITSTREAM_INTEGRATION.md +++ b/BITSTREAM_INTEGRATION.md @@ -18,7 +18,7 @@ All changes in this spec have been implemented. Status per change: | 8. CLI | `dreamstack stream` command | ✅ Done | `main.rs` | | 9. Layout | `to_bytes`/`from_bytes` on `LayoutRect` | ✅ Done | `solver.rs` | -**Test counts**: 82 tests passing across full workspace (ds-stream: 38, ds-parser: 15, ds-analyzer: 6, ds-types: 11, plus others). +**Test counts**: 136 tests passing across full workspace (ds-stream: 38, ds-parser: 15, ds-analyzer: 6, ds-types: 11, ds-codegen: 6, ds-layout: 5, plus integration tests). 48 `.ds` examples compile successfully. ## Background diff --git a/DREAMSTACK.md b/DREAMSTACK.md index d089327..7312595 100644 --- a/DREAMSTACK.md +++ b/DREAMSTACK.md @@ -6,7 +6,7 @@ ## Implementation Status ✅ -DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB output + WASM physics. +DreamStack is **real and running** — 7 Rust crates, 136 tests, 48 compilable examples, 14 registry components, ~7KB runtime. ``` .ds source → ds-parser → ds-analyzer → ds-codegen → JavaScript @@ -20,27 +20,116 @@ DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB | Feature | Syntax | Status | |---------|--------|--------| | Signals | `let count = 0` | ✅ Fine-grained, auto-tracked | -| Derived | `let doubled = count * 2` | ✅ Lazy | -| Interpolation | `"Count: {count}"` | ✅ Reactive | +| Derived | `let doubled = count * 2` | ✅ `DS.derived()`, lazy evaluation | +| Interpolation | `"Count: {count}"` | ✅ Reactive `DS.effect()` | | Conditional | `when count > 5 -> text "hi"` | ✅ Mount/unmount | | If/else | `if x then a else b` | ✅ | | Match | `match state \| Loading -> ...` | ✅ | -| List rendering | `for item in items -> text item` | ✅ Reactive | -| Components | `component Card(title) = ...` | ✅ Props | +| List rendering | `for item in items -> text item` | ✅ Reactive, keyed | +| Components | `component Card(title) = ...` | ✅ Reactive props (getter fns) | +| Component import | `import { Badge } from "./badge"` | ✅ Recursive resolution | | Effects | `effect fetch(id): Result` / `perform` | ✅ Algebraic | -| Streams | `debounce`, `throttle`, `distinct` | ✅ | -| Springs | `spring(target: 0, stiffness: 300)` | ✅ Physics | -| Layout | Cassowary constraint solver | ✅ | -| Types | `Signal`, `Derived` | ✅ Hindley-Milner | -| Dev server | `dreamstack dev app.ds` | ✅ HMR | -| Streaming (source) | `stream main on "ws://..."` | ✅ Signal diffs | -| Streaming (receiver) | `stream from "ws://..."` | ✅ Auto-reconnect | +| Springs | `spring(target: 0, stiffness: 300)` | ✅ RK4 physics | +| Layout | `layout { sidebar.width == 250 }` | ✅ Cassowary solver | +| Types | `type PositiveInt = Int where x > 0` | ✅ Refinement types | | Router | `route "/path" -> body` / `navigate` | ✅ Hash-based | | Two-way binding | `input { bind: name }` | ✅ Signal ↔ input | | Async resources | `DS.resource()` / `DS.fetchJSON()` | ✅ Loading/Ok/Err | -| Springs | `let x = spring(200)` | ✅ RK4 physics | | Physics scene | `scene { gravity_y: g } [ circle {...} ]` | ✅ Rapier2D WASM | | Constraints | `constrain el.width = expr` | ✅ Reactive solver | +| Streaming (signal) | `stream on "ws://..." { mode: signal }` | ✅ Bidirectional, versioned diffs | +| Streaming (receiver) | `stream from "ws://..."` | ✅ Auto-reconnect, field select | +| Streaming (WebRTC) | `transport: webrtc` | ✅ P2P data channel + WS fallback | +| Stream output filter | `output: paddleX, ballY, score` | ✅ Skip internal signals | +| Dev server | `dreamstack dev app.ds` | ✅ File watcher + HMR polling | +| Playground | `dreamstack playground` | ✅ Monaco editor + live preview | +| CLI check | `dreamstack check app.ds` | ✅ Signal graph visualization | +| TSX converter | `dreamstack convert component.tsx` | ✅ React → DreamStack | +| Component registry | `dreamstack add badge` | ✅ 14 components | +| Timer merging | `every 33 -> ...` (multiple) | ✅ Single `setInterval` per interval | +| Sound | `play_tone(440, 60)` | ✅ Built-in oscillator | + +### CLI Commands + +```bash +dreamstack build app.ds # Compile to HTML+JS +dreamstack dev app.ds # Dev server with hot reload (port 3000) +dreamstack check app.ds # Analyze signal graph, type check +dreamstack stream app.ds # Compile + serve with streaming +dreamstack playground # Monaco editor playground (port 4000) +dreamstack add badge # Add a registry component +dreamstack add --list # List all available components +dreamstack convert file.tsx # Convert React/TSX → DreamStack +dreamstack convert button --shadcn # Convert from shadcn/ui registry +dreamstack init my-app # Initialize new project +``` + +### Registry Components (14) + +| Component | File | Description | +|-----------|------|-------------| +| Alert | `alert.ds` | Status messages with variants | +| Avatar | `avatar.ds` | Profile images with sizes | +| Badge | `badge.ds` | Status indicators (success/warning/error/info) | +| Button | `button.ds` | Click actions with variants | +| Card | `card.ds` | Content containers | +| Dialog | `dialog.ds` | Modal overlays | +| Input | `input.ds` | Text inputs with binding | +| Progress | `progress.ds` | Progress bars | +| Select | `select.ds` | Dropdown selects | +| Separator | `separator.ds` | Visual dividers | +| Stat | `stat.ds` | Statistic displays (value, label, trend) | +| Tabs | `tabs.ds` | Tabbed navigation | +| Toast | `toast.ds` | Notification toasts | +| Toggle | `toggle.ds` | On/off switches | + +### Examples (48 compilable .ds files) + +**Games:** +- `game-pong.ds` — Multiplayer Pong with keyboard, sound, streaming +- `game-breakout.ds` — Breakout with 5 rows, collision, score, streaming +- `game-snake.ds` — Snake game +- `game-reaction.ds` — Reaction time test + +**Streaming:** +- `streaming-counter.ds` — Counter synced across tabs +- `streaming-clock.ds` — Clock streamed to viewers +- `streaming-dashboard.ds` — Real-time dashboard via relay +- `streaming-stats.ds` — Signal stats viewer +- `streaming-mood.ds` — Collaborative mood board +- `streaming-physics.ds` — Physics scene streaming +- `streaming-webrtc.ds` — P2P WebRTC streaming +- `streaming-receiver.ds` — Generic stream receiver +- `pong-viewer.ds` — Spectator view for Pong +- `game-viewer.ds` — Generic game viewer + +**Audio:** +- `step-sequencer.ds` — 16-step drum machine with Web Audio API +- `beats-viewer.ds` — Spectator view for step sequencer + +**UI Patterns:** +- `counter.ds` — 3-line counter +- `list.ds` — Dynamic list with add/remove +- `todo.ds` / `todomvc.ds` — TodoMVC +- `form.ds` — Form with validation +- `router-demo.ds` — Multi-page routing +- `dashboard.ds` — Dashboard layout +- `component-gallery.ds` — Component showcase +- `showcase.ds` — Full feature showcase +- `project-manager.ds` — Kanban-style project tracker +- `springs.ds` — Spring physics animations + +**Language Features:** +- `language-features.ds` — Comprehensive syntax demo +- `refined-types.ds` — Refinement type guards +- `import-demo.ds` — Component imports +- `slot-demo.ds` — Component children/slots +- `each-demo.ds` — `each` loop rendering +- `when-else-demo.ds` — Conditional rendering +- `multi-action.ds` / `timer-multi-action.ds` — Multi-statement handlers +- `callback-demo.ds` — Callback props +- `physics.ds` — Rapier2D physics scene +- `bench-signals.ds` — Signal performance benchmarks ### DreamStack vs React @@ -57,9 +146,10 @@ DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB | Animation | Built-in springs | framer-motion (external) | | Physics | Built-in Rapier2D scene | matter.js (external) | | Layout | Built-in Cassowary | CSS only | -| Types | Native HM, `Signal` | TypeScript (external) | +| Streaming | Built-in bidirectional | WebSocket libraries (external) | +| Types | Native refinement types | TypeScript (external) | | Bundle | **~7KB** | **~175KB** | -| Ecosystem | New | Massive | +| Ecosystem | 48 examples, 14 components | Massive | ### Benchmarks (signal propagation) @@ -71,9 +161,157 @@ DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB | Batch Update (50) | 61K | 50 | | Mixed Read/Write | 242K | 10 | -### Examples +--- -`counter.ds` · `list.ds` · `router.ds` · `form.ds` · `springs.ds` · `physics.ds` · `todomvc.html` · `search.html` · `dashboard.html` · `playground.html` · `showcase.html` · `benchmarks.html` +## Quick Start + +### Counter (3 lines of logic) + +``` +let count = 0 + +view counter = + column [ + text "Count: {count}" + button "+" { click: count += 1 } + ] +``` + +### Multiplayer Pong (40 lines) + +``` +let ballX = 300 +let ballY = 200 +let bvx = 4 +let bvy = 3 +let p1y = 180 + +on keydown(ev) -> p1y = if ev.key == "ArrowUp" then p1y - 30 else p1y + +every 33 -> ballX = ballX + bvx +every 33 -> ballY = ballY + bvy +every 33 -> bvx = if ballX > 590 then -4 else bvx +every 33 -> bvy = if ballY < 5 then 3 else bvy + +stream pong on "ws://localhost:9100/peer/pong" { + mode: signal, + output: ballX, ballY, p1y +} + +view pong = + stack { style: "position:relative; width:600px; height:400px; background:#1a1a2e" } [ + column { style: "...ball styles...", top: ballY, left: ballX } [] + column { style: "...paddle...", top: p1y } [] + ] +``` + +### Spectator View (5 lines) + +``` +let game = stream from "ws://localhost:9100/peer/pong" + select [ballX, ballY, p1y] + +view viewer = + stack { style: "..." } [ + column { style: "...ball...", top: game.ballY, left: game.ballX } [] + column { style: "...paddle...", top: game.p1y } [] + ] +``` + +### Run It + +```bash +# Install +cargo install --path compiler/ds-cli + +# Build +dreamstack build examples/counter.ds -o dist + +# Dev server (with hot reload) +dreamstack dev examples/game-pong.ds + +# With streaming +cargo run -p ds-stream # Tab 1: start relay +dreamstack dev examples/game-pong.ds # Tab 2: play +open dist/index.html # Tab 3: spectate (pong-viewer.ds) +``` + +--- + +## Architecture + +### Compiler Pipeline + +``` + ┌─────────────┐ + .ds source ───────►│ ds-parser │──► AST (Program) + └──────┬──────┘ + │ + ┌──────▼──────┐ + │ ds-analyzer │──► SignalGraph (DAG) + └──────┬──────┘ + DomBindings + │ + SignalManifest + ┌──────▼──────┐ + │ ds-codegen │──► Single-file HTML+JS + └──────┬──────┘ (~7KB runtime) + │ + ┌──────▼──────┐ + │ ds-cli │──► dev/build/stream/check + └─────────────┘ +``` + +### Signal Graph (Compile-Time) + +``` +Signal: count (Source) + ├──► Derived: doubled (DS.derived) + │ └──► [DOM] TextNode "Doubled: {doubled}" + ├──► [DOM] TextNode "Count: {count}" + └──► [Stream] _streamDiff("count", count.value) +``` + +- **No virtual DOM.** Compiler maps signals → specific DOM nodes at build time +- **No re-rendering.** Components execute once, set up reactive bindings +- **No dependency arrays.** Compiler infers all dependencies from code +- **Timer merging.** All `every 33` statements share one `setInterval` + +### Reactive Runtime + +```javascript +// Source signal +const count = DS.signal(0); + +// Derived signal (auto-updates) +const doubled = DS.derived(() => count.value * 2); + +// Effect (auto-tracked, re-runs on dependency change) +DS.effect(() => { el.textContent = `Count: ${count.value}`; }); + +// Component props: getter functions for live reactivity +DS_Badge({ label: () => `Score: ${score.value}` }); +``` + +### Stream Protocol (16-byte binary header) + +``` +┌────────┬─────────┬──────────┬────────────┬───────┬────────┬────────┐ +│ type │ flags │ seq │ timestamp │ width │ height │ length │ +│ u8 │ u8 │ u16 │ u32 │ u16 │ u16 │ u32 │ +└────────┴─────────┴──────────┴────────────┴───────┴────────┴────────┘ +``` + +| Mode | What's Sent | Bandwidth | +|------|-------------|-----------| +| `signal` (default) | JSON diffs of changed signals | ~2 KB/s | +| `delta` | XOR + RLE compressed pixel deltas | ~50 KB/s | +| `pixel` | Raw RGBA framebuffer | ~30 MB/s | + +Features: +- **Bidirectional sync** with per-signal version counters +- **Output filtering** — only listed signals are broadcast +- **Exponential reconnect** — 2s → 4s → 8s, max 30s, reset on success +- **WebRTC data channel** with WebSocket fallback +- **Deduplication** — skip unchanged values --- @@ -94,471 +332,211 @@ Not Clojure. Not TypeScript. A new language that steals the best ideas from ever | Property | Inspiration | Why | |---|---|---| | **Homoiconic** | Clojure, Lisp | UI = data. Code = data. Everything is transformable | -| **Dependent types** | Idris, Agda | Types that express "this button is disabled *when* the form is invalid" at the type level | +| **Refinement types** | Liquid Haskell | `type Percent = Int where value >= 0 and value <= 100` | | **Algebraic effects** | Eff, Koka, OCaml 5 | Side effects as first-class, composable, interceptable values | -| **Reactive by default** | Svelte, Solid, Excel | No `useState`, no subscriptions. Values *are* reactive. Assignment *is* the API | +| **Reactive by default** | Svelte, Solid, Excel | No `useState`, no subscriptions. Assignment *is* the API | | **Structural typing** | TypeScript, Go | Flexible composition without class hierarchies | -| **Compiled to native + WASM** | Rust, Zig | Near-zero runtime overhead. No GC pauses during animations | +| **Compiled to JS** | Svelte, Solid | Near-zero runtime overhead | -### Syntax +### Syntax Reference ``` --- Signals are the primitive. Assignment propagates automatically. +-- Signals (mutable state) let count = 0 -let doubled = count * 2 -- derived, auto-tracked -let label = "Count: {doubled}" -- derived, auto-tracked +let name = "world" +let items = [1, 2, 3] --- UI is data. No JSX, no templates, no virtual DOM. -view counter = +-- Derived signals (auto-updated) +let doubled = count * 2 +let greeting = "Hello, {name}!" + +-- Refinement types +type PositiveInt = Int where value > 0 +let score: PositiveInt = 10 + +-- Event handlers +on keydown(ev) -> count = if ev.key == "ArrowUp" then count + 1 else count + +-- Timers +every 33 -> ballX = ballX + velocity + +-- Conditional rendering +when count > 10 -> text "High!" + +-- Pattern matching +match state + "loading" -> spinner + "error" -> text "Failed" + _ -> text "Ready" + +-- For loops +for item, i in items -> text "{i}: {item}" + +-- Components +component Card(title, children) = + column { variant: "card" } [ + text title { variant: "card-title" } + slot + ] + +-- Imports +import { Badge, Card } from "../registry/components/badge" + +-- Routes +route "/" -> text "Home" +route "/about" -> text "About" + +-- Springs +let x = spring(target: 0, stiffness: 300, damping: 30) + +-- Streaming +stream app on "ws://localhost:9100/peer/app" { + mode: signal, + output: count, name +} + +-- Receiving streams +let remote = stream from "ws://localhost:9100/peer/app" + select [count, name] + +-- Views +view main = column [ - text label -- auto-updates when count changes - button "+" { click: count += 1 } - - -- Conditional UI is just pattern matching on signals - when count > 10 -> - text "🔥 On fire!" | animate fade-in 200ms + text "Hello {name}" + button "Click" { click: count += 1 } ] ``` -No hooks. No dependency arrays. No re-renders. **The compiler builds a fine-grained reactive graph at compile time.** - --- -## Reactivity: Compile-Time Signal Graph - -This is the biggest departure from React. React's model is fundamentally **pull-based** — re-render the tree, diff it, patch the DOM. That's backwards. - -### Push-Based with Compile-Time Analysis +## Effect System: Algebraic Effects ``` -Signal: count - ├──► Derived: doubled - │ └──► Derived: label - │ └──► [direct DOM binding] TextNode #47 - └──► Condition: count > 10 - └──► [mount/unmount] text "🔥 On fire!" -``` - -**Key principles:** - -- **No virtual DOM.** The compiler knows at build time exactly which DOM nodes depend on which signals. When `count` changes, it updates *only* the specific text node and evaluates the condition. Nothing else runs. -- **No re-rendering.** Components don't re-execute. They execute *once* and set up reactive bindings. -- **No dependency arrays.** The compiler infers all dependencies from the code. You literally cannot forget one. -- **No stale closures.** Since there are no closures over render cycles, the entire class of bugs vanishes. - -This extends Solid.js's approach with full compile-time analysis in a purpose-built language. - ---- - -## Effect System: Algebraic Effects for Everything - -Every side-effect — HTTP, animations, time, user input, clipboard, drag-and-drop — is an **algebraic effect**. Effects are values you can compose, intercept, retry, and test. - -``` --- An effect is declared, not executed effect fetchUser(id: UserId): Result --- A component "performs" an effect — doesn't control HOW it runs view profile(id: UserId) = let user = perform fetchUser(id) - match user Loading -> skeleton-loader - Ok(u) -> column [ - avatar u.photo - text u.name - ] + Ok(u) -> column [ avatar u.photo, text u.name ] Err(e) -> error-card e | with retry: perform fetchUser(id) - --- At the app boundary, you provide the HANDLER -handle app with - fetchUser(id, resume) -> - let result = http.get "/api/users/{id}" - resume(result) ``` -### Why This is Powerful - -- **Testing:** Swap the handler. `handle app with mockFetchUser(...)` — no mocking libraries, no dependency injection frameworks. -- **Composition:** Effects compose naturally. An animation effect + a data-fetch effect combine without callback hell or `useEffect` chains. -- **Interceptors:** Want to add logging, caching, or retry logic? Add a handler layer. The component code never changes. -- **Time travel:** Since effects are values, you can record and replay them. Free undo/redo. Free debugging. - --- -## Data Flow: Everything is a Stream - -Forget the distinction between "state", "props", "events", and "side effects". **Everything is a stream of values over time.** +## Layout: Constraint-Based ``` --- User input is a stream -let clicks = stream from button.click -let keypresses = stream from input.keydown - --- Derived streams with temporal operators -let search_query = keypresses - | map .value - | debounce 300ms - | distinct - --- API results are streams -let results = search_query - | flatmap (q -> http.get "/search?q={q}") - | catch-with [] - --- UI binds to streams directly -view search = - column [ - input { on-keydown: keypresses } - - match results - Loading -> spinner - Data(rs) -> list rs (r -> search-result-card r) - Empty -> text "No results" - ] +layout dashboard { + sidebar.width == 250 + main.x == sidebar.x + sidebar.width + main.width == parent.width - sidebar.width [strong] +} ``` -### Unification - -This single abstraction covers everything: - -| Concept | Traditional | DreamStack | -|---|---|---| -| User input | Event handlers | Stream | -| Network responses | Promises / async-await | Stream | -| Animations | CSS transitions / JS libraries | Stream of interpolated values | -| Timers | `setInterval` / `setTimeout` | Stream | -| WebSockets | Callback-based | Stream | -| Drag events | Complex event handler state machines | Stream of positions | - -One abstraction. One composition model. Everything snaps together. +Uses a Cassowary constraint solver. Constraints are reactive — when viewport changes, layout re-solves in a single pass. --- -## Layout: Constraint-Based, Not Box-Based - -CSS's box model is from 1996. Flexbox and Grid are patches on a fundamentally limited system. DreamStack starts over with a **constraint solver**. +## Animation: Physics-Based Springs ``` --- Declare relationships, not boxes -layout dashboard = - let sidebar.width = clamp(200px, 20vw, 350px) - let main.width = viewport.width - sidebar.width - let header.height = 64px - let content.height = viewport.height - header.height - - -- Constraints, not nesting - sidebar.left = 0 - sidebar.top = header.height - sidebar.height = content.height - - main.left = sidebar.right - main.top = header.height - main.right = viewport.right - - -- Responsive is just different constraints - when viewport.width < 768px -> - sidebar.width = 0 -- collapses - main.left = 0 -- takes full width -``` - -Inspired by Apple's **Auto Layout** (Cassowary constraint solver), but made reactive. When `viewport.width` changes, the constraint solver re-solves and updates positions in a single pass. No layout thrashing. No "CSS specificity wars." - -### Advantages Over CSS - -- **No cascade conflicts** — constraints are explicit and local -- **No z-index hell** — layering is declarative -- **No media query breakpoints** — responsiveness emerges from constraints -- **Animations are free** — animating a constraint target is the same as setting it - ---- - -## Animation: First-Class, Physics-Based, Interruptible - -Animations aren't CSS transitions bolted on after the fact. They're part of the reactive graph. - -``` --- A spring is a signal that animates toward its target let panel_x = spring(target: 0, stiffness: 300, damping: 30) --- Change the target → the spring animates automatically on toggle_sidebar -> panel_x.target = if open then 250 else 0 --- Gestures feed directly into springs -on drag(event) -> - panel_x.target = event.x -- spring follows finger - -on drag-end(event) -> - panel_x.target = snap-to-nearest [0, 250] event.x - --- UI just reads the spring's current value view sidebar = - panel { x: panel_x } [ - nav-items + panel { x: panel_x } [ nav-items ] +``` + +Springs are interruptible, gesture-driven, and composable through the same signal system. + +--- + +## Physics: Rapier2D WASM + +``` +let gravity_y = 980 + +view main = + scene { width: 700, height: 450, gravity_y: gravity_y } [ + circle { x: 200, y: 80, radius: 35, color: "#8b5cf6" } + rect { x: 500, y: 100, width: 80, height: 50, color: "#10b981" } ] + + button "Anti-Gravity" { click: gravity_y = -500 } ``` -### Design Principles - -- **Interruptible:** Start a new animation mid-flight. The spring handles the physics — no jarring jumps, no "wait for the current animation to finish." -- **Gesture-driven:** Touch/mouse input feeds directly into the animation model. No separate "gesture handler" → "state update" → "CSS transition" pipeline. -- **60fps guaranteed:** Springs resolve on the GPU. The main thread never blocks. -- **Composable:** Combine springs, easing curves, and physics simulations using the same stream operators as everything else. - --- -## Runtime: Tiny, Compiled, No GC Pauses +## Streaming: Built-In Multiplayer + +Any DreamStack app becomes multiplayer with one declaration: ``` -┌──────────────────────────────────────────┐ -│ Compiled Output │ -├──────────────────────────────────────────┤ -│ Signal Graph (static DAG) ~2KB │ -│ Constraint Solver (layout) ~4KB │ -│ Spring Physics (animations) ~1KB │ -│ Effect Runtime (handlers) ~2KB │ -│ DOM Patcher (surgical updates) ~3KB │ -├──────────────────────────────────────────┤ -│ Total Runtime: ~12KB │ -│ (vs React ~45KB + ReactDOM ~130KB) │ -└──────────────────────────────────────────┘ +stream pong on "ws://localhost:9100/peer/pong" { + mode: signal, + output: ballX, ballY, paddleY, score +} ``` -The compiler does the heavy lifting. The runtime is tiny because there is: +The compiler automatically: +1. Connects to the relay via WebSocket +2. Wraps signal mutations with `_streamDiff()` calls +3. Sends versioned JSON diffs (conflict resolution via version counters) +4. Filters to only broadcast `output` signals (skips internals like velocity) +5. Reconnects with exponential backoff on disconnect -- No virtual DOM diffing algorithm -- No fiber scheduler -- No hook state management system -- No reconciliation algorithm -- No garbage collector pauses during animation frames +Viewers connect with `stream from`: +``` +let game = stream from "ws://localhost:9100/peer/pong" + select [ballX, ballY, paddleY, score] +``` --- -## The Killer Feature: Live Structural Editing - -Because the language is homoiconic (code = data), the **editor IS the runtime:** - -``` -┌─────────────────────────────────┐ -│ Live Editor │ -│ │ -│ view counter = │ ← Edit this... -│ column [ │ -│ text "Count: {count}" │ -│ button "+" { ... } │ -│ ] │ -│ │ -│ ───────────────────────────── │ -│ │ -│ ┌─────────────┐ │ ← ...see this update -│ │ Count: 42 │ │ instantly, with state -│ │ [ + ] │ │ preserved. -│ └─────────────┘ │ -│ │ -└─────────────────────────────────┘ -``` - -- Drag and drop UI elements in the preview → the **code updates**. -- Edit the code → the **preview updates**. -- Both directions, simultaneously, with state preserved. -- Not a design tool that generates code — **the code IS the design tool**. - -This is possible because: -1. **Homoiconicity** means UI structure is inspectable and modifiable data -2. **Immutable signals** mean state survives code changes -3. **Compile-time signal graph** means changes are surgical, not full-page reloads - ---- - -## Comparison to Existing Approaches +## Comparison | Capability | React | Svelte 5 | Solid.js | Flutter | **DreamStack** | |---|---|---|---|---|---| -| Reactivity | Pull (re-render + diff) | Compile-time runes | Fine-grained signals | Widget rebuild | **Compile-time signal DAG** | -| Side effects | `useEffect` + deps array | `$effect` | `createEffect` | Lifecycle methods | **Algebraic effects** | -| Layout | CSS (external) | CSS (scoped) | CSS (external) | Constraint-based | **Constraint solver (native)** | -| Animation | 3rd party libs | 3rd party libs | 3rd party libs | Physics-based (native) | **Physics-based (native)** | -| SSR | Yes (complex) | Yes | Yes | No (web) | **Yes (compiled)** | -| Runtime size | ~175KB | ~2KB | ~7KB | ~2MB (web) | **~12KB** | -| Live editing | Fast Refresh (lossy) | HMR | HMR | Hot reload | **Bidirectional structural** | -| Type safety | TypeScript (bolt-on) | TypeScript (bolt-on) | TypeScript (bolt-on) | Dart (native) | **Dependent types (native)** | -| UI = Data | No (JSX compiled away) | No (templates) | No (JSX compiled) | No (widget classes) | **Yes (homoiconic)** | - ---- - -## Fragments of the Future, Today - -The closest approximations to pieces of this vision: - -- **Solid.js** — fine-grained reactivity, no VDOM, ~7KB runtime -- **Svelte 5 Runes** — compiler-driven reactivity, tiny output -- **Elm** — algebraic effects-adjacent, immutable state, strong types -- **Flutter** — constraint layout, physics-based animation, hot reload -- **Clojure/Reagent** — homoiconicity, UI as data, ratom reactivity -- **Koka** — algebraic effect system in a practical language -- **Apple Auto Layout** — Cassowary constraint solver for UI -- **Excel** — reactive by default, dependency auto-tracking - -Nobody has unified them. That's the opportunity. +| Reactivity | Pull (VDOM diff) | Runes | Signals | Rebuild | **Compile-time DAG** | +| Side effects | `useEffect` + deps | `$effect` | `createEffect` | Lifecycle | **Algebraic effects** | +| Layout | CSS (external) | CSS (scoped) | CSS (external) | Constraints | **Cassowary (native)** | +| Animation | 3rd party | 3rd party | 3rd party | Physics (native) | **Springs (native)** | +| Streaming | WebSocket libs | WebSocket libs | WebSocket libs | None | **Built-in bidirectional** | +| Runtime | ~175KB | ~2KB | ~7KB | ~2MB | **~7KB** | +| Type safety | TypeScript | TypeScript | TypeScript | Dart | **Refinement types** | +| UI = Data | No (JSX) | No (templates) | No (JSX) | No (widgets) | **Yes (homoiconic)** | --- ## Design Philosophy -1. **The compiler is the framework.** Move work from runtime to compile time. The less code that runs in the browser, the faster the UI. -2. **Reactivity is not a feature, it's the default.** Every value is live. Every binding is automatic. You opt *out* of reactivity, not *in*. -3. **Effects are values, not side-channels.** Making side effects first-class and composable eliminates the largest source of bugs in modern UIs. -4. **Layout and animation are not afterthoughts.** They're core primitives, not CSS bolt-ons or third-party libraries. -5. **The editor and the runtime are the same thing.** Bidirectional editing collapses the design-develop gap entirely. -6. **UI is data, all the way down.** If you can't `map` over your UI structure, your abstraction is wrong. -7. **Any input bitstream → any output bitstream.** The UI is just one codec. Tomorrow's neural nets generate the pixels directly. +1. **The compiler is the framework.** Move work from runtime to compile time +2. **Reactivity is the default.** Every value is live. You opt *out*, not *in* +3. **Effects are values.** First-class, composable, interceptable +4. **Layout and animation are core.** Not CSS bolt-ons or third-party libraries +5. **The editor and runtime are one.** Bidirectional structural editing +6. **UI is data, all the way down.** If you can't `map` over your UI, your abstraction is wrong +7. **Any bitstream in → any bitstream out.** The UI is just one codec --- -## Phase 7: Universal Bitstream Streaming +## Next Steps -> *Stream the whole UI as bytes. Neural nets will generate the pixels, acoustics, and actuator commands.* +### Phase 1: Polish (estimated ~2h) +- [ ] **Better error messages** — source context with line + caret in parser errors +- [ ] **Route integration test** — multi-page example exercising `route` + `navigate` +- [ ] **WebRTC integration test** — verify P2P data channel transport +- [ ] **Layout constraint test** — example using Cassowary solver -DreamStack's `engine/ds-stream` crate implements a universal binary protocol for streaming any I/O: +### Phase 2: New Examples (estimated ~2h) +- [ ] **Tetris** — stress-test grid, timers, collision, streaming +- [ ] **Todo with sync** — CRUD + bidirectional streaming sync +- [ ] **Snake polish** — cleanup existing game, add streaming -``` -┌──────────┐ WebSocket / WebRTC ┌──────────┐ -│ Source │ ──────frames (bytes)──► │ Receiver │ -│ (renders) │ ◄──────inputs (bytes)── │ (~250 LOC)│ -└──────────┘ └──────────┘ -``` - -### Binary Protocol (16-byte header) - -| Field | Size | Description | -|-------|------|-------------| -| type | u8 | Frame/input type (pixels, audio, haptic, neural, BCI) | -| flags | u8 | Input flag, keyframe flag, compression flag | -| seq | u16 | Sequence number | -| timestamp | u32 | Relative ms since stream start | -| width | u16 | Frame width or channel count | -| height | u16 | Frame height or sample rate | -| length | u32 | Payload length | - -### Output Types -- `Pixels` (0x01) — raw RGBA framebuffer -- `Audio` (0x10) — PCM audio samples -- `Haptic` (0x20) — vibration/actuator commands -- `NeuralFrame` (0x40) — neural-generated pixels *(future)* -- `NeuralAudio` (0x41) — neural speech/music synthesis *(future)* -- `NeuralActuator` (0x42) — learned motor control *(future)* - -### Input Types -- `Pointer` (0x01) — mouse/touch position + buttons -- `Key` (0x10) — keyboard events -- `Gamepad` (0x30) — controller axes + buttons -- `BciInput` (0x90) — brain-computer interface *(future)* - -### Demos -- `examples/stream-source.html` — Springs demo captures canvas → streams pixels at 30fps -- `examples/stream-receiver.html` — Thin client (~250 lines, no framework) renders bytes - -### Run It -```bash -cargo run -p ds-stream # start relay on :9100 -open examples/stream-source.html # source: renders + streams -open examples/stream-receiver.html # receiver: displays bytes -``` - -### Compiler-Native Streaming - -The `stream` keyword makes any `.ds` app streamable with one line. The compiler generates all WebSocket connection, binary protocol encoding, and signal diff broadcasting automatically. - -#### Source: `stream` declaration - -``` -let count = 0 -let doubled = count * 2 - -stream counter on "ws://localhost:9100" { mode: signal } - -view counter = - column [ - text count - text doubled - button "+" { click: count += 1 } - ] -``` - -The compiler: -1. Calls `DS._initStream(url, mode)` on load → connects to relay -2. Wraps every signal mutation with `DS._streamDiff("count", count.value)` → sends JSON diff frames -3. Sends scene body positions at 60fps when a physics scene is streaming - -#### Receiver: `stream from` expression - -``` -let remote = stream from "ws://localhost:9100" - -view main = - column [ - text remote.count - text remote.doubled - ] -``` - -Compiles to `DS._connectStream(url)` — returns a reactive `Signal` that merges incoming `FRAME_SIGNAL_SYNC` and `FRAME_SIGNAL_DIFF` frames. - -#### Streaming Modes - -| Mode | Keyword | What's Sent | Bandwidth | -|------|---------|-------------|----------| -| Signal | `signal` (default) | JSON diffs of changed signals | ~2 KB/s | -| Delta | `delta` | XOR + RLE compressed pixel deltas | ~50 KB/s | -| Pixel | `pixel` | Raw RGBA framebuffer every frame | ~30 MB/s | - -#### CLI - -```bash -# Compile and serve with streaming enabled -dreamstack stream app.ds --relay ws://localhost:9100 --mode signal - -# Or explicitly declare streaming in the .ds file and use dev server -dreamstack dev app.ds -``` - ---- - -## Phase 8: Physics Scene — Rapier2D in the Language - -> *Declare physics bodies in `.ds` syntax. The compiler generates WASM-backed canvas with rigid body simulation.* - -```ds -let gravity_y = 980 - -view main = - column [ - scene { width: 700, height: 450, gravity_y: gravity_y } [ - circle { x: 200, y: 80, radius: 35, color: "#8b5cf6" } - circle { x: 350, y: 50, radius: 50, color: "#7c3aed" } - rect { x: 500, y: 100, width: 80, height: 50, color: "#10b981" } - ] - button "Anti-Gravity" { click: gravity_y = -500 } - button "Normal" { click: gravity_y = 980 } - ] -``` - -### Architecture - -``` -.ds source → parser (scene/circle/rect) → codegen → JS - ↓ - canvas + async WASM init - ds-physics (Rapier2D) ← pkg/ds_physics_bg.wasm - ↓ - requestAnimationFrame loop - step → render → repeat -``` - -### Features -- **Rigid body physics** — circles and rectangles with collision, rotation, restitution -- **Reactive gravity** — signal changes wrapped in `DS.effect()`, bodies wake on gravity change -- **Mouse drag** — click and drag bodies with impulse-based interaction -- **Compile-time colors** — hex color strings parsed at compile time → `set_body_color(r, g, b, a)` -- **Zero JS overhead** — physics runs in WASM, rendering in canvas, signals bridge both +### Phase 3: Ecosystem (estimated ~1.5h) +- [ ] **Component gallery** — live showcase of all 14 components +- [ ] **Crates.io publish** — `cargo install dreamstack` +- [ ] **Self-hosted playground** — deploy playground as web app diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md index 59bfa90..12b58b6 100644 --- a/IMPLEMENTATION_PLAN.md +++ b/IMPLEMENTATION_PLAN.md @@ -1,208 +1,189 @@ # DreamStack Implementation Plan -Build a working prototype of the DreamStack vision — a new UI framework with compile-time reactivity, algebraic effects, constraint layout, and physics animation. The strategy is to **bootstrap pragmatically**: use Rust for the compiler/runtime core, target WASM + DOM, and prove each pillar with working demos before unifying them. +Build a working prototype of the DreamStack vision — a new UI framework with compile-time reactivity, algebraic effects, constraint layout, and physics animation. The strategy is to **bootstrap pragmatically**: use Rust for the compiler/runtime core, target JS + DOM, and prove each pillar with working demos before unifying them. --- -## Phase 0 — Foundation (Weeks 1–3) +## Status Overview -**Goal:** A working compiler that turns DreamStack syntax into executable WASM + JS glue. +| Phase | Goal | Status | +|-------|------|--------| +| 0 — Foundation | Compiler pipeline, CLI | ✅ Complete | +| 1 — Reactive Core | Signals, derived, effects, DOM bindings | ✅ Complete | +| 2 — Effects & Streams | Algebraic effects, stream operators | ✅ Complete | +| 3 — Layout & Animation | Cassowary constraints, spring physics | ✅ Complete | +| 4 — Type System | Refinement types, runtime guards | ✅ Partial (refinement types done, full HM deferred) | +| 5 — Live Editor | Playground, HMR dev server | ✅ Complete | +| 6 — Production | Component registry, imports, TSX converter | ✅ Complete | +| 7 — Bitstream Streaming | Source/receiver, relay, signal diffs | ✅ Complete | +| 8 — Physics Scene | Rapier2D WASM integration | ✅ Complete | +| 9 — Bidirectional Sync | Peer-to-peer, conflict resolution, WebRTC | ✅ Complete | +| 10 — Games & Audio | Pong, Breakout, Sequencer, Sound | ✅ Complete | -### Compiler Pipeline (Rust) - -#### [NEW] `compiler/crates/ds-parser/` -- Hand-written recursive descent parser for DreamStack syntax -- Produces an AST of `view`, `let`, `when`, `match`, `effect`, `on`, `stream`, `layout` constructs -- Homoiconic representation: AST nodes are also the runtime data structure (tagged vectors/maps) - -#### [NEW] `compiler/crates/ds-analyzer/` -- **Signal graph extraction:** Static analysis pass that identifies signals, derived values, and their dependencies → builds a DAG -- **Effect collection:** Catalogs all `perform` sites and their expected effect signatures -- **Type inference:** Hindley-Milner base with structural typing; dependent types deferred to Phase 4 - -#### [NEW] `compiler/crates/ds-codegen/` -- Emits **Rust → WASM** for: - - Signal graph runtime (push-based notification) - - Constraint solver (Cassowary algorithm) - - Spring physics engine -- Emits **JS glue** for DOM bindings — surgical `textContent`, `setAttribute`, `insertBefore`, `removeChild` calls (no VDOM) -- Output format: single `.wasm` + single `.js` module - -#### [NEW] `compiler/crates/ds-cli/` -- `dreamstack build ` → produces `.wasm` + `.js` + `.html` -- `dreamstack dev ` → dev server with file watching + hot-reload -- `dreamstack check ` → type check + signal graph validation - -### Deliverable -A counter app (`examples/counter.ds`) compiles and runs in the browser with reactive signal updates, no VDOM. +**Current:** 7 Rust crates, 136 tests, 48 compilable `.ds` examples, 14 registry components. --- -## Phase 1 — Reactive Core (Weeks 4–6) +## Phase 0 — Foundation ✅ -**Goal:** Fine-grained reactivity with compile-time dependency tracking, proving the "no re-render" model. +**Compiler Pipeline (Rust)** -### Signal Runtime (Rust → WASM) +| Crate | Purpose | Status | +|-------|---------|--------| +| `ds-parser` | Recursive descent parser → AST | ✅ 15 tests | +| `ds-analyzer` | Signal graph extraction (DAG) | ✅ 6 tests | +| `ds-codegen` | JS emitter — single-file HTML+JS output | ✅ | +| `ds-types` | Refinement type checker | ✅ 11 tests | +| `ds-layout` | Cassowary constraint solver | ✅ | +| `ds-cli` | CLI: build, dev, check, stream, playground, add, convert, init | ✅ | +| `ds-stream` | WebSocket relay server + binary protocol | ✅ 38 tests | -#### [NEW] `runtime/crates/ds-signals/` +**Deliverable:** `counter.ds` compiles and runs. ✅ + +--- + +## Phase 1 — Reactive Core ✅ + +**Signal Runtime (pure JS, ~3KB):** - `Signal` — mutable source, notifies dependents on write -- `Derived` — computed from signals, lazy + cached, auto-invalidated -- `Effect` — runs side-effect when dependencies change (for DOM updates only — user-facing effects use algebraic effects) -- Batching: multiple signal writes in one tick coalesce into a single propagation pass -- Topological sort of the DAG ensures glitch-free propagation (no intermediate states visible) +- `Derived` — computed from signals via `DS.derived()`, lazy + cached +- `Effect` — runs side-effect when dependencies change (surgical DOM updates) +- Batching: multiple signal writes coalesce into single propagation pass +- Topological sort ensures glitch-free propagation -### DOM Binding Layer (JS) +**DOM Binding Layer:** +- `DS.effect(() => { el.textContent = signal.value; })` — auto-tracked +- Reactive `style`, `class`, `variant` props on containers +- Component props: getter fns for live reactivity (`label: () => expr`) +- Timer merging: all same-interval `every` statements share one `setInterval` -#### [NEW] `runtime/js/dom-bindings.js` -- Minimal JS layer (~3KB) that the codegen targets -- Functions: `bindText(nodeId, signalId)`, `bindAttr(nodeId, attr, signalId)`, `bindVisible(nodeId, signalId)`, `mountChildren(parentId, listSignalId, templateFn)` -- Each binding registers itself as an effect on the signal graph -- No framework runtime — just a bag of DOM mutation functions - -### Benchmark Suite - -#### [NEW] `bench/` -- Signal propagation: 1K / 10K / 100K signals, measure update latency -- DOM updates: compare against Solid.js, Svelte 5, vanilla JS -- Target: <1ms for 1K signal cascade, <16ms for 10K - -### Deliverable -A TodoMVC app (`examples/todomvc.ds`) with add/remove/toggle/filter — all reactive, no VDOM, benchmarked against Solid.js. +**Deliverable:** TodoMVC, counter, list, form — all reactive, no VDOM. ✅ --- -## Phase 2 — Effect System & Streams (Weeks 7–10) +## Phase 2 — Effects & Streams ✅ -**Goal:** Algebraic effects for side-effects, stream primitives for temporal data. +- `effect fetchUser(id): Result` — declared effects +- `perform fetchUser(id)` — suspends to nearest handler +- `DS.resource()` / `DS.fetchJSON()` — async resources with Loading/Ok/Err states +- Stream operators: `debounce`, `throttle`, `distinct` for temporal data -### Algebraic Effects (Rust → WASM) - -#### [NEW] `runtime/crates/ds-effects/` -- Effect declarations: `effect fetchUser(id: UserId): Result` -- `perform` keyword: suspends the current continuation, delegates to the nearest handler -- `handle ... with` blocks: install effect handlers at any level of the component tree -- Continuation-based: handlers receive a `resume` function to continue execution -- Built-in effects: `Http`, `Time`, `Random`, `Storage`, `Console` - -### Stream Engine - -#### [NEW] `runtime/crates/ds-streams/` -- `Stream` — push-based async sequence -- Core operators: `map`, `filter`, `flatmap`, `merge`, `combine`, `debounce`, `throttle`, `distinct`, `take`, `skip`, `scan` -- `stream from ` — creates a stream from DOM events -- Streams integrate with the signal graph: a stream's latest value is a signal -- Backpressure: drop or buffer strategies, configurable per stream - -### Deliverable -A search-with-autocomplete app (`examples/search.ds`) using `debounce`, `flatmap` over an HTTP effect, with the effect handler swappable for testing. +**Deliverable:** `search.html`, `dashboard.html` with live data. ✅ --- -## Phase 3 — Layout & Animation (Weeks 11–14) +## Phase 3 — Layout & Animation ✅ -**Goal:** Constraint-based layout engine and physics-based animation, both running in WASM. +**Cassowary Constraint Solver:** +- `layout dashboard { sidebar.width == 250 }` — declarative constraints +- Reactive: constraints reference signals → solver re-runs on change +- `constrain el.prop = expr` — imperative constraints -### Constraint Solver +**Spring Physics:** +- `let x = spring(target: 0, stiffness: 300, damping: 30)` — RK4 integrator +- Interruptible, gesture-driven, composable through signal system +- `requestAnimationFrame` scheduler, sleeps when idle -#### [NEW] `runtime/crates/ds-layout/` -- Implement the **Cassowary** simplex-based constraint solving algorithm in Rust -- Constraint types: `eq`, `gte`, `lte`, `clamp`, `ratio` -- Reactive integration: constraints reference signals → solver re-runs when inputs change -- Output: absolute `(x, y, width, height)` for each element, applied directly to DOM via `transform: translate()` (avoids reflow) -- Performance target: solve 500 constraints in <2ms - -### Spring Physics - -#### [NEW] `runtime/crates/ds-springs/` -- `Spring { value, velocity, target, stiffness, damping, mass }` -- RK4 integrator, fixed timestep (1/120s), interpolated for display -- `spring.target = x` triggers animation to new target (interruptible by default) -- Springs are signals — anything that reads `spring.value` auto-updates -- Gesture integration: direct-manipulation by setting `spring.target` to pointer position - -### Animation Scheduler - -#### [NEW] `runtime/js/raf-scheduler.js` -- Drives springs via `requestAnimationFrame` -- When no springs are active, scheduler sleeps (zero CPU when idle) -- Coordinates with signal graph: spring value changes go through normal propagation - -### Deliverable -A draggable panel layout (`examples/dashboard.ds`) with resizable sidebar (constraints) and spring-animated panel transitions. +**Deliverable:** `springs.ds`, `dashboard.ds` with constraint layout. ✅ --- -## Phase 4 — Type System (Weeks 15–18) +## Phase 4 — Type System ✅ (Partial) -**Goal:** Dependent types for UI correctness guarantees. +**Implemented:** +- Refinement types: `type PositiveInt = Int where value > 0` +- Runtime guards: compiler emits `if (!(pred)) throw new Error(...)` +- Basic type annotations on let declarations -### Type Checker - -#### [NEW] `compiler/crates/ds-types/` -- **Base:** Hindley-Milner with let-generalization, structural records -- **Refinement types:** `type PositiveInt = { n: Int | n > 0 }` — checked at compile time where possible, runtime where not -- **Signal-aware types:** `Signal` is a first-class type; the compiler tracks which values are reactive -- **Effect types:** functions declare which effects they may perform → compiler verifies all effects are handled -- **UI types:** `View` type ensures only valid UI expressions appear in `view` blocks - -### Error Messages -- Inspired by Elm: suggest fixes, show the specific constraint that failed, include code context -- Signal graph visualization in error output for dependency-related errors - -### Deliverable -The compiler catches: unhandled effects, type mismatches in signal derivations, invalid UI expressions — all with helpful error messages. +**Deferred:** +- Full Hindley-Milner inference +- Effect types (verified all effects handled) +- Signal-aware types (`Signal` as first-class) --- -## Phase 5 — Live Editor (Weeks 19–24) +## Phase 5 — Live Editor ✅ -**Goal:** Bidirectional structural editing — the killer feature. +- **Dev server:** `dreamstack dev` — file watcher + HMR polling (500ms) +- **Playground:** `dreamstack playground` — Monaco editor + live preview + error panel +- **Compile errors:** displayed in-browser with styled error overlay +- **Auto-reload:** version counter + polling, full page refresh on change -### Incremental Compiler - -#### [NEW] `compiler/crates/ds-incremental/` -- Tracks which AST nodes changed between edits -- Re-analyzes only affected signal subgraph -- Re-generates only changed DOM bindings -- Target: <50ms from keystroke to live update - -### Editor Integration - -#### [NEW] `editor/` -- Web-based editor (Monaco or CodeMirror 6 with custom language mode) -- Split pane: code ↔ live preview -- **Code → Preview:** incremental compile on every keystroke -- **Preview → Code:** click an element in preview → highlights source; drag to reorder → AST rewrite → code updates -- State preservation: signals retain their current values across edits - -### Visual Inspector -- Overlay on the live preview showing signal dependencies, constraint wires, spring states -- Click a DOM element → see its signal bindings, constraint relationships, effect dependencies - -### Deliverable -A working playground at `dreamstack.dev` where users can write DreamStack code and see live results with bidirectional editing. +**Deliverable:** Working playground with Monaco. ✅ --- -## Phase 6 — Production Readiness (Weeks 25–30) +## Phase 6 — Production ✅ -**Goal:** SSR, ecosystem tooling, documentation, and real-world validation. +- **Component registry:** 14 components (Badge, Dialog, Tabs, Toast, Card, etc.) +- **`dreamstack add badge`** — install from registry +- **`dreamstack convert file.tsx`** — React/TSX → DreamStack converter +- **`dreamstack convert button --shadcn`** — shadcn/ui integration +- **`import { Badge } from "./badge"`** — recursive module resolution +- **`export component Card(title) = ...`** — named exports +- **Slot system:** `component Card(title, children) = ... slot ...` -### Server-Side Rendering -- Compile DreamStack to static HTML + hydration script -- Streaming SSR: emit HTML as constraint solver resolves layout -- Effect handlers for server context (no DOM, no springs) +--- -### Package System -- `dreamstack add ` — package registry for reusable views and effect handlers -- Standard library: common UI patterns (buttons, inputs, lists, modals, toasts) -- Interop: import/export with JavaScript modules +## Phase 7 — Bitstream Streaming ✅ -### Documentation & Examples -- Language reference -- Tutorial: "Build a real app in 30 minutes" -- Cookbook: common patterns (forms, routing, data fetching, auth) -- Migration guide: "Coming from React" / "Coming from Svelte" +**Full specification**: see [BITSTREAM_INTEGRATION.md](file:///home/amir/code/dreamstack/BITSTREAM_INTEGRATION.md) + +- 16-byte binary protocol, 3 streaming modes (signal/delta/pixel) +- Bidirectional sync with per-signal version counters +- Output filtering: `output: x, y, score` — skips internal signals +- Exponential reconnect backoff (2s → 4s → 8s, max 30s) +- WebRTC data channel + WebSocket fallback +- `stream from` expression for receivers with `select` clause + +--- + +## Phase 8 — Physics Scene ✅ + +- Rapier2D compiled to WASM via `wasm-pack` +- `scene { gravity_y: g } [ circle {...}, rect {...} ]` +- Reactive gravity, mouse drag, collision, restitution +- Compile-time hex color parsing → `set_body_color(r, g, b, a)` +- `streaming-physics.ds` — physics scene streamed to viewers + +--- + +## Phase 9 — Bidirectional Sync ✅ + +- **Peer mode:** `stream on ".../peer/channel"` — both source and receiver +- **Conflict resolution:** version counters, last-write-wins with stale rejection +- **Deduplication:** JSON.stringify comparison, skip unchanged values +- **Composition:** receiver can derive new signals and re-stream them + +**Full specification**: see [STREAM_COMPOSITION.md](file:///home/amir/code/dreamstack/STREAM_COMPOSITION.md) + +--- + +## Phase 10 — Games & Audio ✅ + +- **Pong:** multiplayer with keyboard controls, sound effects, streaming +- **Breakout:** 5 brick rows, collision, score/lives badges, streaming +- **Snake:** grid-based movement, growth, collision +- **Reaction game:** reaction time measurement +- **Step sequencer:** 16-step drum machine with Web Audio API +- **Beats viewer:** spectator view for step sequencer +- **Sound:** built-in `play_tone(freq, duration)` oscillator + +--- + +## Next Steps + +### Near-Term Polish +- [ ] Better error messages — source context with line + caret +- [ ] Integration tests for routes, WebRTC, layout constraints + +### New Examples +- [ ] Tetris game (grid + timers + line clearing) +- [ ] Todo app with bidirectional streaming sync + +### Ecosystem +- [ ] Crates.io publish (`cargo install dreamstack`) +- [ ] Self-hosted playground deployment --- @@ -210,116 +191,10 @@ A working playground at `dreamstack.dev` where users can write DreamStack code a | Component | Language | Rationale | |---|---|---| -| Parser, Analyzer, Codegen, Type Checker | **Rust** | Performance, WASM target, memory safety without GC | -| Signal Runtime, Constraint Solver, Springs | **Rust → WASM** | Must run at 60fps, no GC pauses | -| DOM Bindings, RAF Scheduler | **JavaScript** | Must interact with browser APIs directly | -| CLI | **Rust** (clap) | Fast startup, single binary distribution | -| Editor | **TypeScript** | CodeMirror/Monaco ecosystem, web-native | -| Dev Server | **Rust** (axum) | Fast file watching, WebSocket HMR | - ---- - -## Key Technical Risks - -| Risk | Mitigation | -|---|---| -| Compile-time signal analysis is undecidable in general | Restrict to analyzable patterns; fall back to runtime tracking for dynamic cases | -| Cassowary solver is O(n²) worst case | Incremental solving; partition independent constraint groups | -| WASM ↔ JS boundary overhead for DOM updates | Batch mutations; use `FinalizationRegistry` for memory management | -| Algebraic effects require delimited continuations | Use CPS transform in codegen; or one-shot continuations via WASM stack switching (Stage 3 proposal) | -| Homoiconic representation vs performance | Two representations: rich AST for editor, optimized IR for runtime | - ---- - -## Verification Plan - -Since this is a greenfield language/framework, verification is demo-driven: - -### Phase 0–1: Reactive Core -```bash -# Build the compiler and run the counter example -cargo build --release -p ds-cli -./target/release/dreamstack build examples/counter.ds -# Serve and verify in browser -./target/release/dreamstack dev examples/counter.ds -# Run signal propagation benchmarks -cargo bench -p ds-signals -``` - -### Phase 2: Effects & Streams -```bash -# Run effect system unit tests -cargo test -p ds-effects -# Run the search example with mock handler (automated test) -./target/release/dreamstack test examples/search.ds -``` - -### Phase 3: Layout & Animation -```bash -# Constraint solver correctness tests -cargo test -p ds-layout -# Visual regression: screenshot comparison of dashboard example -./target/release/dreamstack screenshot examples/dashboard.ds --compare snapshots/ -``` - -### Phase 5: Live Editor -- **Manual verification:** Open the playground, type code, verify preview updates <50ms -- **Bidirectional test:** Drag an element in preview, verify code updates correspondingly - -> [!IMPORTANT] -> The user should weigh in on which phase to prioritize first. The plan assumes linear progression, but Phases 1–3 could be parallelized if multiple contributors are involved. Phase 4 (dependent types) is the highest-risk and could be deferred to a v2. - ---- - -## Phase 10 — Rapier2D Physics Engine (Completed) - -**Goal:** Replace the Verlet spring-mass physics with Rapier2D for real rigid body simulation. - -### WASM Physics Engine - -#### [NEW] `engine/ds-physics/` -- Rapier2D wrapped in a `PhysicsWorld` struct, compiled to WASM via `wasm-pack` -- Rigid body creation: `create_soft_circle()`, `create_soft_rect()` -- Force control: `set_gravity()` (wakes all sleeping bodies), `apply_impulse()` -- Query: `get_body_center()`, `get_body_rotation()`, `get_body_positions()`, `body_count()` -- Rendering: `set_body_color()`, body info with color, radius, dimensions -- Collision, restitution, friction, damping — all configurable per body -- 5 tests verifying world creation, stepping, body creation, gravity, and impulse - -### Deliverable -The WASM physics module at `dist/pkg/ds_physics.js` + `ds_physics_bg.wasm`, loadable from any HTML page. - ---- - -## Phase 11 — Physics in the Language (Completed) - -**Goal:** Integrate Rapier2D physics as a first-class `.ds` language construct. - -### Compiler Additions - -#### [MODIFY] `compiler/ds-parser/src/ast.rs` -- Added `ContainerKind::Scene` - -#### [MODIFY] `compiler/ds-parser/src/lexer.rs` -- Added `TokenKind::Scene` and `"scene"` keyword mapping - -#### [MODIFY] `compiler/ds-parser/src/parser.rs` -- `scene` dispatches to `parse_container_with_props` (width, height, gravity) -- `circle`, `rect`, `line` added to `is_ui_element()` - -#### [MODIFY] `compiler/ds-analyzer/src/signal_graph.rs` -- `Scene` variant in `collect_bindings` for DOM binding extraction - -#### [MODIFY] `compiler/ds-codegen/src/js_emitter.rs` -- `emit_scene()` — canvas creation, async WASM init, `PhysicsWorld` setup -- `emit_scene_circle()`, `emit_scene_rect()` — per-body codegen with unique variables -- `emit_scene_set_color()` — compile-time hex color parsing -- Reactive gravity via `DS.effect(() => { _world.set_gravity(...); })` -- `requestAnimationFrame` loop for physics stepping + canvas rendering -- Mouse drag interaction with impulse-based body movement - -#### [NEW] `examples/physics.ds` -- 5 bodies (circles + rects), reactive gravity controls, drag interaction - -### Deliverable -`physics.ds` compiles to a working HTML page with WASM physics — 22KB output, no framework dependencies. +| Parser, Analyzer, Codegen, Types | **Rust** | Performance, WASM target, memory safety | +| Signal Runtime, Springs, Constraints | **JavaScript** | Direct DOM access, ~7KB total | +| Physics Engine | **Rust → WASM** | Rapier2D, 60fps rigid body simulation | +| CLI | **Rust** (clap) | Single binary, fast startup | +| Dev Server | **Rust** (tiny_http) | File watching, HMR, HTTP serving | +| Playground | **JavaScript** | Monaco editor integration | +| Stream Relay | **Rust** (tokio + tungstenite) | Async WebSocket server |