amir/dreamstack
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: comprehensive documentation update

Browse source 
- DREAMSTACK.md: rewritten with accurate counts (48 examples, 14
  components, 136 tests), full CLI reference, architecture diagrams,
  quick start guide, comparison table, and phased roadmap
- IMPLEMENTATION_PLAN.md: rewritten with all 10 phases showing
  accurate completion status, current capabilities, and next steps
- BITSTREAM_INTEGRATION.md: updated test count (82 → 136)
- USE_CASES.md and STREAM_COMPOSITION.md: already current, unchanged
This commit is contained in:
enzotar 2026-02-27 11:15:54 -08:00
parent d4c7ba2385
commit ebf11889a3
3 changed files with 545 additions and 692 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
BITSTREAM_INTEGRATION.md
View file

@ -18,7 +18,7 @@ All changes in this spec have been implemented. Status per change:
| 8. CLI | `dreamstack stream` command | ✅ Done | `main.rs` | | 8. CLI | `dreamstack stream` command | ✅ Done | `main.rs` |
| 9. Layout | `to_bytes`/`from_bytes` on `LayoutRect` | ✅ Done | `solver.rs` | | 9. Layout | `to_bytes`/`from_bytes` on `LayoutRect` | ✅ Done | `solver.rs` |
**Test counts**: 82 tests passing across full workspace (ds-stream: 38, ds-parser: 15, ds-analyzer: 6, ds-types: 11, plus others). **Test counts**: 136 tests passing across full workspace (ds-stream: 38, ds-parser: 15, ds-analyzer: 6, ds-types: 11, ds-codegen: 6, ds-layout: 5, plus integration tests). 48 `.ds` examples compile successfully.
## Background ## Background

812
DREAMSTACK.md
View file

@ -6,7 +6,7 @@
## Implementation Status ✅ ## Implementation Status ✅
DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB output + WASM physics. DreamStack is **real and running** — 7 Rust crates, 136 tests, 48 compilable examples, 14 registry components, ~7KB runtime.
``` ```
.ds source → ds-parser → ds-analyzer → ds-codegen → JavaScript .ds source → ds-parser → ds-analyzer → ds-codegen → JavaScript
@ -20,27 +20,116 @@ DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB
| Feature | Syntax | Status | | Feature | Syntax | Status |
|---------|--------|--------| |---------|--------|--------|
| Signals | `let count = 0` | ✅ Fine-grained, auto-tracked | | Signals | `let count = 0` | ✅ Fine-grained, auto-tracked |
| Derived | `let doubled = count * 2` | ✅ Lazy | | Derived | `let doubled = count * 2` | ✅ `DS.derived()`, lazy evaluation |
| Interpolation | `"Count: {count}"` | ✅ Reactive | | Interpolation | `"Count: {count}"` | ✅ Reactive `DS.effect()` |
| Conditional | `when count > 5 -> text "hi"` | ✅ Mount/unmount | | Conditional | `when count > 5 -> text "hi"` | ✅ Mount/unmount |
| If/else | `if x then a else b` | ✅ | | If/else | `if x then a else b` | ✅ |
| Match | `match state \| Loading -> ...` | ✅ | | Match | `match state \| Loading -> ...` | ✅ |
| List rendering | `for item in items -> text item` | ✅ Reactive | | List rendering | `for item in items -> text item` | ✅ Reactive, keyed |
| Components | `component Card(title) = ...` | ✅ Props | | Components | `component Card(title) = ...` | ✅ Reactive props (getter fns) |
| Component import | `import { Badge } from "./badge"` | ✅ Recursive resolution |
| Effects | `effect fetch(id): Result` / `perform` | ✅ Algebraic | | Effects | `effect fetch(id): Result` / `perform` | ✅ Algebraic |
| Streams | `debounce`, `throttle`, `distinct` | ✅ | | Springs | `spring(target: 0, stiffness: 300)` | ✅ RK4 physics |
| Springs | `spring(target: 0, stiffness: 300)` | ✅ Physics | | Layout | `layout { sidebar.width == 250 }` | ✅ Cassowary solver |
| Layout | Cassowary constraint solver | ✅ | | Types | `type PositiveInt = Int where x > 0` | ✅ Refinement types |
| Types | `Signal<Int>`, `Derived<Bool>` | ✅ Hindley-Milner |
| Dev server | `dreamstack dev app.ds` | ✅ HMR |
| Streaming (source) | `stream main on "ws://..."` | ✅ Signal diffs |
| Streaming (receiver) | `stream from "ws://..."` | ✅ Auto-reconnect |
| Router | `route "/path" -> body` / `navigate` | ✅ Hash-based | | Router | `route "/path" -> body` / `navigate` | ✅ Hash-based |
| Two-way binding | `input { bind: name }` | ✅ Signal ↔ input | | Two-way binding | `input { bind: name }` | ✅ Signal ↔ input |
| Async resources | `DS.resource()` / `DS.fetchJSON()` | ✅ Loading/Ok/Err | | Async resources | `DS.resource()` / `DS.fetchJSON()` | ✅ Loading/Ok/Err |
| Springs | `let x = spring(200)` | ✅ RK4 physics |
| Physics scene | `scene { gravity_y: g } [ circle {...} ]` | ✅ Rapier2D WASM | | Physics scene | `scene { gravity_y: g } [ circle {...} ]` | ✅ Rapier2D WASM |
| Constraints | `constrain el.width = expr` | ✅ Reactive solver | | Constraints | `constrain el.width = expr` | ✅ Reactive solver |
| Streaming (signal) | `stream on "ws://..." { mode: signal }` | ✅ Bidirectional, versioned diffs |
| Streaming (receiver) | `stream from "ws://..."` | ✅ Auto-reconnect, field select |
| Streaming (WebRTC) | `transport: webrtc` | ✅ P2P data channel + WS fallback |
| Stream output filter | `output: paddleX, ballY, score` | ✅ Skip internal signals |
| Dev server | `dreamstack dev app.ds` | ✅ File watcher + HMR polling |
| Playground | `dreamstack playground` | ✅ Monaco editor + live preview |
| CLI check | `dreamstack check app.ds` | ✅ Signal graph visualization |
| TSX converter | `dreamstack convert component.tsx` | ✅ React → DreamStack |
| Component registry | `dreamstack add badge` | ✅ 14 components |
| Timer merging | `every 33 -> ...` (multiple) | ✅ Single `setInterval` per interval |
| Sound | `play_tone(440, 60)` | ✅ Built-in oscillator |
### CLI Commands
```bash
dreamstack build app.ds # Compile to HTML+JS
dreamstack dev app.ds # Dev server with hot reload (port 3000)
dreamstack check app.ds # Analyze signal graph, type check
dreamstack stream app.ds # Compile + serve with streaming
dreamstack playground # Monaco editor playground (port 4000)
dreamstack add badge # Add a registry component
dreamstack add --list # List all available components
dreamstack convert file.tsx # Convert React/TSX → DreamStack
dreamstack convert button --shadcn # Convert from shadcn/ui registry
dreamstack init my-app # Initialize new project
```
### Registry Components (14)
| Component | File | Description |
|-----------|------|-------------|
| Alert | `alert.ds` | Status messages with variants |
| Avatar | `avatar.ds` | Profile images with sizes |
| Badge | `badge.ds` | Status indicators (success/warning/error/info) |
| Button | `button.ds` | Click actions with variants |
| Card | `card.ds` | Content containers |
| Dialog | `dialog.ds` | Modal overlays |
| Input | `input.ds` | Text inputs with binding |
| Progress | `progress.ds` | Progress bars |
| Select | `select.ds` | Dropdown selects |
| Separator | `separator.ds` | Visual dividers |
| Stat | `stat.ds` | Statistic displays (value, label, trend) |
| Tabs | `tabs.ds` | Tabbed navigation |
| Toast | `toast.ds` | Notification toasts |
| Toggle | `toggle.ds` | On/off switches |
### Examples (48 compilable .ds files)
**Games:**
- `game-pong.ds` — Multiplayer Pong with keyboard, sound, streaming
- `game-breakout.ds` — Breakout with 5 rows, collision, score, streaming
- `game-snake.ds` — Snake game
- `game-reaction.ds` — Reaction time test
**Streaming:**
- `streaming-counter.ds` — Counter synced across tabs
- `streaming-clock.ds` — Clock streamed to viewers
- `streaming-dashboard.ds` — Real-time dashboard via relay
- `streaming-stats.ds` — Signal stats viewer
- `streaming-mood.ds` — Collaborative mood board
- `streaming-physics.ds` — Physics scene streaming
- `streaming-webrtc.ds` — P2P WebRTC streaming
- `streaming-receiver.ds` — Generic stream receiver
- `pong-viewer.ds` — Spectator view for Pong
- `game-viewer.ds` — Generic game viewer
**Audio:**
- `step-sequencer.ds` — 16-step drum machine with Web Audio API
- `beats-viewer.ds` — Spectator view for step sequencer
**UI Patterns:**
- `counter.ds` — 3-line counter
- `list.ds` — Dynamic list with add/remove
- `todo.ds` / `todomvc.ds` — TodoMVC
- `form.ds` — Form with validation
- `router-demo.ds` — Multi-page routing
- `dashboard.ds` — Dashboard layout
- `component-gallery.ds` — Component showcase
- `showcase.ds` — Full feature showcase
- `project-manager.ds` — Kanban-style project tracker
- `springs.ds` — Spring physics animations
**Language Features:**
- `language-features.ds` — Comprehensive syntax demo
- `refined-types.ds` — Refinement type guards
- `import-demo.ds` — Component imports
- `slot-demo.ds` — Component children/slots
- `each-demo.ds` — `each` loop rendering
- `when-else-demo.ds` — Conditional rendering
- `multi-action.ds` / `timer-multi-action.ds` — Multi-statement handlers
- `callback-demo.ds` — Callback props
- `physics.ds` — Rapier2D physics scene
- `bench-signals.ds` — Signal performance benchmarks
### DreamStack vs React ### DreamStack vs React
@ -57,9 +146,10 @@ DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB
| Animation | Built-in springs | framer-motion (external) | | Animation | Built-in springs | framer-motion (external) |
| Physics | Built-in Rapier2D scene | matter.js (external) | | Physics | Built-in Rapier2D scene | matter.js (external) |
| Layout | Built-in Cassowary | CSS only | | Layout | Built-in Cassowary | CSS only |
| Types | Native HM, `Signal<T>` | TypeScript (external) | | Streaming | Built-in bidirectional | WebSocket libraries (external) |
| Types | Native refinement types | TypeScript (external) |
| Bundle | **~7KB** | **~175KB** | | Bundle | **~7KB** | **~175KB** |
| Ecosystem | New | Massive | | Ecosystem | 48 examples, 14 components | Massive |
### Benchmarks (signal propagation) ### Benchmarks (signal propagation)
@ -71,9 +161,157 @@ DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB
| Batch Update (50) | 61K | 50 | | Batch Update (50) | 61K | 50 |
| Mixed Read/Write | 242K | 10 | | Mixed Read/Write | 242K | 10 |
### Examples ---
`counter.ds` · `list.ds` · `router.ds` · `form.ds` · `springs.ds` · `physics.ds` · `todomvc.html` · `search.html` · `dashboard.html` · `playground.html` · `showcase.html` · `benchmarks.html` ## Quick Start
### Counter (3 lines of logic)
```
let count = 0
view counter =
column [
text "Count: {count}"
button "+" { click: count += 1 }
]
```
### Multiplayer Pong (40 lines)
```
let ballX = 300
let ballY = 200
let bvx = 4
let bvy = 3
let p1y = 180
on keydown(ev) -> p1y = if ev.key == "ArrowUp" then p1y - 30 else p1y
every 33 -> ballX = ballX + bvx
every 33 -> ballY = ballY + bvy
every 33 -> bvx = if ballX > 590 then -4 else bvx
every 33 -> bvy = if ballY < 5 then 3 else bvy
stream pong on "ws://localhost:9100/peer/pong" {
mode: signal,
output: ballX, ballY, p1y
}
view pong =
stack { style: "position:relative; width:600px; height:400px; background:#1a1a2e" } [
column { style: "...ball styles...", top: ballY, left: ballX } []
column { style: "...paddle...", top: p1y } []
]
```
### Spectator View (5 lines)
```
let game = stream from "ws://localhost:9100/peer/pong"
select [ballX, ballY, p1y]
view viewer =
stack { style: "..." } [
column { style: "...ball...", top: game.ballY, left: game.ballX } []
column { style: "...paddle...", top: game.p1y } []
]
```
### Run It
```bash
# Install
cargo install --path compiler/ds-cli
# Build
dreamstack build examples/counter.ds -o dist
# Dev server (with hot reload)
dreamstack dev examples/game-pong.ds
# With streaming
cargo run -p ds-stream # Tab 1: start relay
dreamstack dev examples/game-pong.ds # Tab 2: play
open dist/index.html # Tab 3: spectate (pong-viewer.ds)
```
---
## Architecture
### Compiler Pipeline
```
┌─────────────┐
.ds source ───────►│ ds-parser │──► AST (Program)
└──────┬──────┘
│
┌──────▼──────┐
│ ds-analyzer │──► SignalGraph (DAG)
└──────┬──────┘ + DomBindings
│ + SignalManifest
┌──────▼──────┐
│ ds-codegen │──► Single-file HTML+JS
└──────┬──────┘ (~7KB runtime)
│
┌──────▼──────┐
│ ds-cli │──► dev/build/stream/check
└─────────────┘
```
### Signal Graph (Compile-Time)
```
Signal: count (Source)
├──► Derived: doubled (DS.derived)
│ └──► [DOM] TextNode "Doubled: {doubled}"
├──► [DOM] TextNode "Count: {count}"
└──► [Stream] _streamDiff("count", count.value)
```
- **No virtual DOM.** Compiler maps signals → specific DOM nodes at build time
- **No re-rendering.** Components execute once, set up reactive bindings
- **No dependency arrays.** Compiler infers all dependencies from code
- **Timer merging.** All `every 33` statements share one `setInterval`
### Reactive Runtime
```javascript
// Source signal
const count = DS.signal(0);
// Derived signal (auto-updates)
const doubled = DS.derived(() => count.value * 2);
// Effect (auto-tracked, re-runs on dependency change)
DS.effect(() => { el.textContent = `Count: ${count.value}`; });
// Component props: getter functions for live reactivity
DS_Badge({ label: () => `Score: ${score.value}` });
```
### Stream Protocol (16-byte binary header)
```
┌────────┬─────────┬──────────┬────────────┬───────┬────────┬────────┐
│ type │ flags │ seq │ timestamp │ width │ height │ length │
│ u8 │ u8 │ u16 │ u32 │ u16 │ u16 │ u32 │
└────────┴─────────┴──────────┴────────────┴───────┴────────┴────────┘
```
| Mode | What's Sent | Bandwidth |
|------|-------------|-----------|
| `signal` (default) | JSON diffs of changed signals | ~2 KB/s |
| `delta` | XOR + RLE compressed pixel deltas | ~50 KB/s |
| `pixel` | Raw RGBA framebuffer | ~30 MB/s |
Features:
- **Bidirectional sync** with per-signal version counters
- **Output filtering** — only listed signals are broadcast
- **Exponential reconnect** — 2s → 4s → 8s, max 30s, reset on success
- **WebRTC data channel** with WebSocket fallback
- **Deduplication** — skip unchanged values
--- ---
@ -94,471 +332,211 @@ Not Clojure. Not TypeScript. A new language that steals the best ideas from ever
| Property | Inspiration | Why | | Property | Inspiration | Why |
|---|---|---| |---|---|---|
| **Homoiconic** | Clojure, Lisp | UI = data. Code = data. Everything is transformable | | **Homoiconic** | Clojure, Lisp | UI = data. Code = data. Everything is transformable |
| **Dependent types** | Idris, Agda | Types that express "this button is disabled *when* the form is invalid" at the type level | | **Refinement types** | Liquid Haskell | `type Percent = Int where value >= 0 and value <= 100` |
| **Algebraic effects** | Eff, Koka, OCaml 5 | Side effects as first-class, composable, interceptable values | | **Algebraic effects** | Eff, Koka, OCaml 5 | Side effects as first-class, composable, interceptable values |
| **Reactive by default** | Svelte, Solid, Excel | No `useState`, no subscriptions. Values *are* reactive. Assignment *is* the API | | **Reactive by default** | Svelte, Solid, Excel | No `useState`, no subscriptions. Assignment *is* the API |
| **Structural typing** | TypeScript, Go | Flexible composition without class hierarchies | | **Structural typing** | TypeScript, Go | Flexible composition without class hierarchies |
| **Compiled to native + WASM** | Rust, Zig | Near-zero runtime overhead. No GC pauses during animations | | **Compiled to JS** | Svelte, Solid | Near-zero runtime overhead |
### Syntax ### Syntax Reference
``` ```
-- Signals are the primitive. Assignment propagates automatically. -- Signals (mutable state)
let count = 0 let count = 0
let doubled = count * 2 -- derived, auto-tracked let name = "world"
let label = "Count: {doubled}" -- derived, auto-tracked let items = [1, 2, 3]
-- UI is data. No JSX, no templates, no virtual DOM. -- Derived signals (auto-updated)
view counter = let doubled = count * 2
let greeting = "Hello, {name}!"
-- Refinement types
type PositiveInt = Int where value > 0
let score: PositiveInt = 10
-- Event handlers
on keydown(ev) -> count = if ev.key == "ArrowUp" then count + 1 else count
-- Timers
every 33 -> ballX = ballX + velocity
-- Conditional rendering
when count > 10 -> text "High!"
-- Pattern matching
match state
"loading" -> spinner
"error" -> text "Failed"
_ -> text "Ready"
-- For loops
for item, i in items -> text "{i}: {item}"
-- Components
component Card(title, children) =
column { variant: "card" } [
text title { variant: "card-title" }
slot
]
-- Imports
import { Badge, Card } from "../registry/components/badge"
-- Routes
route "/" -> text "Home"
route "/about" -> text "About"
-- Springs
let x = spring(target: 0, stiffness: 300, damping: 30)
-- Streaming
stream app on "ws://localhost:9100/peer/app" {
mode: signal,
output: count, name
}
-- Receiving streams
let remote = stream from "ws://localhost:9100/peer/app"
select [count, name]
-- Views
view main =
column [ column [
text label -- auto-updates when count changes text "Hello {name}"
button "+" { click: count += 1 } button "Click" { click: count += 1 }
-- Conditional UI is just pattern matching on signals
when count > 10 ->
text "🔥 On fire!" | animate fade-in 200ms
] ]
``` ```
No hooks. No dependency arrays. No re-renders. **The compiler builds a fine-grained reactive graph at compile time.**
--- ---
## Reactivity: Compile-Time Signal Graph ## Effect System: Algebraic Effects
This is the biggest departure from React. React's model is fundamentally **pull-based** — re-render the tree, diff it, patch the DOM. That's backwards.
### Push-Based with Compile-Time Analysis
``` ```
Signal: count
├──► Derived: doubled
│ └──► Derived: label
│ └──► [direct DOM binding] TextNode #47
└──► Condition: count > 10
└──► [mount/unmount] text "🔥 On fire!"
```
**Key principles:**
- **No virtual DOM.** The compiler knows at build time exactly which DOM nodes depend on which signals. When `count` changes, it updates *only* the specific text node and evaluates the condition. Nothing else runs.
- **No re-rendering.** Components don't re-execute. They execute *once* and set up reactive bindings.
- **No dependency arrays.** The compiler infers all dependencies from the code. You literally cannot forget one.
- **No stale closures.** Since there are no closures over render cycles, the entire class of bugs vanishes.
This extends Solid.js's approach with full compile-time analysis in a purpose-built language.
---
## Effect System: Algebraic Effects for Everything
Every side-effect — HTTP, animations, time, user input, clipboard, drag-and-drop — is an **algebraic effect**. Effects are values you can compose, intercept, retry, and test.
```
-- An effect is declared, not executed
effect fetchUser(id: UserId): Result<User, ApiError> effect fetchUser(id: UserId): Result<User, ApiError>
-- A component "performs" an effect — doesn't control HOW it runs
view profile(id: UserId) = view profile(id: UserId) =
let user = perform fetchUser(id) let user = perform fetchUser(id)
match user match user
Loading -> skeleton-loader Loading -> skeleton-loader
Ok(u) -> column [ Ok(u) -> column [ avatar u.photo, text u.name ]
avatar u.photo
text u.name
]
Err(e) -> error-card e | with retry: perform fetchUser(id) Err(e) -> error-card e | with retry: perform fetchUser(id)
-- At the app boundary, you provide the HANDLER
handle app with
fetchUser(id, resume) ->
let result = http.get "/api/users/{id}"
resume(result)
``` ```
### Why This is Powerful
- **Testing:** Swap the handler. `handle app with mockFetchUser(...)` — no mocking libraries, no dependency injection frameworks.
- **Composition:** Effects compose naturally. An animation effect + a data-fetch effect combine without callback hell or `useEffect` chains.
- **Interceptors:** Want to add logging, caching, or retry logic? Add a handler layer. The component code never changes.
- **Time travel:** Since effects are values, you can record and replay them. Free undo/redo. Free debugging.
--- ---
## Data Flow: Everything is a Stream ## Layout: Constraint-Based
Forget the distinction between "state", "props", "events", and "side effects". **Everything is a stream of values over time.**
``` ```
-- User input is a stream layout dashboard {
let clicks = stream from button.click sidebar.width == 250
let keypresses = stream from input.keydown main.x == sidebar.x + sidebar.width
main.width == parent.width - sidebar.width [strong]
-- Derived streams with temporal operators }
let search_query = keypresses
| map .value
| debounce 300ms
| distinct
-- API results are streams
let results = search_query
| flatmap (q -> http.get "/search?q={q}")
| catch-with []
-- UI binds to streams directly
view search =
column [
input { on-keydown: keypresses }
match results
Loading -> spinner
Data(rs) -> list rs (r -> search-result-card r)
Empty -> text "No results"
]
``` ```
### Unification Uses a Cassowary constraint solver. Constraints are reactive — when viewport changes, layout re-solves in a single pass.
This single abstraction covers everything:
| Concept | Traditional | DreamStack |
|---|---|---|
| User input | Event handlers | Stream |
| Network responses | Promises / async-await | Stream |
| Animations | CSS transitions / JS libraries | Stream of interpolated values |
| Timers | `setInterval` / `setTimeout` | Stream |
| WebSockets | Callback-based | Stream |
| Drag events | Complex event handler state machines | Stream of positions |
One abstraction. One composition model. Everything snaps together.
--- ---
## Layout: Constraint-Based, Not Box-Based ## Animation: Physics-Based Springs
CSS's box model is from 1996. Flexbox and Grid are patches on a fundamentally limited system. DreamStack starts over with a **constraint solver**.
``` ```
-- Declare relationships, not boxes
layout dashboard =
let sidebar.width = clamp(200px, 20vw, 350px)
let main.width = viewport.width - sidebar.width
let header.height = 64px
let content.height = viewport.height - header.height
-- Constraints, not nesting
sidebar.left = 0
sidebar.top = header.height
sidebar.height = content.height
main.left = sidebar.right
main.top = header.height
main.right = viewport.right
-- Responsive is just different constraints
when viewport.width < 768px ->
sidebar.width = 0 -- collapses
main.left = 0 -- takes full width
```
Inspired by Apple's **Auto Layout** (Cassowary constraint solver), but made reactive. When `viewport.width` changes, the constraint solver re-solves and updates positions in a single pass. No layout thrashing. No "CSS specificity wars."
### Advantages Over CSS
- **No cascade conflicts** — constraints are explicit and local
- **No z-index hell** — layering is declarative
- **No media query breakpoints** — responsiveness emerges from constraints
- **Animations are free** — animating a constraint target is the same as setting it
---
## Animation: First-Class, Physics-Based, Interruptible
Animations aren't CSS transitions bolted on after the fact. They're part of the reactive graph.
```
-- A spring is a signal that animates toward its target
let panel_x = spring(target: 0, stiffness: 300, damping: 30) let panel_x = spring(target: 0, stiffness: 300, damping: 30)
-- Change the target → the spring animates automatically
on toggle_sidebar -> on toggle_sidebar ->
panel_x.target = if open then 250 else 0 panel_x.target = if open then 250 else 0
-- Gestures feed directly into springs
on drag(event) ->
panel_x.target = event.x -- spring follows finger
on drag-end(event) ->
panel_x.target = snap-to-nearest [0, 250] event.x
-- UI just reads the spring's current value
view sidebar = view sidebar =
panel { x: panel_x } [ panel { x: panel_x } [ nav-items ]
nav-items ```
Springs are interruptible, gesture-driven, and composable through the same signal system.
---
## Physics: Rapier2D WASM
```
let gravity_y = 980
view main =
scene { width: 700, height: 450, gravity_y: gravity_y } [
circle { x: 200, y: 80, radius: 35, color: "#8b5cf6" }
rect { x: 500, y: 100, width: 80, height: 50, color: "#10b981" }
] ]
button "Anti-Gravity" { click: gravity_y = -500 }
``` ```
### Design Principles
- **Interruptible:** Start a new animation mid-flight. The spring handles the physics — no jarring jumps, no "wait for the current animation to finish."
- **Gesture-driven:** Touch/mouse input feeds directly into the animation model. No separate "gesture handler" → "state update" → "CSS transition" pipeline.
- **60fps guaranteed:** Springs resolve on the GPU. The main thread never blocks.
- **Composable:** Combine springs, easing curves, and physics simulations using the same stream operators as everything else.
--- ---
## Runtime: Tiny, Compiled, No GC Pauses ## Streaming: Built-In Multiplayer
Any DreamStack app becomes multiplayer with one declaration:
``` ```
┌──────────────────────────────────────────┐ stream pong on "ws://localhost:9100/peer/pong" {
│ Compiled Output │ mode: signal,
├──────────────────────────────────────────┤ output: ballX, ballY, paddleY, score
│ Signal Graph (static DAG) ~2KB │ }
│ Constraint Solver (layout) ~4KB │
│ Spring Physics (animations) ~1KB │
│ Effect Runtime (handlers) ~2KB │
│ DOM Patcher (surgical updates) ~3KB │
├──────────────────────────────────────────┤
│ Total Runtime: ~12KB │
│ (vs React ~45KB + ReactDOM ~130KB) │
└──────────────────────────────────────────┘
``` ```
The compiler does the heavy lifting. The runtime is tiny because there is: The compiler automatically:
1. Connects to the relay via WebSocket
2. Wraps signal mutations with `_streamDiff()` calls
3. Sends versioned JSON diffs (conflict resolution via version counters)
4. Filters to only broadcast `output` signals (skips internals like velocity)
5. Reconnects with exponential backoff on disconnect
- No virtual DOM diffing algorithm Viewers connect with `stream from`:
- No fiber scheduler ```
- No hook state management system let game = stream from "ws://localhost:9100/peer/pong"
- No reconciliation algorithm select [ballX, ballY, paddleY, score]
- No garbage collector pauses during animation frames ```
--- ---
## The Killer Feature: Live Structural Editing ## Comparison
Because the language is homoiconic (code = data), the **editor IS the runtime:**
```
┌─────────────────────────────────┐
│ Live Editor │
│ │
│ view counter = │ ← Edit this...
│ column [ │
│ text "Count: {count}" │
│ button "+" { ... } │
│ ] │
│ │
│ ───────────────────────────── │
│ │
│ ┌─────────────┐ │ ← ...see this update
│ │ Count: 42 │ │ instantly, with state
│ │ [ + ] │ │ preserved.
│ └─────────────┘ │
│ │
└─────────────────────────────────┘
```
- Drag and drop UI elements in the preview → the **code updates**.
- Edit the code → the **preview updates**.
- Both directions, simultaneously, with state preserved.
- Not a design tool that generates code — **the code IS the design tool**.
This is possible because:
1. **Homoiconicity** means UI structure is inspectable and modifiable data
2. **Immutable signals** mean state survives code changes
3. **Compile-time signal graph** means changes are surgical, not full-page reloads
---
## Comparison to Existing Approaches
| Capability | React | Svelte 5 | Solid.js | Flutter | **DreamStack** | | Capability | React | Svelte 5 | Solid.js | Flutter | **DreamStack** |
|---|---|---|---|---|---| |---|---|---|---|---|---|
| Reactivity | Pull (re-render + diff) | Compile-time runes | Fine-grained signals | Widget rebuild | **Compile-time signal DAG** | | Reactivity | Pull (VDOM diff) | Runes | Signals | Rebuild | **Compile-time DAG** |
| Side effects | `useEffect` + deps array | `$effect` | `createEffect` | Lifecycle methods | **Algebraic effects** | | Side effects | `useEffect` + deps | `$effect` | `createEffect` | Lifecycle | **Algebraic effects** |
| Layout | CSS (external) | CSS (scoped) | CSS (external) | Constraint-based | **Constraint solver (native)** | | Layout | CSS (external) | CSS (scoped) | CSS (external) | Constraints | **Cassowary (native)** |
| Animation | 3rd party libs | 3rd party libs | 3rd party libs | Physics-based (native) | **Physics-based (native)** | | Animation | 3rd party | 3rd party | 3rd party | Physics (native) | **Springs (native)** |
| SSR | Yes (complex) | Yes | Yes | No (web) | **Yes (compiled)** | | Streaming | WebSocket libs | WebSocket libs | WebSocket libs | None | **Built-in bidirectional** |
| Runtime size | ~175KB | ~2KB | ~7KB | ~2MB (web) | **~12KB** | | Runtime | ~175KB | ~2KB | ~7KB | ~2MB | **~7KB** |
| Live editing | Fast Refresh (lossy) | HMR | HMR | Hot reload | **Bidirectional structural** | | Type safety | TypeScript | TypeScript | TypeScript | Dart | **Refinement types** |
| Type safety | TypeScript (bolt-on) | TypeScript (bolt-on) | TypeScript (bolt-on) | Dart (native) | **Dependent types (native)** | | UI = Data | No (JSX) | No (templates) | No (JSX) | No (widgets) | **Yes (homoiconic)** |
| UI = Data | No (JSX compiled away) | No (templates) | No (JSX compiled) | No (widget classes) | **Yes (homoiconic)** |
---
## Fragments of the Future, Today
The closest approximations to pieces of this vision:
- **Solid.js** — fine-grained reactivity, no VDOM, ~7KB runtime
- **Svelte 5 Runes** — compiler-driven reactivity, tiny output
- **Elm** — algebraic effects-adjacent, immutable state, strong types
- **Flutter** — constraint layout, physics-based animation, hot reload
- **Clojure/Reagent** — homoiconicity, UI as data, ratom reactivity
- **Koka** — algebraic effect system in a practical language
- **Apple Auto Layout** — Cassowary constraint solver for UI
- **Excel** — reactive by default, dependency auto-tracking
Nobody has unified them. That's the opportunity.
--- ---
## Design Philosophy ## Design Philosophy
1. **The compiler is the framework.** Move work from runtime to compile time. The less code that runs in the browser, the faster the UI. 1. **The compiler is the framework.** Move work from runtime to compile time
2. **Reactivity is not a feature, it's the default.** Every value is live. Every binding is automatic. You opt *out* of reactivity, not *in*. 2. **Reactivity is the default.** Every value is live. You opt *out*, not *in*
3. **Effects are values, not side-channels.** Making side effects first-class and composable eliminates the largest source of bugs in modern UIs. 3. **Effects are values.** First-class, composable, interceptable
4. **Layout and animation are not afterthoughts.** They're core primitives, not CSS bolt-ons or third-party libraries. 4. **Layout and animation are core.** Not CSS bolt-ons or third-party libraries
5. **The editor and the runtime are the same thing.** Bidirectional editing collapses the design-develop gap entirely. 5. **The editor and runtime are one.** Bidirectional structural editing
6. **UI is data, all the way down.** If you can't `map` over your UI structure, your abstraction is wrong. 6. **UI is data, all the way down.** If you can't `map` over your UI, your abstraction is wrong
7. **Any input bitstream → any output bitstream.** The UI is just one codec. Tomorrow's neural nets generate the pixels directly. 7. **Any bitstream in → any bitstream out.** The UI is just one codec
--- ---
## Phase 7: Universal Bitstream Streaming ## Next Steps
> *Stream the whole UI as bytes. Neural nets will generate the pixels, acoustics, and actuator commands.* ### Phase 1: Polish (estimated ~2h)
- [ ] **Better error messages** — source context with line + caret in parser errors
- [ ] **Route integration test** — multi-page example exercising `route` + `navigate`
- [ ] **WebRTC integration test** — verify P2P data channel transport
- [ ] **Layout constraint test** — example using Cassowary solver
DreamStack's `engine/ds-stream` crate implements a universal binary protocol for streaming any I/O: ### Phase 2: New Examples (estimated ~2h)
- [ ] **Tetris** — stress-test grid, timers, collision, streaming
- [ ] **Todo with sync** — CRUD + bidirectional streaming sync
- [ ] **Snake polish** — cleanup existing game, add streaming
``` ### Phase 3: Ecosystem (estimated ~1.5h)
┌──────────┐ WebSocket / WebRTC ┌──────────┐ - [ ] **Component gallery** — live showcase of all 14 components
│ Source │ ──────frames (bytes)──► │ Receiver │ - [ ] **Crates.io publish** — `cargo install dreamstack`
│ (renders) │ ◄──────inputs (bytes)── │ (~250 LOC)│ - [ ] **Self-hosted playground** — deploy playground as web app
└──────────┘ └──────────┘
```
### Binary Protocol (16-byte header)
| Field | Size | Description |
|-------|------|-------------|
| type | u8 | Frame/input type (pixels, audio, haptic, neural, BCI) |
| flags | u8 | Input flag, keyframe flag, compression flag |
| seq | u16 | Sequence number |
| timestamp | u32 | Relative ms since stream start |
| width | u16 | Frame width or channel count |
| height | u16 | Frame height or sample rate |
| length | u32 | Payload length |
### Output Types
- `Pixels` (0x01) — raw RGBA framebuffer
- `Audio` (0x10) — PCM audio samples
- `Haptic` (0x20) — vibration/actuator commands
- `NeuralFrame` (0x40) — neural-generated pixels *(future)*
- `NeuralAudio` (0x41) — neural speech/music synthesis *(future)*
- `NeuralActuator` (0x42) — learned motor control *(future)*
### Input Types
- `Pointer` (0x01) — mouse/touch position + buttons
- `Key` (0x10) — keyboard events
- `Gamepad` (0x30) — controller axes + buttons
- `BciInput` (0x90) — brain-computer interface *(future)*
### Demos
- `examples/stream-source.html` — Springs demo captures canvas → streams pixels at 30fps
- `examples/stream-receiver.html` — Thin client (~250 lines, no framework) renders bytes
### Run It
```bash
cargo run -p ds-stream # start relay on :9100
open examples/stream-source.html # source: renders + streams
open examples/stream-receiver.html # receiver: displays bytes
```
### Compiler-Native Streaming
The `stream` keyword makes any `.ds` app streamable with one line. The compiler generates all WebSocket connection, binary protocol encoding, and signal diff broadcasting automatically.
#### Source: `stream` declaration
```
let count = 0
let doubled = count * 2
stream counter on "ws://localhost:9100" { mode: signal }
view counter =
column [
text count
text doubled
button "+" { click: count += 1 }
]
```
The compiler:
1. Calls `DS._initStream(url, mode)` on load → connects to relay
2. Wraps every signal mutation with `DS._streamDiff("count", count.value)` → sends JSON diff frames
3. Sends scene body positions at 60fps when a physics scene is streaming
#### Receiver: `stream from` expression
```
let remote = stream from "ws://localhost:9100"
view main =
column [
text remote.count
text remote.doubled
]
```
Compiles to `DS._connectStream(url)` — returns a reactive `Signal` that merges incoming `FRAME_SIGNAL_SYNC` and `FRAME_SIGNAL_DIFF` frames.
#### Streaming Modes
| Mode | Keyword | What's Sent | Bandwidth |
|------|---------|-------------|----------|
| Signal | `signal` (default) | JSON diffs of changed signals | ~2 KB/s |
| Delta | `delta` | XOR + RLE compressed pixel deltas | ~50 KB/s |
| Pixel | `pixel` | Raw RGBA framebuffer every frame | ~30 MB/s |
#### CLI
```bash
# Compile and serve with streaming enabled
dreamstack stream app.ds --relay ws://localhost:9100 --mode signal
# Or explicitly declare streaming in the .ds file and use dev server
dreamstack dev app.ds
```
---
## Phase 8: Physics Scene — Rapier2D in the Language
> *Declare physics bodies in `.ds` syntax. The compiler generates WASM-backed canvas with rigid body simulation.*
```ds
let gravity_y = 980
view main =
column [
scene { width: 700, height: 450, gravity_y: gravity_y } [
circle { x: 200, y: 80, radius: 35, color: "#8b5cf6" }
circle { x: 350, y: 50, radius: 50, color: "#7c3aed" }
rect { x: 500, y: 100, width: 80, height: 50, color: "#10b981" }
]
button "Anti-Gravity" { click: gravity_y = -500 }
button "Normal" { click: gravity_y = 980 }
]
```
### Architecture
```
.ds source → parser (scene/circle/rect) → codegen → JS
↓
canvas + async WASM init
ds-physics (Rapier2D) ← pkg/ds_physics_bg.wasm
↓
requestAnimationFrame loop
step → render → repeat
```
### Features
- **Rigid body physics** — circles and rectangles with collision, rotation, restitution
- **Reactive gravity** — signal changes wrapped in `DS.effect()`, bodies wake on gravity change
- **Mouse drag** — click and drag bodies with impulse-based interaction
- **Compile-time colors** — hex color strings parsed at compile time → `set_body_color(r, g, b, a)`
- **Zero JS overhead** — physics runs in WASM, rendering in canvas, signals bridge both

423
IMPLEMENTATION_PLAN.md
View file

@ -1,208 +1,189 @@
# DreamStack Implementation Plan # DreamStack Implementation Plan
Build a working prototype of the DreamStack vision — a new UI framework with compile-time reactivity, algebraic effects, constraint layout, and physics animation. The strategy is to **bootstrap pragmatically**: use Rust for the compiler/runtime core, target WASM + DOM, and prove each pillar with working demos before unifying them. Build a working prototype of the DreamStack vision — a new UI framework with compile-time reactivity, algebraic effects, constraint layout, and physics animation. The strategy is to **bootstrap pragmatically**: use Rust for the compiler/runtime core, target JS + DOM, and prove each pillar with working demos before unifying them.
--- ---
## Phase 0 — Foundation (Weeks 1–3) ## Status Overview
**Goal:** A working compiler that turns DreamStack syntax into executable WASM + JS glue. | Phase | Goal | Status |
|-------|------|--------|
| 0 — Foundation | Compiler pipeline, CLI | ✅ Complete |
| 1 — Reactive Core | Signals, derived, effects, DOM bindings | ✅ Complete |
| 2 — Effects & Streams | Algebraic effects, stream operators | ✅ Complete |
| 3 — Layout & Animation | Cassowary constraints, spring physics | ✅ Complete |
| 4 — Type System | Refinement types, runtime guards | ✅ Partial (refinement types done, full HM deferred) |
| 5 — Live Editor | Playground, HMR dev server | ✅ Complete |
| 6 — Production | Component registry, imports, TSX converter | ✅ Complete |
| 7 — Bitstream Streaming | Source/receiver, relay, signal diffs | ✅ Complete |
| 8 — Physics Scene | Rapier2D WASM integration | ✅ Complete |
| 9 — Bidirectional Sync | Peer-to-peer, conflict resolution, WebRTC | ✅ Complete |
| 10 — Games & Audio | Pong, Breakout, Sequencer, Sound | ✅ Complete |
### Compiler Pipeline (Rust) **Current:** 7 Rust crates, 136 tests, 48 compilable `.ds` examples, 14 registry components.
#### [NEW] `compiler/crates/ds-parser/`
- Hand-written recursive descent parser for DreamStack syntax
- Produces an AST of `view`, `let`, `when`, `match`, `effect`, `on`, `stream`, `layout` constructs
- Homoiconic representation: AST nodes are also the runtime data structure (tagged vectors/maps)
#### [NEW] `compiler/crates/ds-analyzer/`
- **Signal graph extraction:** Static analysis pass that identifies signals, derived values, and their dependencies → builds a DAG
- **Effect collection:** Catalogs all `perform` sites and their expected effect signatures
- **Type inference:** Hindley-Milner base with structural typing; dependent types deferred to Phase 4
#### [NEW] `compiler/crates/ds-codegen/`
- Emits **Rust → WASM** for:
- Signal graph runtime (push-based notification)
- Constraint solver (Cassowary algorithm)
- Spring physics engine
- Emits **JS glue** for DOM bindings — surgical `textContent`, `setAttribute`, `insertBefore`, `removeChild` calls (no VDOM)
- Output format: single `.wasm` + single `.js` module
#### [NEW] `compiler/crates/ds-cli/`
- `dreamstack build <file.ds>` → produces `.wasm` + `.js` + `.html`
- `dreamstack dev <file.ds>` → dev server with file watching + hot-reload
- `dreamstack check <file.ds>` → type check + signal graph validation
### Deliverable
A counter app (`examples/counter.ds`) compiles and runs in the browser with reactive signal updates, no VDOM.
--- ---
## Phase 1 — Reactive Core (Weeks 4–6) ## Phase 0 — Foundation ✅
**Goal:** Fine-grained reactivity with compile-time dependency tracking, proving the "no re-render" model. **Compiler Pipeline (Rust)**
### Signal Runtime (Rust → WASM) | Crate | Purpose | Status |
|-------|---------|--------|
| `ds-parser` | Recursive descent parser → AST | ✅ 15 tests |
| `ds-analyzer` | Signal graph extraction (DAG) | ✅ 6 tests |
| `ds-codegen` | JS emitter — single-file HTML+JS output | ✅ |
| `ds-types` | Refinement type checker | ✅ 11 tests |
| `ds-layout` | Cassowary constraint solver | ✅ |
| `ds-cli` | CLI: build, dev, check, stream, playground, add, convert, init | ✅ |
| `ds-stream` | WebSocket relay server + binary protocol | ✅ 38 tests |
#### [NEW] `runtime/crates/ds-signals/` **Deliverable:** `counter.ds` compiles and runs. ✅
---
## Phase 1 — Reactive Core ✅
**Signal Runtime (pure JS, ~3KB):**
- `Signal<T>` — mutable source, notifies dependents on write - `Signal<T>` — mutable source, notifies dependents on write
- `Derived<T>` — computed from signals, lazy + cached, auto-invalidated - `Derived<T>` — computed from signals via `DS.derived()`, lazy + cached
- `Effect` — runs side-effect when dependencies change (for DOM updates only — user-facing effects use algebraic effects) - `Effect` — runs side-effect when dependencies change (surgical DOM updates)
- Batching: multiple signal writes in one tick coalesce into a single propagation pass - Batching: multiple signal writes coalesce into single propagation pass
- Topological sort of the DAG ensures glitch-free propagation (no intermediate states visible) - Topological sort ensures glitch-free propagation
### DOM Binding Layer (JS) **DOM Binding Layer:**
- `DS.effect(() => { el.textContent = signal.value; })` — auto-tracked
- Reactive `style`, `class`, `variant` props on containers
- Component props: getter fns for live reactivity (`label: () => expr`)
- Timer merging: all same-interval `every` statements share one `setInterval`
#### [NEW] `runtime/js/dom-bindings.js` **Deliverable:** TodoMVC, counter, list, form — all reactive, no VDOM. ✅
- Minimal JS layer (~3KB) that the codegen targets
- Functions: `bindText(nodeId, signalId)`, `bindAttr(nodeId, attr, signalId)`, `bindVisible(nodeId, signalId)`, `mountChildren(parentId, listSignalId, templateFn)`
- Each binding registers itself as an effect on the signal graph
- No framework runtime — just a bag of DOM mutation functions
### Benchmark Suite
#### [NEW] `bench/`
- Signal propagation: 1K / 10K / 100K signals, measure update latency
- DOM updates: compare against Solid.js, Svelte 5, vanilla JS
- Target: <1ms for 1K signal cascade, <16ms for 10K
### Deliverable
A TodoMVC app (`examples/todomvc.ds`) with add/remove/toggle/filter — all reactive, no VDOM, benchmarked against Solid.js.
--- ---
## Phase 2 — Effect System & Streams (Weeks 7–10) ## Phase 2 — Effects & Streams ✅
**Goal:** Algebraic effects for side-effects, stream primitives for temporal data. - `effect fetchUser(id): Result<User, ApiError>` — declared effects
- `perform fetchUser(id)` — suspends to nearest handler
- `DS.resource()` / `DS.fetchJSON()` — async resources with Loading/Ok/Err states
- Stream operators: `debounce`, `throttle`, `distinct` for temporal data
### Algebraic Effects (Rust → WASM) **Deliverable:** `search.html`, `dashboard.html` with live data. ✅
#### [NEW] `runtime/crates/ds-effects/`
- Effect declarations: `effect fetchUser(id: UserId): Result<User, ApiError>`
- `perform` keyword: suspends the current continuation, delegates to the nearest handler
- `handle ... with` blocks: install effect handlers at any level of the component tree
- Continuation-based: handlers receive a `resume` function to continue execution
- Built-in effects: `Http`, `Time`, `Random`, `Storage`, `Console`
### Stream Engine
#### [NEW] `runtime/crates/ds-streams/`
- `Stream<T>` — push-based async sequence
- Core operators: `map`, `filter`, `flatmap`, `merge`, `combine`, `debounce`, `throttle`, `distinct`, `take`, `skip`, `scan`
- `stream from <event>` — creates a stream from DOM events
- Streams integrate with the signal graph: a stream's latest value is a signal
- Backpressure: drop or buffer strategies, configurable per stream
### Deliverable
A search-with-autocomplete app (`examples/search.ds`) using `debounce`, `flatmap` over an HTTP effect, with the effect handler swappable for testing.
--- ---
## Phase 3 — Layout & Animation (Weeks 11–14) ## Phase 3 — Layout & Animation ✅
**Goal:** Constraint-based layout engine and physics-based animation, both running in WASM. **Cassowary Constraint Solver:**
- `layout dashboard { sidebar.width == 250 }` — declarative constraints
- Reactive: constraints reference signals → solver re-runs on change
- `constrain el.prop = expr` — imperative constraints
### Constraint Solver **Spring Physics:**
- `let x = spring(target: 0, stiffness: 300, damping: 30)` — RK4 integrator
- Interruptible, gesture-driven, composable through signal system
- `requestAnimationFrame` scheduler, sleeps when idle
#### [NEW] `runtime/crates/ds-layout/` **Deliverable:** `springs.ds`, `dashboard.ds` with constraint layout. ✅
- Implement the **Cassowary** simplex-based constraint solving algorithm in Rust
- Constraint types: `eq`, `gte`, `lte`, `clamp`, `ratio`
- Reactive integration: constraints reference signals → solver re-runs when inputs change
- Output: absolute `(x, y, width, height)` for each element, applied directly to DOM via `transform: translate()` (avoids reflow)
- Performance target: solve 500 constraints in <2ms
### Spring Physics
#### [NEW] `runtime/crates/ds-springs/`
- `Spring { value, velocity, target, stiffness, damping, mass }`
- RK4 integrator, fixed timestep (1/120s), interpolated for display
- `spring.target = x` triggers animation to new target (interruptible by default)
- Springs are signals — anything that reads `spring.value` auto-updates
- Gesture integration: direct-manipulation by setting `spring.target` to pointer position
### Animation Scheduler
#### [NEW] `runtime/js/raf-scheduler.js`
- Drives springs via `requestAnimationFrame`
- When no springs are active, scheduler sleeps (zero CPU when idle)
- Coordinates with signal graph: spring value changes go through normal propagation
### Deliverable
A draggable panel layout (`examples/dashboard.ds`) with resizable sidebar (constraints) and spring-animated panel transitions.
--- ---
## Phase 4 — Type System (Weeks 15–18) ## Phase 4 — Type System ✅ (Partial)
**Goal:** Dependent types for UI correctness guarantees. **Implemented:**
- Refinement types: `type PositiveInt = Int where value > 0`
- Runtime guards: compiler emits `if (!(pred)) throw new Error(...)`
- Basic type annotations on let declarations
### Type Checker **Deferred:**
- Full Hindley-Milner inference
#### [NEW] `compiler/crates/ds-types/` - Effect types (verified all effects handled)
- **Base:** Hindley-Milner with let-generalization, structural records - Signal-aware types (`Signal<T>` as first-class)
- **Refinement types:** `type PositiveInt = { n: Int | n > 0 }` — checked at compile time where possible, runtime where not
- **Signal-aware types:** `Signal<T>` is a first-class type; the compiler tracks which values are reactive
- **Effect types:** functions declare which effects they may perform → compiler verifies all effects are handled
- **UI types:** `View` type ensures only valid UI expressions appear in `view` blocks
### Error Messages
- Inspired by Elm: suggest fixes, show the specific constraint that failed, include code context
- Signal graph visualization in error output for dependency-related errors
### Deliverable
The compiler catches: unhandled effects, type mismatches in signal derivations, invalid UI expressions — all with helpful error messages.
--- ---
## Phase 5 — Live Editor (Weeks 19–24) ## Phase 5 — Live Editor ✅
**Goal:** Bidirectional structural editing — the killer feature. - **Dev server:** `dreamstack dev` — file watcher + HMR polling (500ms)
- **Playground:** `dreamstack playground` — Monaco editor + live preview + error panel
- **Compile errors:** displayed in-browser with styled error overlay
- **Auto-reload:** version counter + polling, full page refresh on change
### Incremental Compiler **Deliverable:** Working playground with Monaco. ✅
#### [NEW] `compiler/crates/ds-incremental/`
- Tracks which AST nodes changed between edits
- Re-analyzes only affected signal subgraph
- Re-generates only changed DOM bindings
- Target: <50ms from keystroke to live update
### Editor Integration
#### [NEW] `editor/`
- Web-based editor (Monaco or CodeMirror 6 with custom language mode)
- Split pane: code ↔ live preview
- **Code → Preview:** incremental compile on every keystroke
- **Preview → Code:** click an element in preview → highlights source; drag to reorder → AST rewrite → code updates
- State preservation: signals retain their current values across edits
### Visual Inspector
- Overlay on the live preview showing signal dependencies, constraint wires, spring states
- Click a DOM element → see its signal bindings, constraint relationships, effect dependencies
### Deliverable
A working playground at `dreamstack.dev` where users can write DreamStack code and see live results with bidirectional editing.
--- ---
## Phase 6 — Production Readiness (Weeks 25–30) ## Phase 6 — Production ✅
**Goal:** SSR, ecosystem tooling, documentation, and real-world validation. - **Component registry:** 14 components (Badge, Dialog, Tabs, Toast, Card, etc.)
- **`dreamstack add badge`** — install from registry
- **`dreamstack convert file.tsx`** — React/TSX → DreamStack converter
- **`dreamstack convert button --shadcn`** — shadcn/ui integration
- **`import { Badge } from "./badge"`** — recursive module resolution
- **`export component Card(title) = ...`** — named exports
- **Slot system:** `component Card(title, children) = ... slot ...`
### Server-Side Rendering ---
- Compile DreamStack to static HTML + hydration script
- Streaming SSR: emit HTML as constraint solver resolves layout
- Effect handlers for server context (no DOM, no springs)
### Package System ## Phase 7 — Bitstream Streaming ✅
- `dreamstack add <package>` — package registry for reusable views and effect handlers
- Standard library: common UI patterns (buttons, inputs, lists, modals, toasts)
- Interop: import/export with JavaScript modules
### Documentation & Examples **Full specification**: see [BITSTREAM_INTEGRATION.md](file:///home/amir/code/dreamstack/BITSTREAM_INTEGRATION.md)
- Language reference
- Tutorial: "Build a real app in 30 minutes" - 16-byte binary protocol, 3 streaming modes (signal/delta/pixel)
- Cookbook: common patterns (forms, routing, data fetching, auth) - Bidirectional sync with per-signal version counters
- Migration guide: "Coming from React" / "Coming from Svelte" - Output filtering: `output: x, y, score` — skips internal signals
- Exponential reconnect backoff (2s → 4s → 8s, max 30s)
- WebRTC data channel + WebSocket fallback
- `stream from` expression for receivers with `select` clause
---
## Phase 8 — Physics Scene ✅
- Rapier2D compiled to WASM via `wasm-pack`
- `scene { gravity_y: g } [ circle {...}, rect {...} ]`
- Reactive gravity, mouse drag, collision, restitution
- Compile-time hex color parsing → `set_body_color(r, g, b, a)`
- `streaming-physics.ds` — physics scene streamed to viewers
---
## Phase 9 — Bidirectional Sync ✅
- **Peer mode:** `stream on ".../peer/channel"` — both source and receiver
- **Conflict resolution:** version counters, last-write-wins with stale rejection
- **Deduplication:** JSON.stringify comparison, skip unchanged values
- **Composition:** receiver can derive new signals and re-stream them
**Full specification**: see [STREAM_COMPOSITION.md](file:///home/amir/code/dreamstack/STREAM_COMPOSITION.md)
---
## Phase 10 — Games & Audio ✅
- **Pong:** multiplayer with keyboard controls, sound effects, streaming
- **Breakout:** 5 brick rows, collision, score/lives badges, streaming
- **Snake:** grid-based movement, growth, collision
- **Reaction game:** reaction time measurement
- **Step sequencer:** 16-step drum machine with Web Audio API
- **Beats viewer:** spectator view for step sequencer
- **Sound:** built-in `play_tone(freq, duration)` oscillator
---
## Next Steps
### Near-Term Polish
- [ ] Better error messages — source context with line + caret
- [ ] Integration tests for routes, WebRTC, layout constraints
### New Examples
- [ ] Tetris game (grid + timers + line clearing)
- [ ] Todo app with bidirectional streaming sync
### Ecosystem
- [ ] Crates.io publish (`cargo install dreamstack`)
- [ ] Self-hosted playground deployment
--- ---
@ -210,116 +191,10 @@ A working playground at `dreamstack.dev` where users can write DreamStack code a
| Component | Language | Rationale | | Component | Language | Rationale |
|---|---|---| |---|---|---|
| Parser, Analyzer, Codegen, Type Checker | **Rust** | Performance, WASM target, memory safety without GC | | Parser, Analyzer, Codegen, Types | **Rust** | Performance, WASM target, memory safety |
| Signal Runtime, Constraint Solver, Springs | **Rust → WASM** | Must run at 60fps, no GC pauses | | Signal Runtime, Springs, Constraints | **JavaScript** | Direct DOM access, ~7KB total |
| DOM Bindings, RAF Scheduler | **JavaScript** | Must interact with browser APIs directly | | Physics Engine | **Rust → WASM** | Rapier2D, 60fps rigid body simulation |
| CLI | **Rust** (clap) | Fast startup, single binary distribution | | CLI | **Rust** (clap) | Single binary, fast startup |
| Editor | **TypeScript** | CodeMirror/Monaco ecosystem, web-native | | Dev Server | **Rust** (tiny_http) | File watching, HMR, HTTP serving |
| Dev Server | **Rust** (axum) | Fast file watching, WebSocket HMR | | Playground | **JavaScript** | Monaco editor integration |
| Stream Relay | **Rust** (tokio + tungstenite) | Async WebSocket server |
---
## Key Technical Risks
| Risk | Mitigation |
|---|---|
| Compile-time signal analysis is undecidable in general | Restrict to analyzable patterns; fall back to runtime tracking for dynamic cases |
| Cassowary solver is O(n²) worst case | Incremental solving; partition independent constraint groups |
| WASM ↔ JS boundary overhead for DOM updates | Batch mutations; use `FinalizationRegistry` for memory management |
| Algebraic effects require delimited continuations | Use CPS transform in codegen; or one-shot continuations via WASM stack switching (Stage 3 proposal) |
| Homoiconic representation vs performance | Two representations: rich AST for editor, optimized IR for runtime |
---
## Verification Plan
Since this is a greenfield language/framework, verification is demo-driven:
### Phase 0–1: Reactive Core
```bash
# Build the compiler and run the counter example
cargo build --release -p ds-cli
./target/release/dreamstack build examples/counter.ds
# Serve and verify in browser
./target/release/dreamstack dev examples/counter.ds
# Run signal propagation benchmarks
cargo bench -p ds-signals
```
### Phase 2: Effects & Streams
```bash
# Run effect system unit tests
cargo test -p ds-effects
# Run the search example with mock handler (automated test)
./target/release/dreamstack test examples/search.ds
```
### Phase 3: Layout & Animation
```bash
# Constraint solver correctness tests
cargo test -p ds-layout
# Visual regression: screenshot comparison of dashboard example
./target/release/dreamstack screenshot examples/dashboard.ds --compare snapshots/
```
### Phase 5: Live Editor
- **Manual verification:** Open the playground, type code, verify preview updates <50ms
- **Bidirectional test:** Drag an element in preview, verify code updates correspondingly
> [!IMPORTANT]
> The user should weigh in on which phase to prioritize first. The plan assumes linear progression, but Phases 1–3 could be parallelized if multiple contributors are involved. Phase 4 (dependent types) is the highest-risk and could be deferred to a v2.
---
## Phase 10 — Rapier2D Physics Engine (Completed)
**Goal:** Replace the Verlet spring-mass physics with Rapier2D for real rigid body simulation.
### WASM Physics Engine
#### [NEW] `engine/ds-physics/`
- Rapier2D wrapped in a `PhysicsWorld` struct, compiled to WASM via `wasm-pack`
- Rigid body creation: `create_soft_circle()`, `create_soft_rect()`
- Force control: `set_gravity()` (wakes all sleeping bodies), `apply_impulse()`
- Query: `get_body_center()`, `get_body_rotation()`, `get_body_positions()`, `body_count()`
- Rendering: `set_body_color()`, body info with color, radius, dimensions
- Collision, restitution, friction, damping — all configurable per body
- 5 tests verifying world creation, stepping, body creation, gravity, and impulse
### Deliverable
The WASM physics module at `dist/pkg/ds_physics.js` + `ds_physics_bg.wasm`, loadable from any HTML page.
---
## Phase 11 — Physics in the Language (Completed)
**Goal:** Integrate Rapier2D physics as a first-class `.ds` language construct.
### Compiler Additions
#### [MODIFY] `compiler/ds-parser/src/ast.rs`
- Added `ContainerKind::Scene`
#### [MODIFY] `compiler/ds-parser/src/lexer.rs`
- Added `TokenKind::Scene` and `"scene"` keyword mapping
#### [MODIFY] `compiler/ds-parser/src/parser.rs`
- `scene` dispatches to `parse_container_with_props` (width, height, gravity)
- `circle`, `rect`, `line` added to `is_ui_element()`
#### [MODIFY] `compiler/ds-analyzer/src/signal_graph.rs`
- `Scene` variant in `collect_bindings` for DOM binding extraction
#### [MODIFY] `compiler/ds-codegen/src/js_emitter.rs`
- `emit_scene()` — canvas creation, async WASM init, `PhysicsWorld` setup
- `emit_scene_circle()`, `emit_scene_rect()` — per-body codegen with unique variables
- `emit_scene_set_color()` — compile-time hex color parsing
- Reactive gravity via `DS.effect(() => { _world.set_gravity(...); })`
- `requestAnimationFrame` loop for physics stepping + canvas rendering
- Mouse drag interaction with impulse-based body movement
#### [NEW] `examples/physics.ds`
- 5 bodies (circles + rects), reactive gravity controls, drag interaction
### Deliverable
`physics.ds` compiles to a working HTML page with WASM physics — 22KB output, no framework dependencies.