dreamstack/IMPLEMENTATION_PLAN.md

# DreamStack Implementation Plan

Build a working prototype of the DreamStack vision — a new UI framework with compile-time reactivity, algebraic effects, constraint layout, and physics animation. The strategy is to **bootstrap pragmatically**: use Rust for the compiler/runtime core, target JS + DOM, and prove each pillar with working demos before unifying them.

---

## Status Overview

| Phase | Goal | Status |
|-------|------|--------|
| 0 — Foundation | Compiler pipeline, CLI | ✅ Complete |
| 1 — Reactive Core | Signals, derived, effects, DOM bindings | ✅ Complete |
| 2 — Effects & Streams | Algebraic effects, stream operators | ✅ Complete |
| 3 — Layout & Animation | Cassowary constraints, spring physics | ✅ Complete |
| 4 — Type System | Refinement types, runtime guards | ✅ Partial (refinement types done, full HM deferred) |
| 5 — Live Editor | Playground, HMR dev server | ✅ Complete |
| 6 — Production | Component registry, imports, TSX converter | ✅ Complete |
| 7 — Bitstream Streaming | Source/receiver, relay, signal diffs | ✅ Complete |
| 8 — Physics Scene | Rapier2D WASM integration | ✅ Complete |
| 9 — Bidirectional Sync | Peer-to-peer, conflict resolution, WebRTC | ✅ Complete |
| 10 — Games & Audio | Pong, Breakout, Sequencer, Sound | ✅ Complete |

**Current:** 7 Rust crates, 136 tests, 48 compilable `.ds` examples, 14 registry components.

---

## Phase 0 — Foundation ✅

**Compiler Pipeline (Rust)**

| Crate | Purpose | Status |
|-------|---------|--------|
| `ds-parser` | Recursive descent parser → AST | ✅ 15 tests |
| `ds-analyzer` | Signal graph extraction (DAG) | ✅ 6 tests |
| `ds-codegen` | JS emitter — single-file HTML+JS output | ✅ |
| `ds-types` | Refinement type checker | ✅ 11 tests |
| `ds-layout` | Cassowary constraint solver | ✅ |
| `ds-cli` | CLI: build, dev, check, stream, playground, add, convert, init | ✅ |
| `ds-stream` | WebSocket relay server + binary protocol | ✅ 38 tests |

**Deliverable:** `counter.ds` compiles and runs. ✅

---

## Phase 1 — Reactive Core ✅

**Signal Runtime (pure JS, ~3KB):**
- `Signal<T>` — mutable source, notifies dependents on write
- `Derived<T>` — computed from signals via `DS.derived()`, lazy + cached
- `Effect` — runs side-effect when dependencies change (surgical DOM updates)
- Batching: multiple signal writes coalesce into single propagation pass
- Topological sort ensures glitch-free propagation

**DOM Binding Layer:**
- `DS.effect(() => { el.textContent = signal.value; })` — auto-tracked
- Reactive `style`, `class`, `variant` props on containers
- Component props: getter fns for live reactivity (`label: () => expr`)
- Timer merging: all same-interval `every` statements share one `setInterval`

**Deliverable:** TodoMVC, counter, list, form — all reactive, no VDOM. ✅

---

## Phase 2 — Effects & Streams ✅

- `effect fetchUser(id): Result<User, ApiError>` — declared effects
- `perform fetchUser(id)` — suspends to nearest handler
- `DS.resource()` / `DS.fetchJSON()` — async resources with Loading/Ok/Err states
- Stream operators: `debounce`, `throttle`, `distinct` for temporal data

**Deliverable:** `search.html`, `dashboard.html` with live data. ✅

---

## Phase 3 — Layout & Animation ✅

**Cassowary Constraint Solver:**
- `layout dashboard { sidebar.width == 250 }` — declarative constraints
- Reactive: constraints reference signals → solver re-runs on change
- `constrain el.prop = expr` — imperative constraints

**Spring Physics:**
- `let x = spring(target: 0, stiffness: 300, damping: 30)` — RK4 integrator
- Interruptible, gesture-driven, composable through signal system
- `requestAnimationFrame` scheduler, sleeps when idle

**Deliverable:** `springs.ds`, `dashboard.ds` with constraint layout. ✅

---

## Phase 4 — Type System ✅

**Implemented:**
- Hindley-Milner unification with occurs check (prevents infinite types)
- Type variable substitution (`apply_subst` chases bindings)
- Refinement types: `type PositiveInt = Int where value > 0`
- Runtime guards: compiler emits `if (!(pred)) throw new Error(...)`
- Signal-aware inference: `SignalInfo` + `check_program_with_signals()`
- Effect handler scoping: `Dom` effect auto-handled in view blocks
- Type alias resolution with cycle detection
- Numeric coercion: `Int` unifies with `Float`
- 34 tests (unification, occurs check, signal graph, effect scoping)

---

## Phase 5 — Live Editor ✅

- **Dev server:** `dreamstack dev` — file watcher + HMR polling (500ms)
- **Playground:** `dreamstack playground` — Monaco editor + live preview + error panel
- **Compile errors:** displayed in-browser with styled error overlay
- **Auto-reload:** version counter + polling, full page refresh on change

**Deliverable:** Working playground with Monaco. ✅

---

## Phase 6 — Production ✅

- **Component registry:** 14 components (Badge, Dialog, Tabs, Toast, Card, etc.)
- **`dreamstack add badge`** — install from registry
- **`dreamstack convert file.tsx`** — React/TSX → DreamStack converter
- **`dreamstack convert button --shadcn`** — shadcn/ui integration
- **`import { Badge } from "./badge"`** — recursive module resolution
- **`export component Card(title) = ...`** — named exports
- **Slot system:** `component Card(title, children) = ... slot ...`

---

## Phase 7 — Bitstream Streaming ✅

**Full specification**: see [BITSTREAM_INTEGRATION.md](file:///home/amir/code/dreamstack/BITSTREAM_INTEGRATION.md)

- 16-byte binary protocol, 3 streaming modes (signal/delta/pixel)
- Bidirectional sync with per-signal version counters
- Output filtering: `output: x, y, score` — skips internal signals
- Exponential reconnect backoff (2s → 4s → 8s, max 30s)
- WebRTC data channel + WebSocket fallback
- `stream from` expression for receivers with `select` clause

---

## Phase 8 — Physics Scene ✅

- Rapier2D compiled to WASM via `wasm-pack`
- `scene { gravity_y: g } [ circle {...}, rect {...} ]`
- Reactive gravity, mouse drag, collision, restitution
- Compile-time hex color parsing → `set_body_color(r, g, b, a)`
- `streaming-physics.ds` — physics scene streamed to viewers

---

## Phase 9 — Bidirectional Sync ✅

- **Peer mode:** `stream on ".../peer/channel"` — both source and receiver
- **Conflict resolution:** version counters, last-write-wins with stale rejection
- **Deduplication:** JSON.stringify comparison, skip unchanged values
- **Composition:** receiver can derive new signals and re-stream them

**Full specification**: see [STREAM_COMPOSITION.md](file:///home/amir/code/dreamstack/STREAM_COMPOSITION.md)

---

## Phase 10 — Games & Audio ✅

- **Pong:** multiplayer with keyboard controls, sound effects, streaming
- **Breakout:** 5 brick rows, collision, score/lives badges, streaming
- **Snake:** grid-based movement, growth, collision
- **Reaction game:** reaction time measurement
- **Step sequencer:** 16-step drum machine with Web Audio API
- **Beats viewer:** spectator view for step sequencer
- **Sound:** built-in `play_tone(freq, duration)` oscillator

---

## Next Steps

### Near-Term Polish
- [ ] Better error messages — source context with line + caret
- [ ] Integration tests for routes, WebRTC, layout constraints

### New Examples
- [ ] Tetris game (grid + timers + line clearing)
- [ ] Todo app with bidirectional streaming sync

### Ecosystem
- [ ] Crates.io publish (`cargo install dreamstack`)
- [ ] Self-hosted playground deployment

---

## Implementation Language Choices

| Component | Language | Rationale |
|---|---|---|
| Parser, Analyzer, Codegen, Types | **Rust** | Performance, WASM target, memory safety |
| Signal Runtime, Springs, Constraints | **JavaScript** | Direct DOM access, ~7KB total |
| Physics Engine | **Rust → WASM** | Rapier2D, 60fps rigid body simulation |
| CLI | **Rust** (clap) | Single binary, fast startup |
| Dev Server | **Rust** (tiny_http) | File watching, HMR, HTTP serving |
| Playground | **JavaScript** | Monaco editor integration |
| Stream Relay | **Rust** (tokio + tungstenite) | Async WebSocket server |