dreamstack/IMPLEMENTATION_PLAN.md

# DreamStack Implementation Plan

Build a working prototype of the DreamStack vision — a new UI framework with compile-time reactivity, algebraic effects, constraint layout, and physics animation. The strategy is to **bootstrap pragmatically**: use Rust for the compiler/runtime core, target WASM + DOM, and prove each pillar with working demos before unifying them.

---

## Phase 0 — Foundation (Weeks 1–3)

**Goal:** A working compiler that turns DreamStack syntax into executable WASM + JS glue.

### Compiler Pipeline (Rust)

#### [NEW] `compiler/crates/ds-parser/`
- Hand-written recursive descent parser for DreamStack syntax
- Produces an AST of `view`, `let`, `when`, `match`, `effect`, `on`, `stream`, `layout` constructs
- Homoiconic representation: AST nodes are also the runtime data structure (tagged vectors/maps)

#### [NEW] `compiler/crates/ds-analyzer/`
- **Signal graph extraction:** Static analysis pass that identifies signals, derived values, and their dependencies → builds a DAG
- **Effect collection:** Catalogs all `perform` sites and their expected effect signatures
- **Type inference:** Hindley-Milner base with structural typing; dependent types deferred to Phase 4

#### [NEW] `compiler/crates/ds-codegen/`
- Emits **Rust → WASM** for:
  - Signal graph runtime (push-based notification)
  - Constraint solver (Cassowary algorithm)
  - Spring physics engine
- Emits **JS glue** for DOM bindings — surgical `textContent`, `setAttribute`, `insertBefore`, `removeChild` calls (no VDOM)
- Output format: single `.wasm` + single `.js` module

#### [NEW] `compiler/crates/ds-cli/`
- `dreamstack build <file.ds>` → produces `.wasm` + `.js` + `.html`
- `dreamstack dev <file.ds>` → dev server with file watching + hot-reload
- `dreamstack check <file.ds>` → type check + signal graph validation

### Deliverable
A counter app (`examples/counter.ds`) compiles and runs in the browser with reactive signal updates, no VDOM.

---

## Phase 1 — Reactive Core (Weeks 4–6)

**Goal:** Fine-grained reactivity with compile-time dependency tracking, proving the "no re-render" model.

### Signal Runtime (Rust → WASM)

#### [NEW] `runtime/crates/ds-signals/`
- `Signal<T>` — mutable source, notifies dependents on write
- `Derived<T>` — computed from signals, lazy + cached, auto-invalidated
- `Effect` — runs side-effect when dependencies change (for DOM updates only — user-facing effects use algebraic effects)
- Batching: multiple signal writes in one tick coalesce into a single propagation pass
- Topological sort of the DAG ensures glitch-free propagation (no intermediate states visible)

### DOM Binding Layer (JS)

#### [NEW] `runtime/js/dom-bindings.js`
- Minimal JS layer (~3KB) that the codegen targets
- Functions: `bindText(nodeId, signalId)`, `bindAttr(nodeId, attr, signalId)`, `bindVisible(nodeId, signalId)`, `mountChildren(parentId, listSignalId, templateFn)`
- Each binding registers itself as an effect on the signal graph
- No framework runtime — just a bag of DOM mutation functions

### Benchmark Suite

#### [NEW] `bench/`
- Signal propagation: 1K / 10K / 100K signals, measure update latency
- DOM updates: compare against Solid.js, Svelte 5, vanilla JS
- Target: <1ms for 1K signal cascade, <16ms for 10K

### Deliverable
A TodoMVC app (`examples/todomvc.ds`) with add/remove/toggle/filter — all reactive, no VDOM, benchmarked against Solid.js.

---

## Phase 2 — Effect System & Streams (Weeks 7–10)

**Goal:** Algebraic effects for side-effects, stream primitives for temporal data.

### Algebraic Effects (Rust → WASM)

#### [NEW] `runtime/crates/ds-effects/`
- Effect declarations: `effect fetchUser(id: UserId): Result<User, ApiError>`
- `perform` keyword: suspends the current continuation, delegates to the nearest handler
- `handle ... with` blocks: install effect handlers at any level of the component tree
- Continuation-based: handlers receive a `resume` function to continue execution
- Built-in effects: `Http`, `Time`, `Random`, `Storage`, `Console`

### Stream Engine

#### [NEW] `runtime/crates/ds-streams/`
- `Stream<T>` — push-based async sequence
- Core operators: `map`, `filter`, `flatmap`, `merge`, `combine`, `debounce`, `throttle`, `distinct`, `take`, `skip`, `scan`
- `stream from <event>` — creates a stream from DOM events
- Streams integrate with the signal graph: a stream's latest value is a signal
- Backpressure: drop or buffer strategies, configurable per stream

### Deliverable
A search-with-autocomplete app (`examples/search.ds`) using `debounce`, `flatmap` over an HTTP effect, with the effect handler swappable for testing.

---

## Phase 3 — Layout & Animation (Weeks 11–14)

**Goal:** Constraint-based layout engine and physics-based animation, both running in WASM.

### Constraint Solver

#### [NEW] `runtime/crates/ds-layout/`
- Implement the **Cassowary** simplex-based constraint solving algorithm in Rust
- Constraint types: `eq`, `gte`, `lte`, `clamp`, `ratio`
- Reactive integration: constraints reference signals → solver re-runs when inputs change
- Output: absolute `(x, y, width, height)` for each element, applied directly to DOM via `transform: translate()` (avoids reflow)
- Performance target: solve 500 constraints in <2ms

### Spring Physics

#### [NEW] `runtime/crates/ds-springs/`
- `Spring { value, velocity, target, stiffness, damping, mass }`
- RK4 integrator, fixed timestep (1/120s), interpolated for display
- `spring.target = x` triggers animation to new target (interruptible by default)
- Springs are signals — anything that reads `spring.value` auto-updates
- Gesture integration: direct-manipulation by setting `spring.target` to pointer position

### Animation Scheduler

#### [NEW] `runtime/js/raf-scheduler.js`
- Drives springs via `requestAnimationFrame`
- When no springs are active, scheduler sleeps (zero CPU when idle)
- Coordinates with signal graph: spring value changes go through normal propagation

### Deliverable
A draggable panel layout (`examples/dashboard.ds`) with resizable sidebar (constraints) and spring-animated panel transitions.

---

## Phase 4 — Type System (Weeks 15–18)

**Goal:** Dependent types for UI correctness guarantees.

### Type Checker

#### [NEW] `compiler/crates/ds-types/`
- **Base:** Hindley-Milner with let-generalization, structural records
- **Refinement types:** `type PositiveInt = { n: Int | n > 0 }` — checked at compile time where possible, runtime where not
- **Signal-aware types:** `Signal<T>` is a first-class type; the compiler tracks which values are reactive
- **Effect types:** functions declare which effects they may perform → compiler verifies all effects are handled
- **UI types:** `View` type ensures only valid UI expressions appear in `view` blocks

### Error Messages
- Inspired by Elm: suggest fixes, show the specific constraint that failed, include code context
- Signal graph visualization in error output for dependency-related errors

### Deliverable
The compiler catches: unhandled effects, type mismatches in signal derivations, invalid UI expressions — all with helpful error messages.

---

## Phase 5 — Live Editor (Weeks 19–24)

**Goal:** Bidirectional structural editing — the killer feature.

### Incremental Compiler

#### [NEW] `compiler/crates/ds-incremental/`
- Tracks which AST nodes changed between edits
- Re-analyzes only affected signal subgraph
- Re-generates only changed DOM bindings
- Target: <50ms from keystroke to live update

### Editor Integration

#### [NEW] `editor/`
- Web-based editor (Monaco or CodeMirror 6 with custom language mode)
- Split pane: code ↔ live preview
- **Code → Preview:** incremental compile on every keystroke
- **Preview → Code:** click an element in preview → highlights source; drag to reorder → AST rewrite → code updates
- State preservation: signals retain their current values across edits

### Visual Inspector
- Overlay on the live preview showing signal dependencies, constraint wires, spring states
- Click a DOM element → see its signal bindings, constraint relationships, effect dependencies

### Deliverable
A working playground at `dreamstack.dev` where users can write DreamStack code and see live results with bidirectional editing.

---

## Phase 6 — Production Readiness (Weeks 25–30)

**Goal:** SSR, ecosystem tooling, documentation, and real-world validation.

### Server-Side Rendering
- Compile DreamStack to static HTML + hydration script
- Streaming SSR: emit HTML as constraint solver resolves layout
- Effect handlers for server context (no DOM, no springs)

### Package System
- `dreamstack add <package>` — package registry for reusable views and effect handlers
- Standard library: common UI patterns (buttons, inputs, lists, modals, toasts)
- Interop: import/export with JavaScript modules

### Documentation & Examples
- Language reference
- Tutorial: "Build a real app in 30 minutes"
- Cookbook: common patterns (forms, routing, data fetching, auth)
- Migration guide: "Coming from React" / "Coming from Svelte"

---

## Implementation Language Choices

| Component | Language | Rationale |
|---|---|---|
| Parser, Analyzer, Codegen, Type Checker | **Rust** | Performance, WASM target, memory safety without GC |
| Signal Runtime, Constraint Solver, Springs | **Rust → WASM** | Must run at 60fps, no GC pauses |
| DOM Bindings, RAF Scheduler | **JavaScript** | Must interact with browser APIs directly |
| CLI | **Rust** (clap) | Fast startup, single binary distribution |
| Editor | **TypeScript** | CodeMirror/Monaco ecosystem, web-native |
| Dev Server | **Rust** (axum) | Fast file watching, WebSocket HMR |

---

## Key Technical Risks

| Risk | Mitigation |
|---|---|
| Compile-time signal analysis is undecidable in general | Restrict to analyzable patterns; fall back to runtime tracking for dynamic cases |
| Cassowary solver is O(n²) worst case | Incremental solving; partition independent constraint groups |
| WASM ↔ JS boundary overhead for DOM updates | Batch mutations; use `FinalizationRegistry` for memory management |
| Algebraic effects require delimited continuations | Use CPS transform in codegen; or one-shot continuations via WASM stack switching (Stage 3 proposal) |
| Homoiconic representation vs performance | Two representations: rich AST for editor, optimized IR for runtime |

---

## Verification Plan

Since this is a greenfield language/framework, verification is demo-driven:

### Phase 0–1: Reactive Core
```bash
# Build the compiler and run the counter example
cargo build --release -p ds-cli
./target/release/dreamstack build examples/counter.ds
# Serve and verify in browser
./target/release/dreamstack dev examples/counter.ds
# Run signal propagation benchmarks
cargo bench -p ds-signals
```

### Phase 2: Effects & Streams
```bash
# Run effect system unit tests
cargo test -p ds-effects
# Run the search example with mock handler (automated test)
./target/release/dreamstack test examples/search.ds
```

### Phase 3: Layout & Animation
```bash
# Constraint solver correctness tests
cargo test -p ds-layout
# Visual regression: screenshot comparison of dashboard example
./target/release/dreamstack screenshot examples/dashboard.ds --compare snapshots/
```

### Phase 5: Live Editor
- **Manual verification:** Open the playground, type code, verify preview updates <50ms
- **Bidirectional test:** Drag an element in preview, verify code updates correspondingly

> [!IMPORTANT]
> The user should weigh in on which phase to prioritize first. The plan assumes linear progression, but Phases 1–3 could be parallelized if multiple contributors are involved. Phase 4 (dependent types) is the highest-risk and could be deferred to a v2.

---

## Phase 10 — Rapier2D Physics Engine (Completed)

**Goal:** Replace the Verlet spring-mass physics with Rapier2D for real rigid body simulation.

### WASM Physics Engine

#### [NEW] `engine/ds-physics/`
- Rapier2D wrapped in a `PhysicsWorld` struct, compiled to WASM via `wasm-pack`
- Rigid body creation: `create_soft_circle()`, `create_soft_rect()`
- Force control: `set_gravity()` (wakes all sleeping bodies), `apply_impulse()`
- Query: `get_body_center()`, `get_body_rotation()`, `get_body_positions()`, `body_count()`
- Rendering: `set_body_color()`, body info with color, radius, dimensions
- Collision, restitution, friction, damping — all configurable per body
- 5 tests verifying world creation, stepping, body creation, gravity, and impulse

### Deliverable
The WASM physics module at `dist/pkg/ds_physics.js` + `ds_physics_bg.wasm`, loadable from any HTML page.

---

## Phase 11 — Physics in the Language (Completed)

**Goal:** Integrate Rapier2D physics as a first-class `.ds` language construct.

### Compiler Additions

#### [MODIFY] `compiler/ds-parser/src/ast.rs`
- Added `ContainerKind::Scene`

#### [MODIFY] `compiler/ds-parser/src/lexer.rs`
- Added `TokenKind::Scene` and `"scene"` keyword mapping

#### [MODIFY] `compiler/ds-parser/src/parser.rs`
- `scene` dispatches to `parse_container_with_props` (width, height, gravity)
- `circle`, `rect`, `line` added to `is_ui_element()`

#### [MODIFY] `compiler/ds-analyzer/src/signal_graph.rs`
- `Scene` variant in `collect_bindings` for DOM binding extraction

#### [MODIFY] `compiler/ds-codegen/src/js_emitter.rs`
- `emit_scene()` — canvas creation, async WASM init, `PhysicsWorld` setup
- `emit_scene_circle()`, `emit_scene_rect()` — per-body codegen with unique variables
- `emit_scene_set_color()` — compile-time hex color parsing
- Reactive gravity via `DS.effect(() => { _world.set_gravity(...); })`
- `requestAnimationFrame` loop for physics stepping + canvas rendering
- Mouse drag interaction with impulse-based body movement

#### [NEW] `examples/physics.ds`
- 5 bodies (circles + rects), reactive gravity controls, drag interaction

### Deliverable
`physics.ds` compiles to a working HTML page with WASM physics — 22KB output, no framework dependencies.