dreamstack/IMPLEMENTATION_PLAN.md

DreamStack Implementation Plan

Build a working prototype of the DreamStack vision — a new UI framework with compile-time reactivity, algebraic effects, constraint layout, and physics animation. The strategy is to bootstrap pragmatically: use Rust for the compiler/runtime core, target WASM + DOM, and prove each pillar with working demos before unifying them.

---

Phase 0 — Foundation (Weeks 1–3)

Goal: A working compiler that turns DreamStack syntax into executable WASM + JS glue.

Compiler Pipeline (Rust)

[NEW] compiler/crates/ds-parser/

• Hand-written recursive descent parser for DreamStack syntax
• Produces an AST of view, let, when, match, effect, on, stream, layout constructs
• Homoiconic representation: AST nodes are also the runtime data structure (tagged vectors/maps)

[NEW] compiler/crates/ds-analyzer/

• Signal graph extraction: Static analysis pass that identifies signals, derived values, and their dependencies → builds a DAG
• Effect collection: Catalogs all perform sites and their expected effect signatures
• Type inference: Hindley-Milner base with structural typing; dependent types deferred to Phase 4

[NEW] compiler/crates/ds-codegen/

• Emits Rust → WASM for:
  • Signal graph runtime (push-based notification)
  • Constraint solver (Cassowary algorithm)
  • Spring physics engine
• Emits JS glue for DOM bindings — surgical textContent, setAttribute, insertBefore, removeChild calls (no VDOM)
• Output format: single .wasm + single .js module

[NEW] compiler/crates/ds-cli/

• dreamstack build <file.ds> → produces .wasm + .js + .html
• dreamstack dev <file.ds> → dev server with file watching + hot-reload
• dreamstack check <file.ds> → type check + signal graph validation

Deliverable

A counter app (examples/counter.ds) compiles and runs in the browser with reactive signal updates, no VDOM.

---

Phase 1 — Reactive Core (Weeks 4–6)

Goal: Fine-grained reactivity with compile-time dependency tracking, proving the "no re-render" model.

Signal Runtime (Rust → WASM)

[NEW] runtime/crates/ds-signals/

• Signal<T> — mutable source, notifies dependents on write
• Derived<T> — computed from signals, lazy + cached, auto-invalidated
• Effect — runs side-effect when dependencies change (for DOM updates only — user-facing effects use algebraic effects)
• Batching: multiple signal writes in one tick coalesce into a single propagation pass
• Topological sort of the DAG ensures glitch-free propagation (no intermediate states visible)

DOM Binding Layer (JS)

[NEW] runtime/js/dom-bindings.js

• Minimal JS layer (~3KB) that the codegen targets
• Functions: bindText(nodeId, signalId), bindAttr(nodeId, attr, signalId), bindVisible(nodeId, signalId), mountChildren(parentId, listSignalId, templateFn)
• Each binding registers itself as an effect on the signal graph
• No framework runtime — just a bag of DOM mutation functions

Benchmark Suite

[NEW] bench/

• Signal propagation: 1K / 10K / 100K signals, measure update latency
• DOM updates: compare against Solid.js, Svelte 5, vanilla JS
• Target: <1ms for 1K signal cascade, <16ms for 10K

Deliverable

A TodoMVC app (examples/todomvc.ds) with add/remove/toggle/filter — all reactive, no VDOM, benchmarked against Solid.js.

---

Phase 2 — Effect System & Streams (Weeks 7–10)

Goal: Algebraic effects for side-effects, stream primitives for temporal data.

Algebraic Effects (Rust → WASM)

[NEW] runtime/crates/ds-effects/

• Effect declarations: effect fetchUser(id: UserId): Result<User, ApiError>
• perform keyword: suspends the current continuation, delegates to the nearest handler
• handle ... with blocks: install effect handlers at any level of the component tree
• Continuation-based: handlers receive a resume function to continue execution
• Built-in effects: Http, Time, Random, Storage, Console

Stream Engine

[NEW] runtime/crates/ds-streams/

• Stream<T> — push-based async sequence
• Core operators: map, filter, flatmap, merge, combine, debounce, throttle, distinct, take, skip, scan
• stream from <event> — creates a stream from DOM events
• Streams integrate with the signal graph: a stream's latest value is a signal
• Backpressure: drop or buffer strategies, configurable per stream

Deliverable

A search-with-autocomplete app (examples/search.ds) using debounce, flatmap over an HTTP effect, with the effect handler swappable for testing.

---

Phase 3 — Layout & Animation (Weeks 11–14)

Goal: Constraint-based layout engine and physics-based animation, both running in WASM.

Constraint Solver

[NEW] runtime/crates/ds-layout/

• Implement the Cassowary simplex-based constraint solving algorithm in Rust
• Constraint types: eq, gte, lte, clamp, ratio
• Reactive integration: constraints reference signals → solver re-runs when inputs change
• Output: absolute (x, y, width, height) for each element, applied directly to DOM via transform: translate() (avoids reflow)
• Performance target: solve 500 constraints in <2ms

Spring Physics

[NEW] runtime/crates/ds-springs/

• Spring { value, velocity, target, stiffness, damping, mass }
• RK4 integrator, fixed timestep (1/120s), interpolated for display
• spring.target = x triggers animation to new target (interruptible by default)
• Springs are signals — anything that reads spring.value auto-updates
• Gesture integration: direct-manipulation by setting spring.target to pointer position

Animation Scheduler

[NEW] runtime/js/raf-scheduler.js

• Drives springs via requestAnimationFrame
• When no springs are active, scheduler sleeps (zero CPU when idle)
• Coordinates with signal graph: spring value changes go through normal propagation

Deliverable

A draggable panel layout (examples/dashboard.ds) with resizable sidebar (constraints) and spring-animated panel transitions.

---

Phase 4 — Type System (Weeks 15–18)

Goal: Dependent types for UI correctness guarantees.

Type Checker

[NEW] compiler/crates/ds-types/

• Base: Hindley-Milner with let-generalization, structural records
• Refinement types: type PositiveInt = { n: Int | n > 0 } — checked at compile time where possible, runtime where not
• Signal-aware types: Signal<T> is a first-class type; the compiler tracks which values are reactive
• Effect types: functions declare which effects they may perform → compiler verifies all effects are handled
• UI types: View type ensures only valid UI expressions appear in view blocks

Error Messages

• Inspired by Elm: suggest fixes, show the specific constraint that failed, include code context
• Signal graph visualization in error output for dependency-related errors

Deliverable

The compiler catches: unhandled effects, type mismatches in signal derivations, invalid UI expressions — all with helpful error messages.

---

Phase 5 — Live Editor (Weeks 19–24)

Goal: Bidirectional structural editing — the killer feature.

Incremental Compiler

[NEW] compiler/crates/ds-incremental/

• Tracks which AST nodes changed between edits
• Re-analyzes only affected signal subgraph
• Re-generates only changed DOM bindings
• Target: <50ms from keystroke to live update

Editor Integration

[NEW] editor/

• Web-based editor (Monaco or CodeMirror 6 with custom language mode)
• Split pane: code ↔ live preview
• Code → Preview: incremental compile on every keystroke
• Preview → Code: click an element in preview → highlights source; drag to reorder → AST rewrite → code updates
• State preservation: signals retain their current values across edits

Visual Inspector

• Overlay on the live preview showing signal dependencies, constraint wires, spring states
• Click a DOM element → see its signal bindings, constraint relationships, effect dependencies

Deliverable

A working playground at dreamstack.dev where users can write DreamStack code and see live results with bidirectional editing.

---

Phase 6 — Production Readiness (Weeks 25–30)

Goal: SSR, ecosystem tooling, documentation, and real-world validation.

Server-Side Rendering

• Compile DreamStack to static HTML + hydration script
• Streaming SSR: emit HTML as constraint solver resolves layout
• Effect handlers for server context (no DOM, no springs)

Package System

• dreamstack add <package> — package registry for reusable views and effect handlers
• Standard library: common UI patterns (buttons, inputs, lists, modals, toasts)
• Interop: import/export with JavaScript modules

Documentation & Examples

• Language reference
• Tutorial: "Build a real app in 30 minutes"
• Cookbook: common patterns (forms, routing, data fetching, auth)
• Migration guide: "Coming from React" / "Coming from Svelte"

---

Implementation Language Choices

Component	Language	Rationale
Parser, Analyzer, Codegen, Type Checker	Rust	Performance, WASM target, memory safety without GC
Signal Runtime, Constraint Solver, Springs	Rust → WASM	Must run at 60fps, no GC pauses
DOM Bindings, RAF Scheduler	JavaScript	Must interact with browser APIs directly
CLI	Rust (clap)	Fast startup, single binary distribution
Editor	TypeScript	CodeMirror/Monaco ecosystem, web-native
Dev Server	Rust (axum)	Fast file watching, WebSocket HMR

---

Key Technical Risks

Risk	Mitigation
Compile-time signal analysis is undecidable in general	Restrict to analyzable patterns; fall back to runtime tracking for dynamic cases
Cassowary solver is O(n²) worst case	Incremental solving; partition independent constraint groups
WASM ↔ JS boundary overhead for DOM updates	Batch mutations; use FinalizationRegistry for memory management
Algebraic effects require delimited continuations	Use CPS transform in codegen; or one-shot continuations via WASM stack switching (Stage 3 proposal)
Homoiconic representation vs performance	Two representations: rich AST for editor, optimized IR for runtime

---

Verification Plan

Since this is a greenfield language/framework, verification is demo-driven:

Phase 0–1: Reactive Core

# Build the compiler and run the counter example
cargo build --release -p ds-cli
./target/release/dreamstack build examples/counter.ds
# Serve and verify in browser
./target/release/dreamstack dev examples/counter.ds
# Run signal propagation benchmarks
cargo bench -p ds-signals

Phase 2: Effects & Streams

# Run effect system unit tests
cargo test -p ds-effects
# Run the search example with mock handler (automated test)
./target/release/dreamstack test examples/search.ds

Phase 3: Layout & Animation

# Constraint solver correctness tests
cargo test -p ds-layout
# Visual regression: screenshot comparison of dashboard example
./target/release/dreamstack screenshot examples/dashboard.ds --compare snapshots/

Phase 5: Live Editor

• Manual verification: Open the playground, type code, verify preview updates <50ms
• Bidirectional test: Drag an element in preview, verify code updates correspondingly

Important

The user should weigh in on which phase to prioritize first. The plan assumes linear progression, but Phases 1–3 could be parallelized if multiple contributors are involved. Phase 4 (dependent types) is the highest-risk and could be deferred to a v2.

---

Phase 10 — Rapier2D Physics Engine (Completed)

Goal: Replace the Verlet spring-mass physics with Rapier2D for real rigid body simulation.

WASM Physics Engine

[NEW] engine/ds-physics/

• Rapier2D wrapped in a PhysicsWorld struct, compiled to WASM via wasm-pack
• Rigid body creation: create_soft_circle(), create_soft_rect()
• Force control: set_gravity() (wakes all sleeping bodies), apply_impulse()
• Query: get_body_center(), get_body_rotation(), get_body_positions(), body_count()
• Rendering: set_body_color(), body info with color, radius, dimensions
• Collision, restitution, friction, damping — all configurable per body
• 5 tests verifying world creation, stepping, body creation, gravity, and impulse

Deliverable

The WASM physics module at dist/pkg/ds_physics.js + ds_physics_bg.wasm, loadable from any HTML page.

---

Phase 11 — Physics in the Language (Completed)

Goal: Integrate Rapier2D physics as a first-class .ds language construct.

Compiler Additions

[MODIFY] compiler/ds-parser/src/ast.rs

• Added ContainerKind::Scene

[MODIFY] compiler/ds-parser/src/lexer.rs

• Added TokenKind::Scene and "scene" keyword mapping

[MODIFY] compiler/ds-parser/src/parser.rs

• scene dispatches to parse_container_with_props (width, height, gravity)
• circle, rect, line added to is_ui_element()

[MODIFY] compiler/ds-analyzer/src/signal_graph.rs

• Scene variant in collect_bindings for DOM binding extraction

[MODIFY] compiler/ds-codegen/src/js_emitter.rs

• emit_scene() — canvas creation, async WASM init, PhysicsWorld setup
• emit_scene_circle(), emit_scene_rect() — per-body codegen with unique variables
• emit_scene_set_color() — compile-time hex color parsing
• Reactive gravity via DS.effect(() => { _world.set_gravity(...); })
• requestAnimationFrame loop for physics stepping + canvas rendering
• Mouse drag interaction with impulse-based body movement

[NEW] examples/physics.ds

• 5 bodies (circles + rects), reactive gravity controls, drag interaction

Deliverable

physics.ds compiles to a working HTML page with WASM physics — 22KB output, no framework dependencies.