feat: add DreamStack project vision and detailed implementation plan documentation.

This commit is contained in:
enzotar 2026-02-24 23:49:59 -08:00
commit cc45557248
2 changed files with 611 additions and 0 deletions

341
DREAMSTACK.md Normal file
View file

@ -0,0 +1,341 @@
# DreamStack: Reinventing the UI from First Principles
> *What if we threw away every assumption about web UI frameworks and started over — with the sole goal of creating ultra-reactive, dynamic interfaces?*
---
## Vision
React was revolutionary in 2013. But it carries a decade of compromises: the virtual DOM, hooks with manual dependency arrays, re-rendering entire subtrees, CSS from 1996, animations bolted on as an afterthought. DreamStack asks: **what would we build today if none of that existed?**
The answer is a unified system where **UI is data, reactivity is automatic, effects are composable values, layout is constraint-based, animation is physics-native, and the editor and runtime are one.**
---
## The Language
Not Clojure. Not TypeScript. A new language that steals the best ideas from everywhere.
### Core Properties
| Property | Inspiration | Why |
|---|---|---|
| **Homoiconic** | Clojure, Lisp | UI = data. Code = data. Everything is transformable |
| **Dependent types** | Idris, Agda | Types that express "this button is disabled *when* the form is invalid" at the type level |
| **Algebraic effects** | Eff, Koka, OCaml 5 | Side effects as first-class, composable, interceptable values |
| **Reactive by default** | Svelte, Solid, Excel | No `useState`, no subscriptions. Values *are* reactive. Assignment *is* the API |
| **Structural typing** | TypeScript, Go | Flexible composition without class hierarchies |
| **Compiled to native + WASM** | Rust, Zig | Near-zero runtime overhead. No GC pauses during animations |
### Syntax
```
-- Signals are the primitive. Assignment propagates automatically.
let count = 0
let doubled = count * 2 -- derived, auto-tracked
let label = "Count: {doubled}" -- derived, auto-tracked
-- UI is data. No JSX, no templates, no virtual DOM.
view counter =
column [
text label -- auto-updates when count changes
button "+" { click: count += 1 }
-- Conditional UI is just pattern matching on signals
when count > 10 ->
text "🔥 On fire!" | animate fade-in 200ms
]
```
No hooks. No dependency arrays. No re-renders. **The compiler builds a fine-grained reactive graph at compile time.**
---
## Reactivity: Compile-Time Signal Graph
This is the biggest departure from React. React's model is fundamentally **pull-based** — re-render the tree, diff it, patch the DOM. That's backwards.
### Push-Based with Compile-Time Analysis
```
Signal: count
├──► Derived: doubled
│ └──► Derived: label
│ └──► [direct DOM binding] TextNode #47
└──► Condition: count > 10
└──► [mount/unmount] text "🔥 On fire!"
```
**Key principles:**
- **No virtual DOM.** The compiler knows at build time exactly which DOM nodes depend on which signals. When `count` changes, it updates *only* the specific text node and evaluates the condition. Nothing else runs.
- **No re-rendering.** Components don't re-execute. They execute *once* and set up reactive bindings.
- **No dependency arrays.** The compiler infers all dependencies from the code. You literally cannot forget one.
- **No stale closures.** Since there are no closures over render cycles, the entire class of bugs vanishes.
This extends Solid.js's approach with full compile-time analysis in a purpose-built language.
---
## Effect System: Algebraic Effects for Everything
Every side-effect — HTTP, animations, time, user input, clipboard, drag-and-drop — is an **algebraic effect**. Effects are values you can compose, intercept, retry, and test.
```
-- An effect is declared, not executed
effect fetchUser(id: UserId): Result<User, ApiError>
-- A component "performs" an effect — doesn't control HOW it runs
view profile(id: UserId) =
let user = perform fetchUser(id)
match user
Loading -> skeleton-loader
Ok(u) -> column [
avatar u.photo
text u.name
]
Err(e) -> error-card e | with retry: perform fetchUser(id)
-- At the app boundary, you provide the HANDLER
handle app with
fetchUser(id, resume) ->
let result = http.get "/api/users/{id}"
resume(result)
```
### Why This is Powerful
- **Testing:** Swap the handler. `handle app with mockFetchUser(...)` — no mocking libraries, no dependency injection frameworks.
- **Composition:** Effects compose naturally. An animation effect + a data-fetch effect combine without callback hell or `useEffect` chains.
- **Interceptors:** Want to add logging, caching, or retry logic? Add a handler layer. The component code never changes.
- **Time travel:** Since effects are values, you can record and replay them. Free undo/redo. Free debugging.
---
## Data Flow: Everything is a Stream
Forget the distinction between "state", "props", "events", and "side effects". **Everything is a stream of values over time.**
```
-- User input is a stream
let clicks = stream from button.click
let keypresses = stream from input.keydown
-- Derived streams with temporal operators
let search_query = keypresses
| map .value
| debounce 300ms
| distinct
-- API results are streams
let results = search_query
| flatmap (q -> http.get "/search?q={q}")
| catch-with []
-- UI binds to streams directly
view search =
column [
input { on-keydown: keypresses }
match results
Loading -> spinner
Data(rs) -> list rs (r -> search-result-card r)
Empty -> text "No results"
]
```
### Unification
This single abstraction covers everything:
| Concept | Traditional | DreamStack |
|---|---|---|
| User input | Event handlers | Stream |
| Network responses | Promises / async-await | Stream |
| Animations | CSS transitions / JS libraries | Stream of interpolated values |
| Timers | `setInterval` / `setTimeout` | Stream |
| WebSockets | Callback-based | Stream |
| Drag events | Complex event handler state machines | Stream of positions |
One abstraction. One composition model. Everything snaps together.
---
## Layout: Constraint-Based, Not Box-Based
CSS's box model is from 1996. Flexbox and Grid are patches on a fundamentally limited system. DreamStack starts over with a **constraint solver**.
```
-- Declare relationships, not boxes
layout dashboard =
let sidebar.width = clamp(200px, 20vw, 350px)
let main.width = viewport.width - sidebar.width
let header.height = 64px
let content.height = viewport.height - header.height
-- Constraints, not nesting
sidebar.left = 0
sidebar.top = header.height
sidebar.height = content.height
main.left = sidebar.right
main.top = header.height
main.right = viewport.right
-- Responsive is just different constraints
when viewport.width < 768px ->
sidebar.width = 0 -- collapses
main.left = 0 -- takes full width
```
Inspired by Apple's **Auto Layout** (Cassowary constraint solver), but made reactive. When `viewport.width` changes, the constraint solver re-solves and updates positions in a single pass. No layout thrashing. No "CSS specificity wars."
### Advantages Over CSS
- **No cascade conflicts** — constraints are explicit and local
- **No z-index hell** — layering is declarative
- **No media query breakpoints** — responsiveness emerges from constraints
- **Animations are free** — animating a constraint target is the same as setting it
---
## Animation: First-Class, Physics-Based, Interruptible
Animations aren't CSS transitions bolted on after the fact. They're part of the reactive graph.
```
-- A spring is a signal that animates toward its target
let panel_x = spring(target: 0, stiffness: 300, damping: 30)
-- Change the target → the spring animates automatically
on toggle_sidebar ->
panel_x.target = if open then 250 else 0
-- Gestures feed directly into springs
on drag(event) ->
panel_x.target = event.x -- spring follows finger
on drag-end(event) ->
panel_x.target = snap-to-nearest [0, 250] event.x
-- UI just reads the spring's current value
view sidebar =
panel { x: panel_x } [
nav-items
]
```
### Design Principles
- **Interruptible:** Start a new animation mid-flight. The spring handles the physics — no jarring jumps, no "wait for the current animation to finish."
- **Gesture-driven:** Touch/mouse input feeds directly into the animation model. No separate "gesture handler" → "state update" → "CSS transition" pipeline.
- **60fps guaranteed:** Springs resolve on the GPU. The main thread never blocks.
- **Composable:** Combine springs, easing curves, and physics simulations using the same stream operators as everything else.
---
## Runtime: Tiny, Compiled, No GC Pauses
```
┌──────────────────────────────────────────┐
│ Compiled Output │
├──────────────────────────────────────────┤
│ Signal Graph (static DAG) ~2KB │
│ Constraint Solver (layout) ~4KB │
│ Spring Physics (animations) ~1KB │
│ Effect Runtime (handlers) ~2KB │
│ DOM Patcher (surgical updates) ~3KB │
├──────────────────────────────────────────┤
│ Total Runtime: ~12KB │
│ (vs React ~45KB + ReactDOM ~130KB) │
└──────────────────────────────────────────┘
```
The compiler does the heavy lifting. The runtime is tiny because there is:
- No virtual DOM diffing algorithm
- No fiber scheduler
- No hook state management system
- No reconciliation algorithm
- No garbage collector pauses during animation frames
---
## The Killer Feature: Live Structural Editing
Because the language is homoiconic (code = data), the **editor IS the runtime:**
```
┌─────────────────────────────────┐
│ Live Editor │
│ │
│ view counter = │ ← Edit this...
│ column [ │
│ text "Count: {count}" │
│ button "+" { ... } │
│ ] │
│ │
│ ───────────────────────────── │
│ │
│ ┌─────────────┐ │ ← ...see this update
│ │ Count: 42 │ │ instantly, with state
│ │ [ + ] │ │ preserved.
│ └─────────────┘ │
│ │
└─────────────────────────────────┘
```
- Drag and drop UI elements in the preview → the **code updates**.
- Edit the code → the **preview updates**.
- Both directions, simultaneously, with state preserved.
- Not a design tool that generates code — **the code IS the design tool**.
This is possible because:
1. **Homoiconicity** means UI structure is inspectable and modifiable data
2. **Immutable signals** mean state survives code changes
3. **Compile-time signal graph** means changes are surgical, not full-page reloads
---
## Comparison to Existing Approaches
| Capability | React | Svelte 5 | Solid.js | Flutter | **DreamStack** |
|---|---|---|---|---|---|
| Reactivity | Pull (re-render + diff) | Compile-time runes | Fine-grained signals | Widget rebuild | **Compile-time signal DAG** |
| Side effects | `useEffect` + deps array | `$effect` | `createEffect` | Lifecycle methods | **Algebraic effects** |
| Layout | CSS (external) | CSS (scoped) | CSS (external) | Constraint-based | **Constraint solver (native)** |
| Animation | 3rd party libs | 3rd party libs | 3rd party libs | Physics-based (native) | **Physics-based (native)** |
| SSR | Yes (complex) | Yes | Yes | No (web) | **Yes (compiled)** |
| Runtime size | ~175KB | ~2KB | ~7KB | ~2MB (web) | **~12KB** |
| Live editing | Fast Refresh (lossy) | HMR | HMR | Hot reload | **Bidirectional structural** |
| Type safety | TypeScript (bolt-on) | TypeScript (bolt-on) | TypeScript (bolt-on) | Dart (native) | **Dependent types (native)** |
| UI = Data | No (JSX compiled away) | No (templates) | No (JSX compiled) | No (widget classes) | **Yes (homoiconic)** |
---
## Fragments of the Future, Today
The closest approximations to pieces of this vision:
- **Solid.js** — fine-grained reactivity, no VDOM, ~7KB runtime
- **Svelte 5 Runes** — compiler-driven reactivity, tiny output
- **Elm** — algebraic effects-adjacent, immutable state, strong types
- **Flutter** — constraint layout, physics-based animation, hot reload
- **Clojure/Reagent** — homoiconicity, UI as data, ratom reactivity
- **Koka** — algebraic effect system in a practical language
- **Apple Auto Layout** — Cassowary constraint solver for UI
- **Excel** — reactive by default, dependency auto-tracking
Nobody has unified them. That's the opportunity.
---
## Design Philosophy
1. **The compiler is the framework.** Move work from runtime to compile time. The less code that runs in the browser, the faster the UI.
2. **Reactivity is not a feature, it's the default.** Every value is live. Every binding is automatic. You opt *out* of reactivity, not *in*.
3. **Effects are values, not side-channels.** Making side effects first-class and composable eliminates the largest source of bugs in modern UIs.
4. **Layout and animation are not afterthoughts.** They're core primitives, not CSS bolt-ons or third-party libraries.
5. **The editor and the runtime are the same thing.** Bidirectional editing collapses the design-develop gap entirely.
6. **UI is data, all the way down.** If you can't `map` over your UI structure, your abstraction is wrong.

270
IMPLEMENTATION_PLAN.md Normal file
View file

@ -0,0 +1,270 @@
# DreamStack Implementation Plan
Build a working prototype of the DreamStack vision — a new UI framework with compile-time reactivity, algebraic effects, constraint layout, and physics animation. The strategy is to **bootstrap pragmatically**: use Rust for the compiler/runtime core, target WASM + DOM, and prove each pillar with working demos before unifying them.
---
## Phase 0 — Foundation (Weeks 13)
**Goal:** A working compiler that turns DreamStack syntax into executable WASM + JS glue.
### Compiler Pipeline (Rust)
#### [NEW] `compiler/crates/ds-parser/`
- Hand-written recursive descent parser for DreamStack syntax
- Produces an AST of `view`, `let`, `when`, `match`, `effect`, `on`, `stream`, `layout` constructs
- Homoiconic representation: AST nodes are also the runtime data structure (tagged vectors/maps)
#### [NEW] `compiler/crates/ds-analyzer/`
- **Signal graph extraction:** Static analysis pass that identifies signals, derived values, and their dependencies → builds a DAG
- **Effect collection:** Catalogs all `perform` sites and their expected effect signatures
- **Type inference:** Hindley-Milner base with structural typing; dependent types deferred to Phase 4
#### [NEW] `compiler/crates/ds-codegen/`
- Emits **Rust → WASM** for:
- Signal graph runtime (push-based notification)
- Constraint solver (Cassowary algorithm)
- Spring physics engine
- Emits **JS glue** for DOM bindings — surgical `textContent`, `setAttribute`, `insertBefore`, `removeChild` calls (no VDOM)
- Output format: single `.wasm` + single `.js` module
#### [NEW] `compiler/crates/ds-cli/`
- `dreamstack build <file.ds>` → produces `.wasm` + `.js` + `.html`
- `dreamstack dev <file.ds>` → dev server with file watching + hot-reload
- `dreamstack check <file.ds>` → type check + signal graph validation
### Deliverable
A counter app (`examples/counter.ds`) compiles and runs in the browser with reactive signal updates, no VDOM.
---
## Phase 1 — Reactive Core (Weeks 46)
**Goal:** Fine-grained reactivity with compile-time dependency tracking, proving the "no re-render" model.
### Signal Runtime (Rust → WASM)
#### [NEW] `runtime/crates/ds-signals/`
- `Signal<T>` — mutable source, notifies dependents on write
- `Derived<T>` — computed from signals, lazy + cached, auto-invalidated
- `Effect` — runs side-effect when dependencies change (for DOM updates only — user-facing effects use algebraic effects)
- Batching: multiple signal writes in one tick coalesce into a single propagation pass
- Topological sort of the DAG ensures glitch-free propagation (no intermediate states visible)
### DOM Binding Layer (JS)
#### [NEW] `runtime/js/dom-bindings.js`
- Minimal JS layer (~3KB) that the codegen targets
- Functions: `bindText(nodeId, signalId)`, `bindAttr(nodeId, attr, signalId)`, `bindVisible(nodeId, signalId)`, `mountChildren(parentId, listSignalId, templateFn)`
- Each binding registers itself as an effect on the signal graph
- No framework runtime — just a bag of DOM mutation functions
### Benchmark Suite
#### [NEW] `bench/`
- Signal propagation: 1K / 10K / 100K signals, measure update latency
- DOM updates: compare against Solid.js, Svelte 5, vanilla JS
- Target: <1ms for 1K signal cascade, <16ms for 10K
### Deliverable
A TodoMVC app (`examples/todomvc.ds`) with add/remove/toggle/filter — all reactive, no VDOM, benchmarked against Solid.js.
---
## Phase 2 — Effect System & Streams (Weeks 710)
**Goal:** Algebraic effects for side-effects, stream primitives for temporal data.
### Algebraic Effects (Rust → WASM)
#### [NEW] `runtime/crates/ds-effects/`
- Effect declarations: `effect fetchUser(id: UserId): Result<User, ApiError>`
- `perform` keyword: suspends the current continuation, delegates to the nearest handler
- `handle ... with` blocks: install effect handlers at any level of the component tree
- Continuation-based: handlers receive a `resume` function to continue execution
- Built-in effects: `Http`, `Time`, `Random`, `Storage`, `Console`
### Stream Engine
#### [NEW] `runtime/crates/ds-streams/`
- `Stream<T>` — push-based async sequence
- Core operators: `map`, `filter`, `flatmap`, `merge`, `combine`, `debounce`, `throttle`, `distinct`, `take`, `skip`, `scan`
- `stream from <event>` — creates a stream from DOM events
- Streams integrate with the signal graph: a stream's latest value is a signal
- Backpressure: drop or buffer strategies, configurable per stream
### Deliverable
A search-with-autocomplete app (`examples/search.ds`) using `debounce`, `flatmap` over an HTTP effect, with the effect handler swappable for testing.
---
## Phase 3 — Layout & Animation (Weeks 1114)
**Goal:** Constraint-based layout engine and physics-based animation, both running in WASM.
### Constraint Solver
#### [NEW] `runtime/crates/ds-layout/`
- Implement the **Cassowary** simplex-based constraint solving algorithm in Rust
- Constraint types: `eq`, `gte`, `lte`, `clamp`, `ratio`
- Reactive integration: constraints reference signals → solver re-runs when inputs change
- Output: absolute `(x, y, width, height)` for each element, applied directly to DOM via `transform: translate()` (avoids reflow)
- Performance target: solve 500 constraints in <2ms
### Spring Physics
#### [NEW] `runtime/crates/ds-springs/`
- `Spring { value, velocity, target, stiffness, damping, mass }`
- RK4 integrator, fixed timestep (1/120s), interpolated for display
- `spring.target = x` triggers animation to new target (interruptible by default)
- Springs are signals — anything that reads `spring.value` auto-updates
- Gesture integration: direct-manipulation by setting `spring.target` to pointer position
### Animation Scheduler
#### [NEW] `runtime/js/raf-scheduler.js`
- Drives springs via `requestAnimationFrame`
- When no springs are active, scheduler sleeps (zero CPU when idle)
- Coordinates with signal graph: spring value changes go through normal propagation
### Deliverable
A draggable panel layout (`examples/dashboard.ds`) with resizable sidebar (constraints) and spring-animated panel transitions.
---
## Phase 4 — Type System (Weeks 1518)
**Goal:** Dependent types for UI correctness guarantees.
### Type Checker
#### [NEW] `compiler/crates/ds-types/`
- **Base:** Hindley-Milner with let-generalization, structural records
- **Refinement types:** `type PositiveInt = { n: Int | n > 0 }` — checked at compile time where possible, runtime where not
- **Signal-aware types:** `Signal<T>` is a first-class type; the compiler tracks which values are reactive
- **Effect types:** functions declare which effects they may perform → compiler verifies all effects are handled
- **UI types:** `View` type ensures only valid UI expressions appear in `view` blocks
### Error Messages
- Inspired by Elm: suggest fixes, show the specific constraint that failed, include code context
- Signal graph visualization in error output for dependency-related errors
### Deliverable
The compiler catches: unhandled effects, type mismatches in signal derivations, invalid UI expressions — all with helpful error messages.
---
## Phase 5 — Live Editor (Weeks 1924)
**Goal:** Bidirectional structural editing — the killer feature.
### Incremental Compiler
#### [NEW] `compiler/crates/ds-incremental/`
- Tracks which AST nodes changed between edits
- Re-analyzes only affected signal subgraph
- Re-generates only changed DOM bindings
- Target: <50ms from keystroke to live update
### Editor Integration
#### [NEW] `editor/`
- Web-based editor (Monaco or CodeMirror 6 with custom language mode)
- Split pane: code ↔ live preview
- **Code → Preview:** incremental compile on every keystroke
- **Preview → Code:** click an element in preview → highlights source; drag to reorder → AST rewrite → code updates
- State preservation: signals retain their current values across edits
### Visual Inspector
- Overlay on the live preview showing signal dependencies, constraint wires, spring states
- Click a DOM element → see its signal bindings, constraint relationships, effect dependencies
### Deliverable
A working playground at `dreamstack.dev` where users can write DreamStack code and see live results with bidirectional editing.
---
## Phase 6 — Production Readiness (Weeks 2530)
**Goal:** SSR, ecosystem tooling, documentation, and real-world validation.
### Server-Side Rendering
- Compile DreamStack to static HTML + hydration script
- Streaming SSR: emit HTML as constraint solver resolves layout
- Effect handlers for server context (no DOM, no springs)
### Package System
- `dreamstack add <package>` — package registry for reusable views and effect handlers
- Standard library: common UI patterns (buttons, inputs, lists, modals, toasts)
- Interop: import/export with JavaScript modules
### Documentation & Examples
- Language reference
- Tutorial: "Build a real app in 30 minutes"
- Cookbook: common patterns (forms, routing, data fetching, auth)
- Migration guide: "Coming from React" / "Coming from Svelte"
---
## Implementation Language Choices
| Component | Language | Rationale |
|---|---|---|
| Parser, Analyzer, Codegen, Type Checker | **Rust** | Performance, WASM target, memory safety without GC |
| Signal Runtime, Constraint Solver, Springs | **Rust → WASM** | Must run at 60fps, no GC pauses |
| DOM Bindings, RAF Scheduler | **JavaScript** | Must interact with browser APIs directly |
| CLI | **Rust** (clap) | Fast startup, single binary distribution |
| Editor | **TypeScript** | CodeMirror/Monaco ecosystem, web-native |
| Dev Server | **Rust** (axum) | Fast file watching, WebSocket HMR |
---
## Key Technical Risks
| Risk | Mitigation |
|---|---|
| Compile-time signal analysis is undecidable in general | Restrict to analyzable patterns; fall back to runtime tracking for dynamic cases |
| Cassowary solver is O(n²) worst case | Incremental solving; partition independent constraint groups |
| WASM ↔ JS boundary overhead for DOM updates | Batch mutations; use `FinalizationRegistry` for memory management |
| Algebraic effects require delimited continuations | Use CPS transform in codegen; or one-shot continuations via WASM stack switching (Stage 3 proposal) |
| Homoiconic representation vs performance | Two representations: rich AST for editor, optimized IR for runtime |
---
## Verification Plan
Since this is a greenfield language/framework, verification is demo-driven:
### Phase 01: Reactive Core
```bash
# Build the compiler and run the counter example
cargo build --release -p ds-cli
./target/release/dreamstack build examples/counter.ds
# Serve and verify in browser
./target/release/dreamstack dev examples/counter.ds
# Run signal propagation benchmarks
cargo bench -p ds-signals
```
### Phase 2: Effects & Streams
```bash
# Run effect system unit tests
cargo test -p ds-effects
# Run the search example with mock handler (automated test)
./target/release/dreamstack test examples/search.ds
```
### Phase 3: Layout & Animation
```bash
# Constraint solver correctness tests
cargo test -p ds-layout
# Visual regression: screenshot comparison of dashboard example
./target/release/dreamstack screenshot examples/dashboard.ds --compare snapshots/
```
### Phase 5: Live Editor
- **Manual verification:** Open the playground, type code, verify preview updates <50ms
- **Bidirectional test:** Drag an element in preview, verify code updates correspondingly
> [!IMPORTANT]
> The user should weigh in on which phase to prioritize first. The plan assumes linear progression, but Phases 13 could be parallelized if multiple contributors are involved. Phase 4 (dependent types) is the highest-risk and could be deferred to a v2.