dreamstack/DREAMSTACK.md

# DreamStack: Reinventing the UI from First Principles

> *What if we threw away every assumption about web UI frameworks and started over — with the sole goal of creating ultra-reactive, dynamic interfaces?*

---

## Implementation Status ✅

DreamStack is **real and running** — 6 Rust crates, 34 tests, 8 examples, ~7KB output.

```
.ds source → ds-parser → ds-analyzer → ds-codegen → JavaScript
                ↓              ↓
            ds-types       ds-layout
          (type checker)  (Cassowary solver)
```

### What Works Today

| Feature | Syntax | Status |
|---------|--------|--------|
| Signals | `let count = 0` | ✅ Fine-grained, auto-tracked |
| Derived | `let doubled = count * 2` | ✅ Lazy |
| Interpolation | `"Count: {count}"` | ✅ Reactive |
| Conditional | `when count > 5 -> text "hi"` | ✅ Mount/unmount |
| If/else | `if x then a else b` | ✅ |
| Match | `match state \| Loading -> ...` | ✅ |
| List rendering | `for item in items -> text item` | ✅ Reactive |
| Components | `component Card(title) = ...` | ✅ Props |
| Effects | `effect fetch(id): Result` / `perform` | ✅ Algebraic |
| Streams | `debounce`, `throttle`, `distinct` | ✅ |
| Springs | `spring(target: 0, stiffness: 300)` | ✅ Physics |
| Layout | Cassowary constraint solver | ✅ |
| Types | `Signal<Int>`, `Derived<Bool>` | ✅ Hindley-Milner |
| Dev server | `dreamstack dev app.ds` | ✅ HMR |
| Router | `route "/path" -> body` / `navigate` | ✅ Hash-based |
| Two-way binding | `input { bind: name }` | ✅ Signal ↔ input |
| Async resources | `DS.resource()` / `DS.fetchJSON()` | ✅ Loading/Ok/Err |
| Springs | `let x = spring(200)` | ✅ RK4 physics |
| Constraints | `constrain el.width = expr` | ✅ Reactive solver |

### DreamStack vs React

| | **DreamStack** | **React** |
|---|---|---|
| Reactivity | Fine-grained signals, surgical DOM | VDOM diff, re-render subtrees |
| State | `count += 1` direct | `setState(c => c+1)` immutable |
| Derived | `let d = count * 2` auto | `useMemo(() => ..., [deps])` manual |
| Effects | Auto-tracked, algebraic | `useEffect(..., [deps])` manual |
| Conditional | `when x -> text "y"` | `{x && <span>y</span>}` |
| Lists | `for item in items -> ...` | `{items.map(i => ...)}` |
| Router | `route "/path" -> body` | `react-router` (external) |
| Forms | `input { bind: name }` | `useState` + `onChange` (manual) |
| Animation | Built-in springs | framer-motion (external) |
| Layout | Built-in Cassowary | CSS only |
| Types | Native HM, `Signal<T>` | TypeScript (external) |
| Bundle | **~7KB** | **~175KB** |
| Ecosystem | New | Massive |

### Benchmarks (signal propagation)

| Benchmark | Ops/sec | Signals |
|-----------|---------|---------|
| Wide Fan-Out (1→1000) | 46K | 1,001 |
| Deep Chain (100 deep) | 399K | 100 |
| Diamond Dependency | 189K | 4 |
| Batch Update (50) | 61K | 50 |
| Mixed Read/Write | 242K | 10 |

### Examples

`counter.ds` · `list.ds` · `router.ds` · `form.ds` · `springs.ds` · `todomvc.html` · `search.html` · `dashboard.html` · `playground.html` · `showcase.html` · `benchmarks.html`

---

## Vision

React was revolutionary in 2013. But it carries a decade of compromises: the virtual DOM, hooks with manual dependency arrays, re-rendering entire subtrees, CSS from 1996, animations bolted on as an afterthought. DreamStack asks: **what would we build today if none of that existed?**

The answer is a unified system where **UI is data, reactivity is automatic, effects are composable values, layout is constraint-based, animation is physics-native, and the editor and runtime are one.**

---

## The Language

Not Clojure. Not TypeScript. A new language that steals the best ideas from everywhere.

### Core Properties

| Property | Inspiration | Why |
|---|---|---|
| **Homoiconic** | Clojure, Lisp | UI = data. Code = data. Everything is transformable |
| **Dependent types** | Idris, Agda | Types that express "this button is disabled *when* the form is invalid" at the type level |
| **Algebraic effects** | Eff, Koka, OCaml 5 | Side effects as first-class, composable, interceptable values |
| **Reactive by default** | Svelte, Solid, Excel | No `useState`, no subscriptions. Values *are* reactive. Assignment *is* the API |
| **Structural typing** | TypeScript, Go | Flexible composition without class hierarchies |
| **Compiled to native + WASM** | Rust, Zig | Near-zero runtime overhead. No GC pauses during animations |

### Syntax

```
-- Signals are the primitive. Assignment propagates automatically.
let count = 0
let doubled = count * 2          -- derived, auto-tracked
let label = "Count: {doubled}"   -- derived, auto-tracked

-- UI is data. No JSX, no templates, no virtual DOM.
view counter =
  column [
    text label                    -- auto-updates when count changes
    button "+" { click: count += 1 }

    -- Conditional UI is just pattern matching on signals
    when count > 10 ->
      text "🔥 On fire!" | animate fade-in 200ms
  ]
```

No hooks. No dependency arrays. No re-renders. **The compiler builds a fine-grained reactive graph at compile time.**

---

## Reactivity: Compile-Time Signal Graph

This is the biggest departure from React. React's model is fundamentally **pull-based** — re-render the tree, diff it, patch the DOM. That's backwards.

### Push-Based with Compile-Time Analysis

```
Signal: count
  ├──► Derived: doubled
  │       └──► Derived: label
  │               └──► [direct DOM binding] TextNode #47
  └──► Condition: count > 10
          └──► [mount/unmount] text "🔥 On fire!"
```

**Key principles:**

- **No virtual DOM.** The compiler knows at build time exactly which DOM nodes depend on which signals. When `count` changes, it updates *only* the specific text node and evaluates the condition. Nothing else runs.
- **No re-rendering.** Components don't re-execute. They execute *once* and set up reactive bindings.
- **No dependency arrays.** The compiler infers all dependencies from the code. You literally cannot forget one.
- **No stale closures.** Since there are no closures over render cycles, the entire class of bugs vanishes.

This extends Solid.js's approach with full compile-time analysis in a purpose-built language.

---

## Effect System: Algebraic Effects for Everything

Every side-effect — HTTP, animations, time, user input, clipboard, drag-and-drop — is an **algebraic effect**. Effects are values you can compose, intercept, retry, and test.

```
-- An effect is declared, not executed
effect fetchUser(id: UserId): Result<User, ApiError>

-- A component "performs" an effect — doesn't control HOW it runs
view profile(id: UserId) =
  let user = perform fetchUser(id)

  match user
    Loading -> skeleton-loader
    Ok(u)   -> column [
      avatar u.photo
      text u.name
    ]
    Err(e)  -> error-card e | with retry: perform fetchUser(id)

-- At the app boundary, you provide the HANDLER
handle app with
  fetchUser(id, resume) ->
    let result = http.get "/api/users/{id}"
    resume(result)
```

### Why This is Powerful

- **Testing:** Swap the handler. `handle app with mockFetchUser(...)` — no mocking libraries, no dependency injection frameworks.
- **Composition:** Effects compose naturally. An animation effect + a data-fetch effect combine without callback hell or `useEffect` chains.
- **Interceptors:** Want to add logging, caching, or retry logic? Add a handler layer. The component code never changes.
- **Time travel:** Since effects are values, you can record and replay them. Free undo/redo. Free debugging.

---

## Data Flow: Everything is a Stream

Forget the distinction between "state", "props", "events", and "side effects". **Everything is a stream of values over time.**

```
-- User input is a stream
let clicks = stream from button.click
let keypresses = stream from input.keydown

-- Derived streams with temporal operators
let search_query = keypresses
  | map .value
  | debounce 300ms
  | distinct

-- API results are streams
let results = search_query
  | flatmap (q -> http.get "/search?q={q}")
  | catch-with []

-- UI binds to streams directly
view search =
  column [
    input { on-keydown: keypresses }

    match results
      Loading  -> spinner
      Data(rs) -> list rs (r -> search-result-card r)
      Empty    -> text "No results"
  ]
```

### Unification

This single abstraction covers everything:

| Concept | Traditional | DreamStack |
|---|---|---|
| User input | Event handlers | Stream |
| Network responses | Promises / async-await | Stream |
| Animations | CSS transitions / JS libraries | Stream of interpolated values |
| Timers | `setInterval` / `setTimeout` | Stream |
| WebSockets | Callback-based | Stream |
| Drag events | Complex event handler state machines | Stream of positions |

One abstraction. One composition model. Everything snaps together.

---

## Layout: Constraint-Based, Not Box-Based

CSS's box model is from 1996. Flexbox and Grid are patches on a fundamentally limited system. DreamStack starts over with a **constraint solver**.

```
-- Declare relationships, not boxes
layout dashboard =
  let sidebar.width = clamp(200px, 20vw, 350px)
  let main.width = viewport.width - sidebar.width
  let header.height = 64px
  let content.height = viewport.height - header.height

  -- Constraints, not nesting
  sidebar.left = 0
  sidebar.top = header.height
  sidebar.height = content.height

  main.left = sidebar.right
  main.top = header.height
  main.right = viewport.right

  -- Responsive is just different constraints
  when viewport.width < 768px ->
    sidebar.width = 0           -- collapses
    main.left = 0               -- takes full width
```

Inspired by Apple's **Auto Layout** (Cassowary constraint solver), but made reactive. When `viewport.width` changes, the constraint solver re-solves and updates positions in a single pass. No layout thrashing. No "CSS specificity wars."

### Advantages Over CSS

- **No cascade conflicts** — constraints are explicit and local
- **No z-index hell** — layering is declarative
- **No media query breakpoints** — responsiveness emerges from constraints
- **Animations are free** — animating a constraint target is the same as setting it

---

## Animation: First-Class, Physics-Based, Interruptible

Animations aren't CSS transitions bolted on after the fact. They're part of the reactive graph.

```
-- A spring is a signal that animates toward its target
let panel_x = spring(target: 0, stiffness: 300, damping: 30)

-- Change the target → the spring animates automatically
on toggle_sidebar ->
  panel_x.target = if open then 250 else 0

-- Gestures feed directly into springs
on drag(event) ->
  panel_x.target = event.x        -- spring follows finger

on drag-end(event) ->
  panel_x.target = snap-to-nearest [0, 250] event.x

-- UI just reads the spring's current value
view sidebar =
  panel { x: panel_x } [
    nav-items
  ]
```

### Design Principles

- **Interruptible:** Start a new animation mid-flight. The spring handles the physics — no jarring jumps, no "wait for the current animation to finish."
- **Gesture-driven:** Touch/mouse input feeds directly into the animation model. No separate "gesture handler" → "state update" → "CSS transition" pipeline.
- **60fps guaranteed:** Springs resolve on the GPU. The main thread never blocks.
- **Composable:** Combine springs, easing curves, and physics simulations using the same stream operators as everything else.

---

## Runtime: Tiny, Compiled, No GC Pauses

```
┌──────────────────────────────────────────┐
│              Compiled Output             │
├──────────────────────────────────────────┤
│  Signal Graph (static DAG)         ~2KB  │
│  Constraint Solver (layout)        ~4KB  │
│  Spring Physics (animations)       ~1KB  │
│  Effect Runtime (handlers)         ~2KB  │
│  DOM Patcher (surgical updates)    ~3KB  │
├──────────────────────────────────────────┤
│  Total Runtime:                   ~12KB  │
│  (vs React ~45KB + ReactDOM ~130KB)      │
└──────────────────────────────────────────┘
```

The compiler does the heavy lifting. The runtime is tiny because there is:

- No virtual DOM diffing algorithm
- No fiber scheduler
- No hook state management system
- No reconciliation algorithm
- No garbage collector pauses during animation frames

---

## The Killer Feature: Live Structural Editing

Because the language is homoiconic (code = data), the **editor IS the runtime:**

```
┌─────────────────────────────────┐
│  Live Editor                    │
│                                 │
│  view counter =                 │  ← Edit this...
│    column [                     │
│      text "Count: {count}"     │
│      button "+" { ... }         │
│    ]                            │
│                                 │
│  ─────────────────────────────  │
│                                 │
│  ┌─────────────┐               │  ← ...see this update
│  │ Count: 42   │               │     instantly, with state
│  │   [ + ]     │               │     preserved.
│  └─────────────┘               │
│                                 │
└─────────────────────────────────┘
```

- Drag and drop UI elements in the preview → the **code updates**.
- Edit the code → the **preview updates**.
- Both directions, simultaneously, with state preserved.
- Not a design tool that generates code — **the code IS the design tool**.

This is possible because:
1. **Homoiconicity** means UI structure is inspectable and modifiable data
2. **Immutable signals** mean state survives code changes
3. **Compile-time signal graph** means changes are surgical, not full-page reloads

---

## Comparison to Existing Approaches

| Capability | React | Svelte 5 | Solid.js | Flutter | **DreamStack** |
|---|---|---|---|---|---|
| Reactivity | Pull (re-render + diff) | Compile-time runes | Fine-grained signals | Widget rebuild | **Compile-time signal DAG** |
| Side effects | `useEffect` + deps array | `$effect` | `createEffect` | Lifecycle methods | **Algebraic effects** |
| Layout | CSS (external) | CSS (scoped) | CSS (external) | Constraint-based | **Constraint solver (native)** |
| Animation | 3rd party libs | 3rd party libs | 3rd party libs | Physics-based (native) | **Physics-based (native)** |
| SSR | Yes (complex) | Yes | Yes | No (web) | **Yes (compiled)** |
| Runtime size | ~175KB | ~2KB | ~7KB | ~2MB (web) | **~12KB** |
| Live editing | Fast Refresh (lossy) | HMR | HMR | Hot reload | **Bidirectional structural** |
| Type safety | TypeScript (bolt-on) | TypeScript (bolt-on) | TypeScript (bolt-on) | Dart (native) | **Dependent types (native)** |
| UI = Data | No (JSX compiled away) | No (templates) | No (JSX compiled) | No (widget classes) | **Yes (homoiconic)** |

---

## Fragments of the Future, Today

The closest approximations to pieces of this vision:

- **Solid.js** — fine-grained reactivity, no VDOM, ~7KB runtime
- **Svelte 5 Runes** — compiler-driven reactivity, tiny output
- **Elm** — algebraic effects-adjacent, immutable state, strong types
- **Flutter** — constraint layout, physics-based animation, hot reload
- **Clojure/Reagent** — homoiconicity, UI as data, ratom reactivity
- **Koka** — algebraic effect system in a practical language
- **Apple Auto Layout** — Cassowary constraint solver for UI
- **Excel** — reactive by default, dependency auto-tracking

Nobody has unified them. That's the opportunity.

---

## Design Philosophy

1. **The compiler is the framework.** Move work from runtime to compile time. The less code that runs in the browser, the faster the UI.
2. **Reactivity is not a feature, it's the default.** Every value is live. Every binding is automatic. You opt *out* of reactivity, not *in*.
3. **Effects are values, not side-channels.** Making side effects first-class and composable eliminates the largest source of bugs in modern UIs.
4. **Layout and animation are not afterthoughts.** They're core primitives, not CSS bolt-ons or third-party libraries.
5. **The editor and the runtime are the same thing.** Bidirectional editing collapses the design-develop gap entirely.
6. **UI is data, all the way down.** If you can't `map` over your UI structure, your abstraction is wrong.
7. **Any input bitstream → any output bitstream.** The UI is just one codec. Tomorrow's neural nets generate the pixels directly.

---

## Phase 7: Universal Bitstream Streaming

> *Stream the whole UI as bytes. Neural nets will generate the pixels, acoustics, and actuator commands.*

DreamStack's `engine/ds-stream` crate implements a universal binary protocol for streaming any I/O:

```
┌──────────┐    WebSocket / WebRTC    ┌──────────┐
│  Source   │ ──────frames (bytes)──► │ Receiver  │
│ (renders) │ ◄──────inputs (bytes)── │ (~250 LOC)│
└──────────┘                          └──────────┘
```

### Binary Protocol (16-byte header)

| Field | Size | Description |
|-------|------|-------------|
| type | u8 | Frame/input type (pixels, audio, haptic, neural, BCI) |
| flags | u8 | Input flag, keyframe flag, compression flag |
| seq | u16 | Sequence number |
| timestamp | u32 | Relative ms since stream start |
| width | u16 | Frame width or channel count |
| height | u16 | Frame height or sample rate |
| length | u32 | Payload length |

### Output Types
- `Pixels` (0x01) — raw RGBA framebuffer
- `Audio` (0x10) — PCM audio samples
- `Haptic` (0x20) — vibration/actuator commands
- `NeuralFrame` (0x40) — neural-generated pixels *(future)*
- `NeuralAudio` (0x41) — neural speech/music synthesis *(future)*
- `NeuralActuator` (0x42) — learned motor control *(future)*

### Input Types
- `Pointer` (0x01) — mouse/touch position + buttons
- `Key` (0x10) — keyboard events
- `Gamepad` (0x30) — controller axes + buttons
- `BciInput` (0x90) — brain-computer interface *(future)*

### Demos
- `examples/stream-source.html` — Springs demo captures canvas → streams pixels at 30fps
- `examples/stream-receiver.html` — Thin client (~250 lines, no framework) renders bytes

### Run It
```bash
cargo run -p ds-stream              # start relay on :9100
open examples/stream-source.html    # source: renders + streams
open examples/stream-receiver.html  # receiver: displays bytes
```