amir/dreamstack
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
dreamstack/DREAMSTACK.md
enzotar 7f795eac6a docs: add compiler-native streaming syntax to language reference 
- Added streaming (source/receiver) to status table
- Documented 'stream' declaration, 'stream from' expression
- Streaming modes table (signal/delta/pixel)
- CLI 'dreamstack stream' usage
2026-02-25 14:37:16 -08:00

564 lines
22 KiB
Markdown
Raw Blame History

# DreamStack: Reinventing the UI from First Principles
> *What if we threw away every assumption about web UI frameworks and started over — with the sole goal of creating ultra-reactive, dynamic interfaces?*
---
## Implementation Status ✅
DreamStack is **real and running** — 7 Rust crates, 39 tests, 9 examples, ~7KB output + WASM physics.
```
.ds source → ds-parser → ds-analyzer → ds-codegen → JavaScript
↓ ↓
ds-types ds-layout
(type checker) (Cassowary solver)
```
### What Works Today
| Feature | Syntax | Status |
|---------|--------|--------|
| Signals | `let count = 0` | ✅ Fine-grained, auto-tracked |
| Derived | `let doubled = count * 2` | ✅ Lazy |
| Interpolation | `"Count: {count}"` | ✅ Reactive |
| Conditional | `when count > 5 -> text "hi"` | ✅ Mount/unmount |
| If/else | `if x then a else b` | ✅ |
| Match | `match state \| Loading -> ...` | ✅ |
| List rendering | `for item in items -> text item` | ✅ Reactive |
| Components | `component Card(title) = ...` | ✅ Props |
| Effects | `effect fetch(id): Result` / `perform` | ✅ Algebraic |
| Streams | `debounce`, `throttle`, `distinct` | ✅ |
| Springs | `spring(target: 0, stiffness: 300)` | ✅ Physics |
| Layout | Cassowary constraint solver | ✅ |
| Types | `Signal<Int>`, `Derived<Bool>` | ✅ Hindley-Milner |
| Dev server | `dreamstack dev app.ds` | ✅ HMR |
| Streaming (source) | `stream main on "ws://..."` | ✅ Signal diffs |
| Streaming (receiver) | `stream from "ws://..."` | ✅ Auto-reconnect |
| Router | `route "/path" -> body` / `navigate` | ✅ Hash-based |
| Two-way binding | `input { bind: name }` | ✅ Signal ↔ input |
| Async resources | `DS.resource()` / `DS.fetchJSON()` | ✅ Loading/Ok/Err |
| Springs | `let x = spring(200)` | ✅ RK4 physics |
| Physics scene | `scene { gravity_y: g } [ circle {...} ]` | ✅ Rapier2D WASM |
| Constraints | `constrain el.width = expr` | ✅ Reactive solver |
### DreamStack vs React
| | **DreamStack** | **React** |
|---|---|---|
| Reactivity | Fine-grained signals, surgical DOM | VDOM diff, re-render subtrees |
| State | `count += 1` direct | `setState(c => c+1)` immutable |
| Derived | `let d = count * 2` auto | `useMemo(() => ..., [deps])` manual |
| Effects | Auto-tracked, algebraic | `useEffect(..., [deps])` manual |
| Conditional | `when x -> text "y"` | `{x && <span>y</span>}` |
| Lists | `for item in items -> ...` | `{items.map(i => ...)}` |
| Router | `route "/path" -> body` | `react-router` (external) |
| Forms | `input { bind: name }` | `useState` + `onChange` (manual) |
| Animation | Built-in springs | framer-motion (external) |
| Physics | Built-in Rapier2D scene | matter.js (external) |
| Layout | Built-in Cassowary | CSS only |
| Types | Native HM, `Signal<T>` | TypeScript (external) |
| Bundle | **~7KB** | **~175KB** |
| Ecosystem | New | Massive |
### Benchmarks (signal propagation)
| Benchmark | Ops/sec | Signals |
|-----------|---------|---------|
| Wide Fan-Out (1→1000) | 46K | 1,001 |
| Deep Chain (100 deep) | 399K | 100 |
| Diamond Dependency | 189K | 4 |
| Batch Update (50) | 61K | 50 |
| Mixed Read/Write | 242K | 10 |
### Examples
`counter.ds` · `list.ds` · `router.ds` · `form.ds` · `springs.ds` · `physics.ds` · `todomvc.html` · `search.html` · `dashboard.html` · `playground.html` · `showcase.html` · `benchmarks.html`
---
## Vision
React was revolutionary in 2013. But it carries a decade of compromises: the virtual DOM, hooks with manual dependency arrays, re-rendering entire subtrees, CSS from 1996, animations bolted on as an afterthought. DreamStack asks: **what would we build today if none of that existed?**
The answer is a unified system where **UI is data, reactivity is automatic, effects are composable values, layout is constraint-based, animation is physics-native, and the editor and runtime are one.**
---
## The Language
Not Clojure. Not TypeScript. A new language that steals the best ideas from everywhere.
### Core Properties
| Property | Inspiration | Why |
|---|---|---|
| **Homoiconic** | Clojure, Lisp | UI = data. Code = data. Everything is transformable |
| **Dependent types** | Idris, Agda | Types that express "this button is disabled *when* the form is invalid" at the type level |
| **Algebraic effects** | Eff, Koka, OCaml 5 | Side effects as first-class, composable, interceptable values |
| **Reactive by default** | Svelte, Solid, Excel | No `useState`, no subscriptions. Values *are* reactive. Assignment *is* the API |
| **Structural typing** | TypeScript, Go | Flexible composition without class hierarchies |
| **Compiled to native + WASM** | Rust, Zig | Near-zero runtime overhead. No GC pauses during animations |
### Syntax
```
-- Signals are the primitive. Assignment propagates automatically.
let count = 0
let doubled = count * 2 -- derived, auto-tracked
let label = "Count: {doubled}" -- derived, auto-tracked
-- UI is data. No JSX, no templates, no virtual DOM.
view counter =
column [
text label -- auto-updates when count changes
button "+" { click: count += 1 }
-- Conditional UI is just pattern matching on signals
when count > 10 ->
text "🔥 On fire!" | animate fade-in 200ms
]
```
No hooks. No dependency arrays. No re-renders. **The compiler builds a fine-grained reactive graph at compile time.**
---
## Reactivity: Compile-Time Signal Graph
This is the biggest departure from React. React's model is fundamentally **pull-based** — re-render the tree, diff it, patch the DOM. That's backwards.
### Push-Based with Compile-Time Analysis
```
Signal: count
├──► Derived: doubled
│ └──► Derived: label
│ └──► [direct DOM binding] TextNode #47
└──► Condition: count > 10
└──► [mount/unmount] text "🔥 On fire!"
```
**Key principles:**
- **No virtual DOM.** The compiler knows at build time exactly which DOM nodes depend on which signals. When `count` changes, it updates *only* the specific text node and evaluates the condition. Nothing else runs.
- **No re-rendering.** Components don't re-execute. They execute *once* and set up reactive bindings.
- **No dependency arrays.** The compiler infers all dependencies from the code. You literally cannot forget one.
- **No stale closures.** Since there are no closures over render cycles, the entire class of bugs vanishes.
This extends Solid.js's approach with full compile-time analysis in a purpose-built language.
---
## Effect System: Algebraic Effects for Everything
Every side-effect — HTTP, animations, time, user input, clipboard, drag-and-drop — is an **algebraic effect**. Effects are values you can compose, intercept, retry, and test.
```
-- An effect is declared, not executed
effect fetchUser(id: UserId): Result<User, ApiError>
-- A component "performs" an effect — doesn't control HOW it runs
view profile(id: UserId) =
let user = perform fetchUser(id)
match user
Loading -> skeleton-loader
Ok(u) -> column [
avatar u.photo
text u.name
]
Err(e) -> error-card e | with retry: perform fetchUser(id)
-- At the app boundary, you provide the HANDLER
handle app with
fetchUser(id, resume) ->
let result = http.get "/api/users/{id}"
resume(result)
```
### Why This is Powerful
- **Testing:** Swap the handler. `handle app with mockFetchUser(...)` — no mocking libraries, no dependency injection frameworks.
- **Composition:** Effects compose naturally. An animation effect + a data-fetch effect combine without callback hell or `useEffect` chains.
- **Interceptors:** Want to add logging, caching, or retry logic? Add a handler layer. The component code never changes.
- **Time travel:** Since effects are values, you can record and replay them. Free undo/redo. Free debugging.
---
## Data Flow: Everything is a Stream
Forget the distinction between "state", "props", "events", and "side effects". **Everything is a stream of values over time.**
```
-- User input is a stream
let clicks = stream from button.click
let keypresses = stream from input.keydown
-- Derived streams with temporal operators
let search_query = keypresses
| map .value
| debounce 300ms
| distinct
-- API results are streams
let results = search_query
| flatmap (q -> http.get "/search?q={q}")
| catch-with []
-- UI binds to streams directly
view search =
column [
input { on-keydown: keypresses }
match results
Loading -> spinner
Data(rs) -> list rs (r -> search-result-card r)
Empty -> text "No results"
]
```
### Unification
This single abstraction covers everything:
| Concept | Traditional | DreamStack |
|---|---|---|
| User input | Event handlers | Stream |
| Network responses | Promises / async-await | Stream |
| Animations | CSS transitions / JS libraries | Stream of interpolated values |
| Timers | `setInterval` / `setTimeout` | Stream |
| WebSockets | Callback-based | Stream |
| Drag events | Complex event handler state machines | Stream of positions |
One abstraction. One composition model. Everything snaps together.
---
## Layout: Constraint-Based, Not Box-Based
CSS's box model is from 1996. Flexbox and Grid are patches on a fundamentally limited system. DreamStack starts over with a **constraint solver**.
```
-- Declare relationships, not boxes
layout dashboard =
let sidebar.width = clamp(200px, 20vw, 350px)
let main.width = viewport.width - sidebar.width
let header.height = 64px
let content.height = viewport.height - header.height
-- Constraints, not nesting
sidebar.left = 0
sidebar.top = header.height
sidebar.height = content.height
main.left = sidebar.right
main.top = header.height
main.right = viewport.right
-- Responsive is just different constraints
when viewport.width < 768px ->
sidebar.width = 0 -- collapses
main.left = 0 -- takes full width
```
Inspired by Apple's **Auto Layout** (Cassowary constraint solver), but made reactive. When `viewport.width` changes, the constraint solver re-solves and updates positions in a single pass. No layout thrashing. No "CSS specificity wars."
### Advantages Over CSS
- **No cascade conflicts** — constraints are explicit and local
- **No z-index hell** — layering is declarative
- **No media query breakpoints** — responsiveness emerges from constraints
- **Animations are free** — animating a constraint target is the same as setting it
---
## Animation: First-Class, Physics-Based, Interruptible
Animations aren't CSS transitions bolted on after the fact. They're part of the reactive graph.
```
-- A spring is a signal that animates toward its target
let panel_x = spring(target: 0, stiffness: 300, damping: 30)
-- Change the target → the spring animates automatically
on toggle_sidebar ->
panel_x.target = if open then 250 else 0
-- Gestures feed directly into springs
on drag(event) ->
panel_x.target = event.x -- spring follows finger
on drag-end(event) ->
panel_x.target = snap-to-nearest [0, 250] event.x
-- UI just reads the spring's current value
view sidebar =
panel { x: panel_x } [
nav-items
]
```
### Design Principles
- **Interruptible:** Start a new animation mid-flight. The spring handles the physics — no jarring jumps, no "wait for the current animation to finish."
- **Gesture-driven:** Touch/mouse input feeds directly into the animation model. No separate "gesture handler" → "state update" → "CSS transition" pipeline.
- **60fps guaranteed:** Springs resolve on the GPU. The main thread never blocks.
- **Composable:** Combine springs, easing curves, and physics simulations using the same stream operators as everything else.
---
## Runtime: Tiny, Compiled, No GC Pauses
```
┌──────────────────────────────────────────┐
│ Compiled Output │
├──────────────────────────────────────────┤
│ Signal Graph (static DAG) ~2KB │
│ Constraint Solver (layout) ~4KB │
│ Spring Physics (animations) ~1KB │
│ Effect Runtime (handlers) ~2KB │
│ DOM Patcher (surgical updates) ~3KB │
├──────────────────────────────────────────┤
│ Total Runtime: ~12KB │
│ (vs React ~45KB + ReactDOM ~130KB) │
└──────────────────────────────────────────┘
```
The compiler does the heavy lifting. The runtime is tiny because there is:
- No virtual DOM diffing algorithm
- No fiber scheduler
- No hook state management system
- No reconciliation algorithm
- No garbage collector pauses during animation frames
---
## The Killer Feature: Live Structural Editing
Because the language is homoiconic (code = data), the **editor IS the runtime:**
```
┌─────────────────────────────────┐
│ Live Editor │
│ │
│ view counter = │ ← Edit this...
│ column [ │
│ text "Count: {count}" │
│ button "+" { ... } │
│ ] │
│ │
│ ───────────────────────────── │
│ │
│ ┌─────────────┐ │ ← ...see this update
│ │ Count: 42 │ │ instantly, with state
│ │ [ + ] │ │ preserved.
│ └─────────────┘ │
│ │
└─────────────────────────────────┘
```
- Drag and drop UI elements in the preview → the **code updates**.
- Edit the code → the **preview updates**.
- Both directions, simultaneously, with state preserved.
- Not a design tool that generates code — **the code IS the design tool**.
This is possible because:
1. **Homoiconicity** means UI structure is inspectable and modifiable data
2. **Immutable signals** mean state survives code changes
3. **Compile-time signal graph** means changes are surgical, not full-page reloads
---
## Comparison to Existing Approaches
| Capability | React | Svelte 5 | Solid.js | Flutter | **DreamStack** |
|---|---|---|---|---|---|
| Reactivity | Pull (re-render + diff) | Compile-time runes | Fine-grained signals | Widget rebuild | **Compile-time signal DAG** |
| Side effects | `useEffect` + deps array | `$effect` | `createEffect` | Lifecycle methods | **Algebraic effects** |
| Layout | CSS (external) | CSS (scoped) | CSS (external) | Constraint-based | **Constraint solver (native)** |
| Animation | 3rd party libs | 3rd party libs | 3rd party libs | Physics-based (native) | **Physics-based (native)** |
| SSR | Yes (complex) | Yes | Yes | No (web) | **Yes (compiled)** |
| Runtime size | ~175KB | ~2KB | ~7KB | ~2MB (web) | **~12KB** |
| Live editing | Fast Refresh (lossy) | HMR | HMR | Hot reload | **Bidirectional structural** |
| Type safety | TypeScript (bolt-on) | TypeScript (bolt-on) | TypeScript (bolt-on) | Dart (native) | **Dependent types (native)** |
| UI = Data | No (JSX compiled away) | No (templates) | No (JSX compiled) | No (widget classes) | **Yes (homoiconic)** |
---
## Fragments of the Future, Today
The closest approximations to pieces of this vision:
- **Solid.js** — fine-grained reactivity, no VDOM, ~7KB runtime
- **Svelte 5 Runes** — compiler-driven reactivity, tiny output
- **Elm** — algebraic effects-adjacent, immutable state, strong types
- **Flutter** — constraint layout, physics-based animation, hot reload
- **Clojure/Reagent** — homoiconicity, UI as data, ratom reactivity
- **Koka** — algebraic effect system in a practical language
- **Apple Auto Layout** — Cassowary constraint solver for UI
- **Excel** — reactive by default, dependency auto-tracking
Nobody has unified them. That's the opportunity.
---
## Design Philosophy
1. **The compiler is the framework.** Move work from runtime to compile time. The less code that runs in the browser, the faster the UI.
2. **Reactivity is not a feature, it's the default.** Every value is live. Every binding is automatic. You opt *out* of reactivity, not *in*.
3. **Effects are values, not side-channels.** Making side effects first-class and composable eliminates the largest source of bugs in modern UIs.
4. **Layout and animation are not afterthoughts.** They're core primitives, not CSS bolt-ons or third-party libraries.
5. **The editor and the runtime are the same thing.** Bidirectional editing collapses the design-develop gap entirely.
6. **UI is data, all the way down.** If you can't `map` over your UI structure, your abstraction is wrong.
7. **Any input bitstream → any output bitstream.** The UI is just one codec. Tomorrow's neural nets generate the pixels directly.
---
## Phase 7: Universal Bitstream Streaming
> *Stream the whole UI as bytes. Neural nets will generate the pixels, acoustics, and actuator commands.*
DreamStack's `engine/ds-stream` crate implements a universal binary protocol for streaming any I/O:
```
┌──────────┐ WebSocket / WebRTC ┌──────────┐
│ Source │ ──────frames (bytes)──► │ Receiver │
│ (renders) │ ◄──────inputs (bytes)── │ (~250 LOC)│
└──────────┘ └──────────┘
```
### Binary Protocol (16-byte header)
| Field | Size | Description |
|-------|------|-------------|
| type | u8 | Frame/input type (pixels, audio, haptic, neural, BCI) |
| flags | u8 | Input flag, keyframe flag, compression flag |
| seq | u16 | Sequence number |
| timestamp | u32 | Relative ms since stream start |
| width | u16 | Frame width or channel count |
| height | u16 | Frame height or sample rate |
| length | u32 | Payload length |
### Output Types
- `Pixels` (0x01) — raw RGBA framebuffer
- `Audio` (0x10) — PCM audio samples
- `Haptic` (0x20) — vibration/actuator commands
- `NeuralFrame` (0x40) — neural-generated pixels *(future)*
- `NeuralAudio` (0x41) — neural speech/music synthesis *(future)*
- `NeuralActuator` (0x42) — learned motor control *(future)*
### Input Types
- `Pointer` (0x01) — mouse/touch position + buttons
- `Key` (0x10) — keyboard events
- `Gamepad` (0x30) — controller axes + buttons
- `BciInput` (0x90) — brain-computer interface *(future)*
### Demos
- `examples/stream-source.html` — Springs demo captures canvas → streams pixels at 30fps
- `examples/stream-receiver.html` — Thin client (~250 lines, no framework) renders bytes
### Run It
```bash
cargo run -p ds-stream # start relay on :9100
open examples/stream-source.html # source: renders + streams
open examples/stream-receiver.html # receiver: displays bytes
```
### Compiler-Native Streaming
The `stream` keyword makes any `.ds` app streamable with one line. The compiler generates all WebSocket connection, binary protocol encoding, and signal diff broadcasting automatically.
#### Source: `stream` declaration
```
let count = 0
let doubled = count * 2
stream counter on "ws://localhost:9100" { mode: signal }
view counter =
column [
text count
text doubled
button "+" { click: count += 1 }
]
```
The compiler:
1. Calls `DS._initStream(url, mode)` on load → connects to relay
2. Wraps every signal mutation with `DS._streamDiff("count", count.value)` → sends JSON diff frames
3. Sends scene body positions at 60fps when a physics scene is streaming
#### Receiver: `stream from` expression
```
let remote = stream from "ws://localhost:9100"
view main =
column [
text remote.count
text remote.doubled
]
```
Compiles to `DS._connectStream(url)` — returns a reactive `Signal` that merges incoming `FRAME_SIGNAL_SYNC` and `FRAME_SIGNAL_DIFF` frames.
#### Streaming Modes
| Mode | Keyword | What's Sent | Bandwidth |
|------|---------|-------------|----------|
| Signal | `signal` (default) | JSON diffs of changed signals | ~2 KB/s |
| Delta | `delta` | XOR + RLE compressed pixel deltas | ~50 KB/s |
| Pixel | `pixel` | Raw RGBA framebuffer every frame | ~30 MB/s |
#### CLI
```bash
# Compile and serve with streaming enabled
dreamstack stream app.ds --relay ws://localhost:9100 --mode signal
# Or explicitly declare streaming in the .ds file and use dev server
dreamstack dev app.ds
```
---
## Phase 8: Physics Scene — Rapier2D in the Language
> *Declare physics bodies in `.ds` syntax. The compiler generates WASM-backed canvas with rigid body simulation.*
```ds
let gravity_y = 980
view main =
column [
scene { width: 700, height: 450, gravity_y: gravity_y } [
circle { x: 200, y: 80, radius: 35, color: "#8b5cf6" }
circle { x: 350, y: 50, radius: 50, color: "#7c3aed" }
rect { x: 500, y: 100, width: 80, height: 50, color: "#10b981" }
]
button "Anti-Gravity" { click: gravity_y = -500 }
button "Normal" { click: gravity_y = 980 }
]
```
### Architecture
```
.ds source → parser (scene/circle/rect) → codegen → JS
↓
canvas + async WASM init
ds-physics (Rapier2D) ← pkg/ds_physics_bg.wasm
↓
requestAnimationFrame loop
step → render → repeat
```
### Features
- **Rigid body physics** — circles and rectangles with collision, rotation, restitution
- **Reactive gravity** — signal changes wrapped in `DS.effect()`, bodies wake on gravity change
- **Mouse drag** — click and drag bodies with impulse-based interaction
- **Compile-time colors** — hex color strings parsed at compile time → `set_body_color(r, g, b, a)`
- **Zero JS overhead** — physics runs in WASM, rendering in canvas, signals bridge both
Reference in a new issue View git blame Copy permalink