amir/dreamstack
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
112 commits 1 branch 0 tags 1.8 MiB
Commit graph

14 commits

Author SHA1 Message Date
enzotar
35b39a1cf1 feat(ds-stream): v2.0-2.3 composable codec pipeline 
v2.0 — Pipeline Architecture
  - Frame, CodecResult, Codec trait, Pipeline builder
  - 6 adapters: Passthrough, Dedup, Compress, Pacer, Slicer, Stats

v2.1 — Multi-frame & new codecs
  - CodecOutput::Many fan-out, EncryptCodec, FilterCodec
  - Codec::reset(), encode_all/decode_all, real SlicerCodec chunking

v2.2 — Observability & reassembly
  - PipelineResult (frames+errors+consumed), StageMetric
  - ReassemblyCodec, ConditionalCodec, Pipeline presets & metrics

v2.3 — Integrity & rate control
  - ChecksumCodec (CRC32), RateLimitCodec (token bucket), TagCodec
  - Pipeline::chain(), Pipeline::describe()

13 codec adapters, 474 tests (all green, 0 regressions)
 2026-03-11 23:50:35 -07:00
enzotar
4a15e0b70c feat: Bump package versions, add physics body sleeping, revolute motor, and prismatic joints, and enhance type checker exhaustiveness. 2026-03-10 21:07:22 -07:00
enzotar
b0440e2e47 feat(compiler): v0.4 + v0.5 — CLI modularity, diagnostic pipeline, test suite 
v0.4 — CLI Modularity:
- Split monolithic main.rs (2,038 lines) into 8 command modules + slim dispatch (107 lines)
- Add 12 JS codegen tests (signals, derived, views, events, loops, match, enums, components, interpolation, springs, tree-shaking)

v0.5 — Diagnostic Quality + Analyzer Confidence:
- Add From<ParseError> for Diagnostic (E0001) in ds-diagnostic
- Add errors_as_diagnostics() to TypeChecker (E0100–E0110)
- Wire Elm-style diagnostics through dreamstack check and build commands
- Switch incremental compiler to parse_program_resilient() for multi-error collection
- Add 12 analyzer tests (chains, fan-out, diamond deps, empty programs, conditionals, handlers, views)
- Add 2 diagnostic conversion tests

Test suite: 97 → 123 tests (26 new, 0 failures)
 2026-03-10 09:09:02 -07:00
enzotar
cc6aac8697 feat: Add Waveshare P4 panel device integration with display streaming and touch input, alongside core streaming engine and compiler updates. 2026-03-02 16:08:49 -08:00
enzotar
08e36573a5 feat: HTTP /meta API, signal dedup, periodic auto-sync 
Relay HTTP /meta endpoint:
- GET /meta → JSON with all channel stats
- GET /meta/{name} → JSON with specific channel stats, schema, current state
- Uses TCP peek to intercept raw HTTP before WS handshake
- CORS headers for browser access

Signal deduplication:
- _lastSentValues tracks last-sent value per signal
- JSON.stringify comparison skips unchanged values
- Prevents redundant WS frames from derived signals

Periodic auto-sync:
- Every 50 diff batches, source sends full SignalSync (0x30)
- Relay can compact its cache instead of accumulating infinite diffs
- Resets pending_signal_diffs in relay cache

All 57 relay tests pass. All 9 examples pass.
 2026-02-26 18:17:25 -08:00
enzotar
598ecde59c feat: comprehensive streaming improvements 
Runtime _connectStream improvements:
- Connection status as reactive signals: _connected, _latency, _frames, _reconnects
  injected on stream proxy so UIs can show connection health
- Fixed RLE decoder: 2-byte LE count (was 1-byte, mismatched relay encoder)
- Schema caching: 0x32 SchemaAnnounce frames now parsed and cached
- RTT tracking: receivers send periodic pings (5s), measure round-trip latency
- Better reconnect logging: includes URL and attempt count

Relay tests (57 total):
- catchup_merges_multiple_diffs: sync + 3 diffs → 1 merged frame
- catchup_diffs_only_no_sync: diffs without sync → merged frame
- catchup_preserves_version_counters: conflict resolution versions kept

New example:
- timer-multi-action.ds: every timer + multi-action buttons verified

Documentation:
- STREAM_COMPOSITION.md: 4 new sections (Diff Batching, Connection Status, RTT, Relay Merging)
- Updated example table with streaming-dashboard.ds and timer-multi-action.ds

All 9 examples pass regression (44-70KB each)
 2026-02-26 18:09:14 -08:00
enzotar
746b76fe4f perf: streaming core improvements — batched diffs, RTT tracking, relay merging 
Runtime improvements:
- Diff batching: multiple signal changes coalesced into 1 WS frame
  via _pendingDiffs + microtask Promise.resolve()
- Connection status: _streamConnected, _streamLatency, _streamReconnects,
  _streamFrameCount, _streamByteCount tracked for stream health
- RTT tracking: periodic ping/pong every 5s measures round-trip latency
- Removed redundant _streamSync from flush() — diffs are already batched

Relay improvements:
- Diff merging: late joiners receive 1 consolidated sync frame instead
  of replaying hundreds of individual diffs
- Version counters merged: conflict resolution preserved across catchup
- Fallback: if JSON parse fails, falls back to raw sync + all diffs

Test updates:
- state_cache_signal_sync verifies merged payload (count: 1)
- All 54 relay tests pass
- All 8 examples pass regression
 2026-02-26 18:02:31 -08:00
enzotar
a8235c48b3 feat: stream composition API — select, schema, relay filtering 
1. Receiver-side `select` clause:
   - `stream from "url" { select: field1, field2 }`
   - Parser, AST, codegen all updated
   - Emits: `_connectStream(url, ["field1","field2"])`
   - Client-side _csFilter strips unwanted fields

2. Schema announcement (0x32):
   - Sources send output schema on connect
   - Lists registered signal names and mode

3. Relay schema cache:
   - ChannelState stores schema from 0x32
   - Forwarded to late-joining receivers

4. Relay-side subscribe filter (0x33):
   - Receivers send wanted fields after connecting
   - Relay strips unwanted JSON keys from 0x30/0x31
     frames before forwarding — saves bandwidth

Protocol: SchemaAnnounce=0x32, SubscribeFilter=0x33
54 tests pass, all crates build clean.
 2026-02-26 10:07:47 -08:00
enzotar
a943d2e2e9 feat: peer mode relay + self-echo filtering for true bidirectional sync 
- Added /peer/{name} route to relay: all clients are equal peers
- handle_peer: binary broadcast to all other peers, catchup for late joiners
- Simplified runtime: single /peer/ WS replaces dual source+receiver
- _peerId: random 8-char ID prevents self-echo from broadcast
- _pid in each diff JSON, filtered in _applyRemoteDiff
 2026-02-25 21:37:17 -08:00
enzotar
cde84ae270 feat: production hardening — relay v1.0.0, receiver protocol completeness 
Relay v1.0.0:
- Max receivers per channel (default: 64) with rejection counting
- Max channels limit (default: 256) with over-limit rejection
- Channel GC: periodic scan removes idle channels after grace period (30s)
- Source reconnection: cache preserved on disconnect, new input channels
- Stats: peak_receivers, total_connections, rejected_connections, uptime
- ChannelState::is_idle(), grace_period_expired() for lifecycle mgmt
- StateCache::clear(), has_state() for cache introspection
- Banner shows config: max receivers, max channels, grace period
- 54 tests (+8 new: channel_max_limit, idle detection, grace period, cache ops)

Receiver (_connectStream):
- Complete frame type handling: SignalSync (full replace), SignalDiff (merge),
  Pixels/Keyframe (emit stream_frame), DeltaPixels (XOR + RLE decode),
  Ping (keepalive), StreamEnd (emit stream_end)
- RLE decoder for delta pixel frames
- Exponential backoff reconnect (1s → capped at 10s)
- Connection stats: frames, bytes, reconnects
- Event emissions: stream_frame, stream_end for app-level handling

105 tests, 0 failures across workspace
 2026-02-25 18:30:40 -08:00
enzotar
392e478351 feat: WebRTC transport — peer-to-peer data channels with auto-fallback 
Relay:
- /signal/{channel} path for SDP/ICE exchange via WebSocket
- handle_signaling broadcasts text messages between signaling peers
- signaling_tx broadcast channel in ChannelState
- 46 ds-stream tests (+2 signaling path tests)

JS Runtime:
- _initWebRTC(signalingUrl, streamUrl, mode) with RTCPeerConnection
- Unordered DataChannel (ordered:false, maxRetransmits:0) for low latency
- Auto-fallback: WebSocket starts immediately, WebRTC upgrades in ≤5s
- Data channel override of _streamSend for transparent binary protocol

Parser/AST:
- StreamTransport enum (WebSocket, WebRTC)
- transport field in StreamDecl
- Parses: stream x on url { mode: signal, transport: webrtc }

Codegen:
- WebRTC: emits DS._initWebRTC(sigUrl, streamUrl, mode)
- WebSocket: emits DS._initStream(url, mode) (unchanged)

97 tests, 0 failures
 2026-02-25 15:02:31 -08:00
enzotar
0ed76513a6 feat(relay): multi-source routing — /source/{name} and /stream/{name} 
- ChannelState: per-channel broadcast, input mpsc, cache
- RelayState: HashMap<String, Arc<RwLock<ChannelState>>>
- parse_path: /source/{n}, /stream/{n}, legacy / fallback
- Per-channel stats logging with channel name prefix
- Backward compatible: plain /source and /stream use 'default' channel
- 44 ds-stream tests, 95 total workspace (0 failures)
 2026-02-25 14:50:39 -08:00
enzotar
69f39746af feat(ds-stream): RLE compression, input events, keyframe caching 
- protocol.rs: TouchEvent, GamepadAxisEvent, GamepadButtonEvent, ResizeEvent
  with encode/decode and roundtrip tests
- codec.rs: rle_encode/rle_decode for delta frame compression (333x on
  unchanged frames), signal_sync_frame/signal_diff_frame builders,
  touch/gamepad/stream_end convenience builders
- relay.rs: StateCache for late-joining receivers (keyframe + signal sync +
  accumulated diffs), periodic keepalive pings, stats logging every 30s,
  diff accumulator with cap at 1000
- BITSTREAM_INTEGRATION.md: compiler integration spec for another agent
- Tests: 17 → 38 (all passing)
 2026-02-25 11:04:05 -08:00
enzotar
d7961cdc98 feat: universal bitstream streaming — any input → any output 
New crate: engine/ds-stream/
- Binary protocol: 16-byte header, typed frame/input enums
  - Frame types: Pixels, Delta, Audio, Signal, Neural (0x01-0x43)
  - Input types: Pointer, Key, Scroll, Gamepad, MIDI, BCI (0x01-0x90)
- WebSocket relay server (tokio + tungstenite)
  - Source → receivers: frame broadcast
  - Receivers → source: input routing
- Codec: encode/decode, XOR delta compression, RLE, convenience builders
- 17 unit tests, all passing

Streaming modes (stream-source.html):
1. Pixel mode: raw RGBA framebuffer (~28 MB/s)
2. Delta mode: XOR + RLE compression (~1-9 MB/s, 70-95% savings)
3. Signal mode: compact JSON signal diffs (~2 KB/s, 12000x reduction)
4. Neural mode: procedural SDF pixel generator (concept demo)
5. Audio channel: spring velocity→frequency synthesis
6. Multi-receiver: broadcast to all connected clients

Thin receiver client (stream-receiver.html, ~300 lines):
- Zero framework, zero build step
- Renders any incoming bitstream mode
- Local signal-diff renderer for signal mode
- AudioContext playback for audio frames
- Full input capture: click/drag, keyboard, scroll
- Per-channel bitstream bus visualization

DREAMSTACK.md: Phase 7 section with protocol spec
 2026-02-25 10:29:44 -08:00