amir/dreamstack
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
112 commits 1 branch 0 tags 1.8 MiB
Commit graph

23 commits

Author SHA1 Message Date
enzotar
35b39a1cf1 feat(ds-stream): v2.0-2.3 composable codec pipeline 
v2.0 — Pipeline Architecture
  - Frame, CodecResult, Codec trait, Pipeline builder
  - 6 adapters: Passthrough, Dedup, Compress, Pacer, Slicer, Stats

v2.1 — Multi-frame & new codecs
  - CodecOutput::Many fan-out, EncryptCodec, FilterCodec
  - Codec::reset(), encode_all/decode_all, real SlicerCodec chunking

v2.2 — Observability & reassembly
  - PipelineResult (frames+errors+consumed), StageMetric
  - ReassemblyCodec, ConditionalCodec, Pipeline presets & metrics

v2.3 — Integrity & rate control
  - ChecksumCodec (CRC32), RateLimitCodec (token bucket), TagCodec
  - Pipeline::chain(), Pipeline::describe()

13 codec adapters, 474 tests (all green, 0 regressions)
 2026-03-11 23:50:35 -07:00
enzotar
fbbdeb0bc4 feat: add initial interactive compiler and ds-stream demos 2026-03-11 18:25:18 -07:00
enzotar
dfa0c4151c engine: v0.90–v1.0.0 milestone 🎉 
v0.90: World Layers, Stream Encryption V2, Multi-Channel
- ds-physics: set/get layer, gravity scale, angular vel, body type, world gravity, freeze/unfreeze, body tag (183 tests)
- ds-stream: XorCipherV2, ChannelRouter, AckTracker, FramePoolV2, BandwidthEstimatorV2, PriorityMux, NonceGenerator, StreamValidator, RetryQueue (246 tests)
- ds-stream-wasm: 9 exports (156 tests)

v0.95: Scene Graph, Stream Compression V2, Telemetry
- ds-physics: body count all, step count, get gravity, is frozen, get color, AABB, raycast, restitution, emitter count (192 tests)
- ds-stream: Lz4Lite, TelemetrySink, FrameDiffer, BackoffTimer, StreamMirror, QuotaManager, HeartbeatV2, TagFilter, MovingAverage (255 tests)
- ds-stream-wasm: 9 exports (165 tests)

v1.0.0: Production Ready — ECS Foundation, Stream Pipeline, Protocol Finalization
- ds-physics: get tag, body list, impulse, mass, friction, world bounds, body exists, reset world, engine version (201 tests)
- ds-stream: StreamPipeline, ProtocolHeader, FrameSplitterV2, CongestionWindowV2, StreamStatsV2, AckWindow, CodecRegistryV2, FlowControllerV2, VersionNegotiator (264 tests)
- ds-stream-wasm: 9 exports (174 tests)

Total: 639 tests across 3 packages
 2026-03-11 14:58:39 -07:00
enzotar
93cdbb75d7 engine: v0.28–v0.50 milestone 
ds-physics v0.50.0 (138 tests)
- v0.28: apply_body_torque, is_body_sleeping, get_body_angle
- v0.30: set_body_gravity, set_linear_damping, count_awake_bodies
- v0.40: joints (distance/pin), raycast, kinematic, time scale, world stats
- v0.50: point query, explosion, velocity/position set, contacts, gravity, collision groups

ds-stream v0.50.0 (201 tests)
- v0.28: BufferPool, PacketJitterBuffer, RttTracker
- v0.30: FrameRingBuffer, PacketLossDetector, ConnectionQuality
- v0.40: QualityAdapter, SourceMixer, FrameDeduplicator, BackpressureController, HeartbeatMonitor, CompressionTracker, FecEncoder, StreamSnapshot, AdaptivePriorityQueue
- v0.50: StreamCipher, ChannelMux/Demux, FramePacer, CongestionWindow, FlowController, ProtocolNegotiator, ReplayRecorder, BandwidthShaper

ds-stream-wasm v0.50.0 (111 tests)
- WASM bindings for all stream features

ds-screencast v0.50.0
- CLI: --jitter-buffer, --latency-window, --ring-buffer, --loss-threshold, --adaptive, --dedup, --backpressure, --heartbeat-ms, --fec, --encrypt-key, --channels, --pacing-ms, --max-bps, --replay-file
 2026-03-11 12:47:56 -07:00
enzotar
3c14beea50 feat(engine): v0.14-v0.16 releases 
ds-physics 0.16.0 (81 tests):
- v0.14: proximity queries, physics regions
- v0.15: event hooks, transform hierarchy, timeline
- v0.16: deterministic seed/checksum, collision manifolds, distance+hinge constraints

ds-stream 0.16.0 (143 tests):
- v0.14: FrameCompressor (RLE), MultiClientSync, BandwidthThrottle, FrameType::Compressed
- v0.15: AdaptiveBitrate, MetricsSnapshot, FramePipeline
- v0.16: FrameEncryptor (XOR), StreamMigration, FrameDedup, PriorityQueue

ds-stream-wasm 0.16.0 (54 tests):
- v0.14: RLE compress/decompress, sync drift, bandwidth limiting
- v0.15: adaptive quality, metrics snapshot, frame transforms
- v0.16: encrypt/decrypt frames, migration handoff, frame dedup

ds-screencast 0.16.0:
- v0.14: --roi, /clients, --compress, --migrate-on-crash
- v0.15: --adaptive-bitrate, /metrics, --viewport-transform, --cdn-push
- v0.16: /tabs, --encrypt-key, --watermark, --graceful-shutdown
 2026-03-10 22:47:44 -07:00
enzotar
4a15e0b70c feat: Bump package versions, add physics body sleeping, revolute motor, and prismatic joints, and enhance type checker exhaustiveness. 2026-03-10 21:07:22 -07:00
enzotar
9cc395d2a7 feat(demo): pixel streaming demo with delta+RLE encoding 
- Source (index.html): physics sim → pixel capture → delta XOR + RLE encode → WebSocket
- Receiver (receiver.html): WebSocket → decode delta/keyframe → render + click interaction + RTT
- Relay (relay.js): bidirectional WebSocket relay (pixels ↓ signals ↑)
- 97% compression (18KB/frame vs 675KB raw RGBA)
- Interactive: click on receiver → impulse force on source → round-trip latency measured
- Frame types: 0x11 keyframe (every 60 frames), 0x12 delta+RLE
- No buffer bloat: drops frames when pipe is busy
 2026-03-10 20:49:35 -07:00
enzotar
b0440e2e47 feat(compiler): v0.4 + v0.5 — CLI modularity, diagnostic pipeline, test suite 
v0.4 — CLI Modularity:
- Split monolithic main.rs (2,038 lines) into 8 command modules + slim dispatch (107 lines)
- Add 12 JS codegen tests (signals, derived, views, events, loops, match, enums, components, interpolation, springs, tree-shaking)

v0.5 — Diagnostic Quality + Analyzer Confidence:
- Add From<ParseError> for Diagnostic (E0001) in ds-diagnostic
- Add errors_as_diagnostics() to TypeChecker (E0100–E0110)
- Wire Elm-style diagnostics through dreamstack check and build commands
- Switch incremental compiler to parse_program_resilient() for multi-error collection
- Add 12 analyzer tests (chains, fan-out, diamond deps, empty programs, conditionals, handlers, views)
- Add 2 diagnostic conversion tests

Test suite: 97 → 123 tests (26 new, 0 failures)
 2026-03-10 09:09:02 -07:00
enzotar
878e55b962 chore: add per-package versioning, changesets, and clean changelogs 
- Add cliff.toml for git-cliff changelog generation (one-line entries,
  no commit body dumps, improve/refine prefixes mapped)
- Add @changesets/cli config and README in .changeset/
- Add release.sh script with per-package version bumps from changesets,
  changeset-driven per-crate changelog updates, and --all/--dry-run flags
- Switch all crates from workspace version to independent version = "0.1.0"
- Generate clean root CHANGELOG.md and per-crate CHANGELOGs with [0.1.0]
- Retag v1.0.0 → v0.1.0 to match actual crate versions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-08 23:12:54 -07:00
enzotar
9e2cb29dd9 feat: Implement ds-screencast engine, panel preview, and Waveshare ESP-NOW communication. 2026-03-07 02:34:57 -08:00
enzotar
cc6aac8697 feat: Add Waveshare P4 panel device integration with display streaming and touch input, alongside core streaming engine and compiler updates. 2026-03-02 16:08:49 -08:00
enzotar
08e36573a5 feat: HTTP /meta API, signal dedup, periodic auto-sync 
Relay HTTP /meta endpoint:
- GET /meta → JSON with all channel stats
- GET /meta/{name} → JSON with specific channel stats, schema, current state
- Uses TCP peek to intercept raw HTTP before WS handshake
- CORS headers for browser access

Signal deduplication:
- _lastSentValues tracks last-sent value per signal
- JSON.stringify comparison skips unchanged values
- Prevents redundant WS frames from derived signals

Periodic auto-sync:
- Every 50 diff batches, source sends full SignalSync (0x30)
- Relay can compact its cache instead of accumulating infinite diffs
- Resets pending_signal_diffs in relay cache

All 57 relay tests pass. All 9 examples pass.
 2026-02-26 18:17:25 -08:00
enzotar
598ecde59c feat: comprehensive streaming improvements 
Runtime _connectStream improvements:
- Connection status as reactive signals: _connected, _latency, _frames, _reconnects
  injected on stream proxy so UIs can show connection health
- Fixed RLE decoder: 2-byte LE count (was 1-byte, mismatched relay encoder)
- Schema caching: 0x32 SchemaAnnounce frames now parsed and cached
- RTT tracking: receivers send periodic pings (5s), measure round-trip latency
- Better reconnect logging: includes URL and attempt count

Relay tests (57 total):
- catchup_merges_multiple_diffs: sync + 3 diffs → 1 merged frame
- catchup_diffs_only_no_sync: diffs without sync → merged frame
- catchup_preserves_version_counters: conflict resolution versions kept

New example:
- timer-multi-action.ds: every timer + multi-action buttons verified

Documentation:
- STREAM_COMPOSITION.md: 4 new sections (Diff Batching, Connection Status, RTT, Relay Merging)
- Updated example table with streaming-dashboard.ds and timer-multi-action.ds

All 9 examples pass regression (44-70KB each)
 2026-02-26 18:09:14 -08:00
enzotar
746b76fe4f perf: streaming core improvements — batched diffs, RTT tracking, relay merging 
Runtime improvements:
- Diff batching: multiple signal changes coalesced into 1 WS frame
  via _pendingDiffs + microtask Promise.resolve()
- Connection status: _streamConnected, _streamLatency, _streamReconnects,
  _streamFrameCount, _streamByteCount tracked for stream health
- RTT tracking: periodic ping/pong every 5s measures round-trip latency
- Removed redundant _streamSync from flush() — diffs are already batched

Relay improvements:
- Diff merging: late joiners receive 1 consolidated sync frame instead
  of replaying hundreds of individual diffs
- Version counters merged: conflict resolution preserved across catchup
- Fallback: if JSON parse fails, falls back to raw sync + all diffs

Test updates:
- state_cache_signal_sync verifies merged payload (count: 1)
- All 54 relay tests pass
- All 8 examples pass regression
 2026-02-26 18:02:31 -08:00
enzotar
a8235c48b3 feat: stream composition API — select, schema, relay filtering 
1. Receiver-side `select` clause:
   - `stream from "url" { select: field1, field2 }`
   - Parser, AST, codegen all updated
   - Emits: `_connectStream(url, ["field1","field2"])`
   - Client-side _csFilter strips unwanted fields

2. Schema announcement (0x32):
   - Sources send output schema on connect
   - Lists registered signal names and mode

3. Relay schema cache:
   - ChannelState stores schema from 0x32
   - Forwarded to late-joining receivers

4. Relay-side subscribe filter (0x33):
   - Receivers send wanted fields after connecting
   - Relay strips unwanted JSON keys from 0x30/0x31
     frames before forwarding — saves bandwidth

Protocol: SchemaAnnounce=0x32, SubscribeFilter=0x33
54 tests pass, all crates build clean.
 2026-02-26 10:07:47 -08:00
enzotar
a943d2e2e9 feat: peer mode relay + self-echo filtering for true bidirectional sync 
- Added /peer/{name} route to relay: all clients are equal peers
- handle_peer: binary broadcast to all other peers, catchup for late joiners
- Simplified runtime: single /peer/ WS replaces dual source+receiver
- _peerId: random 8-char ID prevents self-echo from broadcast
- _pid in each diff JSON, filtered in _applyRemoteDiff
 2026-02-25 21:37:17 -08:00
enzotar
cde84ae270 feat: production hardening — relay v1.0.0, receiver protocol completeness 
Relay v1.0.0:
- Max receivers per channel (default: 64) with rejection counting
- Max channels limit (default: 256) with over-limit rejection
- Channel GC: periodic scan removes idle channels after grace period (30s)
- Source reconnection: cache preserved on disconnect, new input channels
- Stats: peak_receivers, total_connections, rejected_connections, uptime
- ChannelState::is_idle(), grace_period_expired() for lifecycle mgmt
- StateCache::clear(), has_state() for cache introspection
- Banner shows config: max receivers, max channels, grace period
- 54 tests (+8 new: channel_max_limit, idle detection, grace period, cache ops)

Receiver (_connectStream):
- Complete frame type handling: SignalSync (full replace), SignalDiff (merge),
  Pixels/Keyframe (emit stream_frame), DeltaPixels (XOR + RLE decode),
  Ping (keepalive), StreamEnd (emit stream_end)
- RLE decoder for delta pixel frames
- Exponential backoff reconnect (1s → capped at 10s)
- Connection stats: frames, bytes, reconnects
- Event emissions: stream_frame, stream_end for app-level handling

105 tests, 0 failures across workspace
 2026-02-25 18:30:40 -08:00
enzotar
392e478351 feat: WebRTC transport — peer-to-peer data channels with auto-fallback 
Relay:
- /signal/{channel} path for SDP/ICE exchange via WebSocket
- handle_signaling broadcasts text messages between signaling peers
- signaling_tx broadcast channel in ChannelState
- 46 ds-stream tests (+2 signaling path tests)

JS Runtime:
- _initWebRTC(signalingUrl, streamUrl, mode) with RTCPeerConnection
- Unordered DataChannel (ordered:false, maxRetransmits:0) for low latency
- Auto-fallback: WebSocket starts immediately, WebRTC upgrades in ≤5s
- Data channel override of _streamSend for transparent binary protocol

Parser/AST:
- StreamTransport enum (WebSocket, WebRTC)
- transport field in StreamDecl
- Parses: stream x on url { mode: signal, transport: webrtc }

Codegen:
- WebRTC: emits DS._initWebRTC(sigUrl, streamUrl, mode)
- WebSocket: emits DS._initStream(url, mode) (unchanged)

97 tests, 0 failures
 2026-02-25 15:02:31 -08:00
enzotar
0ed76513a6 feat(relay): multi-source routing — /source/{name} and /stream/{name} 
- ChannelState: per-channel broadcast, input mpsc, cache
- RelayState: HashMap<String, Arc<RwLock<ChannelState>>>
- parse_path: /source/{n}, /stream/{n}, legacy / fallback
- Per-channel stats logging with channel name prefix
- Backward compatible: plain /source and /stream use 'default' channel
- 44 ds-stream tests, 95 total workspace (0 failures)
 2026-02-25 14:50:39 -08:00
enzotar
2b2b4ffaec feat(wasm): add ds-stream-wasm crate — browser codec via WebAssembly 
- 18KB WASM binary with wasm-bindgen exports
- Header encode/decode, RLE compression, XOR delta
- Message builders for signal diff/sync and input events
- 7 native tests, all passing
- Total workspace: 89 tests, 0 failures
 2026-02-25 14:45:51 -08:00
enzotar
69f39746af feat(ds-stream): RLE compression, input events, keyframe caching 
- protocol.rs: TouchEvent, GamepadAxisEvent, GamepadButtonEvent, ResizeEvent
  with encode/decode and roundtrip tests
- codec.rs: rle_encode/rle_decode for delta frame compression (333x on
  unchanged frames), signal_sync_frame/signal_diff_frame builders,
  touch/gamepad/stream_end convenience builders
- relay.rs: StateCache for late-joining receivers (keyframe + signal sync +
  accumulated diffs), periodic keepalive pings, stats logging every 30s,
  diff accumulator with cap at 1000
- BITSTREAM_INTEGRATION.md: compiler integration spec for another agent
- Tests: 17 → 38 (all passing)
 2026-02-25 11:04:05 -08:00
enzotar
ea64617569 feat: physics language integration — scene container with Rapier2D WASM 
- Add scene container to AST, lexer, parser, analyzer, and codegen
- Add circle/rect/line as UI elements for physics body declaration
- Compile scene {} to canvas + async WASM init + Rapier2D PhysicsWorld
- Reactive gravity via DS.effect() — bodies wake on gravity change
- Mouse drag interaction with impulse-based body movement
- Compile-time hex color parsing for body colors
- Fix is_signal_ref matching numeric literals (700.value bug)
- Fix body variable uniqueness (next_node_id per body)
- Fix gravity signal detection (check AST Ident before emit_expr)
- Add physics.ds example with 5 bodies + 4 gravity control buttons
- Update DREAMSTACK.md and IMPLEMENTATION_PLAN.md with Phase 10-11
- 39 tests pass across all crates, 22KB output
 2026-02-25 10:58:43 -08:00
enzotar
d7961cdc98 feat: universal bitstream streaming — any input → any output 
New crate: engine/ds-stream/
- Binary protocol: 16-byte header, typed frame/input enums
  - Frame types: Pixels, Delta, Audio, Signal, Neural (0x01-0x43)
  - Input types: Pointer, Key, Scroll, Gamepad, MIDI, BCI (0x01-0x90)
- WebSocket relay server (tokio + tungstenite)
  - Source → receivers: frame broadcast
  - Receivers → source: input routing
- Codec: encode/decode, XOR delta compression, RLE, convenience builders
- 17 unit tests, all passing

Streaming modes (stream-source.html):
1. Pixel mode: raw RGBA framebuffer (~28 MB/s)
2. Delta mode: XOR + RLE compression (~1-9 MB/s, 70-95% savings)
3. Signal mode: compact JSON signal diffs (~2 KB/s, 12000x reduction)
4. Neural mode: procedural SDF pixel generator (concept demo)
5. Audio channel: spring velocity→frequency synthesis
6. Multi-receiver: broadcast to all connected clients

Thin receiver client (stream-receiver.html, ~300 lines):
- Zero framework, zero build step
- Renders any incoming bitstream mode
- Local signal-diff renderer for signal mode
- AudioContext playback for audio frames
- Full input capture: click/drag, keyboard, scroll
- Per-channel bitstream bus visualization

DREAMSTACK.md: Phase 7 section with protocol spec
 2026-02-25 10:29:44 -08:00