16 KiB
DreamStack Hardware Explorations
Research notes on form factors, display technologies, and touch input methods for DreamStack-powered surfaces.
1. USB Dongle (Chromecast-like)
The DreamStack relay protocol + delta codec already does 90% of the work. A dongle receives the bitstream and outputs to HDMI.
Path A: ESP32-S3 HDMI Dongle (~$15 DIY)
| Component | Part | Cost |
|---|---|---|
| SoC | ESP32-S3-WROOM-1 (N16R8) | ~$4 |
| HDMI output | CH7035B or ADV7513 HDMI encoder IC | ~$3 |
| USB-C power | Standard power-only connector | ~$0.50 |
| PCB + passives | Custom PCB (JLCPCB) | ~$5 for 5 boards |
| HDMI connector | Type-A male or mini-HDMI | ~$1 |
- ESP32-S3 LCD parallel interface → HDMI encoder IC → HDMI out
- WiFi connects to DreamStack relay, receives delta-compressed frames
- Resolution limit: ~480×320 smooth, 800×600 at lower FPS
- Input via BLE HID remote or HDMI CEC (pin 13)
Path B: Linux Stick / Allwinner (~$25-40)
MangoPi MQ-Pro / Radxa Zero form factor:
| Component | Part | Cost |
|---|---|---|
| SoC | Allwinner H616/H618 (HDMI built-in) | ~$15 module |
| RAM | 512MB DDR3 onboard | included |
| WiFi | RTL8723CS | ~$3 |
| Storage | 8GB eMMC or SD | ~$3 |
- Runs minimal Linux (Buildroot), headless browser or C receiver writing to
/dev/fb0 - Native HDMI — no encoder IC needed
- Full DreamStack JS runtime in headless Chromium/WPE-WebKit
- CEC for remote control
Path C: Pi Zero 2 W (~$15, recommended MVP)
Best for proving the concept immediately — $15, mini-HDMI out, WiFi, runs DreamStack natively.
Laptop WiFi/LAN Pi Zero 2 W (in 3D-printed HDMI case)
────── ───────── ─────────────────────────────────────
dreamstack dev app.ds headless browser / ds_runtime
→ relay-bridge ──────── WebSocket ───────→ → HDMI out to TV
←── CEC/BLE ←──── remote control
Off-the-shelf stick computers
| Device | Price | HDMI | WiFi | Notes |
|---|---|---|---|---|
| Raspberry Pi Zero 2 W | $15 | Mini-HDMI ✅ | ✅ | Best form factor |
| MangoPi MQ-Pro (RISC-V) | $20 | HDMI ✅ | ✅ | Stick form factor |
| Radxa Zero | $25 | Micro-HDMI ✅ | ✅ | Amlogic S905Y2 |
| T-Dongle S3 (LilyGO) | $12 | No (LCD only) | ✅ | ESP32-S3, tiny LCD |
2. Projected Touch Wall
Architecture
SOURCE (laptop/Pi) RELAY (:9100) WALL
─────────────── ───────────── ────
DreamStack app WebSocket hub UST Projector
800×1280 canvas + touch overlay
pixels → XOR delta → RLE ────→ relay ─────────────────→ decode → project
←── touch {x,y,phase} ←── touch sensor
Ultra-Short-Throw Projectors
| Product | Price (new) | Price (used) | Notes |
|---|---|---|---|
| Xiaomi Laser Cinema 2 | ~$1,200 | ~$400 | Good value |
| BenQ V7050i | ~$2,500 | ~$800 | 4K HDR |
| JMGO U2 | ~$1,000 | ~$300 | Budget friendly |
| Epson LS500 | ~$2,000 | ~$600 | Bright |
Wall prep: screen paint (Silver Ticket / Rust-Oleum, ~$30).
3. Touch Input Technologies
Ranked from fastest to slowest latency:
3a. Piezoelectric Sensors (<1ms, ~$7)
Stick 3-4 piezo discs on the back of the wall. Finger taps create vibrations; time-difference-of-arrival (TDOA) triangulates X,Y.
| Part | Cost |
|---|---|
| 4× piezo disc (35mm) | ~$2 |
| ESP32-S3 (built-in ADC, 40kHz+ sampling) | ~$5 |
Wall (drywall, glass, wood, whiteboard)
┌───────────────────────────────────────┐
│ P1 ● ● P2 │
│ 👆 TAP │
│ P3 ● ● P4 │
└───────────────────────────────────────┘
└────── ESP32 (TDOA → x,y) ─────┘
Pros: Near-instant, invisible, dirt cheap, works through paint Cons: Only detects taps (not drag/hover), needs hard surface
3b. Capacitive Wire/Paint Grid (1-3ms, ~$21 DIY)
Grid of conductors (copper tape or conductive paint) behind the wall. Measures capacitance change when a finger approaches.
| Part | Cost |
|---|---|
| Copper tape grid (30 channels) or graphite paint | ~$10 |
| MPR121 capacitive controller ×3 | ~$6 |
| ESP32 | ~$5 |
Supports: Touch ✅ Drag ✅ Multi-touch ✅ Hover (~1-2cm) ✅ Resolution: Depends on grid pitch — 3cm pitch ≈ 30×40 nodes over 100"
How Mutual Capacitance Works
Two layers of conductors (rows + columns) cross each other, separated by a thin insulator. Each intersection forms a capacitor. A finger near any intersection absorbs electric field, reducing measured capacitance.
The controller scans one row at a time (AC drive), reads all columns simultaneously. Full scan of 30×40 grid: ~0.5-1ms. Continuous scanning gives automatic drag/swipe detection. Sub-pixel interpolation from adjacent node readings gives ~1mm accuracy from 5mm pitch.
Recommended ICs
| IC | Grid Size | Touch Points | Price |
|---|---|---|---|
| MTCH6303 (Microchip) | 15×49 | 10 | ~$5 |
| IQS7211A (Azoteq) | 15×22 | 5 | ~$3 |
| GT911 (Goodix) | 26×14 | 5 | ~$2 |
| FT5x06 (FocalTech) | 24×14 | 5 | ~$2 |
3c. FTIR — Frustrated Total Internal Reflection (3-8ms, ~$110-250)
Acrylic sheet on wall with IR LEDs on edges (total internal reflection). Finger touch "frustrates" the reflection → bright spots detected by IR camera.
| Part | Cost |
|---|---|
| 4mm acrylic sheet (100") | ~$80-150 |
| IR LED strip (850nm) on edges | ~$10 |
| IR camera (120fps, no IR filter) | ~$15 |
| ESP32-S3 or Pi | ~$5-75 |
Pros: Multi-touch, precise, pressure-sensitive (brighter blob = more pressure) Cons: Needs smooth flat surface (acrylic)
3d. IR Touch Frame (8-15ms, ~$250-500)
Aluminum frame with IR LEDs + sensors on 4 edges. Finger breaks IR beams → X,Y.
| Size | Price | Touch Points |
|---|---|---|
| 65" | ~$250 | 6-10 pt |
| 82" | ~$350 | 10-20 pt |
| 100" | ~$500 | 10-20 pt |
| 120"+ | ~$800+ | 20+ pt |
Premium: Neonode zForce (~6-8ms, 200Hz). Budget: generic Chinese frames (~15-30ms, 100Hz).
Pros: USB HID plug-and-play, works on any surface Cons: Physical border/bezel on wall
3e. Depth Camera (15-30ms, ~$80-450)
| Camera | FPS | Latency | Range | Price |
|---|---|---|---|---|
| Intel RealSense D405 | 90fps | ~11ms | 7cm-50cm | ~$80 |
| RealSense D435i | 90fps | ~11ms | 10cm-10m | ~$200 |
| OAK-D SR (Short Range) | 60fps | ~12ms | 2cm-100cm | ~$150 |
| OAK-D Pro | 30fps depth | ~15ms | 20cm-15m | ~$200 |
| Stereolabs ZED Mini | 100fps | ~10ms | 10cm-12m | ~$300 |
| Stereolabs ZED X Mini | 120fps | ~8ms | 10cm-15m | ~$450 |
| Orbbec Gemini 2 | 60fps | ~16ms | 15cm-10m | ~$130 |
RealSense D405 is ideal for wall touch — 90fps hardware stereo depth, 7cm minimum distance, global shutter. No ML needed for touch detection: just threshold the depth map (depth < 5mm → TOUCH, < 150mm → HOVER).
Layer MediaPipe on top (parallel) for gesture classification.
3f. Hybrid: Best of All Worlds
| Input | Method | Latency |
|---|---|---|
| Tap detection + pressure | Piezo (4 corners) | <1ms |
| Touch + drag + hover | Capacitive grid | 1-3ms |
| Hand gestures (air) | RealSense D405 | ~15ms |
4. Gesture / Hand Tracking
DIY Approaches (Ultraleap alternative)
Stereo IR Camera + MediaPipe (~$30-50)
Two OV2710 IR USB cameras (stereo pair, $15 each) + 850nm IR LED strip ($5). MediaPipe Hands on Pi 5 or Jetson: 21 landmarks per hand, 30-120fps. Stereo triangulation gives Z. Latency: ~20-30ms.
Single Depth Camera (~$80-150)
Use RealSense D405 or OAK-D SR (see above). Hardware depth gives Z-distance from wall.
ESP32-S3 + IR Matrix (~$20, lowest latency)
IR LEDs flood the area in front of the wall. 2-3 IR cameras do blob detection at 120fps on ESP32-S3. Z estimated from blob size. No ML needed. Latency: 5-10ms.
5. Conductive Paint Recipes
For capacitive grid electrodes painted directly on walls.
Graphite Paint (easiest, ~$5)
| Ingredient | Amount | Source |
|---|---|---|
| Graphite powder (<45μm) | 3 tbsp | Art supply, Amazon (~$8/lb) |
| PVA glue (white school glue) | 2 tbsp | Any store |
| Water | 1 tbsp | Tap |
~60% graphite, 30% glue, 10% water by volume. Resistance: ~500-2000 Ω/sq. Good enough for capacitive sensing.
Carbon Black + Acrylic (~$15)
20-25% carbon black powder (conductive grade) in 75-80% acrylic medium. Resistance: ~200-800 Ω/sq. Better adhesion. Wear mask + gloves.
Nickel Paint (~$20)
MG Chemicals 841, premade. Resistance: ~5-50 Ω/sq. Mid-range.
Silver Paint (~$30-50)
Premade: Bare Conductive (~$25/50ml), MG Chemicals 842. DIY: 70-80% silver flake powder (<10μm), 15-20% acrylic medium, 5-10% butyl acetate. Resistance: ~0.5-5 Ω/sq. Near-wire conductivity.
For capacitive sensing: graphite is sufficient
Capacitive touch doesn't need low resistance — just enough conductivity to couple with a finger. Paint lines with tape masking at 3-5cm spacing.
6. Pixel Paint — Paint-On Displays
Electroluminescent (EL) Paint Display
Real and buildable. A stack of painted layers that glow when AC voltage is applied.
Layer stack (painted in order):
5. Clear topcoat
4. Transparent conductor (PEDOT:PSS) ← rows
3. Phosphor layer (ZnS:Cu in acrylic) ← glows
2. Dielectric (BaTiO₃ in acrylic) ← insulator
1. Base conductor (silver/carbon paint) ← columns
─── Wall surface ───
Row/column intersection = one pixel. AC across a specific row+column → only that intersection glows (passive matrix).
| Layer | Material | Cost/m² |
|---|---|---|
| Base conductor (columns) | Silver paint, painted in strips | ~$50 |
| Dielectric | Barium titanate (BaTiO₃) in acrylic | ~$30 |
| Phosphor | ZnS:Cu powder in acrylic | ~$20 |
| Top conductor (rows) | PEDOT:PSS | ~$40 |
| Driver electronics | HV507 shift registers + ESP32 | ~$30 |
| Total | ~$170/m² |
Resolution at different pitches
| Pixel Pitch | Pixels (100" wall) | Comparable To |
|---|---|---|
| 20mm | 110×65 = 7,150 | LED sign |
| 10mm | 220×130 = 28,600 | Scoreboard |
| 5mm | 440×260 = 114,400 | ~400×260 display ✅ |
| 2mm | 1100×650 = 715,000 | Near SD |
At 5mm pitch: 440×260 — enough for DreamStack UIs, dashboards, snake game.
Color
- ZnS:Cu → green (brightest)
- ZnS:Cu,Mn → amber/orange
- ZnS:Cu,Al → blue-green
- Full RGB requires 3 sub-pixels per pixel (3× driver count)
- Monochrome green is practical and looks great
Built-in touch (free!)
The row/column electrodes double as capacitive sensing electrodes via time-multiplexing:
- Sense phase (1ms): measure capacitance = touch position
- Drive phase (15ms): apply AC = illuminate pixels
Same paint layers, no extra hardware.
Driver IC
HV507 — 64-channel high-voltage shift register. Drives 100V+ outputs from 3.3V SPI. Chain several for full display.
Other Display Paint Technologies (Future)
| Technology | Status | Color | Speed |
|---|---|---|---|
| Electrochromic (PEDOT:PSS, WO₃) | Real | Grayscale | 1-30s (too slow for video) |
| Thermochromic + resistive grid | Hackable | Limited | 1-5s |
| Perovskite spray-on LEDs | Lab only | Full color | ~ms |
| QD-LED inkjet | Lab only | Full color | ~ms |
Perovskite / QD-LED spray-on is the future (~2028-2030) but not available today.
7. Off-the-Shelf Solutions
Capacitive Touch Overlays (stick-on film)
| Product | Max Size | Touch Points | Latency | Price |
|---|---|---|---|---|
| Displax Skin Ultra | 105" | 40 | ~6ms | ~$800-1500 |
| Visual Planet TouchFoil | 100"+ | 40 | ~8ms | ~$600-1200 |
| PQ Labs iTouch Plus | 150"+ | 32 | ~8ms | ~$400-900 |
| AliExpress "PCAP touch foil" | 100"+ | 10 | ~10-15ms | ~$200-400 |
Displax Skin Ultra: transparent polymer film with nano-wire grid, adhesive-backed, works through 6mm of material, USB HID, detects hover at ~2cm. Stick on wall, plug USB, done.
All-in-One Interactive Projectors
| Product | Size | Touch | Latency | Price (new) |
|---|---|---|---|---|
| Epson BrightLink 770Fi | 100" | 10pt + pen | ~10ms | ~$2,500 |
| Epson BrightLink 735Fi | 100" | 10pt + pen | ~10ms | ~$2,000 |
| BenQ LW890UST | 100" | 10pt | ~12ms | ~$1,800 |
| Boxlight Mimio MiXX | 100" | 20pt | ~8ms | ~$2,200 |
Used education projectors (schools constantly upgrade):
| Used Option | Price |
|---|---|
| Epson BrightLink 695Wi/696Ui | $300-600 |
| BenQ MW855UST+ with PointWrite | $400-700 |
| Promethean UST + ActivBoard | $300-500 |
Interactive Flat Panels (giant touchscreen monitors)
| Product | Size | Price (new) | Price (used) |
|---|---|---|---|
| SMART Board MX | 65-86" | $3,000-6,000 | $500-1,500 |
| Promethean ActivPanel | 65-86" | $3,000-5,000 | $600-1,200 |
| ViewSonic ViewBoard | 65-98" | $2,000-8,000 | $500-1,500 |
| Samsung Flip | 55-85" | $2,000-4,000 | $800-2,000 |
| Microsoft Surface Hub 2S | 50-85" | $5,000-12,000 | $1,500-3,000 |
8. Recommended Builds
Budget: $675
| Component | Source | Price |
|---|---|---|
| Used SMART Board 65" | eBay | ~$600 |
| Pi 5 | Official | ~$75 |
Plug HDMI + USB, run DreamStack, done.
Mid-Range: $700
| Component | Price |
|---|---|
| UST projector (used) | ~$300 |
| PCAP touch foil 100" (AliExpress) | ~$300 |
| Pi 5 | ~$75 |
| Screen paint | ~$30 |
Premium: $1,050
| Component | Price |
|---|---|
| UST projector (used) | ~$400 |
| 100" IR touch frame | ~$350 |
| RealSense D405 (gestures + hover) | ~$80 |
| Pi 5 | ~$75 |
| Piezo sensors (4 corners, tap confirm) | ~$7 |
| Screen paint | ~$30 |
Touch at 8-15ms + hover/gestures at 15ms + tap confirmation at <1ms.
DIY Maximum: ~$200 + wall paint
| Component | Price |
|---|---|
| Conductive graphite paint (capacitive grid) | ~$10 |
| MPR121/MTCH6303 cap-touch IC | ~$5 |
| ESP32-S3 | ~$5 |
| UST projector (used) | ~$300 |
Paint your own touch grid on the wall, 1-3ms latency, no frame needed.
9. DreamStack Integration
All touch methods feed into the existing relay protocol:
Touch sensor (any method above)
→ ESP32 or Pi reads touch events
→ Encodes as DreamStack protocol:
0x01 Pointer move (x, y)
0x02 Pointer down (x, y, buttons)
0x03 Pointer up
0x10 KeyDown (keyCode)
0x20 Hover (x, y, z_distance) ← new
0x21 Swipe (direction, velocity) ← new
0x22 Pinch/Grab (state) ← new
→ WebSocket → DreamStack relay
→ App receives as signal updates
DreamStack syntax for handling:
on hover(ev) -> opacity = lerp(0.5, 1.0, ev.z)
on swipe(ev) -> navigate(if ev.dir == "left" then "/next" else "/prev")
on grab(ev) -> scale = if ev.closed then 0.9 else 1.0