449 lines
16 KiB
Markdown
449 lines
16 KiB
Markdown
# DreamStack Hardware Explorations
|
||
|
||
> Research notes on form factors, display technologies, and touch input methods for DreamStack-powered surfaces.
|
||
|
||
---
|
||
|
||
## 1. USB Dongle (Chromecast-like)
|
||
|
||
The DreamStack relay protocol + delta codec already does 90% of the work. A dongle receives the bitstream and outputs to HDMI.
|
||
|
||
### Path A: ESP32-S3 HDMI Dongle (~$15 DIY)
|
||
|
||
| Component | Part | Cost |
|
||
|-----------|------|------|
|
||
| SoC | ESP32-S3-WROOM-1 (N16R8) | ~$4 |
|
||
| HDMI output | CH7035B or ADV7513 HDMI encoder IC | ~$3 |
|
||
| USB-C power | Standard power-only connector | ~$0.50 |
|
||
| PCB + passives | Custom PCB (JLCPCB) | ~$5 for 5 boards |
|
||
| HDMI connector | Type-A male or mini-HDMI | ~$1 |
|
||
|
||
- ESP32-S3 LCD parallel interface → HDMI encoder IC → HDMI out
|
||
- WiFi connects to DreamStack relay, receives delta-compressed frames
|
||
- Resolution limit: ~480×320 smooth, 800×600 at lower FPS
|
||
- Input via BLE HID remote or HDMI CEC (pin 13)
|
||
|
||
### Path B: Linux Stick / Allwinner (~$25-40)
|
||
|
||
MangoPi MQ-Pro / Radxa Zero form factor:
|
||
|
||
| Component | Part | Cost |
|
||
|-----------|------|------|
|
||
| SoC | Allwinner H616/H618 (HDMI built-in) | ~$15 module |
|
||
| RAM | 512MB DDR3 onboard | included |
|
||
| WiFi | RTL8723CS | ~$3 |
|
||
| Storage | 8GB eMMC or SD | ~$3 |
|
||
|
||
- Runs minimal Linux (Buildroot), headless browser or C receiver writing to `/dev/fb0`
|
||
- Native HDMI — no encoder IC needed
|
||
- Full DreamStack JS runtime in headless Chromium/WPE-WebKit
|
||
- CEC for remote control
|
||
|
||
### Path C: Pi Zero 2 W (~$15, recommended MVP)
|
||
|
||
Best for proving the concept immediately — $15, mini-HDMI out, WiFi, runs DreamStack natively.
|
||
|
||
```
|
||
Laptop WiFi/LAN Pi Zero 2 W (in 3D-printed HDMI case)
|
||
────── ───────── ─────────────────────────────────────
|
||
dreamstack dev app.ds headless browser / ds_runtime
|
||
→ relay-bridge ──────── WebSocket ───────→ → HDMI out to TV
|
||
←── CEC/BLE ←──── remote control
|
||
```
|
||
|
||
### Off-the-shelf stick computers
|
||
|
||
| Device | Price | HDMI | WiFi | Notes |
|
||
|--------|-------|------|------|-------|
|
||
| Raspberry Pi Zero 2 W | $15 | Mini-HDMI ✅ | ✅ | Best form factor |
|
||
| MangoPi MQ-Pro (RISC-V) | $20 | HDMI ✅ | ✅ | Stick form factor |
|
||
| Radxa Zero | $25 | Micro-HDMI ✅ | ✅ | Amlogic S905Y2 |
|
||
| T-Dongle S3 (LilyGO) | $12 | No (LCD only) | ✅ | ESP32-S3, tiny LCD |
|
||
|
||
---
|
||
|
||
## 2. Projected Touch Wall
|
||
|
||
### Architecture
|
||
|
||
```
|
||
SOURCE (laptop/Pi) RELAY (:9100) WALL
|
||
─────────────── ───────────── ────
|
||
DreamStack app WebSocket hub UST Projector
|
||
800×1280 canvas + touch overlay
|
||
|
||
pixels → XOR delta → RLE ────→ relay ─────────────────→ decode → project
|
||
←── touch {x,y,phase} ←── touch sensor
|
||
```
|
||
|
||
### Ultra-Short-Throw Projectors
|
||
|
||
| Product | Price (new) | Price (used) | Notes |
|
||
|---------|-------------|-------------|-------|
|
||
| Xiaomi Laser Cinema 2 | ~$1,200 | ~$400 | Good value |
|
||
| BenQ V7050i | ~$2,500 | ~$800 | 4K HDR |
|
||
| JMGO U2 | ~$1,000 | ~$300 | Budget friendly |
|
||
| Epson LS500 | ~$2,000 | ~$600 | Bright |
|
||
|
||
Wall prep: screen paint (Silver Ticket / Rust-Oleum, ~$30).
|
||
|
||
---
|
||
|
||
## 3. Touch Input Technologies
|
||
|
||
Ranked from fastest to slowest latency:
|
||
|
||
### 3a. Piezoelectric Sensors (<1ms, ~$7)
|
||
|
||
Stick 3-4 piezo discs on the **back** of the wall. Finger taps create vibrations; time-difference-of-arrival (TDOA) triangulates X,Y.
|
||
|
||
| Part | Cost |
|
||
|------|------|
|
||
| 4× piezo disc (35mm) | ~$2 |
|
||
| ESP32-S3 (built-in ADC, 40kHz+ sampling) | ~$5 |
|
||
|
||
```
|
||
Wall (drywall, glass, wood, whiteboard)
|
||
┌───────────────────────────────────────┐
|
||
│ P1 ● ● P2 │
|
||
│ 👆 TAP │
|
||
│ P3 ● ● P4 │
|
||
└───────────────────────────────────────┘
|
||
└────── ESP32 (TDOA → x,y) ─────┘
|
||
```
|
||
|
||
**Pros:** Near-instant, invisible, dirt cheap, works through paint
|
||
**Cons:** Only detects **taps** (not drag/hover), needs hard surface
|
||
|
||
---
|
||
|
||
### 3b. Capacitive Wire/Paint Grid (1-3ms, ~$21 DIY)
|
||
|
||
Grid of conductors (copper tape or conductive paint) behind the wall. Measures capacitance change when a finger approaches.
|
||
|
||
| Part | Cost |
|
||
|------|------|
|
||
| Copper tape grid (30 channels) or graphite paint | ~$10 |
|
||
| MPR121 capacitive controller ×3 | ~$6 |
|
||
| ESP32 | ~$5 |
|
||
|
||
**Supports:** Touch ✅ Drag ✅ Multi-touch ✅ Hover (~1-2cm) ✅
|
||
**Resolution:** Depends on grid pitch — 3cm pitch ≈ 30×40 nodes over 100"
|
||
|
||
#### How Mutual Capacitance Works
|
||
|
||
Two layers of conductors (rows + columns) cross each other, separated by a thin insulator. Each intersection forms a capacitor. A finger near any intersection absorbs electric field, reducing measured capacitance.
|
||
|
||
The controller scans one row at a time (AC drive), reads all columns simultaneously. Full scan of 30×40 grid: ~0.5-1ms. Continuous scanning gives automatic drag/swipe detection. Sub-pixel interpolation from adjacent node readings gives ~1mm accuracy from 5mm pitch.
|
||
|
||
#### Recommended ICs
|
||
|
||
| IC | Grid Size | Touch Points | Price |
|
||
|----|-----------|-------------|-------|
|
||
| MTCH6303 (Microchip) | 15×49 | 10 | ~$5 |
|
||
| IQS7211A (Azoteq) | 15×22 | 5 | ~$3 |
|
||
| GT911 (Goodix) | 26×14 | 5 | ~$2 |
|
||
| FT5x06 (FocalTech) | 24×14 | 5 | ~$2 |
|
||
|
||
---
|
||
|
||
### 3c. FTIR — Frustrated Total Internal Reflection (3-8ms, ~$110-250)
|
||
|
||
Acrylic sheet on wall with IR LEDs on edges (total internal reflection). Finger touch "frustrates" the reflection → bright spots detected by IR camera.
|
||
|
||
| Part | Cost |
|
||
|------|------|
|
||
| 4mm acrylic sheet (100") | ~$80-150 |
|
||
| IR LED strip (850nm) on edges | ~$10 |
|
||
| IR camera (120fps, no IR filter) | ~$15 |
|
||
| ESP32-S3 or Pi | ~$5-75 |
|
||
|
||
**Pros:** Multi-touch, precise, pressure-sensitive (brighter blob = more pressure)
|
||
**Cons:** Needs smooth flat surface (acrylic)
|
||
|
||
---
|
||
|
||
### 3d. IR Touch Frame (8-15ms, ~$250-500)
|
||
|
||
Aluminum frame with IR LEDs + sensors on 4 edges. Finger breaks IR beams → X,Y.
|
||
|
||
| Size | Price | Touch Points |
|
||
|------|-------|-------------|
|
||
| 65" | ~$250 | 6-10 pt |
|
||
| 82" | ~$350 | 10-20 pt |
|
||
| 100" | ~$500 | 10-20 pt |
|
||
| 120"+ | ~$800+ | 20+ pt |
|
||
|
||
Premium: Neonode zForce (~6-8ms, 200Hz). Budget: generic Chinese frames (~15-30ms, 100Hz).
|
||
|
||
**Pros:** USB HID plug-and-play, works on any surface
|
||
**Cons:** Physical border/bezel on wall
|
||
|
||
---
|
||
|
||
### 3e. Depth Camera (15-30ms, ~$80-450)
|
||
|
||
| Camera | FPS | Latency | Range | Price |
|
||
|--------|-----|---------|-------|-------|
|
||
| **Intel RealSense D405** | 90fps | ~11ms | 7cm-50cm | ~$80 |
|
||
| RealSense D435i | 90fps | ~11ms | 10cm-10m | ~$200 |
|
||
| OAK-D SR (Short Range) | 60fps | ~12ms | 2cm-100cm | ~$150 |
|
||
| OAK-D Pro | 30fps depth | ~15ms | 20cm-15m | ~$200 |
|
||
| Stereolabs ZED Mini | 100fps | ~10ms | 10cm-12m | ~$300 |
|
||
| Stereolabs ZED X Mini | 120fps | ~8ms | 10cm-15m | ~$450 |
|
||
| Orbbec Gemini 2 | 60fps | ~16ms | 15cm-10m | ~$130 |
|
||
|
||
**RealSense D405** is ideal for wall touch — 90fps hardware stereo depth, 7cm minimum distance, global shutter. No ML needed for touch detection: just threshold the depth map (`depth < 5mm → TOUCH`, `< 150mm → HOVER`).
|
||
|
||
Layer MediaPipe on top (parallel) for gesture classification.
|
||
|
||
---
|
||
|
||
### 3f. Hybrid: Best of All Worlds
|
||
|
||
| Input | Method | Latency |
|
||
|-------|--------|---------|
|
||
| Tap detection + pressure | Piezo (4 corners) | <1ms |
|
||
| Touch + drag + hover | Capacitive grid | 1-3ms |
|
||
| Hand gestures (air) | RealSense D405 | ~15ms |
|
||
|
||
---
|
||
|
||
## 4. Gesture / Hand Tracking
|
||
|
||
### DIY Approaches (Ultraleap alternative)
|
||
|
||
#### Stereo IR Camera + MediaPipe (~$30-50)
|
||
|
||
Two OV2710 IR USB cameras (stereo pair, ~$15 each) + 850nm IR LED strip (~$5). MediaPipe Hands on Pi 5 or Jetson: 21 landmarks per hand, 30-120fps. Stereo triangulation gives Z. **Latency: ~20-30ms.**
|
||
|
||
#### Single Depth Camera (~$80-150)
|
||
|
||
Use RealSense D405 or OAK-D SR (see above). Hardware depth gives Z-distance from wall.
|
||
|
||
#### ESP32-S3 + IR Matrix (~$20, lowest latency)
|
||
|
||
IR LEDs flood the area in front of the wall. 2-3 IR cameras do blob detection at 120fps on ESP32-S3. Z estimated from blob size. No ML needed. **Latency: 5-10ms.**
|
||
|
||
---
|
||
|
||
## 5. Conductive Paint Recipes
|
||
|
||
For capacitive grid electrodes painted directly on walls.
|
||
|
||
### Graphite Paint (easiest, ~$5)
|
||
|
||
| Ingredient | Amount | Source |
|
||
|-----------|--------|--------|
|
||
| Graphite powder (<45μm) | 3 tbsp | Art supply, Amazon (~$8/lb) |
|
||
| PVA glue (white school glue) | 2 tbsp | Any store |
|
||
| Water | 1 tbsp | Tap |
|
||
|
||
~60% graphite, 30% glue, 10% water by volume. **Resistance: ~500-2000 Ω/sq.** Good enough for capacitive sensing.
|
||
|
||
### Carbon Black + Acrylic (~$15)
|
||
|
||
20-25% carbon black powder (conductive grade) in 75-80% acrylic medium. **Resistance: ~200-800 Ω/sq.** Better adhesion. Wear mask + gloves.
|
||
|
||
### Nickel Paint (~$20)
|
||
|
||
MG Chemicals 841, premade. **Resistance: ~5-50 Ω/sq.** Mid-range.
|
||
|
||
### Silver Paint (~$30-50)
|
||
|
||
Premade: Bare Conductive (~$25/50ml), MG Chemicals 842.
|
||
DIY: 70-80% silver flake powder (<10μm), 15-20% acrylic medium, 5-10% butyl acetate.
|
||
**Resistance: ~0.5-5 Ω/sq.** Near-wire conductivity.
|
||
|
||
### For capacitive sensing: graphite is sufficient
|
||
|
||
Capacitive touch doesn't need low resistance — just enough conductivity to couple with a finger. Paint lines with tape masking at 3-5cm spacing.
|
||
|
||
---
|
||
|
||
## 6. Pixel Paint — Paint-On Displays
|
||
|
||
### Electroluminescent (EL) Paint Display
|
||
|
||
Real and buildable. A stack of painted layers that glow when AC voltage is applied.
|
||
|
||
```
|
||
Layer stack (painted in order):
|
||
|
||
5. Clear topcoat
|
||
4. Transparent conductor (PEDOT:PSS) ← rows
|
||
3. Phosphor layer (ZnS:Cu in acrylic) ← glows
|
||
2. Dielectric (BaTiO₃ in acrylic) ← insulator
|
||
1. Base conductor (silver/carbon paint) ← columns
|
||
─── Wall surface ───
|
||
```
|
||
|
||
Row/column intersection = one pixel. AC across a specific row+column → only that intersection glows (passive matrix).
|
||
|
||
| Layer | Material | Cost/m² |
|
||
|-------|----------|---------|
|
||
| Base conductor (columns) | Silver paint, painted in strips | ~$50 |
|
||
| Dielectric | Barium titanate (BaTiO₃) in acrylic | ~$30 |
|
||
| Phosphor | ZnS:Cu powder in acrylic | ~$20 |
|
||
| Top conductor (rows) | PEDOT:PSS | ~$40 |
|
||
| Driver electronics | HV507 shift registers + ESP32 | ~$30 |
|
||
| **Total** | | **~$170/m²** |
|
||
|
||
#### Resolution at different pitches
|
||
|
||
| Pixel Pitch | Pixels (100" wall) | Comparable To |
|
||
|------------|---------------------|---------------|
|
||
| 20mm | 110×65 = 7,150 | LED sign |
|
||
| 10mm | 220×130 = 28,600 | Scoreboard |
|
||
| 5mm | 440×260 = 114,400 | ~400×260 display ✅ |
|
||
| 2mm | 1100×650 = 715,000 | Near SD |
|
||
|
||
At 5mm pitch: 440×260 — enough for DreamStack UIs, dashboards, snake game.
|
||
|
||
#### Color
|
||
|
||
- ZnS:Cu → green (brightest)
|
||
- ZnS:Cu,Mn → amber/orange
|
||
- ZnS:Cu,Al → blue-green
|
||
- Full RGB requires 3 sub-pixels per pixel (3× driver count)
|
||
- Monochrome green is practical and looks great
|
||
|
||
#### Built-in touch (free!)
|
||
|
||
The row/column electrodes double as capacitive sensing electrodes via time-multiplexing:
|
||
1. **Sense phase** (1ms): measure capacitance = touch position
|
||
2. **Drive phase** (15ms): apply AC = illuminate pixels
|
||
|
||
Same paint layers, no extra hardware.
|
||
|
||
#### Driver IC
|
||
|
||
HV507 — 64-channel high-voltage shift register. Drives 100V+ outputs from 3.3V SPI. Chain several for full display.
|
||
|
||
### Other Display Paint Technologies (Future)
|
||
|
||
| Technology | Status | Color | Speed |
|
||
|-----------|--------|-------|-------|
|
||
| Electrochromic (PEDOT:PSS, WO₃) | Real | Grayscale | 1-30s (too slow for video) |
|
||
| Thermochromic + resistive grid | Hackable | Limited | 1-5s |
|
||
| Perovskite spray-on LEDs | Lab only | Full color | ~ms |
|
||
| QD-LED inkjet | Lab only | Full color | ~ms |
|
||
|
||
Perovskite / QD-LED spray-on is the future (~2028-2030) but not available today.
|
||
|
||
---
|
||
|
||
## 7. Off-the-Shelf Solutions
|
||
|
||
### Capacitive Touch Overlays (stick-on film)
|
||
|
||
| Product | Max Size | Touch Points | Latency | Price |
|
||
|---------|----------|-------------|---------|-------|
|
||
| **Displax Skin Ultra** | 105" | 40 | ~6ms | ~$800-1500 |
|
||
| Visual Planet TouchFoil | 100"+ | 40 | ~8ms | ~$600-1200 |
|
||
| PQ Labs iTouch Plus | 150"+ | 32 | ~8ms | ~$400-900 |
|
||
| AliExpress "PCAP touch foil" | 100"+ | 10 | ~10-15ms | ~$200-400 |
|
||
|
||
Displax Skin Ultra: transparent polymer film with nano-wire grid, adhesive-backed, works through 6mm of material, USB HID, detects hover at ~2cm. Stick on wall, plug USB, done.
|
||
|
||
### All-in-One Interactive Projectors
|
||
|
||
| Product | Size | Touch | Latency | Price (new) |
|
||
|---------|------|-------|---------|-------------|
|
||
| **Epson BrightLink 770Fi** | 100" | 10pt + pen | ~10ms | ~$2,500 |
|
||
| Epson BrightLink 735Fi | 100" | 10pt + pen | ~10ms | ~$2,000 |
|
||
| BenQ LW890UST | 100" | 10pt | ~12ms | ~$1,800 |
|
||
| Boxlight Mimio MiXX | 100" | 20pt | ~8ms | ~$2,200 |
|
||
|
||
**Used education projectors** (schools constantly upgrade):
|
||
|
||
| Used Option | Price |
|
||
|-------------|-------|
|
||
| Epson BrightLink 695Wi/696Ui | $300-600 |
|
||
| BenQ MW855UST+ with PointWrite | $400-700 |
|
||
| Promethean UST + ActivBoard | $300-500 |
|
||
|
||
### Interactive Flat Panels (giant touchscreen monitors)
|
||
|
||
| Product | Size | Price (new) | Price (used) |
|
||
|---------|------|-------------|-------------|
|
||
| **SMART Board MX** | 65-86" | $3,000-6,000 | $500-1,500 |
|
||
| Promethean ActivPanel | 65-86" | $3,000-5,000 | $600-1,200 |
|
||
| ViewSonic ViewBoard | 65-98" | $2,000-8,000 | $500-1,500 |
|
||
| Samsung Flip | 55-85" | $2,000-4,000 | $800-2,000 |
|
||
| Microsoft Surface Hub 2S | 50-85" | $5,000-12,000 | $1,500-3,000 |
|
||
|
||
---
|
||
|
||
## 8. Recommended Builds
|
||
|
||
### Budget: $675
|
||
|
||
| Component | Source | Price |
|
||
|-----------|--------|-------|
|
||
| Used SMART Board 65" | eBay | ~$600 |
|
||
| Pi 5 | Official | ~$75 |
|
||
|
||
Plug HDMI + USB, run DreamStack, done.
|
||
|
||
### Mid-Range: $700
|
||
|
||
| Component | Price |
|
||
|-----------|-------|
|
||
| UST projector (used) | ~$300 |
|
||
| PCAP touch foil 100" (AliExpress) | ~$300 |
|
||
| Pi 5 | ~$75 |
|
||
| Screen paint | ~$30 |
|
||
|
||
### Premium: $1,050
|
||
|
||
| Component | Price |
|
||
|-----------|-------|
|
||
| UST projector (used) | ~$400 |
|
||
| 100" IR touch frame | ~$350 |
|
||
| RealSense D405 (gestures + hover) | ~$80 |
|
||
| Pi 5 | ~$75 |
|
||
| Piezo sensors (4 corners, tap confirm) | ~$7 |
|
||
| Screen paint | ~$30 |
|
||
|
||
Touch at 8-15ms + hover/gestures at 15ms + tap confirmation at <1ms.
|
||
|
||
### DIY Maximum: ~$200 + wall paint
|
||
|
||
| Component | Price |
|
||
|-----------|-------|
|
||
| Conductive graphite paint (capacitive grid) | ~$10 |
|
||
| MPR121/MTCH6303 cap-touch IC | ~$5 |
|
||
| ESP32-S3 | ~$5 |
|
||
| UST projector (used) | ~$300 |
|
||
|
||
Paint your own touch grid on the wall, 1-3ms latency, no frame needed.
|
||
|
||
---
|
||
|
||
## 9. DreamStack Integration
|
||
|
||
All touch methods feed into the existing relay protocol:
|
||
|
||
```
|
||
Touch sensor (any method above)
|
||
→ ESP32 or Pi reads touch events
|
||
→ Encodes as DreamStack protocol:
|
||
0x01 Pointer move (x, y)
|
||
0x02 Pointer down (x, y, buttons)
|
||
0x03 Pointer up
|
||
0x10 KeyDown (keyCode)
|
||
0x20 Hover (x, y, z_distance) ← new
|
||
0x21 Swipe (direction, velocity) ← new
|
||
0x22 Pinch/Grab (state) ← new
|
||
→ WebSocket → DreamStack relay
|
||
→ App receives as signal updates
|
||
```
|
||
|
||
DreamStack syntax for handling:
|
||
|
||
```
|
||
on hover(ev) -> opacity = lerp(0.5, 1.0, ev.z)
|
||
on swipe(ev) -> navigate(if ev.dir == "left" then "/next" else "/prev")
|
||
on grab(ev) -> scale = if ev.closed then 0.9 else 1.0
|
||
```
|