canvas/docs/plan-gesture-system-v2.md

Gesture System v2 Plan

Recommendation

Do a hard cutoff.

Do not spend time on adapters, dual dispatch, legacy callbacks, or persisted settings compatibility. v2 should replace the current gesture surface with one coherent shared-input runtime and delete the old stack in the same migration.

No deprecation window. No compatibility bridge. One breaking migration.

This is still not a blind greenfield rewrite. Reuse low-level pieces that are already correct, but stop designing around compatibility.

---

Cutoff Decision

Keep

• core/input-store.ts as the global pointer/source truth
• core/interaction-store.ts as the source of input modes
• core/selection-path-store.ts for lasso / rect-select state
• @use-gesture/react for drag / pinch / wheel recognition
• local gesture scopes where pointer capture or special ownership is required

Replace

• core/gesture-rules.ts
• core/gesture-rule-store.ts
• core/settings-types.ts
• core/settings-store.ts
• core/action-registry.ts
• core/action-executor.ts
• hooks/useCanvasSettings.ts
• components/SettingsPanel.tsx
• gesture logic embedded in Viewport.tsx, Node.tsx, and useNodeDrag.ts

Delete At Cutoff

• gestureRules prop
• per-event compatibility callbacks such as onNodeClick, onNodeDoubleClick, onBackgroundClick, etc.
• CanvasEventType
• presets and event-to-action settings persistence
• any legacy adapter or bridge layer

---

Goals

1. Replace the current split gesture logic with one shared-input runtime.
2. Fix lifecycle bugs around pointercancel, hidden tabs, stale stylus state, and orphaned timers.
3. Move from event-name mapping to input-event mapping with context layering and phase-aware actions.
4. Keep local scopes only where they are structurally necessary.
5. End with less code, fewer parallel abstractions, and one public gesture API.

Non-Goals

• Maintaining old props, old settings, or old storage formats
• Preserving old click / double-click semantics exactly
• Rebuilding the settings UI in the same phase as the runtime cutoff
• Unifying resize, minimap, edge creation, and split into the central runtime on day one

---

Current Problems

Area	Problem
Viewport.tsx	Pan, zoom, inertia, long-press, threshold logic, and gesture ownership are all inline
Node.tsx	Rendering is mixed with click counting, touch long-press timers, and context menu handling
useNodeDrag.ts	Drag mechanics are coupled to gesture policy and right-drag suppression
gesture-rules + settings-types + action-registry	The system is split between gesture matching, event mappings, and action execution with overlapping concerns
settings UI	The current UI is built around discrete event names, not gesture streams

The main mistake to avoid is preserving this split under a new name.

---

Architecture

v2 should have a single shared-input pipeline with four layers.

Layer 1: Normalize

Normalize raw DOM input into pointer events plus shared keyboard state:

• source
• button
• modifiers
• screen position
• pressure
• timestamp
• pointer lifecycle including cancel
• coalesced points when available

Layer 2: Recognize

Turn normalized pointers into pointer gestures and model keyboard as a parallel input event stream:

• drag / pinch / wheel via @use-gesture/react
• tap / double-tap / triple-tap / long-press via a small timed state machine
• explicit gesture ownership rules
• source-aware thresholds in one shared module

Layer 3: Resolve

Resolve recognized pointer gestures and keyboard events against one context stack:

• default bindings
• palm rejection context
• input-mode context
• local blocking contexts
• consumer-pushed contexts

Layer 4: Dispatch

Dispatch all input events through one action model:

• discrete gestures use phase: 'instant'
• continuous gestures use start / move / end / cancel
• no separate CanvasEventType bridge

---

Core Runtime Types

type GestureSubject =
  | { kind: 'background' }
  | { kind: 'node'; nodeId: string }
  | { kind: 'edge'; edgeId: string }
  | { kind: 'port'; nodeId: string; portId: string };

type GestureType =
  | 'tap'
  | 'double-tap'
  | 'triple-tap'
  | 'long-press'
  | 'drag'
  | 'pinch'
  | 'scroll';

type GesturePhase = 'start' | 'move' | 'end' | 'instant' | 'cancel';

interface GestureEvent {
  type: GestureType;
  phase: GesturePhase;
  subject: GestureSubject;
  source: 'mouse' | 'finger' | 'pencil';
  button: 0 | 1 | 2;
  modifiers: { shift: boolean; ctrl: boolean; alt: boolean; meta: boolean };
  pointerId?: number;
  screenPosition: { x: number; y: number };
  worldPosition: { x: number; y: number };
  delta?: { x: number; y: number };
  velocity?: { x: number; y: number };
  scale?: number;
  originalEvent?: Event;
}

interface Binding {
  id: string;
  pattern: Partial<Pick<GestureEvent, 'type' | 'source' | 'button' | 'phase'>> & {
    subjectKind?: GestureSubject['kind'];
    modifiers?: Partial<GestureEvent['modifiers']>;
  };
  actionId: string;
  consumeInput?: boolean;
  when?: (ctx: GuardContext) => boolean;
}

interface MappingContext {
  id: string;
  priority: number;
  enabled: boolean;
  bindings: Binding[];
}

This replaces the old CanvasEventType -> actionId model completely.

---

Design Decisions

1. Replace Event-Mapping With Gesture-Mapping

The current settings/event system is built around labels like node:click and background:long-press.

That model is too narrow for:

• continuous phases
• pointer ownership
• source-aware gesture behavior
• context stacking

v2 should map actual gesture patterns to actions, not synthetic event names.

2. Own Click And Tap Recognition In One Place

Do not keep browser click, dblclick, and contextmenu as a separate path.

v2 should own:

• tap
• double-tap / double-click
• triple-tap
• long-press
• right-click

That gives one recognition model across mouse, touch, and pencil.

3. Keep Local Scopes Where The Physics Differ

These stay local in v2:

Interaction	Why
Resize handles	direct geometry updates + pointer capture
Minimap drag	separate coordinate system
Edge creation drag	direct hit testing + pointer capture
Split gesture	specialized two-finger ownership

These local scopes still interact with the runtime by:

• setting guard flags
• pushing temporary blocking contexts
• dispatching high-level actions when needed

4. Input Mode Must Use The Real Store Shape

interaction-store.ts already models input mode as a discriminated union. Keep that.

Guard context should carry:

• the full input mode object, or
• a derived inputMode.type

Do not flatten it into an invented string union.

5. Context Sync Belongs In Hooks

Input mode and local blocking contexts should be pushed from explicit hooks.

Do not hide that work in effectful derived atoms.

6. No Legacy Surface At The End

When v2 lands:

• the old public props are removed
• the old settings UI is removed or disabled
• the old stores and registry files are deleted

No compatibility release.

---

New File Plan

Create a dedicated packages/canvas/src/gestures/ module:

• types.ts
• normalize.ts
• timed-state.ts
• specificity.ts
• mapper.ts
• contexts.ts
• dispatcher.ts
• inertia.ts
• useCanvasGestures.ts
• useNodeGestures.ts
• useGestureSystem.ts
• useInputModeGestureContext.ts

Extend or keep:

• core/input-store.ts
• core/interaction-store.ts
• core/selection-path-store.ts
• hooks/useNodeResize.ts
• hooks/useSplitGesture.ts

Delete after cutoff:

• core/gesture-rules.ts
• core/gesture-rule-store.ts
• core/settings-types.ts
• core/settings-store.ts
• core/action-registry.ts
• core/action-executor.ts
• hooks/useCanvasSettings.ts
• components/SettingsPanel.tsx

---

Public API Direction

Replace the old callback-heavy surface with a gesture-native API.

Remove

• gestureRules
• onGestureAction
• onNodeClick
• onNodeDoubleClick
• onNodeTripleClick
• onNodeRightClick
• onNodeLongPress
• onBackgroundClick
• onBackgroundDoubleClick
• onBackgroundRightClick
• onBackgroundLongPress

Add

interface CanvasProps {
  gestureConfig?: {
    contexts?: MappingContext[];
    debug?: boolean;
    palmRejection?: boolean;
  };
  onAction?: (event: GestureEvent, resolution: ResolvedAction) => void;
}

If consumers need click-specific behavior, they express it as bindings and action handlers, not bespoke props.

---

Phased Implementation Plan

Phase 0: Capture Expected Behavior

Before rewriting, document the behavior we still care about.

Deliverables:

• tests for Viewport pan / pinch / wheel / background gesture behavior
• tests for Node tap, double-tap, triple-tap, right-click, and long-press behavior
• tests for useNodeDrag modifier and button behavior
• tests for stylus palm rejection
• tests for split-vs-pinch ownership on nodes

Success criteria:

• we have a concrete acceptance matrix
• we know what to preserve functionally and what can change

Phase 1: Lifecycle Hardening

Fix the bugs that should survive the rewrite.

Deliverables:

• handle pointercancel in all pointer-tracking code
• clear active pointers on visibilitychange
• cancel timers and inertia on hidden tab, cancel, and unmount
• extract pure source-threshold and inertia helpers
• add cancellation handling to useSplitGesture.ts

Success criteria:

• no stale pointer state
• no orphaned timers
• no runaway inertia

Phase 2: Build The New Pure Runtime

Implement the new system in gestures/ without touching public components yet.

Deliverables:

• normalization layer
• timed state machine
• specificity scoring
• context stack resolver
• phase-aware dispatcher
• inertia engine

Success criteria:

• all pure pieces are independently testable
• the runtime has no dependency on old event-mapping types

Phase 3: Add React Integration

Wire the runtime into hooks and stores.

Deliverables:

• useCanvasGestures.ts
• useNodeGestures.ts
• useGestureSystem.ts
• useInputModeGestureContext.ts
• hook-driven context sync for input modes and local blockers

Success criteria:

• hooks can drive gestures end-to-end without using the old rule or settings stack

Phase 4: Cut Off Components

Replace the old component gesture paths directly.

Deliverables:

• Viewport.tsx uses useCanvasGestures.ts
• Node.tsx uses useNodeGestures.ts
• useNodeDrag.ts is reduced to drag mechanics
• DOM click / dblclick / contextmenu paths are removed
• old callback props are removed from Canvas.tsx

Success criteria:

• one runtime owns gesture interpretation
• components mostly render and forward gesture props

Phase 5: Delete The Old Stack

Remove everything that only existed for the old model.

Deliverables:

• delete old rules and settings files
• delete the settings panel
• delete the old action registry / executor
• delete unused exports
• update docs and examples to the new API

Success criteria:

• no dead compatibility code remains
• there is one obvious path for gesture behavior

---

Cutover Risks

1. Desktop Semantics Will Change

Owning double-click and context menu behavior ourselves means web behavior may differ from native browser expectations.

That is acceptable, but it must be deliberate.

2. Settings UI Disappears Until Rebuilt

If the current settings panel is still valuable, it should be rebuilt later against gesture bindings and contexts. It should not block the runtime rewrite.

3. @use-gesture Ownership Must Be Verified

Do not assume pinch cancellation and drag ownership work exactly as the new runtime wants.

Test:

• pinch over background
• split over node
• drag interrupted by second pointer
• cancel during active gesture

4. Local Scopes Can Drift

Resize, minimap, edge creation, and split are intentionally local at first. If they do not publish guard state consistently, gesture conflicts will remain.

---

Verification

Automated:

1. pnpm --filter @blinksgg/canvas vitest run
2. pnpm --filter @blinksgg/canvas check-types
3. pnpm --filter @blinksgg/canvas build

Manual:

• mouse: tap, double-tap, right-click, drag, wheel zoom
• touch: pan, pinch zoom, long-press
• pencil: tap, drag, lasso
• stylus + palm rejection
• node drag with modifiers and alternate buttons
• split gesture on node vs pinch on background
• resize handles
• minimap drag
• edge creation
• input modes overriding normal behavior
• visibility change during active gesture

Exit criteria:

• all supported gestures route through the new runtime
• old settings and callback code is deleted
• no stale pointer or timer bugs remain
• Viewport.tsx, Node.tsx, and useNodeDrag.ts are materially smaller

---

Summary

The right v2 is a hard cutoff, not a compatibility bridge.

Build one gesture-native runtime, cut the old event-mapping stack, keep only the low-level stores that still make sense, and delete the rest in the same migration.