The problem
Distracted driving from phone use kills thousands yearly, largely because drivers reach for tasks their phone won't do reliably hands-free.
Apple and other platforms block third-party software from driving the phone hands-free, and built-in voice assistants fail on the long tail of app interactions.
Any real solution has to operate arbitrary apps the way a human passenger would — tapping, swiping, typing, multi-touch — without special OS APIs.
The approach
Sidestep platform API restrictions entirely by physically operating the phone: a frame around the phone with two articulated robotic arms ending in capacitive stylus tips (multi-touch capable) plus side-button actuators.
Close the loop with vision-language models: an inward camera reads the phone's screen state and maps tappable targets to actuator coordinates; an outward camera reads the road for navigator context.
Drive it by voice — wake word to ASR to LLM intent parsing to an action planner that emits tap/swipe/type sequences and recovers from unexpected screens, gated by a driver-attention safety layer.
What I built
- Full multi-subsystem product architecture spanning mechanical actuation, inward/outward vision pipelines, voice/intent stack, action planner, memory, and safety/privacy layers
- A locked industrial-design specification for the articulated dual-arm capacitive-stylus actuator and adjustable spring-grip phone frame
- A provisional patent filing plus a supplemental provisional covering the specific embodiment
- Product concept and provisional specs including full system diagram, subsystem breakdowns, and action-planner pseudocode
- A set of high-fidelity industrial-design renders and hero visual materials for the device
- A signup landing page on an all-Cloudflare stack (Pages + Pages Functions + KV) capturing early interest
- Vision-language models (VLM)
- LLM intent parsing
- ASR / wake-word
- TTS
- robotic actuators / servos
- embedded systems
- Cloudflare Pages / Functions / KV
Result
A concept-and-IP-stage project: patent filed, industrial-design specification and renders locked, full system architecture documented, and an early-interest landing page shipped ahead of hardware.