After the wiimote headtracking experiments, I felt that having two extra accessories (wiimote and a sensor bar) just wouldn’t cut it for any practical application of a deep UI. Instead, I downloaded OpenCV (open source face tracking) to track the user’s eyes and perspective transform the UI based on that.
Sorry that there’s no live camera feed with this recording; you’ll just have to trust that I sat here and moved my face back and forth and magically made the UI swing around to that. I did a slow pan from right to left, followed by a slow low pan from left to right, and then some misc moves. Use the green and blue dots as references for the perspective transform; the blue dot is where my face is (in the camera image) and the green dot is canvas center, translated along positive Z axis to show the transform more clearly.
I’ve been very interested in alternative user interfaces for many years. In my voodoopad wiki the topic “My Dream UI” has the tag line “If I just keep throwing ideas together, the big mashed-up blob will eventually become awesomeness” :) So, for my dream UI, I would want:
Natural UI/Multitouch. You all know what multitouch is, and it’s become mainstream now with the iPhone. (When I started using the term, I was referring to a multitouch surface such as Jeff Han’s table. Several companies and groups are claiming this term as their name now, though).
Physical UI: Entities in a Physical UI are tangible, physical, can collide, slide, stick, be heavy. The most well known example is BumpTop, which was recently publically released.
Deep UI: This is my own term for a UI that appears deep, for example by head tracking. The inspiration is Johnny Chung Lee’s excellent wiimote experimentation (bottom of that page). In a Deep UI, when you move your head or rotate the device, you are able to look under foreground objects and see off the side what’s below; e g seeing windows below the foreground window that would be completely obscured if seen head-on.
My dream UI is thus a Deep NUI. From my wiki:
A Deep UI is one that takes the user’s depth perception into account, specifically by using head and/or eye tracking. A Deep classical UI would be a window-managed UI, where one may pan the head to see behind the current window. A Deep NUI is one that uses multitouch surfaces, and applies depth handling. This applies not only to seeing behind flat surfaces in the NUI, but also changing the point-of-view of three-dimensional applications in the NUI.
Addendum: About a month ago, The Astonishing Tribe released a concept of a cell phone with Deep UI! Check it out:
