As an experiment I’ve been submitting myself and my exasperated family to the full  ‘real world’ XBoxOne + Kinect Voice + gesture experience using the HDMI pass-through for the last 6 months.

• The XboxOne is always on.
• All Video input from my Windows Media Center & Apple TV pass-through the XboxOne.

The theory is that you can use voice and gestures to switch between TV and XboxOne Games and Apps.

The reality is that you get an annoying number of false positives which result in the XboxOne dropping you out of TV or bringing up a “What was it you said again?” style prompt.

• Sometimes it will lock onto your hand, because it is there.
• Sometimes it will recognise audio from the TV, conversation in the room or a few times from a video played on my phone about how to control the XboxOne 🙂

A very annoying issue is that once it accidentally swaps you out of TV your voice is competing with audio from the show which continues to play in the background. You find yourself shouting “Stop Listening” or having to pause and do same.

The most annoying issue however is an issue from the 360+Kinect days which is that the coverage of the Voice/Gesture control is not universal. If you accidentally switch to an app that needs an update you get shoved into the update UI which will not accept Voice/Gesture commands and you need to find a controller. Individual app updates happen a lot.

On the positive side I have noticed that with each update, command registration gets better. So given enough time I believe they can make this workable. But we are not there yet.

Also, we do use it.

We use it to switch between games and TV, to Skype my mother and to play/pause content when the remote is hidden under potato chips.

I still believe that this is the future, we are progressing, but it’s not there yet.

As for Android TV, the newly announced revamp of Google TV……

For me personally, using the phone/tablet to control the TV is better than a remote, at least in the experiments I have done.

But not how Google have implemented it.

Phone as Dpad is not the way. It’s just a more expensive, heavier, less convenient, more complex remote control

What works particularly well is if the browsing and control aspects of the interface are split between the tablet and the TV.

My favourite TV experience is pushing content from my phone or laptop to the TV via the Apple TV. There is some kind of joyful usability magic there beyond the technology.

Browse near, play far.

I’ve always found remote controls unnatural. There exists something wrong at the heart of the experience.

I think there is something basic in our sense of proprioception and in particular our sense of extended proprioception that is violated by a distance further than a short stick suitable for fishing termites out of a mound.

Browse near, play far. It feels more natural.

I hope that Google can bring some magic to voice and gesture, but don’t believe it will work in the real world yet.