I’ve Always Found Remote Controls Unnatural: Controlling Android TV and the XboxOne+Kinect

As an experiment I’ve been submitting myself and my exasperated family to the full  ‘real world’ XBoxOne + Kinect Voice + gesture experience using the HDMI pass-through for the last 6 months.

  • The XboxOne is always on.
  • All Video input from my Windows Media Center & Apple TV pass-through the XboxOne.

27-06-2014 10-25-35 AM

The theory is that you can use voice and gestures to switch between TV and XboxOne Games and Apps.

The reality is that you get an annoying number of false positives which result in the XboxOne dropping you out of TV or bringing up a “What was it you said again?” style prompt.

27-06-2014 10-30-04 AM

  • Sometimes it will lock onto your hand, because it is there.
  • Sometimes it will recognise audio from the TV, conversation in the room or a few times from a video played on my phone about how to control the XboxOne 🙂

A very annoying issue is that once it accidentally swaps you out of TV your voice is competing with audio from the show which continues to play in the background. You find yourself shouting “Stop Listening” or having to pause and do same.

27-06-2014 10-32-01 AM

The most annoying issue however is an issue from the 360+Kinect days which is that the coverage of the Voice/Gesture control is not universal. If you accidentally switch to an app that needs an update you get shoved into the update UI which will not accept Voice/Gesture commands and you need to find a controller. Individual app updates happen a lot.

On the positive side I have noticed that with each update, command registration gets better. So given enough time I believe they can make this workable. But we are not there yet.

Also, we do use it.

We use it to switch between games and TV, to Skype my mother and to play/pause content when the remote is hidden under potato chips.

I still believe that this is the future, we are progressing, but it’s not there yet.

As for Android TV, the newly announced revamp of Google TV……

For me personally, using the phone/tablet to control the TV is better than a remote, at least in the experiments I have done.

But not how Google have implemented it.

Phone as Dpad is not the way. It’s just a more expensive, heavier, less convenient, more complex remote control

What works particularly well is if the browsing and control aspects of the interface are split between the tablet and the TV.

My favourite TV experience is pushing content from my phone or laptop to the TV via the Apple TV. There is some kind of joyful usability magic there beyond the technology.

Browse near, play far.

I’ve always found remote controls unnatural. There exists something wrong at the heart of the experience.

I think there is something basic in our sense of proprioception and in particular our sense of extended proprioception that is violated by a distance further than a short stick suitable for fishing termites out of a mound.

27-06-2014 11-02-54 AM

Browse near, play far. It feels more natural.

I hope that Google can bring some magic to voice and gesture, but don’t believe it will work in the real world yet.

Posted in IPTV | Tagged | Leave a comment

The Game Progress Report #21

Another video test. Playing this on a loop looking for any improvements I can make. Already have quite a few ideas to make it better.

Video | Posted on by | Leave a comment

The Game Progress Report #20


Sped up the video encoder by an order of magnitude.

Here are some more example encodes. Just playing with random video that I have already split out into images to put the encoder it through its paces.

Going to try it on some larger videos and work on the file/streaming format.







Posted in TheGame | Leave a comment

The Game Progress Report #19

Example video encoding using the hierarchical hexagon encoding.

Video | Posted on by | Leave a comment

The Game Progress Report #18

Got stuck on the train a bit longer than normal today (heatwave signal failures) and added hexagon culling to the digitizer.

So this..

13-01-2014 7-51-18 PM

Now my favourite processing from the last post looks like this (which I believe is an improvement).

16-01-2014 7-00-52 PM

If you look carefully, any sub-hexagon that does not contribute to the detail is culled, this results in an interesting effect and in the end is exactly what I was after. Some contrast is lost, but I think that’s a given seeing that I am applying an exotic ‘blur’. I’m going to generate some colour histograms later and see what spectral distortions I might be adding.

Here are some runs against some default the sample Windows ‘photos’.

16-01-2014 7-01-53 PM 16-01-2014 7-01-39 PM

16-01-2014 6-59-49 PM 16-01-2014 6-59-31 PM

16-01-2014 6-58-10 PM 16-01-2014 6-57-56 PM

16-01-2014 7-15-03 PM

16-01-2014 7-14-53 PM

Posted in TheGame | Tagged , | Leave a comment

The Game Progress Report #17

Some interesting results from the hierarchical hexagonal digitizer.

The original source image..

13-01-2014 7-51-18 PM

The closest match so far.. but a little boring.

15-01-2014 8-52-31 PM

The one I like the most – but a little dark.

15-01-2014 8-05-44 PM

They all seem to pass the squint test 🙂

Posted in TheGame | Tagged , | 1 Comment

The Game Progress Report #16

More work on the L-System generator/mutator/renderer.

6-01-2014 5-54-33 PM 6-01-2014 9-33-44 PM 6-01-2014 9-31-03 PM

More work on basic image display.

13-01-2014 7-51-18 PM

I can also now load hexagon patterns from Hexels via XML export files.

13-01-2014 7-52-28 PM

And the game engine can now generate hexagon bitmaps from images.

13-01-2014 7-46-30 PM

The effect I’m after with the hierarchically tessellated (regular) hexagons is going to take some more work, but it’s looking promising.

13-01-2014 7-38-17 PM

As for the Twine engine, I’m still trying to get a side project up and running to use the game’s tutorial engine which can read a subset of the Twine format. This will allow me to publish some complex Interactive Fiction.  I’m following a few leads from some Twine authors I have found online. Probably going to start a small collective of artists that can contribute to the IF production pipeline for fun and hopefully profit.

13-01-2014 8-10-24 PM

Posted in TheGame | Tagged , | Leave a comment