augmented reality

A costly, but interesting (to my mind), idea would be to have some hardware in the HDMI pipeline which merges camera input with computer input. With this HDMI mixer you could have a sort-of green-screen; for example any pixels with colour value X will be replaced with the relevant pixels from the video stream.

Where analysis of the video-in needs to be performed, you could in some way split the video-in stream, with one copy going into the above hardware, and the other into the PC. The HDMI mixer would need a programmable delay between video-in and mixing, to allow processing to be performed and have the processed signal roughly match up with the video signal.

My guess would be that all this kit, plus low latency cameras, would cost 200-300 USD and would be quite a major bit of work. But it may be the best option for minimising latency between video-in and HMD display.

What do you want to do with AR anyway when full VR is available? Isn't looking the reality through a small floating window in VR enough?

I'm saying this because, extracting meaningful informtaion from real world requires far more processing power than to just passing video stream around...

Wonder what the latency is on the GoPro streaming for the new Hero3 model? Hmm, I wonder if their stereo setup is supported by the base app.

http://gopro.com/hd-hero3-cameras