Raspberry Pi camera module - Could it be modded for Oculus?

"Harley" wrote:

"edzieba" wrote:

"Harley" wrote:
I guess you would still require a separate microcontroller with digital camera interface for USB connection to your PC (without connecting a full Raspberry Pi to use it as the micro controller)

Why NOT use the Pi? It can stream the video over a network connection to the PC.

Because of latency, which is very important for Augmented Reality. I just can't imagine that using a Raspberry Pi as the microcontroller to capture the video data and sending it as a stream over the network would have lower latency than using a USB attached microcrontroller for the camera to connect it directly to the PC which in turn have the Oculus Rift connected to it via HDMI / DVI.

This isn't a webcam, it's a MIPI CSI camera. The bandwidth of the raw pixel stream is an order of magnitude more than what USB could ever hope to carry. You can encode it to h.264 before transmission, though no microcontroller or embedded processor will be grunty enough to do so without a dedicated encoder block. Luckily, the Pi's ARM has just such a dedicated encoder. Have the Pi handle the encoding of the raw video stream, then send this to a PC over ethernet instead of USB. That's not going to add any more latency than encoding on-board and sending it over USB (which you'd have to bodge on, as the Pi can only act as a USB host).

As to whether displaying the video on-board the Pi will have a lower latency, theoretically yes. But the Pi can only handle a single camera, so you'd be stuck with mono video, and performing the warp shader with the ARM (or the ARM's GPU) might take significantly longer than if you did so on a desktop.

Broadcom is known to be one of the more open source friendly companies out there, and they have previously released many drivers, firmwares, and libraries under open source libraries.

I am sure that with enough 'pestering' from the Raspberry Pi community they could be convinced to release all code for these camera modules too.

Ideally, but Broadcom have so far refused to open source anything that operates on the GPU side of things. They've been pretty adamant in not allowing anyone enough access to write their own CSI drivers either. This is why using other camera modules with the Pi is vanishingly unlikely, and similarly why there aren't more barebones camera modules available for a similar cost.