Tutorial: Low cost FPV setup for the rift (130ms latency)

EDIT/UPDATE: Read post 5. Basically, latency has been reduced to only 130ms!


Yesterday night, it finally happened: I got my rift! And one of the first things I tried was flying FPV with my RC quadcopter. I've been wanting to do this since I heard about the rift last year, and I've been researching how to get FPV working since. And now I finally have the confirmation: Although it's not perfect, it definitely works! Basically, what I do is convert the analog signal from the video receiver to an USB webcam video, and feed that to the rift with proper duplication and warping.

What you need:
- An Oculus Rift
- An RC vehicle
- Standard FPV gear (Camera, video transmitter and receiver)
- An RCA to USB video capture device (I use this: http://bit.ly/12daZIR)
- A laptop
- A 12V battery
- 5V voltage regulator (or 5V battery)
- A power cable you don't need that fits the rift
- RCA cable
- Optional, but recommended: A customizable OSD (I uses minimOSD)

First, setup your FPV system if you are new to FPV. I won't cover that in this tutorial, google it. There are however a few things to note: For the rift, a high FOV camera is obviously preferable. I currently use a GoPro which has 110 degrees which matches the rift pretty perfectly. I have however ordered a different camera for reasons I will explain later, and I went with this: http://bit.ly/11sR7Tk . It has roughly the same FOV.
If you are using an OSD, which is really recommend that you do, you'll probably want one that lets you customize the layout, like minimOSD. Otherwise the text may be too hard to read due to placement or size. This is my layout in minimOSD, everything is perfectly readable: http://pastebin.com/qvrGLxya (Save as x.osd and import)

Another note about the capture device: Ebay is riddled with dirt cheap "easycaps". I recommend that you avoid them, because they are most likely fake and will lead only to problems. I first bought one for $5, but it was useless for my purposes, it used some obscure codec that only the program included in the disk would recognize. Fake devices also have worse image quality and latency. As said, I bought this one: http://bit.ly/12daZIR . It uses the same chip as many higher end capture cards, and that site provides proper and updated drivers for download.

After you have installed the capture card drivers and verified that it works, download this program: http://lab.neosolve.de/RemoteEyes/RemoteEyes.zip
(Credit goes to this guy: https://www.spacefish.biz/blog/ ).
It should automatically detect the camera and start outputting in rift format. Press f to go fullscreen and esc to exit. If like me you have a webcam integrated in the laptop, you need to disable it in device manager, because the program won't let you change the camera it uses, it just picks the first one it sees. I've tried to get the guy to put the source on github or something but he hasn't responded. Out of the box the program has wrong warp parameters. Open the file fshader.txt and replace it with this: http://pastebin.com/57YkYYfa . Feel free to experiment with params, I sacrificed a bit of FOV for better overview.

If you turn everything on you should now get the image from the FPV camera to the rift! But the setup is not yet portable. You need to make the rift and video receiver battery powered. For this I soldered together a custom cable: On one end it has a connector for a 12V liPo battery, I had a few of those because I use them on my quadcopter. The cable splits, where one end goes directly to the video receiver with the proper plug, and the other end goes to a 5V regulator with a small heatsink, and then to the rift (I had a spare cable with a plug that fit the rift control box, so I soldered it on).

You might be tempted to run the rift off the laptop's USB power. In fact I tried this first, but after five minutes the computer bluescreened due to too high USB power draw. Luckily the quad was on the ground, if that happens when you are in the air, you're screwed. Also, make sure to disable any power saver settings on the laptop, the program does not stop the screen from dimming after x minutes, and that would also be terrible if it happened while in the air.

So how well does this work? There are a few problems, the biggest is latency. Imeasured my system to around 0.25 seconds. It's flyable, especially in open areas, but it can get a bit annoying. However, my gopro camera is partially at fault, the live feed has a slight delay in itself. That's why I ordered the other camera I mentioned above. Latency is 130ms with the new camera. Much better. Here's a demonstration video: http://youtu.be/6rTkuO_44S8 .
That, and the fact that the image quality is much better, especially in low or high light conditions. The controller radio can also be a factor, I use a very cheap radio (Gotta upgrade that some day), and there might be a delay between me moving the sticks and the motors responding, which would also affect the total latency. Another factor is the software, if someone makes a program that really focuses on low latency, it should shave a few additional milliseconds off. I bet that with proper equipment the latency can definitely become acceptable. Though in fact, I would say it already is acceptable. I tried flying for a few minutes but crashed, but I think this was because it was in a small, bulky area, and this is the first time I've ever flied FPV, so I just need some practice in better conditions.

Another problem is image quality. This would depend on the camera, warp parameter and capture device, but in my case it is not that great, but of course, this is nothing new to FPV. PAL is the highest resolution one can transmit currently, and that's obviously not great. The rift has a limited resolution as well, of course. But again, it's definitely flyable, and I think the image quality will improve a bit when I get the new camera as well.

Two missing features are 3D, and headtracking. Both of these are possible to implement, using a gimbal and two cameras, however, it would be much more expensive. Perhaps someone else has the dough to test it out? :)



Here are some pictures:
http://i.imgur.com/o3Gg366.jpg
http://i.imgur.com/K6KBA9c.jpg
http://i.imgur.com/bsWR9Zq.jpg
I can take a video of a flight once I get to try it properly in an open area.

One a sidenote, I'd like to link to this article, another take on the rift and FPV: http://intuitiveaerial.com/home/2013/6/8/oculus-rift-fpv
These guys are doing it a bit differently. They have two cameras for 3D, and better latency, perhaps better resolution as well. However, I believe there is a few downsides compared to this tutorial:
- It's much more expensive, of course.
- It looks like they are using a digital wireless link. This allows for things like better bandwidth, but it also vastly reduces range, and you'd have to buy a whole new, expensive setup even if you already have traditional FPV gear.
- I'm not sure they would even give the community the details on their hardware and software unless you buy it, because they are a company their solution will most likely be proprietary.