Forum Discussion
Rift Telepresence Robot
Hi all,
I build an 3D telepresence robot based on a double robot and I would like to share my results with you.
The idea was to improve the telepresence experience of the double robot. The double robot can be used from everywhere to create a more natural telepresence experience and gives you a physical presence where ever you want.
I had the idea to combine the telepresence experience of the double robot and the 3D VR experience of the Oculus Rift.
The initial idea was to use two GoPros to create an stereoscopic video signal. There are already mounts for recording in 3D which I could use. I really like the flexibility and the wide angle lenses of GoPro. But there is one big problem which is the reason why I'm not using GoPros. The GoPro is a camera for recording and not for livestreaming, and I need a low latency video stream. You can connect a GoPro to your wifi and stream a low resolution video with about 2-5 seconds delay. But thats not what I was aiming for. The only way to use GoPros would be to send the HDMI output of the GoPro to the Oculus Rift using a video grabber. But that would be to heavy for the double robot and even then, the quality would not be perfect.
In this forum i found an amazing oculus rift player called Live Rift View (https://developer.oculusvr.com/forums/viewtopic.php?f=28&t=11001) mrjazz, the creator of this player also made a really cool iOs app called iPCamera. With iPCamera you can stream the video signal from you iPhone/iPad camera to your PC using WIFI. You are also able to change a lot of settings like the resolution and exposure. So I mounted two iPad mini to the double robot using the GoPro mount system.
(klick to enlarge)
I also taped two 0.67X wide angle lenses on the cameras which increases the field of view a lot.
But you are still not able to look down and see if anything is in your way so i mounted a third iPad mini on the bottom of the double robot.
Now you can drive around without putting down the Oculus Rift to see if there is anything in your way. The "Wheelcam" will appear as a small window on the bottom of your view.
The weight of all the equipment is quite a problem. The double robot is always trying to balance and stabilize itself and more weight means more to balance. Three additional iPad mini and some plastic is already almost the limit. The double robot is not as fast as without all the equipment and its a bit more tricky to control the robot. On the picture below you can see the full dimensions of the modified robot
To have less latency as possible is very important. If you have to much latency and not a video signal with at least 30FPS, it will make you feel uncomfortable and not real enough. At the beginning i used my personal iPhone as a hotspot. With an OSX and Windows at-hoc WIFI the latency an package loss was to high, I have no idea why. Now I'm using an Apple AirPort Extreme which is perfect. There is almost no latency and no package loss. All iPads and the Macbook which is connected to the Oculus Rift are in the same WIFI.
As already mentioned I used LiveRiftView. It's a heavy tool with a lot of functions and settings. It lets you watch different kinds of media with the Oculus Rift. So I just typed in the IPs of the iPads (for each iPad a layer).
Now you have to adjust the position of each layer which is quite difficult. Here you can see the output which is also streamed to the Oculus Rift.
The experience is really nice, but of course the resolution is not the best. I used a DK1 and my DK2 will arrive in the next Weeks. I think that will be a big improvement.
This prototype was really dirty hacked together but it shows very good how the future of telepresence could look like. It is a really impressive feeling to sit at home with the rift on and drive around in the office in 3D.
If you have any questions, feedback or some possible improvements, feel free to leaf a post :)
I can also post a short demo video if you are interested.
Best,
Jonas
I build an 3D telepresence robot based on a double robot and I would like to share my results with you.
The idea was to improve the telepresence experience of the double robot. The double robot can be used from everywhere to create a more natural telepresence experience and gives you a physical presence where ever you want.
I had the idea to combine the telepresence experience of the double robot and the 3D VR experience of the Oculus Rift.
The initial idea was to use two GoPros to create an stereoscopic video signal. There are already mounts for recording in 3D which I could use. I really like the flexibility and the wide angle lenses of GoPro. But there is one big problem which is the reason why I'm not using GoPros. The GoPro is a camera for recording and not for livestreaming, and I need a low latency video stream. You can connect a GoPro to your wifi and stream a low resolution video with about 2-5 seconds delay. But thats not what I was aiming for. The only way to use GoPros would be to send the HDMI output of the GoPro to the Oculus Rift using a video grabber. But that would be to heavy for the double robot and even then, the quality would not be perfect.
In this forum i found an amazing oculus rift player called Live Rift View (https://developer.oculusvr.com/forums/viewtopic.php?f=28&t=11001) mrjazz, the creator of this player also made a really cool iOs app called iPCamera. With iPCamera you can stream the video signal from you iPhone/iPad camera to your PC using WIFI. You are also able to change a lot of settings like the resolution and exposure. So I mounted two iPad mini to the double robot using the GoPro mount system.
(klick to enlarge)I also taped two 0.67X wide angle lenses on the cameras which increases the field of view a lot.
But you are still not able to look down and see if anything is in your way so i mounted a third iPad mini on the bottom of the double robot.
Now you can drive around without putting down the Oculus Rift to see if there is anything in your way. The "Wheelcam" will appear as a small window on the bottom of your view.
The weight of all the equipment is quite a problem. The double robot is always trying to balance and stabilize itself and more weight means more to balance. Three additional iPad mini and some plastic is already almost the limit. The double robot is not as fast as without all the equipment and its a bit more tricky to control the robot. On the picture below you can see the full dimensions of the modified robot
To have less latency as possible is very important. If you have to much latency and not a video signal with at least 30FPS, it will make you feel uncomfortable and not real enough. At the beginning i used my personal iPhone as a hotspot. With an OSX and Windows at-hoc WIFI the latency an package loss was to high, I have no idea why. Now I'm using an Apple AirPort Extreme which is perfect. There is almost no latency and no package loss. All iPads and the Macbook which is connected to the Oculus Rift are in the same WIFI.
As already mentioned I used LiveRiftView. It's a heavy tool with a lot of functions and settings. It lets you watch different kinds of media with the Oculus Rift. So I just typed in the IPs of the iPads (for each iPad a layer).
Now you have to adjust the position of each layer which is quite difficult. Here you can see the output which is also streamed to the Oculus Rift.
The experience is really nice, but of course the resolution is not the best. I used a DK1 and my DK2 will arrive in the next Weeks. I think that will be a big improvement.
This prototype was really dirty hacked together but it shows very good how the future of telepresence could look like. It is a really impressive feeling to sit at home with the rift on and drive around in the office in 3D.
If you have any questions, feedback or some possible improvements, feel free to leaf a post :)
I can also post a short demo video if you are interested.
Best,
Jonas
No RepliesBe the first to reply