Reducing delays: AR use case

Hi,

I measured the latency in the DK1 and DK2 using method described by Jacobs et al.

In principle I ran a very slim OpenGL 4 application which just displayed a full screen quad alternating between full white and full black. Then I recorded the timing values from when the the command is sent until the corresponding change could be recorded by the screen. Note that the Oculus units were used as pure screens, i.e. no lens distortion shaders. For DK2 this means running the unit in extended mode. The values I got were as below (in milliseconds):
Oculus Rift DK1 + Vsync Off Min: 16.8 Max: 60.0 Avg: 37.6
Oculus Rift DK1 + Vsync On Min: 34.0 Max: 79.1 Avg: 48.1
Oculus Rift DK2 + Vsync Off Min: 17.0 Max: 65.0 Avg: 39.6
Oculus Rift DK2 + Vsync On Min: 24.0 Max: 70.1 Avg: 43.6

I do not have any measurements for running with DirectMode.

(Jacobs, M. C., Livingston, M. A., & State, A. (1997). Managing
latency in complex augmented reality systems. In A. van Dam (Ed.),
Proceedings of the 1997 symposium on Interactive 3D graphics.
ACM Press. doi:10.1145/253284.253306)

My experience is the same as yours. A full AR solution ending up around 80-100 ms in total latency.

pixelminer said:

...
My experience is the same as yours. A full AR solution ending up around 80-100 ms in total latency.

Thanks a lot for your response! My measuring method is similar to that described in the paper - light source directed into a camera (laser in our case) and photo-electric sensor pointing at Oculus screen, both connected to oscilloscope. In addition, I'm generating a special serial signal (over FTDI) when I submit a "lighted" frame to Oculus, which also goes to same oscilloscope. As result, I can measure the exact time from "ovr_SubmitFrame" to the moment the image appears on the Rift, and it's around 50-60 msec. BTW, I'm working with the consumer version of Rift, SDK version 1.6.

These numbers are not acceptable for a natural AR experience. So, any hope here? Or we can just conclude that Oculus is not an appropriate device for AR use cases?

Interesting!

May I ask what type of camera you are using? Are you using Mono or Stereo cameras?

Personally I do not think Oculus are suitable for a natural AR experience. In my humble opinion the need to process the camera image on the CPU is what is ruining the latency. Any successful AR solution would instead merge the images directly inside the HMD and thus bypassing the computer to reduce latency.

One possible future actor may be VR Vana. They claim that their AR-HMD have less than 20 msec latency.
https://www.vrvana.com/

We are currently using two mono cameras. We plan to test the ZED stereo camera, maybe it can upload the images faster to the GPU - at least in the API level they have an option to provide a pointer to CUDA buffer, but still not sure whether they can do it any faster than simple upload to the GPU from the User mode. Another direction we are considering is to test the HTC Vive device, maybe it will allow smaller delays. I believe that overall delay of ~40 msec will be already good enough for some of the use cases.

BTW, I have tried the VRVana HMD at some conference, and it is pretty impressive indeed, still not clear when it will be available, and I would prefer some home-made solution for now.