Skip to main content
wheatgrinder
Up and Coming
January 5, 2016

create Overlay apps (like steamvr)

Hi,
recently I had to setup steamvr to enable ED Horizons.. anyway, during the setup you can chose to show the "chaperon" bounding box in any game. I tried this for fun and it works. So it seems it is possible to create a VR app that works OVER the top of other running vr apps.


I want to make a simple tool that I can access while in VR without leaving the current game\experience. Where do I start? Is there something in the SDK\runtime that enables this for steamVR?

Thanks

      9 replies

      jherico
      Inspiring
      January 5, 2016
      The SDK would have to expose a mechanism for Application A to inject an overlay that should be rendered inside Application B. This would be tricky because Application A doesn't have access to the rendering context for B. Steam doesn't really have this issue because their runtime will always have access to rendering context for a running application via the compositor, so they can do whatever rendering they need to for their overlay.
      Brad Davis - Developer for High Fidelity Co-author of Oculus Rift in Action
      joan
      joan
      Up and Coming
      January 5, 2016
      The runtime itself does it with the debug HUDs though. They are always on top and can be turned ON/OFF from outside the current application.

      Being able to share the OVR session between several applications would open some interesting mashups, for example injecting a camera feed on a quad locked in the virtual world. Like a peephole into reality that you can use to talk to people that are in the same physical space but outside your personal world.

      At the initialization step the runtime could send back an existing session and hmd description that you would use to create your own render targets and submit as usual. Even better if you could somehow insert your layers at arbitrary places in the existing layer stack, using names to identify layers of other apps.
      thewhiteambit
      Inspiring
      January 5, 2016
      It was easy in extended mode (when that was still possible), if you know the right way of creating a per pixel alpha-transparent direct3d overlay. No rendering context was needed, and I would also say you don't need to know any rendering context for this - the oculus runtime itself has a reference to it, and you could use shared resources. But as with the current Oculus SDK 0.8 it is not possible.
      jherico
      Inspiring
      January 6, 2016
      "joan" wrote:
      The runtime itself does it with the debug HUDs though. They are always on top and can be turned ON/OFF from outside the current application.


      The runtime is generating the overlay and controls the final rendering context, so it's trivial for the runtime to do so. This is entirely different from having a completely unrelated application be able to talk to the runtime, hand it a texture and say 'inject this into whatever's running right now'.
      Brad Davis - Developer for High Fidelity Co-author of Oculus Rift in Action
      thewhiteambit
      Inspiring
      January 7, 2016
      "jherico" wrote:

      The runtime is generating the overlay and controls the final rendering context, so it's trivial for the runtime to do so. This is entirely different from having a completely unrelated application be able to talk to the runtime, hand it a texture and say 'inject this into whatever's running right now'.


      Technically, I don't see much of a problem with handing over a texture and inject this into whatever's running right now. There is great interop at least within Direct3D:

      https://msdn.microsoft.com/en-us/library/windows/desktop/ee913554(v=vs.85).aspx

      "The rendering devices using synchronized shared surfaces can be Direct3D 10.1 or Direct3D 11 devices, each running in the same process or cross-processes."

      Normally it is more a problem of HR getting people being capable of doing this right, and the structures within a company of letting people capable of understanding their decision decide. I guess Carmack is not much involved in the PC SDK, and I have seen to many people hire experts to tell them what to do... instead of having the experts tell them how to do it right.
      jherico
      Inspiring
      January 8, 2016
      I'm an OpenGL man myself, so I'm mostly speaking from that context. However, given that I believe the runtime is doing all the main rendering in D3D, it sounds like it should be possible based on what you're saying.

      Of course technical feasibility is only the first hurdle. How do you implement something like this without spawning an industry of unwanted overlays trying to sell you stuff? It would be the new 'internet explorer toolbar' that unscrupulous companies would try to subvert to get your attention. The only real solution to that is whitelisting apps that can create overlays.
      Brad Davis - Developer for High Fidelity Co-author of Oculus Rift in Action
      thewhiteambit
      Inspiring
      January 8, 2016
      "jherico" wrote:
      I'm an OpenGL man myself, so I'm mostly speaking from that context. However, given that I believe the runtime is doing all the main rendering in D3D, it sounds like it should be possible based on what you're saying.

      It definately is possible. I understand the desire for client site rendering and not having extended desktop anymore (beside I would like to still have this option for developers). I also see that in 90% it is ok to have its rendertarget textures given by oculus runtime. Just wanted to say it is not impossible to hand over own texture sets.


      Of course technical feasibility is only the first hurdle. How do you implement something like this without spawning an industry of unwanted overlays trying to sell you stuff? It would be the new 'internet explorer toolbar' that unscrupulous companies would try to subvert to get your attention. The only real solution to that is whitelisting apps that can create overlays.


      I personally would prevent these kind of Apps the same way I take care of Toolbars - by not installing them. Of course there are stupidos out there willing to install everything. But this should in the end be no bigger problem than Programms cluttering your Desktop with ads... I mean they also could, since you allow multiple Programs to use it also...
      Anonymous
      January 26, 2017
      V Dashboard manages it somehow.
      SkyeEyks
      New Participant
      June 6, 2022

      Ping!

      Has there been any updates on this?

        Terms & ConditionsAccessibility statement