Skip to main content
knchaffin
Up and Coming
March 20, 2019

wglDXRegisterObjectNV returns 0 but no error ( NV_DX_interop2 )

Has anyone successfully added NV_DX_interop2 functionality to a PC Win10 OVR SDK app with NVIDIA GTX 1080 GPU?  Just to be clear, my goal is to be able to execute arbitrary GL code and have it
write to the DX eye render targets, with the proper wglDXLockObjectsNV()
and wglDXUnlockObjectsNV() surrounds.


The problem I am having is that when I call

gl_handles[0] = wglDXRegisterObjectNV(gl_handleD3D, dxColorbuffer,  gl_names[0], GL_RENDERBUFFER,  WGL_ACCESS_READ_WRITE_NV);

the handle is set to 0, but there is no error detected by GetLastError();  So, I have 2 renderbuffer objects, color and depth, that are derived from the OVR eye render targets/buffers.  The dxColorbuffer object appears correct, as does the dxDepthbuffer object.  The call succeeds on the dxDepthbuffer object and sets the handle to a non-null value.

I think I am doing all the required steps correctly, including:

gl_handleD3D = wglDXOpenDeviceNVFunc((void *)((OVR::Render::D3D11::RenderDevice*)pRender)->Device); 
glGenFramebuffers(2, gl_fbonames);
glGenRenderbuffers(2, gl_names);
gl_handles[0] = wglDXRegisterObjectNV(gl_handleD3D, dxColorbuffer, gl_names[0],   GL_RENDERBUFFER,      WGL_ACCESS_READ_WRITE_NV);

... subsequent calls such as glBindFramebuffer(), etc. are present but not shown.

Thanks.












    57 replies

    Show previous replies
    knchaffinAuthor
    Up and Coming
    April 30, 2019
    @thewhiteambit
    Yeah, it's an adventure game.  I guess that is why it is so much fun ;)   Honestly I do better if I can view these obstacles and obfuscations as game challenges.  Otherwise they are just frustrations.

    I have a bare bones HLSL compute shader set up and working correctly using the DX/GL shared color texture as a RWTexture2D and compiled and initialized for Dispatch in my C++ code.  I did have to manually create an Unordered Access View for this texture since just setting  desc.BindFlags |= D3D11_BIND_UNORDERED_ACCESS is not sufficient.  I think that just sets up GPU access permissions.  I could be misunderstanding though.  The resources in the compute shader have to be associated with resources in the C++ code, and that can only be done via an appropriate "View".


    knchaffinAuthor
    Up and Coming
    May 7, 2019
    @thewhiteambit

    After only 10 days of work ;) , I have most of the interop working.  This time it was MS HLSL computer shader and C++ dispatching that were the challenges.  In the attached image, the Tuscany scene with Touch controllers are rendered as usual by DirectX and the colored triangle on the left eye view is rendered by OpenGL to a shared interop device and texture and then composited by the compute shader which combines the pre-rendered DX scene and controllers eye view render target texture and the shared DX/GL texture using two intermediate textures.  I had to use 2 intermediate textures in order to have SRV input texture reads and UAV output texture writes, preceded and followed by Context->CopyResource() calls.  In the compute shader, for now I am not using the depth textures but rather just doing a stencil overlay of the GL texture on top of the DX texture.  I hope to involve the actual depth textures in this next.  I also need to duplicate the code for the right eye path and correctly set up the view matrices in GL.

    So, four months to get to this point...  Thanks for bearing witness to the feasibility of this, which encouraged me to persevere.

    knchaffinAuthor
    Up and Coming
    May 8, 2019
    @thewhiteambit

    Hindsight is 20/20, or at least 20/19 :)
    All of my work of the past two weeks was unnecessary, including the compute shader.  All needed is:



    Per Eye:
    -Create a "proper" DX color and depth texture manually and register them with NV_DX_interop2 shared device.
    -glGenFramebuffers(2, gl_fbonames);
    -glBindFramebuffer(GL_FRAMEBUFFER, gl_fbonames[0]);
    -During the Eye view render:
         -CopyResource() from current eye render target color texture to shared DX/GL color texture,
         -CopyResource() from current eye render target depth texture to shared DX/GL depth texture,
         -glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,.., gltrextures[0], 0)
         -glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_STENCIL,..., m_gltextures[1], 0)
         -wglDXLockObjectsNV() the shared DX/GL color texture and depth texture
         -Do GL rendering to the shared and set (registered) DX/GL textures without clearing color or depth,
         -wglDXUnLockObjectsNV() the shared DX/GL color texture and depth texture
         -CopyResource() from  shared DX/GL color texture to current eye render target color texture
         -CopyResource() from  shared DX/GL depth texture to current eye render target depth texture

    In my case I'm just having GL render a rotating triangle in 3D space and the end result is the same with this simplified flow and with the much more complex compute shader and additional intermediate texture buffers.  I still have work to do to make sure the DX and GL viewports and view projections match as well as making sure the depth values agree between the DX and GL renders.and that the depth test parameters are correct on the GL side.










    knchaffinAuthor
    Up and Coming
    May 19, 2019

    Well, after several long months of effort, I have the OVR PC SDK  along with the OVR Avatar SDK working together  to render the default avatar using  an NV_DX_interop2 enabled OpenGL to  DirectX rendered windows application.  In the attached image you can see that the avatar models are rendering upside down, which I have to sort out.  For testing I have the avatar rendering with GL to the left eye render target texture via DX and the Tuscany scene and both Touch controllers rendering with DX to the right eye render target texture, just to show that both are functional ~simultaneously.  I still need to get depth buffer and color buffer blending working correctly between DX and GL so that the GL avatar can be placed in the DX 3D scene.

    As mentioned earlier, one of my layers comes from an OVRvision 3D camera for which the pan and tilt is controlled by the OVR HMD head tracking via UDP socket communications with a Max/MSP program controlling an Arduino, controlling a robotics pan and tilt turret on which the camera is mounted.  The Touch tracking is also sent out to Max/MSP and then via MIDI to my music studio network.

    My end goal is to have a VR live music performance environment.  All for fun!



    knchaffinAuthor
    Up and Coming
    May 19, 2019

    Well,
    after several long months of effort, I have the OVR PC SDK  along with
    the OVR Avatar SDK working together  to render the default avatar using 
    an NV_DX_interop2 enabled OpenGL to  DirectX rendered windows
    application.  In the attached image you can see that the avatar models
    are rendering upside down, which I have to sort out.  For testing I have
    the avatar rendering with GL to the left eye render target texture via
    DX and the Tuscany scene and both Touch controllers rendering with DX to
    the right eye render target texture, just to show that both are
    functional ~simultaneously.  I still need to get depth buffer and color
    buffer blending working correctly between DX and GL so that the GL
    avatar can be placed in the DX 3D scene.

    As
    mentioned earlier, one of my layers comes from an OVRvision 3D camera
    for which the pan and tilt is controlled by the OVR HMD head tracking
    via UDP socket communications with a Max/MSP program controlling an
    Arduino, controlling a robotics pan and tilt turret on which the camera
    is mounted.  The Touch tracking is also sent out to Max/MSP and then via
    MIDI to my music studio network.

    My end goal is to have a VR live music performance environment.  All for fun!

    Well, this will not Post with my image attached.  Image is in next message post.
    knchaffinAuthor
    Up and Coming
    May 19, 2019

    knchaffinAuthor
    Up and Coming
    May 21, 2019
    Here I am with the culmination of several months of work. In this image, I have my stereoscopic video camera capturing a live 3D room of me and my music studio. Over the top of that is a realtime 3D avatar that is synchronized to my head and hands movements and orientation. I do not have the VR headset on as it is too difficult to capture a screen with it on. In this image my first person avatar virtual hand is reaching into the scene and shaking hands with my 3rd person avatar virtual hand. So, the Cyberpunken Virtual Trio is live!

    Terms & ConditionsAccessibility statement