Up and Coming

OpenSceneGraph

54 replies
19067 views

I've started working on an integration with OpenSceneGraph (http://www.openscenegraph.org/) and although I don't have a Rift to test with yet what I've done looks pretty close to the Oculus World demo so I'm hoping I've done it right. If there are any OSG developers out there who could test out my code I'd really appreciate it. I found GLSL versions of the shaders in the OpenHMD project (http://openhmd.net/) which are referenced as vertexShaderSource and fragmentShaderSource. I based my work on the night vision effect example from the OpenSceneGraph Cookbook (http://www.packtpub.com/openscenegrap-3-for-advanced-3d-programming-using-api-cookbook/book) and my code uses the functions from that example to create the RTT & HUD cameras as well as the quads. The source is available at https://github.com/xarray/osgRecipes. I have pseudocode where the scene is added to the cameras (step five in the cookbook) and where the cameras are added to the root node (step 7 in the cookbook) because I'm also integrating with a derived type of renderer but you should be able to replace them with what's in the example.



   // Because I don't have a Rift yet I just made a OVR::HMDInfo and OVR::Util::Render::StereoConfig and guessed at some values.
      myHMDInfo.HResolution = 1280;
      myHMDInfo.VResolution = 800;
      myHMDInfo.HScreenSize = 0.14976;
      myHMDInfo.VScreenSize = 0.0935;
      myHMDInfo.VScreenCenter = myHMDInfo.VScreenSize / 2;
      myHMDInfo.EyeToScreenDistance = 0.05; //????
      myHMDInfo.LensSeparationDistance = 0.03; //????
      myHMDInfo.InterpupillaryDistance = 0.025; //?????????

      myStereoConfig.SetHMDInfo(myHMDInfo);


      const int textureWidth = myHMDInfo.HResolution / 2;
      const int textureHeight = myHMDInfo.VResolution;

      // Setup textures for the RTT cameras
      osg::ref_ptr<osg::Texture2D> leftEyeTex2D = new osg::Texture2D;
      leftEyeTex2D->setTextureSize( textureWidth, textureHeight );
      leftEyeTex2D->setInternalFormat( GL_RGBA );
      osg::ref_ptr<osg::Texture2D> rightEyeTex2D = new osg::Texture2D;
      rightEyeTex2D->setTextureSize( textureWidth, textureHeight );
      rightEyeTex2D->setInternalFormat( GL_RGBA );
     
      // Initialize RTT cameras for each eye
      osg::ref_ptr<osg::Camera> leftEyeRTTCamera = osgCookBook::createRTTCamera(
         osg::Camera::COLOR_BUFFER, leftEyeTex2D.get());
      OVR::Util::Render::StereoEyeParams leftEyeParams = 
         myStereoConfig.GetEyeRenderParams(OVR::Util::Render::StereoEye_Left);
      osg::Matrixf leftEyeTranslation = osg::Matrixf::translate(
         leftEyeParams.ViewAdjust.M[0][3], leftEyeParams.ViewAdjust.M[2][3], 
         leftEyeParams.ViewAdjust.M[2][3]);
      leftEyeRTTCamera->setViewMatrix(leftEyeTranslation);
      leftEyeRTTCamera->setProjectionMatrixAsPerspective(
         myStereoConfig.GetYFOVDegrees(), myStereoConfig.GetAspect(), 0.3, 5000);
      osg::ref_ptr<osg::Camera> rightEyeRTTCamera = osgCookBook::createRTTCamera(
         osg::Camera::COLOR_BUFFER, rightEyeTex2D.get());
      OVR::Util::Render::StereoEyeParams rightEyeParams = 
         myStereoConfig.GetEyeRenderParams(OVR::Util::Render::StereoEye_Right);
      osg::Matrixf rightEyeTranslation = osg::Matrixf::translate(
         rightEyeParams.ViewAdjust.M[0][3], rightEyeParams.ViewAdjust.M[2][3], 
         rightEyeParams.ViewAdjust.M[2][3]);
      rightEyeRTTCamera->setViewMatrix(rightEyeTranslation);
      rightEyeRTTCamera->setProjectionMatrixAsPerspective(
         myStereoConfig.GetYFOVDegrees(), myStereoConfig.GetAspect(), 0.3, 5000);

      // add your scene to the camera, should be something like:
      // leftEyeRTTCamera->addChild( scene.get() );
      // rightEyeRTTCamera->addChild( scene.get() );
	  
      // Create HUD cameras for each eye
      osg::ref_ptr<osg::Camera> leftEyeHUDCamera =
         osgCookBook::createHUDCamera(0.0, 1.0, 0.0, 1.0);
      osg::ref_ptr<osg::Camera> rightEyeHUDCamera =
         osgCookBook::createHUDCamera(0.0, 1.0, 0.0, 1.0);

      // Create quads on each camera
      leftEyeHUDCamera->addChild( osgCookBook::createScreenQuad(
         1.0f, 1.0f) );
      leftEyeHUDCamera->setViewport( 
         new osg::Viewport(0, 0, myHMDInfo.HResolution / 2, myHMDInfo.VResolution));
      rightEyeHUDCamera->addChild( osgCookBook::createScreenQuad(
         1.0f, 1.0f) );
      rightEyeHUDCamera->setViewport(
         new osg::Viewport(myHMDInfo.HResolution / 2, 0, myHMDInfo.HResolution / 2, myHMDInfo.VResolution));

      // Set up shaders (from the OpenHMD project)
      osg::ref_ptr<osg::Program> program = new osg::Program;
      program->addShader( new osg::Shader(osg::Shader::VERTEX,
         vertexShaderSource) );
      program->addShader( new osg::Shader(osg::Shader::FRAGMENT,
         fragmentShaderSource) );

      // Configure state sets for both eyes
      osg::StateSet* leftEyeStateSet = leftEyeHUDCamera->getOrCreateStateSet();
      leftEyeStateSet->setTextureAttributeAndModes( 0, leftEyeTex2D.get() );
      leftEyeStateSet->setAttributeAndModes( program.get() );
      leftEyeStateSet->addUniform( new osg::Uniform("warpTexture", 0) );

      osg::StateSet* rightEyeStateSet = rightEyeHUDCamera->getOrCreateStateSet();
      rightEyeStateSet->setTextureAttributeAndModes( 0, rightEyeTex2D.get() );
      rightEyeStateSet->setAttributeAndModes( program.get() );
      rightEyeStateSet->addUniform( new osg::Uniform("warpTexture", 0) );

      
      // Add everything to the root, should be something like:
      //osg::ref_ptr<osg::Group> root = ?;
      //root->addChild( leftEyeRTTCamera.get().get() );
      //root->addChild( leftEyeHUDCamera.get().get() );
      //root->addChild( rightEyeRTTCamera.get().get() );
      //root->addChild( rightEyeHUDCamera.get().get() );
      //root->addChild( hudCamera.get() );
      //root->addChild( scene.get() );

Show previous replies

V

Vrally

Up and Coming

Ok,

I got past that issue. But now I am back at the catch 22 moment:

In what order do you create your RTT cameras in respect to where you create your osg::Texture objects?

Is all this happening inside your initialization function, which you call from your subclassed osg::Drawable?

One other question: Are you using one or two render textures?

C

ConstellationAuthor

Up and Coming

I create my textures immediately before I create the RTT cameras. I do the allocateMipmapLevels trick after the cameras are created. All this is happening from the drawable. I'm using two render textures but unlike some of the OVR sample code I'm not doing two separate render calls since they're both slaved to the same master view.

V

Vrally

Up and Coming

Hi again,

I still have not got anything working but now I feel that I am close. I have my drawable with the drawImplementation() function implemented to:
1. InitializeTextures
2. SetupSlaveCameras
3. AllocateMipmapLevels and apply on textures
4. ConfigureRendering
5. AttachToWindow

But something is still missing. Because I hit an assert in:

OVR::CAPI::GL::GLVersionAndExtensions::ParseGLVersion() Line 582

The first time this function is called is from inside ovrHmd_ConfigureRendering() function. And then I have a valid GL version, i.e. no break for assert.

But later when the first ovrHmd_EndFrame() is called the version is suddenly 0 (NULL) and I get an assert break. I suspect this have something to do with my render textures.

I have subclassed the osg::Texture2D class and made a OculusTexture2D class. This class is identical to the normal texture2d class but takes a graphic context as input and have a extra function which returns a ovrTexture object.

The implementation of that function looks like this:

ovrTexture OculusTexture2D::getOvrTexture()
{
	ovrTexture tex;
	OVR::Sizei newRTSize(getTextureWidth(), getTextureHeight());

	ovrGLTextureData* texData = (ovrGLTextureData*)&tex;
	texData->Header.API = ovrRenderAPI_OpenGL;
	texData->Header.TextureSize = newRTSize;
	texData->Header.RenderViewport = m_viewPort;
	unsigned int contextID = m_context->getState()->getContextID();
	texData->TexId = getTextureObject(contextID)->id();
	return tex;
}

Although if I compare the internals of the returned object with is return from the Get_ovrTexture() function from the OculusWorldDemo they seem to contain the similar information.

V

Vrally

Up and Coming

Hmm.. I connected gDebugger to see if my context was set up correctly and if my render textures were allocated. And it seems like they are. But I guess it is about getting the order right.

A bit frustrating that the "simple" way of doing Oculus rendering proves to be the hardest to get working.

C

ConstellationAuthor

Up and Coming

I also got that ParseGLVersion error and in my case it was my context that was the problem; once I got past that I got a completely different error when my textures weren't fully initialized. I'd suggest allowing your application to draw a frame to make sure the context is fully initialized before using it. That should be what the drawable would do for you but perhaps there's something wrong there.

M

mungewell

Up and Coming

Hi,
Built this on Linux against SDK 0.4.3, couple of problems.
1). GL was not picked up by cmake, thus linking failed. Added 'GL' to hardcoded list in 'src/CMakeList.txt'
2). Composite viewer works OK, but other two show static (1st frame?) image behind flashing/updating segments all over screen area. Could be related to under-powered GFX, but previous versions were OK I believe.

Willing to try any suggestions/tests you have,
Simon.

V

Vrally

Up and Coming

"mungewell" wrote:
Hi,
Built this on Linux against SDK 0.4.3, couple of problems.
1). GL was not picked up by cmake, thus linking failed. Added 'GL' to hardcoded list in 'src/CMakeList.txt'
2). Composite viewer works OK, but other two show static (1st frame?) image behind flashing/updating segments all over screen area. Could be related to under-powered GFX, but previous versions were OK I believe.

Willing to try any suggestions/tests you have,
Simon.

Hi Simon,

1. CMake should be able to pickup your OpenGL library location with the following command (which already is present in the CMakeList.txt file):

FIND_PACKAGE( OpenGL REQUIRED )

CMake tries to find the libGL.so file. Maybe your driver installation has installed that with another name such as libGL.so.1. In that case you need to create a symbolic link from libGL.so.1 to libGL.so.

2. The flickering you see has been reported by other Linux users and I just merged a pull request that hopefully will fix your problems.

Best regards
Björn

V

Vrally

Up and Coming

I found this post regarding context management in the Oculus SDK:
viewtopic.php?f=20&t=19937

The problems that Eskil are having seems very similar to mine. But I added the call to ovr_InitializeRenderingShim() in the beginning of my application. Still without any luck. Still hit the ASSERT inside CAPI_GL_Util.cpp in the CreateShared function(). The call to wglShareLists() fail every time.

C

ConstellationAuthor

Up and Coming

I'm working on upgrading my code to the 0.6.0.0 SDK and it looks like what I'll need to do is create osg::Texture2D's from the textures allocated by ovrHmd_CreateSwapTextureSetGL. Then I can reattach each of my RTT cameras to the proper textures each frame. I think I've found a thread in the OSG forums that may be very helpful with this: http://forum.openscenegraph.org/viewtopic.php?t=14780&highlight=rebind+texture. It might be a while before I'm able to try it but I thought I'd share this in case anyone else is working on the same thing. Let me know how it goes.

V

Vrally

Up and Coming

"JeffBail" wrote:
I'm working on upgrading my code to the 0.6.0.0 SDK and it looks like what I'll need to do is create osg::Texture2D's from the textures allocated by ovrHmd_CreateSwapTextureSetGL. Then I can reattach each of my RTT cameras to the proper textures each frame. I think I've found a thread in the OSG forums that may be very helpful with this: http://forum.openscenegraph.org/viewtopic.php?t=14780&highlight=rebind+texture. It might be a while before I'm able to try it but I thought I'd share this in case anyone else is working on the same thing. Let me know how it goes.

Hi Jeff,

I have a somewhat working example using SDK v0.6.0.0. BUT, I have a lot of flickering. This is probably due to the fact that I am getting a lot of errors (I guess one per rendered frame):

Warning: detected OpenGL error 'invalid framebuffer operation' at start of State::apply()

So something about my framebuffer setup is wrong. But I have yet been unable to pinpoint which exact call that is responsible for the error.

Show more replies

Get Help

Sign up

Login with SSO

Login to the community

Login with SSO