Multisampling (OpenGL, OculusSDK3.0.1)

Anyone?

I'm researching how best to do multi-sampling in order to improve image quality in the final rendered scene. I suspect, but I'm not certain, that you'd want multi-sampling at the point of distortion and nowhere else. In theory this would mean specifying a specific level of multi-sampling in the ovrGLConfig structure, and then forgetting about it.

However, I haven't had much time to devote to this topic as yet, and the documentation I can find on OpenGL multisampling is a confused contradictory mess that makes it appear as if MS is fairly GL version dependent. Further, as far as I can tell, right now the multi-sampling functionality in ovrGLConfig has absolutely zero impact on the scene. I don't believe it's ever read by the SDK other than to copy it around to various structures. it doesn't seem to be used in any of the OpenGL calls or in the shader.

So, yeah at some point it will be covered in my book, but I don't have any guidance for you now. Sorry.

Is the current example that we looked at incompatible with multisampling now? I get a black screen when enabling rift support on the project I am attempting to integrate the Oculus Rift in but not with the example project we worked on together.

@ckoeber: It wouldn't call it incompatible, it's just that nothing seems to be happening. Then again, this is all without a multisampling enabled FBO. Fiddling with the multisampled FBO always leaves me in pure darkness.

// GLFWOculusRiftTest
// (c) cThrough 2014 (Daniel Dekkers)
// Version 2014051300

#include <Windows.h>
#include <GL/glew.h>
#define GLFW_EXPOSE_NATIVE_WIN32
#define GLFW_EXPOSE_NATIVE_WGL
#include <GLFW/glfw3.h>
#include <GLFW/glfw3native.h>
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <OVR_CAPI.h>
#include <OVR_CAPI_GL.h>
#include <OVR.h>

const bool l_FullScreen = false;
const bool l_MultiSampling = false;

ovrHmd l_Hmd;
ovrHmdDesc l_HmdDesc;
ovrEyeDesc l_Eyes[2];
ovrGLConfig l_Cfg;

GLfloat l_VAPoints[] =
{ 
	 0.5f, 0.5f, 0.5f,
	-0.5f, 0.5f, 0.5f,
	-0.5f,-0.5f, 0.5f,
	 0.5f,-0.5f, 0.5f,
	-0.5f,-0.5f,-0.5f,
	-0.5f, 0.5f,-0.5f,
	 0.5f, 0.5f,-0.5f,
	 0.5f,-0.5f,-0.5f,
	 0.5f, 0.5f, 0.5f,
	 0.5f, 0.5f,-0.5f,
	-0.5f, 0.5f,-0.5f,
	-0.5f, 0.5f, 0.5f,
	-0.5f,-0.5f,-0.5f,
	 0.5f,-0.5f,-0.5f,
	 0.5f,-0.5f, 0.5f,
	-0.5f,-0.5f, 0.5f,
	 0.5f, 0.5f, 0.5f,
	 0.5f,-0.5f, 0.5f,
	 0.5f,-0.5f,-0.5f,
	 0.5f, 0.5f,-0.5f,
	-0.5f,-0.5f,-0.5f,
	-0.5f,-0.5f, 0.5f,
	-0.5f, 0.5f, 0.5f,
	-0.5f, 0.5f,-0.5f
};

GLfloat l_VANormals[] =
{
	0.0f, 0.0f, 1.0f,
	0.0f, 0.0f, 1.0f,
	0.0f, 0.0f, 1.0f,
	0.0f, 0.0f, 1.0f,
	0.0f, 0.0f,-1.0f,
	0.0f, 0.0f,-1.0f,
	0.0f, 0.0f,-1.0f,
	0.0f, 0.0f,-1.0f,
	0.0f, 1.0f, 0.0f,
	0.0f, 1.0f, 0.0f,
	0.0f, 1.0f, 0.0f,
	0.0f, 1.0f, 0.0f,
	0.0f,-1.0f, 0.0f,
	0.0f,-1.0f, 0.0f,
	0.0f,-1.0f, 0.0f,
	0.0f,-1.0f, 0.0f,
	1.0f, 0.0f, 0.0f,
	1.0f, 0.0f, 0.0f,
	1.0f, 0.0f, 0.0f,
	1.0f, 0.0f, 0.0f,
   -1.0f, 0.0f, 0.0f,
   -1.0f, 0.0f, 0.0f,
   -1.0f, 0.0f, 0.0f,
   -1.0f, 0.0f, 0.0f
};

GLuint l_VAIndici[] =
{
	0, 1, 2, 3,
	4, 5, 6, 7,
	8, 9, 10, 11,
	12, 13, 14, 15,
	16, 17, 18, 19,
	20, 21, 22, 23
};

// =============================================================================

static void ErrorCallback(int p_Error, const char* p_Description)
{
	fputs(p_Description, stderr);
}

// =============================================================================

static void KeyCallback(GLFWwindow* p_Window, int p_Key, int p_Scancode, int p_Action, int p_Mods)
{
	if (p_Key == GLFW_KEY_ESCAPE && p_Action == GLFW_PRESS) 
		glfwSetWindowShouldClose(p_Window, GL_TRUE);
}

// =============================================================================

static void WindowSizeCallback(GLFWwindow* p_Window, int p_Width, int p_Height)
{
	l_Cfg.OGL.Header.RTSize.w = p_Width; 
	l_Cfg.OGL.Header.RTSize.h = p_Height;

	int l_RenderCaps = 0;
	int l_DistortionCaps = ovrDistortion_Chromatic | ovrDistortion_TimeWarp;
	ovrEyeRenderDesc l_EyeRenderDesc[2];
	ovrHmd_ConfigureRendering(l_Hmd, &l_Cfg.Config, l_RenderCaps, l_DistortionCaps, l_Eyes, l_EyeRenderDesc);
}

// ============================================================================

void RenderCubeVertexArrays(void)
{
	glEnableClientState(GL_VERTEX_ARRAY);
	glVertexPointer(3, GL_FLOAT, 0, l_VAPoints);
	glEnableClientState(GL_NORMAL_ARRAY);
	glNormalPointer(GL_FLOAT, 0, l_VANormals);
	glDrawElements(GL_QUADS, 6*4, GL_UNSIGNED_INT, l_VAIndici);
	glDisableClientState(GL_NORMAL_ARRAY);
	glDisableClientState(GL_VERTEX_ARRAY);
}

// ============================================================================

void RenderCubeFixedFunction(void)
{
	// Obsolete, remains as a fall back for the vertex arrays version...
	glBegin(GL_QUADS);
		glNormal3f( 0.0f, 0.0f, 1.0f);
		glVertex3f( 0.5f, 0.5f, 0.5f);
		glVertex3f(-0.5f, 0.5f, 0.5f);
		glVertex3f(-0.5f,-0.5f, 0.5f);
		glVertex3f( 0.5f,-0.5f, 0.5f);
	glEnd();

	glBegin(GL_QUADS);
		glNormal3f( 0.0f, 0.0f,-1.0f);
		glVertex3f(-0.5f,-0.5f,-0.5f);
		glVertex3f(-0.5f, 0.5f,-0.5f);
		glVertex3f( 0.5f, 0.5f,-0.5f);
		glVertex3f( 0.5f,-0.5f,-0.5f);
	glEnd();

	glBegin(GL_QUADS);
		glNormal3f( 0.0f, 1.0f, 0.0f);
		glVertex3f( 0.5f, 0.5f, 0.5f);
		glVertex3f( 0.5f, 0.5f,-0.5f);
		glVertex3f(-0.5f, 0.5f,-0.5f);
		glVertex3f(-0.5f, 0.5f, 0.5f);
	glEnd();

	glBegin(GL_QUADS);
		glNormal3f( 0.0f,-1.0f, 0.0f);
		glVertex3f(-0.5f,-0.5f,-0.5f);
		glVertex3f( 0.5f,-0.5f,-0.5f);
		glVertex3f( 0.5f,-0.5f, 0.5f);
		glVertex3f(-0.5f,-0.5f, 0.5f);
	glEnd();

	glBegin(GL_QUADS);
		glNormal3f( 1.0f, 0.0f, 0.0f);
		glVertex3f( 0.5f, 0.5f, 0.5f);
		glVertex3f( 0.5f,-0.5f, 0.5f);
		glVertex3f( 0.5f,-0.5f,-0.5f);
		glVertex3f( 0.5f, 0.5f,-0.5f);
	glEnd();

	glBegin(GL_QUADS);
		glNormal3f(-1.0f, 0.0f, 0.0f);
		glVertex3f(-0.5f,-0.5f,-0.5f);
		glVertex3f(-0.5f,-0.5f, 0.5f);
		glVertex3f(-0.5f, 0.5f, 0.5f);
		glVertex3f(-0.5f, 0.5f,-0.5f);
	glEnd();
}

// ============================================================================

static void SetOpenGLState(void)
{
	// Some state...
	glEnable(GL_CULL_FACE);
	glEnable(GL_LIGHTING);
	glDisable(GL_TEXTURE_2D);
	glEnable(GL_DEPTH_TEST);
	glShadeModel(GL_SMOOTH);
	glEnable(GL_BLEND);
	glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
	if (l_MultiSampling) glEnable(GL_MULTISAMPLE);
        
	// Some (stationary) lights...
	GLfloat l_Light0Position[] = { 5.0f, 6.0f, 3.0f, 0.0f };
	GLfloat l_Light0Diffuse[] = { 1.0f, 0.8f, 0.6f, 1.0f };
	glLightfv(GL_LIGHT0, GL_POSITION, l_Light0Position);
	glLightfv(GL_LIGHT0, GL_DIFFUSE, l_Light0Diffuse);
	glEnable(GL_LIGHT0);
    
	GLfloat l_Light1Position[] = { -5.0f, -6.0f, 5.0f, 0.0f };
	GLfloat l_Light1Diffuse[] = { 0.6f, 0.8f, 1.0f, 1.0f };
	glLightfv(GL_LIGHT1, GL_POSITION, l_Light1Position);
	glLightfv(GL_LIGHT1, GL_DIFFUSE, l_Light1Diffuse);
	glEnable(GL_LIGHT1);
    
	// Material...
	GLfloat l_MaterialSpecular[] = { 0.3f, 0.3f, 0.3f, 1.0f };
	GLfloat l_MaterialShininess[] = { 10.0f };
	glMaterialfv(GL_FRONT, GL_SPECULAR, l_MaterialSpecular);
	glMaterialfv(GL_FRONT, GL_SHININESS, l_MaterialShininess);
}

// =============================================================================

int main(void)
{
	// Initialize LibOVR...
	ovr_Initialize();

	l_Hmd = ovrHmd_Create(0);
	if (!l_Hmd) l_Hmd = ovrHmd_CreateDebug(ovrHmd_DK1);

	ovrHmd_GetDesc(l_Hmd, &l_HmdDesc);

	ovrHmd_StartSensor(l_Hmd, ovrHmdCap_Orientation, 0);

	GLFWwindow* l_Window;

	glfwSetErrorCallback(ErrorCallback);

	if (!glfwInit()) exit(EXIT_FAILURE);

	if (l_MultiSampling) glfwWindowHint(GLFW_SAMPLES, 4); else glfwWindowHint(GLFW_SAMPLES, 0);

	ovrSizei l_ClientSize;
	if (l_FullScreen)
	{
		l_ClientSize.w = l_HmdDesc.Resolution.w; // 1280 for DK1...
		l_ClientSize.h = l_HmdDesc.Resolution.h; // 800 for DK1...
		// Create a fullscreen window with the Oculus Rift resolution...
		l_Window = glfwCreateWindow(l_ClientSize.w, l_ClientSize.h, "GLFW Oculus Rift Test", glfwGetPrimaryMonitor(), NULL);
	}
	else
	{
		l_ClientSize.w = 640;
		l_ClientSize.h = 480;
		l_Window = glfwCreateWindow(l_ClientSize.w, l_ClientSize.h, "GLFW Oculus Rift Test", NULL, NULL);
	}

	if (!l_Window)
	{
		glfwTerminate();
		exit(EXIT_FAILURE);
	}

	// Make the context current for this window...
	glfwMakeContextCurrent(l_Window);

	// Don't forget to initialize Glew...
	GLenum l_Result = glewInit();
	if (l_Result!=GLEW_OK)	
	{
		printf("glewInit() error.\n");
		exit(EXIT_FAILURE);
	}

	// Print some info about the OpenGL version we are using...
	int l_Major = glfwGetWindowAttrib(l_Window, GLFW_CONTEXT_VERSION_MAJOR);
	int l_Minor = glfwGetWindowAttrib(l_Window, GLFW_CONTEXT_VERSION_MINOR);
	int l_Profile = glfwGetWindowAttrib(l_Window, GLFW_OPENGL_PROFILE);
	printf("OpenGL: %d.%d ", l_Major, l_Minor);
	if (l_Profile==GLFW_OPENGL_COMPAT_PROFILE) printf("GLFW_OPENGL_COMPAT_PROFILE\n"); else printf("GLFW_OPENGL_CORE_PROFILE\n");
	printf("Vendor: %s\n", (char*)glGetString(GL_VENDOR));
	printf("Renderer: %s\n", (char*)glGetString(GL_RENDERER));

	// Create some lights, materials, etc...
	SetOpenGLState();

	// We will do some offscreen rendering, setup FBO...
	ovrSizei l_TextureSizeLeft = ovrHmd_GetFovTextureSize(l_Hmd, ovrEye_Left, l_HmdDesc.DefaultEyeFov[0], 1.0f);
	ovrSizei l_TextureSizeRight = ovrHmd_GetFovTextureSize(l_Hmd, ovrEye_Right, l_HmdDesc.DefaultEyeFov[1], 1.0f);
	ovrSizei l_TextureSize;
	l_TextureSize.w = l_TextureSizeLeft.w + l_TextureSizeRight.w;
	l_TextureSize.h = (l_TextureSizeLeft.h>l_TextureSizeRight.h ? l_TextureSizeLeft.h : l_TextureSizeRight.h);

	// Create FBO...
	GLuint l_FBOId;
	glGenFramebuffers(1, &l_FBOId);
	glBindFramebuffer(GL_FRAMEBUFFER, l_FBOId);

	// The texture we're going to render to...
	GLuint l_TextureId;
	glGenTextures(1, &l_TextureId);
	// "Bind" the newly created texture : all future texture functions will modify this texture...
	glBindTexture(GL_TEXTURE_2D, l_TextureId);
	// Give an empty image to OpenGL (the last "0")
	glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, l_TextureSize.w, l_TextureSize.h, 0, GL_RGBA, GL_UNSIGNED_BYTE, 0);
	// Linear filtering...
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);

	// Create Depth Buffer...
	GLuint l_DepthBufferId;
	glGenRenderbuffers(1, &l_DepthBufferId);
	glBindRenderbuffer(GL_RENDERBUFFER, l_DepthBufferId);
	glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT, l_TextureSize.w, l_TextureSize.h);
	glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, l_DepthBufferId);

	// Set the texture as our colour attachment #0...
	glFramebufferTexture(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, l_TextureId, 0);

	// Set the list of draw buffers...
	GLenum l_GLDrawBuffers[1] = { GL_COLOR_ATTACHMENT0 };
	glDrawBuffers(1, l_GLDrawBuffers); // "1" is the size of DrawBuffers

	// Check if everything is OK...
	GLenum l_Check = glCheckFramebufferStatus(GL_DRAW_FRAMEBUFFER);
	if (l_Check!=GL_FRAMEBUFFER_COMPLETE)
	{
		printf("There is a problem with the FBO.\n");
		exit(EXIT_FAILURE);
	}

	// Unbind...
	glBindRenderbuffer(GL_RENDERBUFFER, 0);
	glBindTexture(GL_TEXTURE_2D, 0);
	glBindFramebuffer(GL_FRAMEBUFFER, 0);

	// Oculus Rift eye configurations...
	l_Eyes[0].Eye = ovrEye_Left;
	l_Eyes[1].Eye = ovrEye_Right;
	l_Eyes[0].Fov = l_HmdDesc.DefaultEyeFov[0];
	l_Eyes[1].Fov = l_HmdDesc.DefaultEyeFov[1];
	l_Eyes[0].TextureSize.w = l_TextureSize.w;
	l_Eyes[0].TextureSize.h = l_TextureSize.h;
	l_Eyes[1].TextureSize.w = l_TextureSize.w;
	l_Eyes[1].TextureSize.h = l_TextureSize.h;
	l_Eyes[0].RenderViewport.Pos.x = 0;
	l_Eyes[0].RenderViewport.Pos.y = 0;
	l_Eyes[1].RenderViewport.Pos.x = (l_TextureSize.w+1)/2;
	l_Eyes[1].RenderViewport.Pos.y = 0;
	l_Eyes[0].RenderViewport.Size.w = l_TextureSize.w/2;
	l_Eyes[0].RenderViewport.Size.h = l_TextureSize.h;
	l_Eyes[1].RenderViewport.Size.w = l_Eyes[0].RenderViewport.Size.w;
	l_Eyes[1].RenderViewport.Size.h = l_Eyes[0].RenderViewport.Size.h;

	l_Cfg.OGL.Header.API = ovrRenderAPI_OpenGL;
	l_Cfg.OGL.Header.Multisample = (l_MultiSampling ? 1 : 0);
	l_Cfg.OGL.Header.RTSize.w = l_ClientSize.w; 
	l_Cfg.OGL.Header.RTSize.h = l_ClientSize.h;
	l_Cfg.OGL.WglContext = glfwGetWGLContext(l_Window);
	l_Cfg.OGL.Window = glfwGetWin32Window(l_Window);
	l_Cfg.OGL.GdiDc = GetDC(l_Cfg.OGL.Window);

	int l_RenderCaps = 0;
	int l_DistortionCaps = ovrDistortion_Chromatic | ovrDistortion_TimeWarp;
	ovrEyeRenderDesc l_EyeRenderDesc[2];
	ovrHmd_ConfigureRendering(l_Hmd, &l_Cfg.Config, l_RenderCaps, l_DistortionCaps, l_Eyes, l_EyeRenderDesc);

	ovrGLTexture l_EyeTexture[2];
	l_EyeTexture[0].OGL.Header.API = ovrRenderAPI_OpenGL;
	l_EyeTexture[0].OGL.Header.TextureSize.w = l_TextureSize.w;
	l_EyeTexture[0].OGL.Header.TextureSize.h = l_TextureSize.h;
	l_EyeTexture[0].OGL.Header.RenderViewport = l_Eyes[0].RenderViewport;
	l_EyeTexture[0].OGL.TexId = l_TextureId;

	// Right eye uses the same texture, but a different rendering viewport...
	l_EyeTexture[1] = l_EyeTexture[0];
	l_EyeTexture[1].OGL.Header.RenderViewport = l_Eyes[1].RenderViewport;    

	glfwSetKeyCallback(l_Window, KeyCallback);
	glfwSetWindowSizeCallback(l_Window, WindowSizeCallback);

	GLfloat l_SpinX;
	GLfloat l_SpinY;

	while (!glfwWindowShouldClose(l_Window))
	{
		l_SpinX = 30.0f; // (GLfloat) fmod(glfwGetTime()*17.0, 360.0);
		l_SpinY = 40.0f; // (GLfloat) fmod(glfwGetTime()*23.0, 360.0);

		ovrFrameTiming m_HmdFrameTiming = ovrHmd_BeginFrame(l_Hmd, 0);

		// Bind the FBO...
		glBindFramebuffer(GL_FRAMEBUFFER, l_FBOId);
		// Clear...
		glClearColor(0.2f, 0.3f, 0.4f, 1.0f);
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

		for (int l_EyeIndex=0; l_EyeIndex<ovrEye_Count; l_EyeIndex++)
		{
			ovrEyeType l_Eye = l_HmdDesc.EyeRenderOrder[l_EyeIndex];
			ovrPosef l_EyePose = ovrHmd_BeginEyeRender(l_Hmd, l_Eye);

			glViewport(l_EyeRenderDesc[l_Eye].Desc.RenderViewport.Pos.x,	// StartX
					   l_EyeRenderDesc[l_Eye].Desc.RenderViewport.Pos.y,	// StartY
					   l_EyeRenderDesc[l_Eye].Desc.RenderViewport.Size.w,	// Width
					   l_EyeRenderDesc[l_Eye].Desc.RenderViewport.Size.h	// Height
					  );

			// Get Projection and ModelView matrici from the device...
			OVR::Matrix4f l_ProjectionMatrix = ovrMatrix4f_Projection(
				l_EyeRenderDesc[l_Eye].Desc.Fov, 0.3f, 100.0f, true);
			OVR::Quatf l_Orientation = OVR::Quatf(l_EyePose.Orientation);
			OVR::Matrix4f l_ModelViewMatrix = OVR::Matrix4f(l_Orientation.Inverted());

			// Pass matrici on to OpenGL...
			glMatrixMode(GL_PROJECTION);
			glLoadIdentity();
			glMultMatrixf(&(l_ProjectionMatrix.Transposed().M[0][0]));
			glMatrixMode(GL_MODELVIEW);
			glLoadIdentity();
			// Translate for specific eye based on IPD...
			glTranslatef(l_EyeRenderDesc[l_Eye].ViewAdjust.x,
						 l_EyeRenderDesc[l_Eye].ViewAdjust.y,
						 l_EyeRenderDesc[l_Eye].ViewAdjust.z);
			// Multiply with orientation retrieved from sensor...
			glMultMatrixf(&(l_ModelViewMatrix.Transposed().M[0][0]));
			// Move back a bit to show scene in front of us...
			glTranslatef(0.0f, 0.0f, -2.0f);
			// Make the cube spin...
			glRotatef(l_SpinX, 1.0f, 0.0f, 0.0f);
			glRotatef(l_SpinY, 0.0f, 1.0f, 0.0f);

			// Render...
			RenderCubeFixedFunction();
			// RenderCubeVertexArrays();

			ovrHmd_EndEyeRender(l_Hmd, l_Eye, l_EyePose, &l_EyeTexture[l_Eye].Texture);
		}

		// Unbind the FBO, back to normal drawing...
		glBindFramebuffer(GL_FRAMEBUFFER, 0);

		glDisable(GL_CULL_FACE); // Oculus wants CW orientations, avoid the problem by turning of culling...
		glDisable(GL_DEPTH_TEST); // Nothing is drawn with depth test on...
		ovrHmd_EndFrame(l_Hmd);
		glBindBuffer(GL_ARRAY_BUFFER, 0); // Unbind GL_ARRAY_BUFFER for my own vertex arrays to work...
		glEnable(GL_CULL_FACE); // Turn back on...
		glEnable(GL_DEPTH_TEST); // Turn back on...
		glClearDepth(1); // Oculus set this to 0 (the near plane), return to normal...
		glUseProgramObjectARB(0); // Oculus shader is still active, turn it off...

		glfwPollEvents();
	}

	glfwDestroyWindow(l_Window);

	glfwTerminate();

	ovrHmd_Destroy(l_Hmd);
	ovr_Shutdown();

	exit(EXIT_SUCCESS);
}

@jherico: Yes. But... you would get some strange effects. The barrel distortion would favor "high density" multisampling near the edges of the screen and less at the center of the screen (where it is most needed). The original texture is cramped near the edges and stretched at the center. Maybe not stretched but at least "less cramped". I think (apart from efficiency) that the distortion shader wouldn't mind if you would pass a multisampled FBO. So you would have a better quality texture to begin with. But I'm not sure if you would really see the difference after distortion. Anyway. Good that you looked at the code to see where the Multisampling field in ovrGLConfig goes. Nowhere. ;-)

Thanks,
Daniel

Any comment from Oculus as to when we can expect the multisampling support to be enabled in the SDK for OpenGL?

Since you were struggling with setting up a multisampled FBO I can give a quick example. It's Scala code but it should trivial to map to other languages. I'm definitely not sure if this is the only/right way to do it but, at least, it works for me. If I understand it correctly, you actually need two FBOs in order to setup a multisampled FBO. The first FBO is the one you activate (bind) before you draw your scene. After drawing the scene, you blit the first FBO to another (regular) FBO. This resolves the "multiple samples" in the first FBO, resulting in an anti-aliased rendering. For the SDK you would specify the textureId of this second FBO in order to tell the SDK about your rendering result (so, the SDK does not know about the 1. FBO). Overall, the setup of the two FBOs looks like this:

// create 1. FBO
val framebufferId1 = glGenFramebuffers()
glBindFramebuffer(GL_FRAMEBUFFER, framebufferId1)
  
// gen & bind texture (for colors)
val textureId1 = glGenTextures()
glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, textureId1)
// Allocate space for the texture
glTexImage2DMultisample(GL_TEXTURE_2D_MULTISAMPLE, numSamples, GL_RGBA, fboW, fboH, false)
  
// gen & bind renderbuffer (for depth)
val depthBufferId = glGenRenderbuffers()
glBindRenderbuffer(GL_RENDERBUFFER, depthBufferId)
glRenderbufferStorageMultisample(GL_RENDERBUFFER, numSamples, GL_DEPTH_COMPONENT, fboW, fboH)
  
// attach texture to FBO
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL32.GL_TEXTURE_2D_MULTISAMPLE, textureId1, 0)
glDrawBuffers(GL_COLOR_ATTACHMENT0)

// attach renderbuffer to FBO
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthBufferId)
  
// unbind, surprisingly this seems to be necessary (otherwise errors in glBlitFramebuffer)
glBindFramebuffer(GL_FRAMEBUFFER, 0)
  
// create 2. FBO
val framebufferId2 = glGenFramebuffers()
glBindFramebuffer(GL_FRAMEBUFFER, framebufferId2)
  
// gen & bind texture (for colors)
val textureId2 = glGenTextures()
glBindTexture(GL_TEXTURE_2D, textureId2)
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR) 
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR) // GL_NEAREST, GL_LINEAR, GL_LINEAR_MIPMAP_LINEAR
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE)
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE)
// Allocate space for the texture
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, fboW, fboH, 0, GL_RGBA, GL_UNSIGNED_BYTE, null.asInstanceOf[java.nio.ByteBuffer])
  
// attach texture to FBO
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureId2, 0)
  
// unbind
glBindFramebuffer(GL_FRAMEBUFFER, 0)

Note that you have to use multisampling versions glTexImage2DMultisample and also glRenderbufferStorageMultisample for the render buffer to initialize the storage. And for the 1. FBO you don't need to specify the texture filtering, since it is never used as a texture. I'm currently experimenting with "trilinear" filtering, so I use GL_LINEAR_MIPMAP_LINEAR for the 2. FBO. If you do not want to generate mipmaps you may want to switch to GL_LINEAR. Now the code for activating your FBO before rendering would look like this:

// before rendering
glBindFramebuffer(GL_FRAMEBUFFER, framebufferId1)
glViewport(0, 0, fboW, fboH)
glEnable(GL_MULTISAMPLE)

After rendering we have to blit the two FBOs:

// maybe it is better to unbind the active FBO first
glBindFramebuffer(GL_FRAMEBUFFER, 0)

// then set the multisampled FBO as input
glBindFramebuffer(GL_READ_FRAMEBUFFER, framebufferId1)
glReadBuffer(GL_COLOR_ATTACHMENT0)

// set the regular FBO as output
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, framebufferId2)
glDrawBuffer(GL_COLOR_ATTACHMENT0)

// and blit; this will "resolve" the multisampling resulting in an anti-aliased rendering
glBlitFramebuffer(0, 0, fboW, fboH, 0, 0, fboW, fboH, GL_COLOR_BUFFER_BIT, GL_NEAREST)

// maybe for clean-up we should unbind the draw/read FBOs
glBindFramebuffer(GL_READ_FRAMEBUFFER, 0)
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0)

// if you want to use mipmaps for your FBO texture:    
if (generateMipMap) {
  glBindTexture(GL_TEXTURE_2D, textureId2)
  glEnable(GL_TEXTURE_2D) // maybe not needed?
  glGenerateMipmap(GL_TEXTURE_2D)
  glBindTexture(GL_TEXTURE_2D, 0)
}    

Hope that helps. Overall, I'm not really satisfied with anti-aliasing yet. Using MSAA in the FBO with a pixel-density-factor (I'm referring to the parameter that is used in the SDK call to determing the FBO size) of 1.0 adds too much blur imho, because now MSAA and the texture filtering both lead to an interpolation. With a pixel-density of 2.0 and above, the MSAA in the FBO makes sense and this is currently the best image quality I can achieve. I was hoping that with a really large pixel-density of like 4.0 and mipmapping enabled, I would get an even better result. But so far I can not really confirm that mipmapping improves the anti-aliasing quality a lot.

Btw: The topic has also been discussed in this thread. So far, I have no idea what the multisampling flag that we can pass to the SDK actually does. Since multisampling in the FBO is in fact something that has to be done on our side, I'm not really sure how this is supposed to work.

Nice example code. :)

I would hope the intention of any multisampling support in the SDK is to handle this additional FBO blitting stage for us.

"StellaArtois" wrote:
Nice example code. :)

I would hope the intention of any multisampling support in the SDK is to handle this additional FBO blitting stage for us.

Unlikely since the SDK currently doesn't manage the FBO creation at all. They've said that the DK2 SDK API won't change much which leads me to believe that the division of labor in OpenGL / Direct3D management isn't likely to change. I do wish they'd allow us to provide a callback mechanism for buffer swapping rather than requiring platform specific window details though.

"jherico" wrote:

Unlikely since the SDK currently doesn't manage the FBO creation at all. They've said that the DK2 SDK API won't change much which leads me to believe that the division of labor in OpenGL / Direct3D management isn't likely to change.

Something that would be possible without changing the API is that we just tell the SDK whether our FBO is a GL_TEXTURE_2D or a GL_TEXTURE_2D_MULTISAMPLE. Oh, maybe that is just the idea behind having this flag that we can pass to the SDK... :)

On client side, this would simplify things because you can just setup a single FBO that uses either GL_TEXTURE_2D/glTexImage2D/glRenderbufferStorage or GL_TEXTURE_2D_MULTISAMPLE/glTexImage2DMultisample/glRenderbufferStorageMultisample. In this case the SDK could setup the second FBO internally and perform the blitting. Alternatively, the SDK distortion rendering could also avoid the internal blitting step by a modified fragment shader that uses a sampler2DMS and texelFetch to directly operate on the multi-sampled texture. Which again makes me wonder: The raw multisampled texels actually provide more information compared with texels of the blitted/resolved texture. Somehow there should be a way to exploit this to improve the final AA quality, but I simply don't know how.