FBO Picking in Oculus Applications

I am writing an Oculus Rift application, I am rendering a very-high-poly mesh that I wish to be able to perform picking on using the Oculus Touch. Ideally, I want to be able to get the "triangle" id and other information attached to it.

Currently, I am using an OBB Tree and ray casting to perform picking on the CPU, It works perfectly, the problem is that even with OBB tree, the picking process is slow. I thought I'd perform picking on the GPU by rendering the view (from the point of view of the Oculus Touch) to an FBO using a custom shader that outputs "triangle information" to the buffer and then use glReadPixels to read the central pixel data. The problem I am facing is that the Oculus does distortion to the scene but there is no way to apply it to the off-screen buffer, so there is significant difference between the on-screen buffer and the off-screen buffer.

My question is, Is ray casting the only feasible way to do picking in Oculus Apps or is there a way to perform the faster FBO picking even when the view distorted?