Forum Discussion
Idea - How to get fast head tracking with any frame rate
Here's an idea of how to get low latency head tracking with almost any variable frame rate. Render the scene from the player's point of view with no distortion, but to a larger render target with a...
Posted this on Reddit, posting here too. From the fact that this would work just as well for on-screen games I suspected this has already been done, and sure enough it has. See this paper and look up other works that reference it (this paper works out a lot of the kinks with your idea, including dealing with translation and occlusion):
Mark et al, Post-rendering 3D warping, 1997 Symposium on Interactive 3D Graphics
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.13.1789&rep=rep1&type=pdf
Abstract: "A pair of rendered images and their Z-buffers contain almost all of the information necessary to re-render from nearby viewpoints. For the small changes in viewpoint that occur in a fraction of a second, this information is sufficient for high quality re-rendering with cost independent of scene complexity. Re-rendering from previously computed views allows an order-of-magnitude increase in apparent frame rate over that provided by conventional rendering alone. It can also compensate for system latency in local or remote display.
We use McMillan and Bishop’s image warping algorithm to re-render, allowing us to compensate for viewpoint translation as well as rotation. We avoid occlusion-related artifacts by warping two different reference images and compositing the results. This paper explains the basic design of our system and provides details of our reconstruction and multi-image compositing algorithms. We present our method for selecting reference image locations and the heuristic we use for any portions of the scene which happen to be occluded in both reference images. We also discuss properties of our technique which make it suitable for real-time implementation, and briefly describe our simpler real-time remote display system."
Mark et al, Post-rendering 3D warping, 1997 Symposium on Interactive 3D Graphics
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.13.1789&rep=rep1&type=pdf
Abstract: "A pair of rendered images and their Z-buffers contain almost all of the information necessary to re-render from nearby viewpoints. For the small changes in viewpoint that occur in a fraction of a second, this information is sufficient for high quality re-rendering with cost independent of scene complexity. Re-rendering from previously computed views allows an order-of-magnitude increase in apparent frame rate over that provided by conventional rendering alone. It can also compensate for system latency in local or remote display.
We use McMillan and Bishop’s image warping algorithm to re-render, allowing us to compensate for viewpoint translation as well as rotation. We avoid occlusion-related artifacts by warping two different reference images and compositing the results. This paper explains the basic design of our system and provides details of our reconstruction and multi-image compositing algorithms. We present our method for selecting reference image locations and the heuristic we use for any portions of the scene which happen to be occluded in both reference images. We also discuss properties of our technique which make it suitable for real-time implementation, and briefly describe our simpler real-time remote display system."