Forum Discussion
are "asymmetric" barrel distortions correct?
By "asymmetric", I mean to look at the distortion applied for just one eye - note that the distortion appears compressed vertically at the outside like in one of the two attached images.
I am wondering - and I see this in various Oculus demos - is it correct to do this? Ideally: the centre of projection, the centre of the lens, and the centre of the eye are all coaxial, and thus a "symmetric" barrel distortion is correct.
Is the purpose of the "asymmetric" barrel distortion (produced by offsetting the "LensCenter" parameter) then specifically to compensate for individuals with an IPD which is distant from the axis formed by the centre of lens and centre of projection points? If this is the case, then why do so many demos use this asymmetric distortion (most people would be near "the mean IPD" and therefore symmetric distortion would be ideal as a fixed parameter)?
If the asymmetric barrel distortion is best - why?
Insights greatly appreciated :mrgreen:
Here is the GLSL fragment shader code I am using, for reference (it was used in the creation of the two images):
I am wondering - and I see this in various Oculus demos - is it correct to do this? Ideally: the centre of projection, the centre of the lens, and the centre of the eye are all coaxial, and thus a "symmetric" barrel distortion is correct.
Is the purpose of the "asymmetric" barrel distortion (produced by offsetting the "LensCenter" parameter) then specifically to compensate for individuals with an IPD which is distant from the axis formed by the centre of lens and centre of projection points? If this is the case, then why do so many demos use this asymmetric distortion (most people would be near "the mean IPD" and therefore symmetric distortion would be ideal as a fixed parameter)?
If the asymmetric barrel distortion is best - why?
Insights greatly appreciated :mrgreen:
Here is the GLSL fragment shader code I am using, for reference (it was used in the creation of the two images):
uniform sampler2D framebuf;
uniform int left_eye;
uniform float kappa_0;
uniform float kappa_1;
uniform float kappa_2;
uniform float kappa_3;
uniform vec2 LensCenter;
uniform vec2 Scale;
uniform vec2 ScaleIn;
varying vec2 vPos;
varying vec2 vTex;
void main()
{
vec2 theta = (vTex - LensCenter) * ScaleIn;
float rSq = theta.x * theta.x + theta.y * theta.y;
vec2 rvector = theta * (kappa_0 + kappa_1 * rSq + kappa_2 * rSq * rSq + kappa_3 * rSq * rSq * rSq);
vec2 tc = LensCenter + Scale * rvector;
//keep within bounds of texture
if ((left_eye == 1 && (tc.x < 0.0 || tc.x > 0.5)) ||
(left_eye == 0 && (tc.x < 0.5 || tc.x > 1.0)) ||
tc.y < 0.0 || tc.y > 1.0) {
discard;
}
//keep on either side of viewport
if (left_eye == 1 && vPos.x > 0.0) {
discard;
}
else if (left_eye == 0 && vPos.x < 0.0) {
discard;
}
gl_FragColor=texture2D(framebuf, tc);
}
9 Replies
- Moving the lens center by any significant amount requires the tangential lens distortion terms and coefficients that are missing from the simplified formula used in the OculusVR SDK. When you look through a lens from an offset position (not through the lens optical axis, such as if your IPD varies significantly from 64mm used in the Rift), tangential distortion correction is needed or there will be unwanted vertical misalignments near the outer edges of the image pair, making stereoscopic convergence in those areas difficult or impossible.
The full correct "Brown's model" lens distortion formula can be found here:
http://en.wikipedia.org/wiki/Distortion_(optics)#Software_correction
Although it is popular for software to leave out the tangential (P) terms in that formula, this only works correctly when you can guarantee that the lens optical center will always be used (such as with a digital camera). For an HMD with fixed IPD (such as the Rift), tangential correction can be very important for anybody with significantly smaller or larger IPD than was designed into the Rift DK.
It has been reported that the consumer Rift will have mechanically adjustable IPD, in which case tangential IPD correction may be optional (but still useful for users who wish to keep their Rift IPD set to a "standard" value for multiple users).
So, to answer your question, asymmetrical barrel distortions are only correct if they take lens axis offset into account, by restoring tangential correction to the lens distortion correction formula. If (and only if) you can guarantee that the user will look through the optical centers of both eyes (64mm IPD), then you can leave out tangential (asymmetrical) correction.
Any other method of software IPD adjustment will have very limited (and not particularly useful) application. I have published this information (in much more detail) in many posts here and even more at MTBS3D, if you want to learn more about it. - To geekmaster - I appreciate the response. Especially the bit about Brown's distortion model. In fact I found a scanned PDF of Brown's old paper and plan to give it a more careful read through today. And if you have some other notes/discussion about this subject that are valuable as you mentioned, please send those my way :)
Other issues/thoughts:
1) A contradiction:- In the attached image, which is an example rendering from the reference Oculus SDK implementation (the Tuscany unity demo binary which comes with SDK release 0.2.2), note that both the IPD value is set to 64mm and asymmetric distortion is occurring. There seems to be a contradiction here. (One possibility is that the true IPD of the Rift device (lens centre to lens centre) is not 64mm. But I have a feeling your knowledge of the dimensioning of the unit is probably right. Perhaps the program is not reporting the right IPD (doubtful). Another theory is a flaw in the implementation, they desire a symmetric distortion, but do not use the "LensCenter" values for each eye which produce a symmetric distortion and then just translate the quads inward. Though I would suspect a gap in my understanding before an implementation mistake like this :) And also, it cannot be denied that the asymmetrically-distorted rendering attached looks great wearing the Rift.)
2) Concerning Brown's distortion correction model and the tangential/"P" terms:- a) Looking through the lens/aperture at its extents, the screen gets progressively blurrier - even the "grid lines" that delineate the pixels themselves become blurred - thus a software correction that can only modify pixel intensities is not a reasonable solution after a given point. (There are diminishing returns labouring to minimize the distortion caused by non-lens axial view directions as the viewing angle off the axis increases.)
- b) The "P" terms of Brown's formula do not seem particularly difficult or computationally expensive to compute. Still, the reference SDK rendering examples use the asymmetric distortion, and do not document incorporating these extra terms. (I suspect here, like for many problems in graphics, their removal yielded a result that was "good enough". On that note, I should implement these extra terms in my fragment shader, and report back my findings using various P_* for curiosity's sake and to gain some intuition on their effect on the final rendering by interactively modifying them)
I very much appreciate the discussion. "JamesMcCrae" wrote:
... On that note, I should implement these extra terms in my fragment shader, and report back my findings using various P_* for curiosity's sake and to gain some intuition on their effect on the final rendering by interactively modifying them)
I very much appreciate the discussion.
The P terms will be dependent on lens offset (viewer IPD). You would need to view (or photograph) with a corresponding offset position through the lens.
And the effect of such tangential distortion correction becames greater and the offset increases. So for people with wide IPD, or with narrow IPD (such as children), tangential correction may be quite important.
It will need testing with a wide variety of "different IPD" users, with some method to remove personal bias (because it is difficult to measure non-subjectively).
This page contains a bunch of great links in the "References" section:
viewtopic.php?f=20&t=32&p=4433#p3962
Lens distortion correction using displacement mapping:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=140&t=17200&p=126095#p120358
And a bit about how displacement map textures can be faster than direct computation:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=140&t=17200&p=126095#p120524
And sometimes you need "lion taming" skill to suggest improvements:
http://www.mtbs3d.com/phpBB/viewtopic.php?f=140&t=17200&p=126095#p121137
Note that when using precomputed displacement mapping, tangential correction incurs no additional run-time overhead.- Great references, lots of information. I see you have been exploring this space for some time :)
Just to be clear on what is happening geometrically - consider the viewer's left eye (call this point L_1), left lens centre (call it L_2) and left centre of projection (L_3) and these are coaxial along some direction vector "d". So what is happening is that L_1 and L_3 are translating together (or perhaps equivalently looking at only one eye, only L_2 is translating sideways). And the Brown correction sorts that out.
Then, for the contradiction stated previously, the reported 64mm IPD is actually the "device" or "lens IPD", but the distortion actually occurring is for a "viewer IPD" which is not 64mm (and the viewer IPD and centre of projection are translated - L_1 and L_3 coaxial in direction "d", but shifted away from L_2). (If that's the case, it's interesting they do not reveal the fixed viewer IPD parameter.)
Does all that sound right? - I'm not sure I'm following exactly what you've observed, but you may be making this much more complicated than it is. The outlines of the rendered areas for the two eyes can be arbitrarily asymmetric even if the (un-)distortion being applied is radially symmetric about the lens center (which I would expect it to be in Rift code). Do you have a specific reason to believe that the *distortion* is asymmetric in the various demos that you mention? Or are you just looking at the shapes of the rendered areas for the two eyes?
The rendered areas are likely asymmetric just because the Rift lenses are not centered over the centers of the two halves of the display (they are inset quite a bit because the display is oversized). "madmethods" wrote:
Do you have a specific reason to believe that the *distortion* is asymmetric in the various demos that you mention? Or are you just looking at the shapes of the rendered areas for the two eyes?
So what I have done in the following is changed only the "LensCenter" shader parameter from left (0.25, 0.5) and right (0.75, 0.5)
distort1.png
to move inward (0.29, 0.5) and (0.71, 0.5). Bringing the values closer to the centre causes the "asymmetric" border that I've been noticing (where the outer edge appears squished vertically). All other program parameters are fixed.
distort2.png
Overlaying the two renderings shows that the final projection (or "undistortion") does in fact appear to change (so there is more going on here than just a change in the shape of the border).
distort3.png
This doesn't come as a surprise, because by translating the LensCenter parameter the "r" value (distance from LensCenter) calculated for all fragments changes. Therefore the texture sampling positions for all fragments changes and you get a different result for the final projection/undistortion.
To make my question more clear - assuming #1 the display is oversized and the LensCenter points are at some distance closer to the inside of the display. And assuming #2 we want to set up the projection for someone with the ideal IPD where the eyes are the same distance apart as the lenses. Why do this undistortion where it is squished vertically on the outside or radially asymmetric? Why not just use the radially symmetric undistortion, and just translate the quads inward so that the centers of projection physically line up with the lens centres on the display? Is there a mistake being made here with all these demos with the asymmetric undistortion, or am I missing something - is there in fact something special or extra going on?
Geekmaster and I had some back and forth - I discovered asymmetric undistortions relate to working for different IPDs (those off the optical axis) and in addition should include additional tangential distortion terms which the SDK does not. But the reference examples use the asymmetric undistortion, yet report being set up for a 64mm IPD - well I haven't found a real resolution to this seeming contradiction yet."JamesMcCrae" wrote:
...
Geekmaster and I had some back and forth - I discovered asymmetric undistortions relate to working for different IPDs (those off the optical axis) and in addition should include additional tangential distortion terms which the SDK does not. But the reference examples use the asymmetric undistortion, yet report being set up for a 64mm IPD - well I haven't found a real resolution to this seeming contradiction yet.
That sounds like your findings agreed with my suggestions, but I am not sure exactly what corrections you applied to the above images.
Question: Were those middle images created with tangential distortion correction? If so, the following applies:
When viewed without optical lens distortion (wide-eye view of above images), the top images look best, and the middle images hurt my eyes a bit when viewing the edges (look at the top of the clouds).
However, this is as it should be. When viewing with offset lenses (not viewing through the optical axis), the lenses should make the middle images look best (if the same amount of tangential distortion is performed by the lenses).
To view tangentially-corrected images properly, the viewing IPD and lens offset must match closely with the images. To view this in the Rift, a person with IPD near 64mm will need little or no tangential correction. People with significantly larger IPD will need more positive correction, and people with sigificantly smaller IPD will need more negative correction (depending on how you define positive and negative correction).
The vertical offset near the edges in the middle pair will exactly compensate for vertical offset caused by offset (wrong IPD) lenses. For that to work correctly, the final output image centers (AFTER lens distortion correction) need to be moved horizontally to match the viewer IPD.
Note that the IPD adjustments are used TWICE, both in the lens offset for tangential distortion correction, AND again when positioning the left and right images onto the Rift LCD panel.
If your vertical offset was caused by some other factor (not tangential correction), then the statements above do not apply (but remain useful)."JamesMcCrae" wrote:
"madmethods" wrote:
Do you have a specific reason to believe that the *distortion* is asymmetric in the various demos that you mention? Or are you just looking at the shapes of the rendered areas for the two eyes?
But the reference examples use the asymmetric undistortion, yet report being set up for a 64mm IPD - well I haven't found a real resolution to this seeming contradiction yet.
This is what I was referring to, not what you are doing with your own shader. In your own code vou can create asymmetry related to off-axis viewing (IPD mismatch) and tangential distortion and such, but I don't think that's what you are seeing in other demos / reference examples. In particular if you haven't specifically configured a non-typical IPD. I think my explanation is much more likely for any off-the-shelf demo / example code. If you haven't specified a non-typical IPD, then the asymmetry that I described would be fine, while an asymmetric distortion would be incorrect."geekmaster" wrote:
Question: Were those middle images created with tangential distortion correction?
They were not created with tangential distortion correction. All images were created with the shader code shown on the first post. The only parameter changed was "LensCenter" between example images.
I am still looking for a reasonable explanation why reference programs demonstrating a lens-matching IPD exhibit a distortion which is radially asymmetric (per eye).
