Skip to main content
Up and Coming
April 11, 2013

Shader Optimization.

I was playing with the shader and wondered if it could be optimized.

Lets start with the standard Shader


float2 HmdWarp(float2 in01)
{
float2 theta = (in01 - LensCenter) * ScaleIn; // Scales to [-1, 1]
float rSq= theta.x * theta.x + theta.y * theta.y;
float2 theta1 = theta * (HmdWarpParam.x + HmdWarpParam.y * rSq +
HmdWarpParam.z * rSq * rSq + HmdWarpParam.w * rSq * rSq * rSq);
return LensCenter + Scale * theta1;
}

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float2 tc = HmdWarp(texCoord);
if (any(clamp(tc, ScreenCenter-float2(0.25,0.5), ScreenCenter+float2(0.25, 0.5)) - tc))
return 0;
return Texture.Sample(Linear, tc);

};


That's 11 Multiplies, 6 Additions and 3 Subtractions per pixel. (eyeballed the numbers sorry, could be off by one or two) and on my GTX670 takes 0.1447264 milliseconds to execute (based on averaging the time of a 100 frames of just the draw with the shader)

Which could be good, could be bad, nothing to compare it to, so lets compare it to a shader that does not warp and just samples the texture.


float4 pixelShader( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float4 c = Texture.Sample(Linear, texCoord);
return c;
}


which takes 0.0272857 milliseconds to execute (again GTX670, 100 frames)

So the warping shader is 5.3 times slower than just bare sampling.

There must be a better way, why are we recalculating the distortion map every frame single frame? seems rather wasteful. Lets render the distortion map to texture instead. We'll use the Red channel for the X distortion, and the Green channel for the Y distortion. If we make a texture of format R32G32_Float it should fit our needs nicely. And the upside is we only have to do the nasty math once!

our shader now looks like this


float2 HmdWarp(float2 in01)
{
/// Same as above, cut to keep the post size down.
}

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float2 tc = HmdWarp(texCoord);
if (any(clamp(tc, ScreenCenter-float2(0.25,0.5), ScreenCenter+float2(0.25, 0.5)) - tc))
return 0;
return float4(tc.x, tc.y,0,0);
};


The resulting texture will look kinda like this (Used bad parameters, had to guess since I don't have a rift yet so the horizontal mapping is somewhat off)



Sweet, almost in business! lets update the warping shader to make use of the new texture. First of all the linear sampler has to go, a MinMagMipPoint sampler will fit our needs better.


SamplerState PointSampler : register(s0);
Texture2D Texture : register(t0);
Texture2D TextureWarp : register(t1);

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float4 c = TextureWarp.Sample(PointSampler, texCoord);
float2 RealCoord = float2(c.r,c.g);
return Texture.Sample(PointSampler, RealCoord);
};


Ahh much simpler now! New execution time 0.0598547 milliseconds and no visual difference! 2.5 times speed up from the original shader! Not bad for 20 minutes of tinkering.

    80 replies

    Up and Coming
    May 12, 2013
    I agree, but only about studying the FoV that other people get. Palmer already posted that some people CAN see all the way to the display edges in all directions, and I experimented a lot with lens variations (with and without foam) until I finally achieved that situation for my own use. And I do not want to see developers (like Valve) intentionally drawing black pixels THAT I CAN SEE! Now, with your image, I see that gluing the foam back on my Rift DK has reduced my FoV again. Bummer. With no foam I was able to see beyond the magenta area in your image (and I could actually see a little of the bezel when ALL 1280x800 pixels were illuminated). But now with the foam on, I can only see magenta when I look a little in the opposite direction. The more to the LEFT I look, the more of the RIGHT magenta I can see. The more I look RIGHT, the more LEFT magenta I see. The more I look DOWN, the more TOP magenta I see. However looking up does not let me see BOTTOM magenta, because the A-cups press against the underside of my eyebrow ridges and do not let me vertically center my eyes over the "sweet spot" (optical axis) of the lenses.

    The magenta boundaries you defined are about correct for FOVEAL vision (looking directly at them), but not for PERIPHERAL vision (what you see out of the "corners of your eyes", also important).

    Our faces and our eyeballs are probably shaped a little differently, and my eyeballs do not touch the lenses unless I remove the foam to get my eyes closer to the lenses.

    That medium ring is about what I see when I look directly at the edges. But when looking forward I see the original much larger FoV in my peripheral vision (plus a little more, if my headstrap is tight or I push the Rift DK into my face a bit with my hands). I normally keep my strap it pretty tight to increase my FoV. And of course, my "coin-slot" adjustments are already placing the lenses as close to my eyes as possible. Different people have their eyeballs at different inset depths relative to how the Rift sits against their face, but I wish the coin-slot adjustments would put the lenses even closer to my eyes so I do not have to keep the head strap so tight.

    My only concern was your recommendation to NOT render outside that inner circle, which would only be acceptable in those cases where the viewer cannot see those pixels, but not in the general case. If the software can be configured to adapt to different lens settings and different FoVs, then (and only then) does it make sense to not render non-visible pixels FOR THAT VIEWER WITH THAT CONFIGURATION.

    I do not want to lose ANY of my valuable peripheral FoV. Thanks! And yes, in some cases I can see slightly into the Rift DK LCD bezel area (where there are no pixels), so intentionally painting black pixels THAT I CAN SEE is not what I want to see recommended to developers (except in a special per-user configuration case).

    Now, your test has made me aware that putting this thick foam back on my Rift DK may be a mistake (for me). To look straight into the Rift DK (perpendicular to the display), I still need some foam at the top between my eyebrows. I may end up with variable-thickness foam, thick at the top but thin at the bottom. Your test shows that I lost significant FoV when I reattached my foam, so I need to do some more experimentation in that area.

    Thanks!

    EDIT: With the original foam installed, I can see some magenta when looking away from it, but if those were black pixels I would probably not notice. However, there are reports of people how CAN see all the way to the edges with the foam. So I think that it is pretty reasonable to replace the magenta areas in the newer images with black (but all of the very-darkened pixels certainly must be rendered, because I see (most of) them even with my foam when looking directly at them).
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    Up and Coming
    May 12, 2013
    You would probably notice that the extra light was missing.


    (╯°□°)╯︵┻━┻
    Up and Coming
    May 12, 2013
    @geekmaster: What's your IPD?
    Up and Coming
    May 12, 2013
    "Capyvara" wrote:
    @geekmaster: What's your IPD?

    About 6.5mm, but my eyebrow ridges prevent me from vertically centering the A-cup lenses "just right". I may have to grind off some of the top-side of my A-cups (and perhaps even some of the lenses) to get them centered properly. Lenses larger than 1.5-inch would be more of a problem unless they were large enough that they could go over my eyebrows instead of under them.

    I see that the latest OculusVR SDK has changed the default Rift DK IPD frin 64mm to 63.5mm, which explains why I cannot look through the horizontal centers of both lenses at the same time.
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    Up and Coming
    May 12, 2013
    I'd be interested in hearing from a few others:

    1.) Are you able to see outside the blue "band" (not the brighter inner blue disc) regardless of cup, dial position or eye distance? (please be objective, it's possible for me but only if I'm manipulating the rift with my hands or don't have it on/centered reasonably)

    2.) Are you able to see the magenta area?
    (Regardless of assembly, cup or eye distance I can't even manage this without going so far as to almost view so far from one side of the lens and on "edge" at a side angle that I get a reversed image effect through the lens)

    In terms of eye distance try to get them as close to the lens as possible that gives you the most FOV. I would imagine if the assembly is dialed all the way in and you push in a bit nearly everyone should be able to do this regardless of the foam. If you feel you aren't able to this, please mention that.
    Up and Coming
    May 12, 2013
    C cups, dialed in all the way and pushed: bright region perfectly describes foveal limits. Peripherally I see the blue ring entirely and parts of the dark area in the corners. Can't get anywhere near the magenta.

    IMHO the ideal case is for the stencil/masking to be adjustable as part of a usual software calibration process since it's going to change even for the same person depending on how they have the Rift configured.
    Up and Coming
    May 12, 2013
    "320x200" wrote:
    C cups, dialed in all the way and pushed: bright region perfectly describes foveal limits. Peripherally I see the blue ring entirely and parts of the dark area in the corners. Can't get anywhere near the magenta.

    IMHO the ideal case is for the stencil/masking to be adjustable as part of a usual software calibration process since it's going to change even for the same person depending on how they have the Rift configured.

    When I used C cups, I got a much better focus, but I lost half my FoV (determined by how many floor tiles across I could see when looking down in the OculusRoomTiny demo). I probably could not even see the brighter blue pixels using those lenses...

    I agree with the adjustment. In cases where the smaller FoV increases my framerate by rendering fewer pixels, I *MIGHT* choose that instead. But when I can have high framerate AND large FoV, that would be my preference. User adjustment is fine, but please do not blacken my valuable pixels without my consent.
    ;)

    I only see magenta when I an NOT looking directly at it (in my peripheral FoV only). However, if I remove my foam, I will be able to see a lot more of the magenta (lenses closer to my eyes). In the photos where magenta was replaced with black, my peripheral vision does not notice black, so it may be a worthwile tradeoff for increased framerate. But not rendering the dark blue? No way!
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    Participant
    May 13, 2013
    With my usual setup (Type A lens and almost all the way in but just far enough that my lashes do not brush against the lens) I can definitely see the entire blue ring and some dark area but no magenta.
    If I adjust it all the way close so my lashes touch the lens I can see most of the outer dark regions in my periphery and the center plastic screen divider and the magenta just showing a little on the sides and bottom when I'm looking the other way. It goes away when I try to look at it. I see a bit less of the top of the screen.
    I'd say the center circle is safe for HUD elements but everything but the magenta is needed for periphery vision.
    Up and Coming
    May 13, 2013
    "vanti" wrote:
    With my usual setup (Type A lens and almost all the way in but just far enough that my lashes do not brush against the lens) I can definitely see the entire blue ring and some dark area but no magenta.
    If I adjust it all the way close so my lashes touch the lens I can see most of the outer dark regions in my periphery and the center plastic screen divider and the magenta just showing a little on the sides and bottom when I'm looking the other way. It goes away when I try to look at it. I see a bit less of the top of the screen.
    I'd say the center circle is safe for HUD elements but everything but the magenta is needed for periphery vision.

    If removing the foam would allow you to get your eyes even closer to the lenses (such that the eyelashes bend as they brush the lenses) you would probably see a lot more of the magenta (like me) including some of the bezel area beyond the magenta when looking the other direction. And for that increased FoV I plan to modify my foam (thinner and/or some portions removed). That is why I do not wish to see advice to blacken pixels some people cannot see with their lens configurations.
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    Up and Coming
    May 13, 2013
    Even with the lenses pushed right up to your eyeballs, tmek's diagram does serve a useful purpose. For anyone familiar with the analog TV world, you've probably heard of the 'title safe area', the area you need to stay inside to guarantee that your on-screen text/graphics/ticker will be visible to all viewers, taking into account worst-case overscan and tube masking.


    Tmek's diagram we can then call a 'HUD safe area': the area in which you should keep all your HUD UI elements to guarantee that all Rift users will be able to see them in their entirety, even in the worst case of C-lenses at their furthest ratchet.


    Of course, if you ask your users (or for future devices, measure internally) what lens they are using and the ratchet position they have set, you can push the HUD further towards the edges, but that might not be desirable anyway due to eye-strain issues of protracted far off-axis viewing.