Skip to main content
Up and Coming
April 11, 2013

Shader Optimization.

I was playing with the shader and wondered if it could be optimized.

Lets start with the standard Shader


float2 HmdWarp(float2 in01)
{
float2 theta = (in01 - LensCenter) * ScaleIn; // Scales to [-1, 1]
float rSq= theta.x * theta.x + theta.y * theta.y;
float2 theta1 = theta * (HmdWarpParam.x + HmdWarpParam.y * rSq +
HmdWarpParam.z * rSq * rSq + HmdWarpParam.w * rSq * rSq * rSq);
return LensCenter + Scale * theta1;
}

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float2 tc = HmdWarp(texCoord);
if (any(clamp(tc, ScreenCenter-float2(0.25,0.5), ScreenCenter+float2(0.25, 0.5)) - tc))
return 0;
return Texture.Sample(Linear, tc);

};


That's 11 Multiplies, 6 Additions and 3 Subtractions per pixel. (eyeballed the numbers sorry, could be off by one or two) and on my GTX670 takes 0.1447264 milliseconds to execute (based on averaging the time of a 100 frames of just the draw with the shader)

Which could be good, could be bad, nothing to compare it to, so lets compare it to a shader that does not warp and just samples the texture.


float4 pixelShader( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float4 c = Texture.Sample(Linear, texCoord);
return c;
}


which takes 0.0272857 milliseconds to execute (again GTX670, 100 frames)

So the warping shader is 5.3 times slower than just bare sampling.

There must be a better way, why are we recalculating the distortion map every frame single frame? seems rather wasteful. Lets render the distortion map to texture instead. We'll use the Red channel for the X distortion, and the Green channel for the Y distortion. If we make a texture of format R32G32_Float it should fit our needs nicely. And the upside is we only have to do the nasty math once!

our shader now looks like this


float2 HmdWarp(float2 in01)
{
/// Same as above, cut to keep the post size down.
}

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float2 tc = HmdWarp(texCoord);
if (any(clamp(tc, ScreenCenter-float2(0.25,0.5), ScreenCenter+float2(0.25, 0.5)) - tc))
return 0;
return float4(tc.x, tc.y,0,0);
};


The resulting texture will look kinda like this (Used bad parameters, had to guess since I don't have a rift yet so the horizontal mapping is somewhat off)



Sweet, almost in business! lets update the warping shader to make use of the new texture. First of all the linear sampler has to go, a MinMagMipPoint sampler will fit our needs better.


SamplerState PointSampler : register(s0);
Texture2D Texture : register(t0);
Texture2D TextureWarp : register(t1);

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float4 c = TextureWarp.Sample(PointSampler, texCoord);
float2 RealCoord = float2(c.r,c.g);
return Texture.Sample(PointSampler, RealCoord);
};


Ahh much simpler now! New execution time 0.0598547 milliseconds and no visual difference! 2.5 times speed up from the original shader! Not bad for 20 minutes of tinkering.

    80 replies

    Up and Coming
    April 21, 2013
    "geekmaster" wrote:
    According to the literature on this subject, it is rarely beneficial to use more than two terms for radial distortion correction.

    That's my experience too, although not with the Rift's lenses so far.
    www.vorpx.com
    Up and Coming
    April 21, 2013
    "brantlew" wrote:
    Just shootin' from the hip here, but the current warp correction has k[0]==1 which seems to be a constant via the definition of Brown's distortion formula and also k[3]==0 which is "practically" a constant for non-fish eye lens. So the general shader could have those terms simplified out for an easy reduction in instructions.

    Yep, you can do this optimization (it saves 2 ALU ops out of 22 total in my pixel shader). I was also able to drop another 3 ALU ops on top of this by making two shaders, a left eye warp and a right eye warp, and using macros to directly specify the shader constant values rather than using actual constant registers. This works because unless you're adjusting your Rift settings at run-time (IPD, eye-lens distance, etc.), these values will never change for the duration of your application.

    Of course you could also drop most of the ALU ops entirely by doing what the original guy said and precomputing the warp values to a texture. :)
    April 22, 2013
    Has anyone else tried this with OpenGL yet? I still haven't been able to get it running faster with a precomputed displacement map. I'm curious if someone with more competence than me has had better luck.

    I was able to speed things up slightly by distorting both eyes at the same time and using imageStore() to write to their respective halves of the screen, but it's still nowhere near as big a gain as others have reported with the precomputed displacement.
    Up and Coming
    April 22, 2013
    Wow. Nice work! How does this play out in a practical scenario, where things like IPD are required? Or is this the practical scenario? I want to try this out in here somewhere...
    Up and Coming
    April 22, 2013
    "Ben" wrote:
    Has anyone else tried this with OpenGL yet? I still haven't been able to get it running faster with a precomputed displacement map. I'm curious if someone with more competence than me has had better luck.

    I'm also surprised at the results of caching the shader to a texture. I got a 2x speedup on my MacBook Air but on my desktop with a 480GTX the performance is exactly the same at 250fps. I made other optimizations that pushed that framerate to 280fps so I know that if the shader really did have an impact on the GTX it would have shown even a little. Still, not complaining!
    Ibex 3D VR Desktop for the Oculus Rift: http://hwahba.com/ibex - https://bitbucket.org/druidsbane/ibex
    Up and Coming
    April 22, 2013
    "hesham" wrote:
    I'm also surprised at the results of caching the shader to a texture. I got a 2x speedup on my MacBook Air but on my desktop with a 480GTX the performance is exactly the same at 250fps. I made other optimizations that pushed that framerate to 280fps so I know that if the shader really did have an impact on the GTX it would have shown even a little. Still, not complaining!
    On your 480GTX, the framerate bottleneck must be elsewhere, and not in the shader. Do you really NEED more than 250fps (or 280fps)?
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    Up and Coming
    April 22, 2013
    "geekmaster" wrote:
    On your 480GTX, the framerate bottleneck must be elsewhere, and not in the shader. Do you really NEED more than 250fps (or 280fps)?

    Not at all. I just wanted a confirmation that the shader was actually an improvement :) I haven't tested on slower Windows video cards so I didn't get the confirmation that it made a huge difference. My current use-case uploads a 2560x1600 texture 42 times per second so that's probably the bottleneck really :)

    (as an aside: it would be nice if I had a virtual driver for Windows or OSX so I could get just the updates to the desktop and update that portion of the texture instead of everything at once. The Linux version is a compositor so it has a sort-of live texture that doesn't really need updating, just mapping once per window)
    Ibex 3D VR Desktop for the Oculus Rift: http://hwahba.com/ibex - https://bitbucket.org/druidsbane/ibex
    Up and Coming
    April 22, 2013
    There's a lot of concern over speed here. :) Certainly of value, but a warp-map should be a faster choice except on compute-optimized hardware rather than graphics.

    The only quibble I have with a warp-map is that it's always present. So whatever memory it takes is just that much less you have for something else. Fatter is more bandwidth too.

    So I'm encoding it as a delta, normalized to bitdepth (currently using 8bits per component), and mirroring the right eye. This means I have it in 640x800x2 = 1024kbytes. Mirroring the right eye involves an unfortunate detail -- the u-component is also flipped. Right now I just do that in the shader, but that's slow (a conditional). Later I'll render a separate quad per eye and change a shader param to negate the u-component.

    These uv manipulations (applying a delta vs direct mapping) could cost a little more, but I haven't tested yet.

    Also, for those of you using the shader calculation, aside from removing unused or constant terms, it's a polynomial, making it susceptible to a simple optimization:
    x + (y + (z + w * r2) * r2) * r2
    lazydodoAuthor
    Up and Coming
    April 22, 2013
    Yeah it's a neat optimalisation trick but it should be taken within context,
    the overal improvement on the shader time is great (up to 3.8 times!) but
    relative to the complete frametime the improvement is not nearly as massive.

    If we take the base frame time of 4 ms (someone posted their code did 250 fps)
    and assume he has identical hardware to mine.

    Raw Frame Time (no distortion shader) : 4 - (0.1447264*2) (shader runs twice) = 3.7105472 ms
    Optimized Frame Time : 3.7105472 + (0.0598547 *2) = 3.8302566

    1000 / 3.8302566 = 261 fps..

    So in the *BEST* case scenario you'll squeeze an extra 11 frames out a second.
    which is still pretty decent but it won't turn a poor performing engine into
    a great one by pure magic. However in VR latency matters and if we can squueze out
    a frame just 0.20ms faster it's worth it especially if it's an optimalisation as
    trivial as this.

    I have my rift now, and dislike that the shader has to run twice, i can probably
    cache both runs in a single texture (Writing some code to test it in the next
    few days) but still even then that would get the framerate to 265 fps in the best
    case scenario. So yeah it's a little speedboost, but it's not a magic bullet that
    make all problems go away.
    Up and Coming
    April 22, 2013
    "lazydodo" wrote:

    I have my rift now, and dislike that the shader has to run twice, i can probably
    cache both runs in a single texture (Writing some code to test it in the next
    few days) but still even then that would get the framerate to 265 fps in the best
    case scenario. So yeah it's a little speedboost, but it's not a magic bullet that
    make all problems go away.


    Ah, my Rift is out for delivery... I await... impatiently. ;)

    Shader has to run twice? You mean to generate the warp texture? I generate
    the texture on CPU as part of initialization. Or you mean once for each eye?
    But that's half the pixel coverage. Overhead of changing a uniform variable
    once should be minimal.