Skip to main content
Up and Coming
April 11, 2013

Shader Optimization.

I was playing with the shader and wondered if it could be optimized.

Lets start with the standard Shader


float2 HmdWarp(float2 in01)
{
float2 theta = (in01 - LensCenter) * ScaleIn; // Scales to [-1, 1]
float rSq= theta.x * theta.x + theta.y * theta.y;
float2 theta1 = theta * (HmdWarpParam.x + HmdWarpParam.y * rSq +
HmdWarpParam.z * rSq * rSq + HmdWarpParam.w * rSq * rSq * rSq);
return LensCenter + Scale * theta1;
}

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float2 tc = HmdWarp(texCoord);
if (any(clamp(tc, ScreenCenter-float2(0.25,0.5), ScreenCenter+float2(0.25, 0.5)) - tc))
return 0;
return Texture.Sample(Linear, tc);

};


That's 11 Multiplies, 6 Additions and 3 Subtractions per pixel. (eyeballed the numbers sorry, could be off by one or two) and on my GTX670 takes 0.1447264 milliseconds to execute (based on averaging the time of a 100 frames of just the draw with the shader)

Which could be good, could be bad, nothing to compare it to, so lets compare it to a shader that does not warp and just samples the texture.


float4 pixelShader( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float4 c = Texture.Sample(Linear, texCoord);
return c;
}


which takes 0.0272857 milliseconds to execute (again GTX670, 100 frames)

So the warping shader is 5.3 times slower than just bare sampling.

There must be a better way, why are we recalculating the distortion map every frame single frame? seems rather wasteful. Lets render the distortion map to texture instead. We'll use the Red channel for the X distortion, and the Green channel for the Y distortion. If we make a texture of format R32G32_Float it should fit our needs nicely. And the upside is we only have to do the nasty math once!

our shader now looks like this


float2 HmdWarp(float2 in01)
{
/// Same as above, cut to keep the post size down.
}

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float2 tc = HmdWarp(texCoord);
if (any(clamp(tc, ScreenCenter-float2(0.25,0.5), ScreenCenter+float2(0.25, 0.5)) - tc))
return 0;
return float4(tc.x, tc.y,0,0);
};


The resulting texture will look kinda like this (Used bad parameters, had to guess since I don't have a rift yet so the horizontal mapping is somewhat off)



Sweet, almost in business! lets update the warping shader to make use of the new texture. First of all the linear sampler has to go, a MinMagMipPoint sampler will fit our needs better.


SamplerState PointSampler : register(s0);
Texture2D Texture : register(t0);
Texture2D TextureWarp : register(t1);

float4 main( float2 texCoord : TEXCOORD0,float4 color : COLOR0 ) : SV_Target
{
float4 c = TextureWarp.Sample(PointSampler, texCoord);
float2 RealCoord = float2(c.r,c.g);
return Texture.Sample(PointSampler, RealCoord);
};


Ahh much simpler now! New execution time 0.0598547 milliseconds and no visual difference! 2.5 times speed up from the original shader! Not bad for 20 minutes of tinkering.

    80 replies

    Up and Coming
    May 13, 2013
    I agree that any HUD content should be in that central (untinted) area. But for those who have a wider peripheral view, all of the tinted pixels should remain visible, but perhaps those in the outer peripheral zone could be rendered at a lower resolution to save processing time.
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    Up and Coming
    May 13, 2013
    But how much time is really being saved? Everybody should remember Amdahl's law.
    (╯°□°)╯︵┻━┻
    Up and Coming
    May 13, 2013
    "jwilkins" wrote:
    But how much time is really being saved? Everybody should remember Amdahl's law.
    Computers have a Mean-Time-Before-Obsolescence of 18 months? :D
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    Up and Coming
    May 13, 2013
    "geekmaster" wrote:
    "jwilkins" wrote:
    But how much time is really being saved? Everybody should remember Amdahl's law.
    Computers have a Mean-Time-Before-Obsolescence of 18 months? :D

    That's Moore's Law or Dennard Scaling. Amdahl's law relates speedup to the percentage of a process you are optimizing.

    I'd be surprised if killing pixels early so you could only render the "title safe" pixels would be even .5% of frame time. And as you and I have argued, those pixels aren't actually invisible. I would actually argue that you shouldn't have the black border, but you should use edge clamping to fill those pixels with light that might be picked up as flashes in your peripheral vision.
    (╯°□°)╯︵┻━┻
    Up and Coming
    May 13, 2013
    "jwilkins" wrote:
    That's Moore's Law or Dennard Scaling. Amdahl's law relates speedup to the percentage of a process you are optimizing.

    Oops! Yeah, I know Moore's law. :mrgreen:
    "jwilkins" wrote:
    I'd be surprised if killing pixels early so you could only render the "title safe" pixels would be even .5% of frame time. And as you and I have argued, those pixels aren't actually invisible. I would actually argue that you shouldn't have the black border, but you should use edge clamping to fill those pixels with light that might be picked up as flashes in your peripheral vision.

    Even better to use light-pipes (lucite/acrylic rods and/or fiber optics) to bring those "unused" pixel areas into the Rift DK face mask along the peripheral rubber sides, "Ambi-light" style, as I suggested in various MTBS3D posts in the past. Then you could see motion way out beyond the lenses, in your peripheral vision. That would make the facemask seem transparent, with VR content visible "through" it.
    ;)
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
    Up and Coming
    May 13, 2013
    "jwilkins" wrote:
    I would actually argue that you shouldn't have the black border, but you should use edge clamping to fill those pixels with light that might be picked up as flashes in your peripheral vision.


    I agree with this, although there is one important reason for blacking-out: The central pixels at the partition between eyes should be black. In the distortion shader I made before having a Rift I did mirroring at the border, but sometimes light-bleed was noticeable from one eye to the other. When I touch that code again, I'm going to change the second-pass to have a few pixels of gap between each eye-quad.
    Up and Coming
    May 14, 2013
    "jwilkins" wrote:
    But how much time is really being saved? Everybody should remember Amdahl's law.


    Well, according to the first post on this thread, the original V0.1 warp shader (before chromatic distortion was added) took 0.1447264 on a GTX670. Comparing the 670's 3DMark Texture Fill score of 86.1 gigatexels/sec to the 6.1 gt/s of the very common IntelHD4000, I can estimate that the Intel takes 0.1447264*86.1/6.1=2.04ms out of the 16.67ms 60fps budget just to run the basic distortion shader. Lots of laptop users are going to want those 2ms back. Masking out the pink and dark areas of the visible area diagram with stencil should at least cut the time in half.
    Up and Coming
    May 14, 2013
    "corysama" wrote:
    "jwilkins" wrote:
    But how much time is really being saved? Everybody should remember Amdahl's law.


    Well, according to the first post on this thread, the original V0.1 warp shader (before chromatic distortion was added) took 0.1447264 on a GTX670. Comparing the 670's 3DMark Texture Fill score of 86.1 gigatexels/sec to the 6.1 gt/s of the very common IntelHD4000, I can estimate that the Intel takes 0.1447264*86.1/6.1=2.04ms out of the 16.67ms 60fps budget just to run the basic distortion shader. Lots of laptop users are going to want those 2ms back. Masking out the pink and dark areas of the visible area diagram with stencil should at least cut the time in half.


    I know I did it myself, but you really are not allowed to use words like "I bet" and "should" when talking about optimizations. Either you've measured an improvement or you are talking out your ass :-) I'd be happy to lose this bet.
    (╯°□°)╯︵┻━┻
    Participant
    May 14, 2013
    @geekmaster - would you suggest that, as some people can see the black areas, that developers should fill those with something post-warp? Eg repeating the colour nearest that edge to at least fill your peripheral vision with something approximating what is there?
    Up and Coming
    May 14, 2013
    "mrklaw" wrote:
    @geekmaster - would you suggest that, as some people can see the black areas, that developers should fill those with something post-warp? Eg repeating the colour nearest that edge to at least fill your peripheral vision with something approximating what is there?

    Exactly! I think the border pixels should be stretched into the black areas. Perhaps a pre-warp filter could leave a copy of the original unwarped outer pixels in place when moving them inward for barrel distortion, at little or no extra processing cost. This would provide an "ambi-light"-like effect at least to the outer edges of the lenses where I can see black pixels in my peripheral vision when looking AWAY from them.

    And you still have the option of drawing ADDITIONAL content (with extreme fisheye distortion) on those black pixels to conduct into the Rift DK face-mask with light pipes, for a true "translucent face-mask" effect. The light-pipe addition would be easier to accomplish when integrated into a new design, but could be pretty DIY using fiber optics instead of custom-molded solid acrylic light-guides like a commercial Rift could benefit from.

    I really want to see all those now-wasted pixels driving internal ambient lighting inside the face-mask, beyond what the lenses can see. But at least, let me see SOMETHING useful there WHEN I CAN see parts of those black pixel areas.

    Here is a post by Nick3DvB with similar suggestions on using otherwise-wasted pixels for ambient lighting effects:
    viewtopic.php?f=28&t=320&p=10204#p3858
    This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.