Skip to main content
Up and Coming
May 10, 2013

Rift Ray-tracing

[EDIT] This now has a demo, but I'll keep the link here up to date.

Download demo
(Updated to 12 Meg. Ver 0.50 on the Twentieth of June in the Year Twenty-Thirteen)).

Hi All,

I haven't got my Rift yet (UK near the end of the Kickstarter), but have been following the goings on with interest.

I've started to have a go at doing some CUDA based ray-tracing for VR and have managed to get some basic stuff going.

Rather than warping a resulting image I've made a start at modeling the lenses in the ray projection code.
(no texturing or shadows yet, just trying to get the distortion and stereo going.)

This is what I've got so far. It's probably waaaay out, but until I get my one, I'm flying blind. :D

Note: It's a bit wide to see in the post, you may need to open the image in another tab to see the entire screen.

    152 replies

    July 6, 2013
    Hello,

    I'm currently trying to add support for the rift to luxvr/slg, but even with all the hacking I can think of I can't reach an acceptable fps.
    warptest.jpg

    Any chance you could share your code? I'd love to have a cuda starting point a little more advanced than the optix examples. I think with 3 590 I could reach the fps I need.
    TeraBitAuthor
    Up and Coming
    July 7, 2013
    "vimax" wrote:
    Any chance you could share your code? I'd love to have a cuda starting point a little more advanced than the optix examples. I think with 3 590 I could reach the fps I need.


    My code is somewhat messy, as I'm just playing around with it at the moment. Also it is a hybrid system, with the bones of it wrapped in one DLL, being called by a conversion of my preexisting .NET based 3D engine and passing in the CUDA code related to the tracing.

    So.. what bit would you be interesting in seeing? The Camera Shader looks like this:
    #include <optix.h>
    #include <optix_math.h>
    #include <optix_world.h>
    #include <optixu_matrix_namespace.h>
    #include <optixu_vector_types.h>
    #include <optixu/optixu_math_namespace.h>
    #include "helpers.h"

    #define is ==
    #define or ||
    #define and &&
    #define then {
    #define endif };
    #define else }else{
    #define endsub }

    using namespace optix;

    struct PerRayData_radiance
    {
    float3 result;
    float importance;
    float r;
    int depth;
    };

    rtDeclareVariable(Matrix4x4, mat, , );
    rtDeclareVariable(float3, eye, , );
    rtDeclareVariable(float, FOV, , );
    rtDeclareVariable(float, IPD, , );
    rtDeclareVariable(int, Stereo, , );
    rtDeclareVariable(float, Aperture, , );
    rtDeclareVariable(float, Distortion, , );
    rtDeclareVariable(float, Aspect,,);
    rtDeclareVariable(float3, bad_color, , );
    rtDeclareVariable(float, scene_epsilon, , );
    rtDeclareVariable(rtObject, top_object, , );
    rtDeclareVariable(unsigned int, radiance_ray_type, , );

    rtDeclareVariable(uint2, launch_index, rtLaunchIndex, );
    rtDeclareVariable(uint2, launch_dim, rtLaunchDim, );
    rtDeclareVariable(float, time_view_scale, , ) = 1e-6f;

    rtBuffer<float4, 2> output_buffer;

    static __inline__ __device__ float2 GetLocalDirection(float Xpix, float Ypix){
    int xof = int(112.0f * ( float(launch_dim.y)/ float(800) ));
    if(Stereo is 0u) then
    return make_float2(Xpix, Ypix) / make_float2(launch_dim.x,launch_dim.y) * 2.f - 1.f;
    else
    int mid = launch_dim.x * 0.5f;
    if(Xpix < mid) then
    return make_float2(Xpix, Ypix) / make_float2(mid+xof,launch_dim.y) * 2.f - 1.f;
    else
    return make_float2(Xpix - mid +xof , Ypix) / make_float2(mid+xof,launch_dim.y) * 2.f - 1.f;
    endif
    endif
    endsub

    static __inline__ __device__ float3 GetWorldOrigin(){
    if(Stereo is 0u) then
    return eye;
    else
    int mid = launch_dim.x * 0.5f;
    if(launch_index.x < mid) then
    return eye+make_float3(make_float4(-IPD,0.0f,0.0f,0.0f)*mat);
    else
    return eye+make_float3(make_float4(IPD,0.0f,0.0f,0.0f)*mat);
    endif
    endif
    endsub

    static __inline__ __device__ float GetRadiusFromCenter(float2 d){
    if(Stereo is 1u) then
    return (sqrt((d.x * d.x) + ((d.y * d.y) * Aspect)) * Distortion);
    else
    return 0.0f;
    endif
    endsub

    static __inline__ __device__ float3 GetWorldDirection(float2 d, float tanTheta, float theta){
    return make_float3(make_float4(d.x,d.y,1.0f*theta,1.0f)* mat);
    endsub

    static __inline__ __device__ PerRayData_radiance CastRay(float2 d, float3 o, float r){
    PerRayData_radiance prd;
    prd.importance = 1.0f;
    prd.depth = 0u;
    prd.r = 0.0f;
    prd.result = make_float3(0.0f);
    if(r < Aperture*Distortion or Distortion is 0.0f) then
    float theta = 1.0f-((r*r)*0.2f);
    float3 ray_direction = GetWorldDirection(d,0.0f,theta); //tanTheta

    Ray ray = make_Ray(o, normalize(ray_direction), radiance_ray_type, scene_epsilon, RT_DEFAULT_MAX);

    rtTrace(top_object, ray, prd);
    endif
    return prd;
    endsub

    RT_PROGRAM void pinhole_camera(){
    float2 d; // Local Direction
    float3 o; // World Origin of Ray
    float r; // Radius of Pixel

    PerRayData_radiance result;

    float offset = 0.25;

    output_buffer[launch_index] = make_float4(0.0f);
    d = GetLocalDirection(float(launch_index.x), float(launch_index.y));
    o = GetWorldOrigin();
    r = GetRadiusFromCenter(d);
    result = CastRay(d,o,r);
    float dif = (Aperture*Distortion - r);
    if(dif < (0.05f * Distortion)) then
    dif = dif / (0.05f * Distortion);
    result.result *= dif;
    endif
    output_buffer[launch_index] += make_float4(result.result,1.0f);
    endsub
    Regards, Lee Page TeraBit Software
    Up and Coming
    July 23, 2013
    "TeraBit" wrote:
    "MarkV" wrote:
    Worked for me (I only had issues with some of the earlier versions)...
    Looks good.


    Glad it's working still. I've done a few new things...
    Cube Map Sky...
    HDR Bloom
    Normal Mapping.
    It's night and day 8-)



    Is that an old Light wave model ....? seems age since i have seen that little beasty,

    Keep up the good work, i think at the end of the day raytracing will provide the nicest eye candy in VR.
    "I love the French language. I have sampled every language, French is my favourite - fantastic language, especially to curse with. ..... It's like wiping your arse with silk, I love it." - The Merovingian
    TeraBitAuthor
    Up and Coming
    July 23, 2013
    "zerowaitstate" wrote:

    Is that an old Light wave model ....? seems age since i have seen that little beasty,

    Keep up the good work, i think at the end of the day raytracing will provide the nicest eye candy in VR.


    Yup, Lightwave Bot. I have LW version 11.5, but still have the model files from earlier incarnations. :D

    Not much going on with the code at the moment, as Kids are off on Summer Holiday... Ah well. :lol:
    Regards, Lee Page TeraBit Software
    Up and Coming
    July 23, 2013
    Hi TeraBit,
    I am very interested in your project. but when I try to run it I immediately get a "External component has thrown an exception" error.
    any idea what is wrong with my system ( running 64bit Win7, i7 X980 4Ghz with 2x 7970 crossfire'd ) ?
    any components that I have to install ?
    TeraBitAuthor
    Up and Coming
    July 23, 2013
    "RayJay" wrote:
    Hi TeraBit,
    I am very interested in your project. but when I try to run it I immediately get a "External component has thrown an exception" error.
    any idea what is wrong with my system ( running 64bit Win7, i7 X980 4Ghz with 2x 7970 crossfire'd ) ?
    any components that I have to install ?


    The problem is that it is based on Optix and CUDA. These are unfortunately Nvidia GPU specific, so it will not work for you. :(
    Regards, Lee Page TeraBit Software
    Participant
    October 14, 2013
    I recently bought an NVidia card and just checked this out. Raytracing definitely looks more real to me than raster-based rendering. The effects with the cube are fun. :D The surrounding area looks so real that seeing the waves just 'frozen' is.. strange. I'm not sure if using raytraced rendering for that is even any different compared to raster, though..

    At any rate, this is a fun demo, it's quick and you can virtually peer through prisms and see raytracing in realtime 3D Rift action.
    TeraBitAuthor
    Up and Coming
    May 14, 2014
    I just had a play with this again. After putting together a quick OpenGL demo to test out the new SDK preview, I come to realise that actually, even though I was joining .NET with c++ with CUDA, I still find it easier to get nice looking results with Ray tracing than with OpenGL. 8-)

    Shame there is no cross GPU support on Optix, or I would have built it out more...
    Regards, Lee Page TeraBit Software
    Up and Coming
    May 15, 2014
    You guys should try foveated rendering. I know it's absolutely no use until we have actual eye tracking, but still - it's between 10x and 20x potential performance increase.

    The idea is that you heavily bias ray density distribution between fovea and peripheral vision, exploiting the fact that only small fovea spot has high resolution and the rest of the retina is very low resolution.

    I didn't really implemented any ray tracer but I threw together show-off demo of restoring very significant amount of lost pixels with voronoi filter: https://drive.google.com/file/d/0B2SiUN3fIbxYcUdDWmkzZEhYdEE/edit?usp=sharing The application of this is that you can render periphery at resolution as low as 10 samples per degree squared (7x7 area in DK1) and even lower if necessary, while only rendering fovea at full sample density. Apparent wobbliness can be further reduced with smoothing filters.

    I've implemented pixel restore filter as a post-procesing shader, but in real ray tracer you can implement this as a in-processing function. Voronoi filter is normally very computationally extensive function, but it will resolve itself automatically (without need for explicit implementation) at low cost if you use following approach: for every sample, instead of a pixel draw a small circle (depending on how large you expect your inter-sample spacing) with gradually decreasning theta (extra channel, which will be discarded at image output stage). However for every actual pixel put you have to check against already rendered image for theta value - only put new pixel if it's theta is larger than already rendered theta. Since around every sample theta will gradually decrease, using this check guarantees that you only put certain pixel for current sample if it has the least distance (biggest theta) to the sample in question, thus satisfying Vononoi definition. As for shader, it would just sample 3x3 area (starting with center) and use the color of first non-transparent pixel it could find, done in multiple passes it eventually fills up all missing pixels with nearest sample color. Additionally, you could integrate with previously rendered frames (on low motion levels) those areas that heavily used Voronoi image reconstruction (has small theta) thus decreasing wobbliness and improving perceived quality. If you would sample multiple rays per pixel, Voronoi function would generically resolve to image identical to regular ray intensity integration.

    Implications are obvious - you can render most of the view area at (much) lower than nominal resoltuion without extra aliasing, thus saving a lot of tracing cost (while cost of filling up missing pixels is nearly nonexistent), whereas in resolution- and quality-critical vision area you could use even bigger than usual ray budget. It's not granted that eye tracking is coming, but you should at least give it a try. Even without eye tracking, just biasing sampling resolution towards vision centre would already benefit since you wouldn't normally look at far edges of your screen anyway, you would turn your head so the thing you're looking at would be closer to view centre.
    TeraBitAuthor
    Up and Coming
    May 15, 2014
    Eye tracking would be useful in this I agree. As it stands, I am only casting rays in the inner circle. The black areas have no rays fired at all. It's possible to vary the rays per pixel, but they are already 0 - Black, 1 - Everywhere else. May end up firing only one ray every so many pixels and then interpolating the results. ;)
    Regards, Lee Page TeraBit Software