Direct HMD Access with Client-Distortion? (Raytracing)

"kojack" wrote:
I was doing something a little like that:
-snip-
The ground is conventional rendering. The green sphere is ray tracing through a cube mesh frame with a sphere intersect equation. The white sphere is ray marching through a cube mesh frame with a distance field. Both generate valid z buffer values so they can intersect with other conventional objects in the scene. Distorting the cube mesh (just 12 triangles) will change the shape of the ray traced / ray marched contents too (stretch the cube and it will draw an ellipse, etc).

Very cool, that's basically the foundation of what I want to accomplish. I am wondering about the differences between using intersection formulas vs ray marching. I've seen several different conclusions about the subject. Which one is faster in your implementation?
How I understand it ray marching is just a fancy name for an iterative method of calculating the intersection formula, plus you don't have to worry about multiple solutions.

"thewhiteambit" wrote:
Pentile AB.png

with raytracing no complicated rotation is needed. you can calculate suppicture A (just green) with just green rays, and then an interleaved (blue/green) one as layer B. As I mentioned it is more like a BRG 4:8:4 or GBR 8:4:4 not exactly YUV 4:2:2 - since it is not chromatic etc and has more BitPerPixel - even BRG 32:16:16 is useful for floats. chromatic aberration is done here also. this techniques could basically also apply for scanline. one can imagine a shader writing to the warped texture doing some multisampling on the fly. this could be a good compromise between convenient scanline techniques and picture quality. you can also implement a stencil to have only samples taken where needed. then one can also calculate the subsample with the position it takes in the warped picture. hey john, something for you in here? ;)

Well, that depends you want to use the hardware rasterizer or not (I do). I thought about that approach as well and I think it's a viable one. The reason I like the rotational solution so much is that you don't have to rasterize to images. So you decrease the smallest possible primitive that can be rasterized efficiently by two. Of course that would make figuring the colors more difficult and I haven't figured out how to do that yet... Using stencil masks might be a thing yes.

"ElectricMucus" wrote:

Well, that depends you want to use the hardware rasterizer or not (I do). I thought about that approach as well and I think it's a viable one. The reason I like the rotational solution so much is that you don't have to rasterize to images. So you decrease the smallest possible primitive that can be rasterized efficiently by two. Of course that would make figuring the colors more difficult and I haven't figured out how to do that yet... Using stencil masks might be a thing yes.

No this should not be done using stencil, because it needs passes then again. I would render full color and pick just the one from the position. But this does not solve still too many samples are needed. I have some thoughts in direction of "irregular shadow mapping" approach to solve. Again this is far from being all day practical scanlining :/

I can't see anything in the SDK that precludes using your own distortion with direct hmd. Instead of theorizing about it, try it yourself. As for the actual distortion process, one way of going about it would be to get the distortion meshes, and generate sample points on them to use for primary ray directions.
Also, you don't need to use the direct HMD mode to achieve 75hz refresh rate... I don't know where you get these arbitrary restrictions from, but again, try it yourself and you will see what works and what doesn't.

"ElectricMucus" wrote:

"kojack" wrote:
I was doing something a little like that:
-snip-
The ground is conventional rendering. The green sphere is ray tracing through a cube mesh frame with a sphere intersect equation. The white sphere is ray marching through a cube mesh frame with a distance field. Both generate valid z buffer values so they can intersect with other conventional objects in the scene. Distorting the cube mesh (just 12 triangles) will change the shape of the ray traced / ray marched contents too (stretch the cube and it will draw an ellipse, etc).

Very cool, that's basically the foundation of what I want to accomplish. I am wondering about the differences between using intersection formulas vs ray marching. I've seen several different conclusions about the subject. Which one is faster in your implementation?
How I understand it ray marching is just a fancy name for an iterative method of calculating the intersection formula, plus you don't have to worry about multiple solutions.

Yep, ray marching is iterating through space. It's often used with a distance field, which is a formula that gives the distance to the nearest surface for any point in space. The distance field can then optimise the marching, if the nearest surface is 5 units away, then you can jump the ray 5 units forward for the next test. This makes empty areas very quick, with the marching narrowing down to more detailed steps as it approaches objects.

There's a couple of good features for ray marched distance fields compared to pure ray tracing with intersect algorithms. The marchng style can do more organic shapes by using multiple overlapped distance fields, which is harder with intersection. It also naturally gives surface normals by sampling points around the march point to test the direction away from the surface.
On the down side, rays that go near an object but miss it are slowed down (the closer you get, the smaller the steps it takes).

Have a look at shadertoy.com website. Most of the demos on there are ray marching distance fields (with source). Some very cool effects.

You could use a volumetric texture as a distance field too, letting the texture hardware filter the result. Kind of voxel render.

I don't want to take it too far off topic, but I just found my shaders for the two spheres.
Probably very inefficient code, it was a quick hack to show my class.

Both take a cube mesh with vertex positions in the 0-1 range)

Ray tracing:

void raytrace_vs(float4 position : POSITION,
						  out float4 oPosition : POSITION,
						  out float3 oPosPS	   : TEXCOORD0,
						  uniform float4x4 worldViewProj)
{
	oPosition = mul(worldViewProj, position);
	oPosPS = position.xyz;
}


float4 raytrace_ps(float3 pos:TEXCOORD0, uniform float3 cameraPos, uniform float4x4 viewproj, uniform float4 lightPos, out float depth:DEPTH) : COLOR 
{
	float3 cen=float3(0.5,0.5,0.5);
	float radius=0.5;
	float3 rd = normalize(pos-cameraPos);
	float a = dot(rd,rd);
	float b = dot(2*rd, cameraPos-cen);
	float c = dot(cen,cen)+dot(cameraPos,cameraPos)+ -2*dot(cen,cameraPos) - radius * radius;
	if(b*b-4*a*c < 0)
		discard;
	float t = (-b - sqrt(b*b-4*a*c))/2*a;
	float3 hit=cameraPos+rd*t;
	float light=saturate(dot(normalize(hit-cen),normalize(lightPos.xyz-hit)));
	float4 clippos=mul(viewproj, float4(hit,1));
	depth=clippos.z/clippos.w;
	return float4(light,light,light,1);
}

Ray marching:

void raymarch_vs(float4 position : POSITION,
						  out float4 oPosition : POSITION,
						  out float3 oPosPS	   : TEXCOORD0,
						  uniform float4x4 worldViewProj)
{
	oPosition = mul(worldViewProj, position);
	oPosPS = position.xyz;
}

float distanceSphere(float3 pos, float3 centre, float radius)
{
	return length(pos-centre)-radius;
}

float distance(float3 pos)
{
	return distanceSphere(pos, float3(0.5,0.5,0.5), 0.5);
}


float4 raymarchps(float3 pos:TEXCOORD0, uniform float3 cameraPos, uniform float4x4 viewproj, uniform float4 lightPos, out float depth:DEPTH) : COLOR 
{
	float3 p=pos;
	float3 rd = normalize(pos-cameraPos);
	float d=0;
	for(int i=0;i<200;++i)
	{
		d = distance(p);
		if(d<0.001)
			break;	
		if(d>100)
			discard;
		p=p+rd*d;
	}
	float3 normal;
	normal.x = distance(p+float3(0.01,0,0)) - distance(p-float3(0.01,0,0));
	normal.y = distance(p+float3(0,0.01,0)) - distance(p-float3(0,0.01,0));
	normal.z = distance(p+float3(0,0,0.01)) - distance(p-float3(0,0,0.01));
	normal = normalize(normal);
	
	float4 clippos=mul(viewproj, float4(p,1));
	depth=clippos.z/clippos.w;
	return float4(0,1,0,1)*saturate(dot(normal, normalize(lightPos.xyz - p)));
}

"nuclear" wrote:
I can't see anything in the SDK that precludes using your own distortion with direct hmd. Instead of theorizing about it, try it yourself. As for the actual distortion process, one way of going about it would be to get the distortion meshes, and generate sample points on them to use for primary ray directions.
Also, you don't need to use the direct HMD mode to achieve 75hz refresh rate... I don't know where you get these arbitrary restrictions from, but again, try it yourself and you will see what works and what doesn't.

Thanks for your advice to try it myself, I have never even thought about it! I did not even have a look at the manual to find Direct HMD always seems distorted automatic. Also thank you for theorizing about a distortion process that was never a question and archived long time ago. Further thank you for ignoring that windows syncs refresh-rate vsync when using multiple monitors... this post was really very helpful :lol: