Split Post Processing

The trick is to realise that the CameraController/Camera remains in its own transform and the POV is updated directly by the rift Plugin relative to the camera location(The rift never updates the cameras transform).

so if you want the rifts location in world space you would add the two vectors together.
ie. cameramanager->camera->WorldLocation + HMDgetOrientationAndPosition->Location.

I beleive HMDOrientationandPosition->Rotation is in world space.

That makes sense, the question is now more along the lines of how to actually feed the two images as a render layer or a post process which is different for each eye.

According to epic this isn't currently possible, but I wonder if there isn't a way around this?

as youve got the location and rotation of the rift in world space you could always have 2 quads positioned in front of the rift
at the correct relative locations and the correct distances and update there transforms every tick.
A bit hacky but it would work.

I take it you've managed to figure out how to get these images updated to Utexture2d's without stalling the game thread?

Im not to clued up on the leap motion but wouldnt it be easier to just have a couple of your own skeletal meshes for the arms and update the bone transforms via an anim blueprint rarther than using some images generated by the leap SDK?

Thanks for the feedback opamp, though its not finger/arm location tracking I'm going for, I've got that working quite well in the plugin

just doing this


gives you this


In addition the plugin already supports real-time image forwarding to UE's Texture2d.


What I'm trying to achieve is a pass-through effect (see initial video). Essentially it gives the Rift AR capabilities by allowing you to see the world around you in IR. Its absolutely good enough for typing on a keyboard or drinking with the rift on. A very compelling experience, which is currently working in unity because they just render straight to the cameras and have easy render layer support, whereas the UE Rift implementation is much weirder (opaque).

Maybe 2 well placed 3d quads might work, if they can be somehow be applied to only the specific eye and if it can be passed through rendering before everything else (so you can overlay 3d objects on top of the real world). I worry that this way will be hard to calibrate for a proper 1:1 control even if you can do split-eye feeding in the vanilla UE engine and why rendering a full 2d texture directly in each eye before the scene would be ideal.

Is there any way to flag an object so that it is only left/right eye rendered? If not, does the rendering pipeline have any flags to indicates which eye is being rendered?

"getnamo" wrote:
Thanks for the feedback opamp, though its not finger/arm location tracking I'm going for, I've got that working quite well in the plugin...

As Ive spent the evening bashing my head against a wall in regards to one of my own projects I thought I'd give myself a break and have a go at a per-eye postprocessing shader.

The below shader should get you started although it will only look correct in the rift.

http://i.imgur.com/AlGoWIo.png

"andrewtek" wrote:
Is there any way to flag an object so that it is only left/right eye rendered? If not, does the rendering pipeline have any flags to indicates which eye is being rendered?

I dont believe so,

There is'nt any sort of camera layer system with UE4.
The only one that exists is bOwnerNoSee. I really think they need to implement something like that.
I use to love the system they've got with Unity as all sorts of cool tricks where possible.

As for flags for eye rendering you'd have to look into the unreal plugin source code(something I really havnt had the time to do). But I doubt it.

"opamp" wrote:

The below shader should get you started although it will only look correct in the rift.
...

Thanks for the blueprint. That is very cool! Something like this could definitely be used to do some interesting things. I tried this trick to create a material that would put a red texture in the right eye, and a gray-scale version of that texture in the left eye. The result was interesting. The brain combines the resulting colors for a grayish/red texture. I could definitely see some inversion puzzle mechanic uses for this.

Getnamo, would you be willing to share the steps you took to integrate with Leap Motion? Thanks!

"andrewtek" wrote:

Thanks for the blueprint. That is very cool! Something like this could definitely be used to do some interesting things. I tried this trick to create a material that would put a red texture in the right eye, and a gray-scale version of that texture in the left eye. The result was interesting. The brain combines the resulting colors for a grayish/red texture. I could definitely see some inversion puzzle mechanic uses for this.

I didnt even think to try this on a surface material.
I just tried this with 2 extremly different textures and the effect was very strange.
Probably not a good thing to look at for too long!

I wonder if anything interesting can be done with normal maps?

P.S Just be aware that you are only rendering the oposite half of each texture(thats why i tiled them horizontally in my example).
To do the postprocess correctly I believe you would start with 2 eye textures of the correct size and use a ScreenAlignPixeltoPixelUV node.