Up and Coming

Experimental changes in VR camera control

49 replies
16343 views

Happy New Year to everybody, btw!

Ok, the first cut is in GitHub:
https://github.com/Oculus-VR/UnrealEngine/tree/4.7-np

This is 4.7-preview with re-factored plugin and improved (I hope) camera control. I haven't added new Blueprint functions yet (like raw accel data or player's metrics, still in work). I am going to document all the changes (documentation is long due anyway)...

So, briefly. I've added PlayerController.bFollowHmd property, which is by default set to 'true' (for compatibility reasons). When it is set in 'true' then the behavior is the same as it was before: HMD drives PlayerController. You can either set it to 'false' or just don't bother, it will be set to false automatically, once you use PlayerCameraManager or CameraComponent approach.

The main approach you guys care the most (I think) is to enable HMD via CameraComponent. This class now has two additional boolean properties - bFollowHmdOrientation and bFollowHmdPosition. In addition, the scale of the CameraComponent now will be a scale of the position change / ipd. The bigger scale is set, the smaller world is perceived. Camera's origin is located in-between eyes, i.e. IPD will be applied at the camera's origin, half of IPD in positive Y-axis direction and another half in negative Y-axis direction (in camera coordinate system).

Besides CameraComponent, the PlayerCameraManager also has boolean properties bFollowHmdOrientation and bFollowHmdPosition. Thus, any camera attached to this PlayerCameraManager via SetViewTarget method will follow the HMD orientation and/or position.

In addition, the HeadMountedDisplay::GetOrientationAndPosition method now has two additional boolean parameters: bUseOrienationForPlayerCamera and bUsePositionForPlayerCamera (by default - both a set to 'false'). Set these to true, if you use this method to get HMD orientation/position to change the camera orientation / position manually. This is necessary to correctly calculate delta orientation/position for timewarp later on a render thread. However, there is no way to specify the proper scale to position in this case (should I add one more parameter to this method? With default value vector(1,1,1)? Probably, will think about this.)

So, if you, guys, have a look on these changes and try them I'd be grateful. I'll meet with Epic Games guys later this month and we will discuss these changes (however, Epic may suggest different approaches, different names for function/parameters, etc), so these changes are not final in any sense. Your opinion will be very important and will be considered in that discussion.

This topic has been closed for replies.

Show previous replies

A

artyom17Author

Up and Coming

.... and updated to 0.5.0 SDK.

S

spyro

Up and Coming

Sounds fantastic. Just a few quick questions:

What BP function should I use now to set the position of the player and initial viewing angle ('beam' him behind the wheels of a car for example)? In previous versions, it wasn't possible to really force the player looking in one specific direction in a reliable way.

Will the initial viewing angle of the player start now work as expected (was 90° or 180° off for me in 4.7.3), see https://answers.unrealengine.com/questi ... 0-off.html

What about this strange issue with VR camera rotation (player is standing on rotating platform): https://answers.unrealengine.com/questi ... tform.html

Is 'late latching' included in this build (https://www.oculus.com/blog/optimizing- ... -latching/). If yes, how do I enable it?

Many thanks,
spyro

A

Anonymous

Hi artyom17,

I've gotten your latest NP branch (via a zip file) and I'm finding it doesn't compile.

I'm going to make educated guesses on how to change it but thought I'd give you the heads up.

http://i.imgur.com/B3ExDbz.png

UPDATE:

I just removed the offending Experimental plugins - SteamVRHMD and SimpleHMD. This is for the GearVR Jam.

A

artyom17Author

Up and Coming

hey guys,

I'm on vacation at the moment, will review this topic once I am back next week.

A

artyom17Author

Up and Coming

"spyro" wrote:
What BP function should I use now to set the position of the player and initial viewing angle ('beam' him behind the wheels of a car for example)? In previous versions, it wasn't possible to really force the player looking in one specific direction in a reliable way.

If you use FillowHmdOrientation/Position on CameraComponent or PlayerCameraManager then I'd expect the initial viewing angle should match the orientation of the camera.

"spyro" wrote:
Will the initial viewing angle of the player start now work as expected (was 90° or 180° off for me in 4.7.3), see https://answers.unrealengine.com/questi ... 0-off.html

This should be taken care of by this branch.

"spyro" wrote:
What about this strange issue with VR camera rotation (player is standing on rotating platform): https://answers.unrealengine.com/questi ... tform.html

Same thing, I'd expect. Since the viewing point is currently linked tightly to the camera, I'd expect it to work correctly now. Would be nice if someone could confirm this on real example.

"spyro" wrote:
Is 'late latching' included in this build (https://www.oculus.com/blog/optimizing- ... -latching/). If yes, how do I enable it?

Late latching is not here yet, we are still working on it (since it requires changes in UE4 core, if we don't want big positional latency).

And again, I have to remind you guys, that all of these changes are still experimental and we still don't have the final solution and agreement with Epic guys about how exactly all of this should be handled. However, we agreed that it is unacceptable to leave it as it was before.

C

chaosgrid

Up and Coming

I hope this gets streamlined soon because (correct me if Im wrong) in the standard UEngine release getting the actual HMD position is currently only possible by calling the GetOrientationAndPosition function and is never translated to i.e. the playercameramanager or the camera actor (but orientation is!).

Havent tried your changes but they sound very sensible.

Freeworlds: Tides of War Blue Drift (Underwater Hydra-based Diving Game)

O

owenwp

Up and Coming

Currently the best way (in the release branches) to get the real camera position is to call camera->GetCameraView followed by GEngine->StereoRenderingDevice->CalculateStereoViewOffset for both eyes. This is the only way to get it using the same code path the renderer uses to compute the view matrix, other methods (short of modifying the engine code yourself) would involve doing the math in your game code and the math might change in future versions/other VR platforms. This method should be future-proof.

My code looks like this:


FMinimalViewInfo ViewInfo;
camera->GetCameraView(0, ViewInfo);

FVector StereoViewLocationLeft = ViewInfo.Location;
FVector StereoViewLocationRight = ViewInfo.Location;

if (GEngine->HMDDevice.IsValid() && GEngine->HMDDevice->IsHeadTrackingAllowed())
{
	GEngine->StereoRenderingDevice->CalculateStereoViewOffset(eSSP_LEFT_EYE, ViewInfo.Rotation, GetWorld()->GetWorldSettings()->WorldToMeters, StereoViewLocationLeft);
	GEngine->StereoRenderingDevice->CalculateStereoViewOffset(eSSP_RIGHT_EYE, ViewInfo.Rotation, GetWorld()->GetWorldSettings()->WorldToMeters, StereoViewLocationRight);
}

FVector eyePos = 0.5f * (StereoViewLocationLeft + StereoViewLocationRight);
FQuat eyeRot = ViewInfo.Rotation.Quaternion();