Why is the UE4 Oculus Plugin still so broken (4.8.1)?

Very sorry about this. Let me see what I can do.

Well. Sorry to see your frustration, I understand it. Fortunately or unfortunately, VR in UE4 is not only Oculus anymore. Epic is working on increasing the number of supported HMDs. Therefore it is becoming harder and harder to introduce Oculus-specific changes into core functionality.

I'll try to respond item by item.
1, 2 and 4. Camera. As you know, I tried my best to introduce changes into CameraManager to be able to get/set real coordinates of the camera (in 4.7 experimental branch (viewtopic.php?f=60&t=19145&e=1), including 'Follow Hmd Position' option). Unfortunately, those changes were banned by Epic and I can't do much about it. They are working on alternative solution for 4.9. I have yet to see the progress, though.

3. This is tough one. There is still on going internal discussion whether we need to re-enable vsync 0 functionality or not. The risk is that devs might release apps with vsync off and this would cause 'bad VR experience'.
As an alternative, I could offer to run your game with -emulatestereo command line option for benchmarking.
We also working on adding a built-in performance HUD into our Runtime. You already can try it with Oculus version of 4.8 from github. Or you can integrate this commit into your branch: https://github.com/Oculus-VR/UnrealEngi ... 9c8f2288bf) and 0.6.0.1 RT.

4. I think I replied to it already, but I can do it again. All the positions returned by the GetPositionalTrackingCameraProperties are relative to the tracking volume. The same coordinate space as you get from GetCurrentOrientationAndPosition. Thus you must apply the same transforms to the results of the function. There is a C++ function that uses this call - DrawDebugTrackingCameraFrustum, I'll try to make a blueprint for demoing it.

5. VR Preview - I'll try to repro the issue, but the last time I tried it, it worked just fine....

"artyom17" wrote:
Well. Sorry to see your frustration, I understand it. Fortunately or unfortunately, VR in UE4 is not only Oculus anymore. Epic is working on increasing the number of supported HMDs. Therefore it is becoming harder and harder to introduce Oculus-specific changes into core functionality.

This is understandable, but unfortunate. How are you dealing with this complication on the Unity side? Is there any way that these processes can be carried over to UE?

"artyom17" wrote:

3. This is tough one. There is still on going internal discussion whether we need to re-enable vsync 0 functionality or not. The risk is that devs might release apps with vsync off and this would cause 'bad VR experience'.
As an alternative, I could offer to run your game with -emulatestereo command line option for benchmarking.
We also working on adding a built-in performance HUD into our Runtime. You already can try it with Oculus version of 4.8 from github. Or you can integrate this commit into your branch: https://github.com/Oculus-VR/UnrealEngi ... 9c8f2288bf) and 0.6.0.1 RT.

Thanks Artyom! That is really cool of you. But, you cannot run all of our applications for us. Not to mention, benchmarking involves toggling functionality until you find the performance bottlenecks. How about simply rejecting any application that sets this flag from being available in Oculus Share or the Oculus Store?

Well, I meant, you can just run your app with the -emulatestereo command line option. :) This is not the same as running with Oculus, sure, but might be somewhat useful.
You also can use 'profile gpu' that should work regardless to Vsync on or off.

"artyom17" wrote:
As an alternative, I could offer to run your game with -emulatestereo command line option for benchmarking.

"artyom17" wrote:
Well, I meant, you can just run your app with the -emulatestereo command line option. :)

HAHA - I totally misread. LOL. Thanks!

Any clue as to whether the relationship between Epic/UE4 and Oculus will resume its previous awesomeness? Are you working with other HMD developers to come up with common solutions? Probably not allowed to say; but I thought I would ask.

EDIT: Also, thanks to Spyro for enumerating these issues here. As a UE4 developer, I want to see UE continue to be a quality VR development platform.

I can't say our relationship with Epic is worse, not at all. I am saying that Epic guys have more stuff to worry about than it was before. At Oculus we can introduce any changes we want for our particular device, however, Epic should worry about adopting those changes for multiple devices (if we ask them to integrate the changes into main UE). We could make some Oculus-specific changes (like, 4.8 introduced OculusLibrary where the Oculus-specific functionality was moved to). But then the game won't work the same way with other HMDs.
We can also introduce changes in our own branch in github. But again, we don't want to create a completely separate branch with different functionality since it won't do any good in a long run.

Hi Artynom17,

first of all, many thanks for your quick, helpful and detailed reaction, really appreciate that.

"artyom17" wrote:
Well. Sorry to see your frustration, I understand it. Fortunately or unfortunately, VR in UE4 is not only Oculus anymore. Epic is working on increasing the number of supported HMDs. Therefore it is becoming harder and harder to introduce Oculus-specific changes into core functionality.

I could (at least from a business standpoint) understand this if their stuff would work as expected....

"artyom17" wrote:
As you know, I tried my best to introduce changes into CameraManager to be able to get/set real coordinates of the camera (in 4.7 experimental branch (viewtopic.php?f=60&t=19145&e=1), including 'Follow Hmd Position' option). Unfortunately, those changes were banned by Epic and I can't do much about it.

This is - pretty frustrating. For all sides...

"artyom17" wrote:

They are working on alternative solution for 4.9. I have yet to see the progress, though.

Great. Why accepting merge requests which help to fix a non-complete implementation if you can just release a broken major version? :roll:

"artyom17" wrote:

There is still on going internal discussion whether we need to re-enable vsync 0 functionality or not. The risk is that devs might release apps with vsync off and this would cause 'bad VR experience'.

I understand the intention (I really do, VR without V-Sync is hell) but we simply need this for profiling. Your upcoming alternatives may be a big help here, sure, but maybe you could just display a hardcoded, red warning message through your composer if V-Sync is disabled.

"artyom17" wrote:

4. I think I replied to it already, but I can do it again. All the positions returned by the GetPositionalTrackingCameraProperties are relative to the tracking volume. The same coordinate space as you get from GetCurrentOrientationAndPosition. Thus you must apply the same transforms to the results of the function. There is a C++ function that uses this call - DrawDebugTrackingCameraFrustum, I'll try to make a blueprint for demoing it.

Hmm, I just can't get this to work, sorry. I have to admit, that I'm not really experienced in matrix transformations and all this stuff. I'm afraid I am more the lookAt-Guy ;)

What exactly do you mean by "relative to the tracking volume"? Isn't this the same as the local space of the HMD itself (which should be the same as the location of the playerCameraManager)? What exactly do I have to do with this coordinates to make the tracker show up at the right place in front of my pawn's camera?

Background:
I would love to build some sort of 'chaperone' system, like DrashVR did in his fantastic Titans Of Space. If the user gets too close to the tracking volume boundaries I would like to fade-in some sort of glowing grid (DrashVR used circles along the sides of the tracking frustum which radii growed up gradually). This is IHMO a way better solution than reacting to a non-valid tracking position when it's too late already. IHMO this should be integrated in the composer as well because getting outside the tracking volume is highly uncomfortable to the user, regardless of the experience.

(If something like this planned anyway for the near future please give me an hint. It takes me ages to create something myself and it would be very frustrating if this will be released in the upcoming weeks anyway... :? )

"artyom17" wrote:

5. VR Preview - I'll try to repro the issue, but the last time I tried it, it worked just fine....

Just open a few tabs and then try again... ;)

Actually, I forgot to mention that there are some 'backdoor' methods left from the Experimental branch. I've just renamed those and moved to OculusLibrary.

Here is the list of those:

	/**
	 * Grabs the current orientation and position for the HMD.  If positional tracking is not available, DevicePosition will be a zero vector
	 *
	 * @param DeviceRotation	(out) The device's current rotation
	 * @param DevicePosition	(out) The device's current position, in its own tracking space
	 * @param NeckPosition		(out) The estimated neck position, calculated using NeckToEye vector from User Profile. Same coordinate space as DevicePosition.
	 * @param bUseOrienationForPlayerCamera	(in) Should be set to 'true' if the orientation is going to be used to update orientation of the camera manually.
	 * @param bUsePositionForPlayerCamera	(in) Should be set to 'true' if the position is going to be used to update position of the camera manually.
	 * @param PositionScale		(in) The 3D scale that will be applied to position.
	 */
	UFUNCTION(BlueprintPure, Category="Input|OculusLibrary")
	static void GetPose(FRotator& DeviceRotation, FVector& DevicePosition, FVector& NeckPosition, bool bUseOrienationForPlayerCamera = false, bool bUsePositionForPlayerCamera = false, const FVector PositionScale = FVector::ZeroVector);

	/**
	* Reports raw sensor data. If HMD doesn't support any of the parameters then it will be set to zero.
	*
	* @param Accelerometer	(out) Acceleration reading in m/s^2.
	* @param Gyro			(out) Rotation rate in rad/s.
	* @param Magnetometer	(out) Magnetic field in Gauss.
	* @param Temperature	(out) Temperature of the sensor in degrees Celsius.
	* @param TimeInSeconds	(out) Time when the reported IMU reading took place, in seconds.
	*/
	UFUNCTION(BlueprintPure, Category = "Input|OculusLibrary")
	static void GetRawSensorData(FVector& Accelerometer, FVector& Gyro, FVector& Magnetometer, float& Temperature, float& TimeInSeconds);

	/**
	* Returns current user profile.
	*
	* @param Profile		(out) Structure to hold current user profile.
	* @return (boolean)	True, if user profile was acquired.
	*/
	UFUNCTION(BlueprintPure, Category = "Input|OculusLibrary")
	static bool GetUserProfile(FHmdUserProfile& Profile);

	/**
	* Sets 'base rotation' - the rotation that will be subtracted from
	* the actual HMD orientation.
	* Sets base position offset (in meters). The base position offset is the distance from the physical (0, 0, 0) position
	* to current HMD position (bringing the (0, 0, 0) point to the current HMD position)
	* Note, this vector is set by ResetPosition call; use this method with care.
	* The axis of the vector are the same as in Unreal: X - forward, Y - right, Z - up.
	*
	* @param Rotation			(in) Rotator object with base rotation
	* @param BaseOffsetInMeters (in) the vector to be set as base offset, in meters.
	* @param Options			(in) specifies either position, orientation or both should be set.
	*/
	UFUNCTION(BlueprintCallable, Category = "Input|OculusLibrary")
	static void SetBaseRotationAndBaseOffsetInMeters(FRotator Rotation, FVector BaseOffsetInMeters, EOrientPositionSelector::Type Options);

	/**
	* Returns current base rotation and base offset.
	* The base offset is currently used base position offset, previously set by the
	* ResetPosition or SetBasePositionOffset calls. It represents a vector that translates the HMD's position
	* into (0,0,0) point, in meters.
	* The axis of the vector are the same as in Unreal: X - forward, Y - right, Z - up.
	*
	* @param OutRotation			(out) Rotator object with base rotation
	* @param OutBaseOffsetInMeters	(out) base position offset, vector, in meters.
	*/
	UFUNCTION(BlueprintPure, Category = "Input|OculusLibrary")
	static void GetBaseRotationAndBaseOffsetInMeters(FRotator& OutRotation, FVector& OutBaseOffsetInMeters);

	/**
	 * Turns on/off default PlayerController's behavior to follow HMD orientation/position
	 */
	UFUNCTION(BlueprintCallable, Category = "Input|OculusLibrary")
	static void EnablePlayerControllerFollowHmd(bool bEnable);

	/**
	 * Returns true if PlayerController follows HMD orientation/position. False, otherwise.
	 */
	UFUNCTION(BlueprintPure, Category = "Input|OculusLibrary")
	static bool IsPlayerControllerFollowHmdEnabled();

	/**
	 * Controls how PlayerCameraManager follows HMD. Note, this works for any active PlayerCameraManager
	 * with bFollowHmdOrientation property set to true. 
	 *
	 * @param bFollowHmdOrientation	(in) True if camera's orientation should be updated by most recent HMD orientation.
	 * @param bFollowHmdPosition	(in) Whether the camera's position should be updated by most recent HMD orientation or not.
	 */
	UFUNCTION(BlueprintCallable, Category = "Input|OculusLibrary")
	static void EnablePlayerCameraManagerFollowHmd(bool bFollowHmdOrientation, bool bFollowHmdPosition);

	/**
	 * Returns current settings for PlayerCameraManager's overrides for following HMD orientation and position.
	 *
	 * @param bFollowHmdOrientation	(out) True if camera's orientation should be updated by most recent HMD orientation.
	 * @param bFollowHmdPosition	(out) Whether the camera's position should be updated by most recent HMD orientation or not.
	 */
	UFUNCTION(BlueprintPure, Category = "Input|OculusLibrary")
	static void GetPlayerCameraManagerFollowHmd(bool& bFollowHmdOrientation, bool& bFollowHmdPosition);

	/**
	 * Sets 'base rotation' - the rotation that will be subtracted from
	 * the actual HMD orientation.
	 * The position offset might be added to current HMD position,
	 * effectively moving the virtual camera by the specified offset. The addition
	 * occurs after the HMD orientation and position are applied.
	 *
	 * @param BaseRot			(in) Rotator object with base rotation
	 * @param PosOffset			(in) the vector to be added to HMD position.
	 * @param Options			(in) specifies either position, orientation or both should be set.
	 */
	UFUNCTION(BlueprintCallable, Category = "Input|OculusLibrary", meta = (DeprecatedFunction, DeprecationMessage = "A hack, proper camera positioning should be used"))
	static void SetBaseRotationAndPositionOffset(FRotator BaseRot, FVector PosOffset, EOrientPositionSelector::Type Options);

	/**
	 * Returns current base rotation and position offset.
	 *
	 * @param OutRot			(out) Rotator object with base rotation
	 * @param OutPosOffset		(out) the vector with previously set position offset.
	 */
	UFUNCTION(BlueprintCallable, Category = "Input|OculusLibrary", meta = (DeprecatedFunction, DeprecationMessage = "A hack, proper camera positioning should be used"))
	static void GetBaseRotationAndPositionOffset(FRotator& OutRot, FVector& OutPosOffset);

The most interesting for you could be: GetPose, EnablePlayerCameraManagerFollowHmd, GetPlayerCameraManagerFollowHmd. These methods basically re-introduce the bFollowHmdPosition property for the CameraManager (not for the CameraComponent, though). You may try to use those and let me know if you have any issues with them.
You may use EnablePlayerControllerFollowHmd to disable HMD following by the PlayerController (to disable the default behavior).

Hmm, I just can't get this to work, sorry. I have to admit, that I'm not really experienced in matrix transformations and all this stuff. What exactly do you mean by "relative to the tracking volume"? Isn't this the same as the local space of the HMD itself (which should be the same as the location of the playerCameraManager)? What exactly do I have to do with this coordinates to make the tracker show up at the right place in front of my pawn's camera?

Yes, this is what I meant. It should be in the same coord space as the HMD's position. I understand that it is not trivial to make that function work, will think how to make it easier to use. I'll show you a blueprint once I have it.

However, if you just need to detect that you are out of the tracking space, you can use HasValidTrackingPosition function call (in Blueprint). This returns false if you are out of the tracking frustum.

And, the good news: I've reproduced the issue with VR Preview. The issue is in wrong "world" being picked up by the plugin (the "Editor" one instead of "PIE" one). Need to think how to resolve this w/o much changes in interfaces.

"artyom17" wrote:
Yes, this is what I meant. It should be in the same coord space as the HMD's position. I understand that it is not trivial to make that function work, will think how to make it easier to use. I'll show you a blueprint once I have it.

Would love to see your BP. A boolean is a bit jarring. Knowing the distance from frustum edge allows us to fade to a "holodeck" style "safe space".

"artyom17" wrote:
And, the good news: I've reproduced the issue with VR Preview. The issue is in wrong "world" being picked up by the plugin (the "Editor" one instead of "PIE" one). Need to think how to resolve this w/o much changes in interfaces.

Excellent! Thanks Artyom!