Forum Discussion
Experimental changes in VR camera control
Happy New Year to everybody, btw!
Ok, the first cut is in GitHub:
https://github.com/Oculus-VR/UnrealEngine/tree/4.7-np
This is 4.7-preview with re-factored plugin and improved (I hope) camera control. I haven't added new Blueprint functions yet (like raw accel data or player's metrics, still in work). I am going to document all the changes (documentation is long due anyway)...
So, briefly. I've added PlayerController.bFollowHmd property, which is by default set to 'true' (for compatibility reasons). When it is set in 'true' then the behavior is the same as it was before: HMD drives PlayerController. You can either set it to 'false' or just don't bother, it will be set to false automatically, once you use PlayerCameraManager or CameraComponent approach.
The main approach you guys care the most (I think) is to enable HMD via CameraComponent. This class now has two additional boolean properties - bFollowHmdOrientation and bFollowHmdPosition. In addition, the scale of the CameraComponent now will be a scale of the position change / ipd. The bigger scale is set, the smaller world is perceived. Camera's origin is located in-between eyes, i.e. IPD will be applied at the camera's origin, half of IPD in positive Y-axis direction and another half in negative Y-axis direction (in camera coordinate system).
Besides CameraComponent, the PlayerCameraManager also has boolean properties bFollowHmdOrientation and bFollowHmdPosition. Thus, any camera attached to this PlayerCameraManager via SetViewTarget method will follow the HMD orientation and/or position.
In addition, the HeadMountedDisplay::GetOrientationAndPosition method now has two additional boolean parameters: bUseOrienationForPlayerCamera and bUsePositionForPlayerCamera (by default - both a set to 'false'). Set these to true, if you use this method to get HMD orientation/position to change the camera orientation / position manually. This is necessary to correctly calculate delta orientation/position for timewarp later on a render thread. However, there is no way to specify the proper scale to position in this case (should I add one more parameter to this method? With default value vector(1,1,1)? Probably, will think about this.)
So, if you, guys, have a look on these changes and try them I'd be grateful. I'll meet with Epic Games guys later this month and we will discuss these changes (however, Epic may suggest different approaches, different names for function/parameters, etc), so these changes are not final in any sense. Your opinion will be very important and will be considered in that discussion.
Ok, the first cut is in GitHub:
https://github.com/Oculus-VR/UnrealEngine/tree/4.7-np
This is 4.7-preview with re-factored plugin and improved (I hope) camera control. I haven't added new Blueprint functions yet (like raw accel data or player's metrics, still in work). I am going to document all the changes (documentation is long due anyway)...
So, briefly. I've added PlayerController.bFollowHmd property, which is by default set to 'true' (for compatibility reasons). When it is set in 'true' then the behavior is the same as it was before: HMD drives PlayerController. You can either set it to 'false' or just don't bother, it will be set to false automatically, once you use PlayerCameraManager or CameraComponent approach.
The main approach you guys care the most (I think) is to enable HMD via CameraComponent. This class now has two additional boolean properties - bFollowHmdOrientation and bFollowHmdPosition. In addition, the scale of the CameraComponent now will be a scale of the position change / ipd. The bigger scale is set, the smaller world is perceived. Camera's origin is located in-between eyes, i.e. IPD will be applied at the camera's origin, half of IPD in positive Y-axis direction and another half in negative Y-axis direction (in camera coordinate system).
Besides CameraComponent, the PlayerCameraManager also has boolean properties bFollowHmdOrientation and bFollowHmdPosition. Thus, any camera attached to this PlayerCameraManager via SetViewTarget method will follow the HMD orientation and/or position.
In addition, the HeadMountedDisplay::GetOrientationAndPosition method now has two additional boolean parameters: bUseOrienationForPlayerCamera and bUsePositionForPlayerCamera (by default - both a set to 'false'). Set these to true, if you use this method to get HMD orientation/position to change the camera orientation / position manually. This is necessary to correctly calculate delta orientation/position for timewarp later on a render thread. However, there is no way to specify the proper scale to position in this case (should I add one more parameter to this method? With default value vector(1,1,1)? Probably, will think about this.)
So, if you, guys, have a look on these changes and try them I'd be grateful. I'll meet with Epic Games guys later this month and we will discuss these changes (however, Epic may suggest different approaches, different names for function/parameters, etc), so these changes are not final in any sense. Your opinion will be very important and will be considered in that discussion.
49 Replies
Replies have been turned off for this discussion
- That sounds pretty good.
I think it is reasonable to assume that if the developer is managing the position of the camera, they can take care of handling scale changes. In most use cases it should not matter anyway, because if you want to explicitly snap the camera to a specific position then you usually do not want any scale taken into account, you just want the camera to be exactly there (assuming the camera will continue to track from that position and not jump back to its "normal" position the next frame). And the developer is the one who changed the scale in the first place, and only the developer knows why or how it should be used in other systems. "owenwp" wrote:
That sounds pretty good.
I think it is reasonable to assume that if the developer is managing the position of the camera, they can take care of handling scale changes. In most use cases it should not matter anyway, because if you want to explicitly snap the camera to a specific position then you usually do not want any scale taken into account, you just want the camera to be exactly there. And the developer is the one who changed the scale in the first place, and only the developer knows why or how it should be used in other systems.
I need to know the exact scale the developer applied, it will be used to correct the position later on a render thread to reduce latency. Anyway, I've added one more optional parameter (PositionScale) to GetOrientationAndPosition function. By default it is (1.f, 1.f, 1.f). Note, I think, GetOrientationAndPosition should return ALREADY scaled position using the provided scale to avoid confusion, so there is no need to apply that scale again manually, and the same scale will be applied on renderthread, if the parameter bUsePositionForPlayerCamera was specified.- Added GetRawSensorData method to HeadMountedDisplay (blueprint & C++). It returns the following:
FVector Accelerometer; // Acceleration reading in m/s^2.
FVector Gyro; // Rotation rate in rad/s.
FVector Magnetometer; // Magnetic field in Gauss.
float Temperature; // Temperature of the sensor in degrees Celsius.
float TimeInSeconds; // Time when the reported IMU reading took place, in seconds.
It is in the branch https://github.com/Oculus-VR/UnrealEngine/tree/4.7-np As before, the disclaimer, that these changes are experimental, and we will discuss them with Epic guys next or next after next week. "artyom17" wrote:
Added GetRawSensorData method to HeadMountedDisplay (blueprint & C++). It returns the following:
FVector Accelerometer; // Acceleration reading in m/s^2.
FVector Gyro; // Rotation rate in rad/s.
FVector Magnetometer; // Magnetic field in Gauss.
float Temperature; // Temperature of the sensor in degrees Celsius.
float TimeInSeconds; // Time when the reported IMU reading took place, in seconds.
It is in the branch https://github.com/Oculus-VR/UnrealEngine/tree/4.7-np As before, the disclaimer, that these changes are experimental, and we will discuss them with Epic guys next or next after next week.
Doing good work artyom, this and the VRPreview mode is going to make 4.7 a really nice productivity/flexibility boost.- Thank you! These changes sound like a very good step towards making the integration more flexible and easier to customize.
- Hey,
I've added a couple more BP/C++ functions. Two of them GetScreenPercentage/SetScreenPercentage - the analog of 'hmd sp xxx' console command.
The third one - GetUserProfile, it returns a structure with the following members:
C++:
struct UserProfile
{
FString Name;
FString Gender;
float PlayerHeight; // Height of the player, in meters
float EyeHeight; // Height of the player's eyes, in meters
float IPD; // Interpupillary distance, in meters
FVector2D EyeToNeckDistance; // Eye-to-neck distance, X - horizontal, Y - vertical, in meters
TMap<FString, FString> ExtraFields; // extra fields in name / value pairs.
};
Blueprint is similar, however, instead of TMap I had to use TArray of FString pairs.
https://github.com/Oculus-VR/UnrealEngine/tree/4.7-np "artyom17" wrote:
FVector2D EyeToNeckDistance; // Eye-to-neck distance, X - horizontal, Y - vertical, in meters
Cool! I'm guessing this is used for the head-neck model when camera tracking is lost, but in my case it will be super useful to fine-tune the navigation method i developed which makes use of the HMD positional tracking:
https://forums.oculus.com/viewtopic.php?f=32&t=18422
The problem is i don't think many users will measure and enter these values manually on the Advanced Configuration tab.. Are you planning to update the user calibration process in order to calculate and store these values semi-automatically? Make the users rotate their heads while staying still for instance?"PatimPatam" wrote:
"artyom17" wrote:
FVector2D EyeToNeckDistance; // Eye-to-neck distance, X - horizontal, Y - vertical, in meters
Cool! I'm guessing this is used for the head-neck model when camera tracking is lost, but in my case it will be super useful to fine-tune the navigation method i developed which makes use of the HMD positional tracking:
https://forums.oculus.com/viewtopic.php?f=32&t=18422
This also can be used to calculate the position of the 'neck origin' from the camera position (which is "eyes position"). I am just thinking: shouldn't this be in unreal units rather than in meters?"PatimPatam" wrote:
The problem is i don't think many users will measure and enter these values manually on the Advanced Configuration tab.. Are you planning to update the user calibration process in order to calculate and store these values semi-automatically? Make the users rotate their heads while staying still for instance?
Honestly, I don't the answer on this question. The importance of the head model is much lower now, since it is implied that most of the time you have positional tracking (well.... not on GearVR, though! but calibration of head model on GearVR is not possible, due to lack of the... positional tracking!). So, hard to tell."artyom17" wrote:
"PatimPatam" wrote:
Cool! I'm guessing this is used for the head-neck model when camera tracking is lost, but in my case it will be super useful to fine-tune the navigation method i developed which makes use of the HMD positional tracking:
https://forums.oculus.com/viewtopic.php?f=32&t=18422
This also can be used to calculate the position of the 'neck origin' from the camera position (which is "eyes position").
Yep that's exactly what i wanted it for, i think controlling your avatar using your torso/neck-origin feels very intuitive, and also helps reducing simulation sickness."artyom17" wrote:
I am just thinking: shouldn't this be in unreal units rather than in meters?
Well i think it's not a big problem if it's not. I guess if we have the player height, IPD, etc in meters it makes sense to keep everything the same.."artyom17" wrote:
"PatimPatam" wrote:
The problem is i don't think many users will measure and enter these values manually on the Advanced Configuration tab.. Are you planning to update the user calibration process in order to calculate and store these values semi-automatically? Make the users rotate their heads while staying still for instance?
Honestly, I don't the answer on this question. The importance of the head model is much lower now, since it is implied that most of the time you have positional tracking (well.... not on GearVR, though! but calibration of head model on GearVR is not possible, due to lack of the... positional tracking!). So, hard to tell.
I understand, personally i think it would be the way to go since it would also help reducing sim-sickness in situations where the tracking is lost, and it would allow to determine precise neck position which can be very useful (like in my case).
It could be as simple as "now look at the ball to your right" and "now look at the ball on top", and only once per user like the IPD measurement.
So yeah for what it's worth you have my vote to implement it :-P
Thanks!- Any chance that the player profile could fill out information not entered using the information that is entered via a lookup table?
