Forum Discussion
New Unity UI + OVR Look-Based Input HOWTO
LookInputModule.jpg
Edit6: version 5 of example project/code:
https://www.dropbox.com/s/g8ptl7w9xdewp ... 5.zip?dl=0
Version removed old Oculus integration and uses Native VR support + 0.1.2 utils so it works with latest runtimes (0.7 or 0.8).
Edit5: version 4 of example project/code (fixed for and tested with Unity 4.6.4f1):
https://www.dropbox.com/s/3bpz5rgimbxdk ... 4.zip?dl=0
This version also adds support for a cursor that scales size with distance so it will always appear the same size but be at the right depth.
Edit4: version 3 of example project/code:
[removed - see updated example above]
Edit3: New version of example project and code posted:
[removed - see updated example above]
Edit2: If you don't have time to read all of this thread and just want the code and a sample project, I uploaded one here:
[removed - see updated example above]
Edit: fixed some issues with the code to handle InputField and also was selecting wrong game object in some cases.
I have been messing around with the new Unity GUI stuff in 4.6 release, and I figured I would share a basic solution to get up and running quickly with a look-based input system. Here is how to get a basic look UI working:
1. Add some kind of UI element to your scene. This will also add a Canvas object and an EventSystem object if you don't already have them in your scene. All UI elements must be parented by a Canvas.
2. On the UI Canvas object, set the Render Mode to World Space
3. Arrange your Canvas in world space and add more UI elements as you would like. To get started, I would just add some buttons.
4. Under the OVRCameraRig->CenterEyeAnchor object, add a regular Unity Camera at position and rotation 0,0,0, and set this camera's Culling Mask to "Nothing" so it won't actually bother rendering anything and waste CPU/GPU cycles. I named it "Look Camera". This camera is just for use by the UI event system. By putting it under the CenterEyeAnchor object, it tracks perfectly with the head. The OVR cameras don't seem to work with the UI system, probably because they utilize render textures and not screen space rendering. This dummy look camera solves that problem.
5. For every UI Canvas object you have in the scene, drag this "Look Camera" object into the "Event Camera" field.
6. create a new script called BasicLookInputModule.cs. Copy in this code:
7. Drag this script onto the EventSystem object in the scene.
8. fix Submit Button Name and Control Axis Name to match what you use in the Input Settings if you don't use Unity defaults.
9. disable or remove Standalone Input Module and Touch Input Module on the Event System object.
That's it. Push play. Look around with headset and you should see it highlight the buttons/sliders/etc. as you are looking around. When you hit the submit button it will trigger that UI element (buttons On Value Changed operations will be called if you set any of those up). Sliders get selected when you click them and can then be manipulated with the axis that was chosen on the BasicLookInputModule component.
I hope someone finds this useful since there isn't much out there on how to do this yet.
ccs
Edit6: version 5 of example project/code:
https://www.dropbox.com/s/g8ptl7w9xdewp ... 5.zip?dl=0
Version removed old Oculus integration and uses Native VR support + 0.1.2 utils so it works with latest runtimes (0.7 or 0.8).
Edit5: version 4 of example project/code (fixed for and tested with Unity 4.6.4f1):
https://www.dropbox.com/s/3bpz5rgimbxdk ... 4.zip?dl=0
This version also adds support for a cursor that scales size with distance so it will always appear the same size but be at the right depth.
Edit4: version 3 of example project/code:
[removed - see updated example above]
Edit3: New version of example project and code posted:
[removed - see updated example above]
Edit2: If you don't have time to read all of this thread and just want the code and a sample project, I uploaded one here:
[removed - see updated example above]
Edit: fixed some issues with the code to handle InputField and also was selecting wrong game object in some cases.
I have been messing around with the new Unity GUI stuff in 4.6 release, and I figured I would share a basic solution to get up and running quickly with a look-based input system. Here is how to get a basic look UI working:
1. Add some kind of UI element to your scene. This will also add a Canvas object and an EventSystem object if you don't already have them in your scene. All UI elements must be parented by a Canvas.
2. On the UI Canvas object, set the Render Mode to World Space
3. Arrange your Canvas in world space and add more UI elements as you would like. To get started, I would just add some buttons.
4. Under the OVRCameraRig->CenterEyeAnchor object, add a regular Unity Camera at position and rotation 0,0,0, and set this camera's Culling Mask to "Nothing" so it won't actually bother rendering anything and waste CPU/GPU cycles. I named it "Look Camera". This camera is just for use by the UI event system. By putting it under the CenterEyeAnchor object, it tracks perfectly with the head. The OVR cameras don't seem to work with the UI system, probably because they utilize render textures and not screen space rendering. This dummy look camera solves that problem.
5. For every UI Canvas object you have in the scene, drag this "Look Camera" object into the "Event Camera" field.
6. create a new script called BasicLookInputModule.cs. Copy in this code:
using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.UI;
using System.Collections;
public class BasicLookInputModule : BaseInputModule {
public const int kLookId = -3;
public string submitButtonName = "Fire1";
public string controlAxisName = "Horizontal";
private PointerEventData lookData;
// use screen midpoint as locked pointer location, enabling look location to be the "mouse"
private PointerEventData GetLookPointerEventData() {
Vector2 lookPosition;
lookPosition.x = Screen.width/2;
lookPosition.y = Screen.height/2;
if (lookData == null) {
lookData = new PointerEventData(eventSystem);
}
lookData.Reset();
lookData.delta = Vector2.zero;
lookData.position = lookPosition;
lookData.scrollDelta = Vector2.zero;
eventSystem.RaycastAll(lookData, m_RaycastResultCache);
lookData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
m_RaycastResultCache.Clear();
return lookData;
}
private bool SendUpdateEventToSelectedObject() {
if (eventSystem.currentSelectedGameObject == null)
return false;
BaseEventData data = GetBaseEventData ();
ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
return data.used;
}
public override void Process() {
// send update events if there is a selected object - this is important for InputField to receive keyboard events
SendUpdateEventToSelectedObject();
PointerEventData lookData = GetLookPointerEventData();
// use built-in enter/exit highlight handler
HandlePointerExitAndEnter(lookData,lookData.pointerCurrentRaycast.gameObject);
if (Input.GetButtonDown (submitButtonName)) {
eventSystem.SetSelectedGameObject(null);
if (lookData.pointerCurrentRaycast.gameObject != null) {
GameObject go = lookData.pointerCurrentRaycast.gameObject;
GameObject newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.submitHandler);
if (newPressed == null) {
// submit handler not found, try select handler instead
newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.selectHandler);
}
if (newPressed != null) {
eventSystem.SetSelectedGameObject(newPressed);
}
}
}
if (eventSystem.currentSelectedGameObject && controlAxisName != null && controlAxisName != "") {
float newVal = Input.GetAxis (controlAxisName);
if (newVal > 0.01f || newVal < -0.01f) {
AxisEventData axisData = GetAxisEventData(newVal,0.0f,0.0f);
ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, axisData, ExecuteEvents.moveHandler);
}
}
}
}
7. Drag this script onto the EventSystem object in the scene.
8. fix Submit Button Name and Control Axis Name to match what you use in the Input Settings if you don't use Unity defaults.
9. disable or remove Standalone Input Module and Touch Input Module on the Event System object.
That's it. Push play. Look around with headset and you should see it highlight the buttons/sliders/etc. as you are looking around. When you hit the submit button it will trigger that UI element (buttons On Value Changed operations will be called if you set any of those up). Sliders get selected when you click them and can then be manipulated with the axis that was chosen on the BasicLookInputModule component.
I hope someone finds this useful since there isn't much out there on how to do this yet.
ccs
105 Replies
Replies have been turned off for this discussion
- Cool, gonna try this out later.
Thanks for the writeup. - Thanks for sharing! Good to see some custom input module code.
- Great post! Thanx : )
- Finally. I've been scouring the internet all night looking for something that works. Thanks for putting this all together with such clear instructions. It's such a relief.
One problem I'm currently facing now is that I can't actually enter any keyboard input into the InputField UI. Do you have any idea how I might go about doing that while still being able to keep what you've made? I've noticed that unity can still read when I press "space" to bring up the VR menu, so I know the keyboard is not disabled completely. - Sorry I haven't tried an input field yet. I'll try that in a minute. There may be something special that needs to happen with keyboard input focus that my basic example doesn't do. I'm only using Submit and Select events in the example code. There are many others to experiment with.
Maybe add the standalone input module back to the Event System object and re-enable? They should be able to work together.
BTW - if you use "Fire1" as the submit button and you haven't changed the Unity default input settings, both mouse click and gamepad button 0 (A on xbox controller) will click buttons, select sliders, etc.
ccs - I fixed a minor bug in the code I posted. It has been corrected in the original post:
Was:
if (newPressed != null) {
eventSystem.SetSelectedGameObject(go);
}
Is now:
if (newPressed != null) {
eventSystem.SetSelectedGameObject(newPressed);
}
I can't seem to get the InputField to work either. It responds to the submit action and the cursor starts blinking so I know it is activating, but the keyboard input is not getting picked up to change the text. - Did you try posting in the Unity 4.6 beta forum about this?
- I figured it out by looking at the StandaloneInputModule code here:
https://gist.github.com/stramit/ce455682b7944bdff0e7
What I was missing was this procedure that handles the update events that need to get passed to the currently selected UI object:
private bool SendUpdateEventToSelectedObject() {
if (eventSystem.currentSelectedGameObject == null)
return false;
BaseEventData data = GetBaseEventData ();
ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
return data.used;
}
I fixed it in the original post. I make a call to this function right at the beginning of the Process() function. This is what passes on the keyboard input events to the selected object.
Keep in mind, this doesn't disable all the key bindings made elsewhere in code, like in the OVR code. You will have to find a way to disable those other scripts from trying to also use the keystroke. OVR SDK has a lot of keyboard shortcuts. - I found something I didn't like too much about the built-in slider/scrollbar axis move handling. It is set at a fixed step of 10%. Unfortunately the step size is not user-settable. So I changed around my Input Module to do custom value setting for Sliders and Scrollbars instead of using the built-in move handlers for these components.
Instead of updating code in original post, I'm just posting a more comprehensive look-based input module in this post. It includes additional features like setting some values to indicate if the input module consumed the button or axis or whether or not the raytracing hit something. This can be useful if you want to multi-purpose a specific axis for other things. I also added a select color. This make it clear which scrollbar/slider is selected when you aren't looking directly at it. You can still manipulate it when you are looking elsewhere with the axis.
To enable the smooth-scrolling sliders and scrollbars, just set useSmoothAxis to true (the default). You can adjust the rate by setting smoothAxisMultiplier to different values.
Here is the more comprehensive look-based input module:
using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.UI;
using System.Collections;
public class LookInputModule : BaseInputModule {
private static LookInputModule _singleton;
public static LookInputModule singleton {
get {
return _singleton;
}
}
public const int kLookId = -3;
public string submitButtonName = "Fire1";
public string controlAxisName = "Horizontal";
public bool useSmoothAxis = true;
public float smoothAxisMultiplier = 0.01f;
// if smooth axis is off - the input is stepped - need an interval
public float steppedAxisStepsPerSecond = 10f;
private bool _guiRaycastHit;
public bool guiRaycastHit {
get {
return _guiRaycastHit;
}
}
private bool _controlAxisUsed;
public bool controlAxisUsed {
get {
return _controlAxisUsed;
}
}
private bool _buttonUsed;
public bool buttonUsed {
get {
return _buttonUsed;
}
}
private PointerEventData lookData;
private Color currentSelectedNormalColor;
private Color currentSelectedHighlightedColor;
public bool useSelectColor = true;
public Color selectColor = Color.blue;
private float nextAxisActionTime;
// use screen midpoint as locked pointer location, enabling look location to be the "mouse"
private PointerEventData GetLookPointerEventData() {
Vector2 lookPosition;
lookPosition.x = Screen.width/2;
lookPosition.y = Screen.height/2;
if (lookData == null) {
lookData = new PointerEventData(eventSystem);
}
lookData.Reset();
lookData.delta = Vector2.zero;
lookData.position = lookPosition;
lookData.scrollDelta = Vector2.zero;
eventSystem.RaycastAll(lookData, m_RaycastResultCache);
lookData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
if (lookData.pointerCurrentRaycast.gameObject != null) {
_guiRaycastHit = true;
} else {
_guiRaycastHit = false;
}
m_RaycastResultCache.Clear();
return lookData;
}
private void SetSelectedColor(GameObject go) {
if (useSelectColor) {
Selectable s = go.GetComponent<Selectable>();
if (s != null) {
ColorBlock cb = s.colors;
currentSelectedNormalColor = cb.normalColor;
currentSelectedHighlightedColor = cb.highlightedColor;
cb.normalColor = selectColor;
cb.highlightedColor = selectColor;
s.colors = cb;
}
}
}
private void RestoreColor(GameObject go) {
if (useSelectColor) {
Selectable s = go.GetComponent<Selectable>();
if (s != null) {
ColorBlock cb = s.colors;
cb.normalColor = currentSelectedNormalColor;
cb.highlightedColor = currentSelectedHighlightedColor;
s.colors = cb;
}
}
}
public void ClearSelection() {
if (eventSystem.currentSelectedGameObject) {
RestoreColor(eventSystem.currentSelectedGameObject);
eventSystem.SetSelectedGameObject(null);
}
}
public void Select(GameObject go) {
ClearSelection();
SetSelectedColor(go);
eventSystem.SetSelectedGameObject(go);
}
private bool SendUpdateEventToSelectedObject() {
if (eventSystem.currentSelectedGameObject == null)
return false;
BaseEventData data = GetBaseEventData ();
ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
return data.used;
}
public override void Process() {
_singleton = this;
// send update events if there is a selected object - this is important for InputField to receive keyboard events
SendUpdateEventToSelectedObject();
PointerEventData lookData = GetLookPointerEventData();
HandlePointerExitAndEnter(lookData,lookData.pointerCurrentRaycast.gameObject);
_buttonUsed = false;
if (Input.GetButtonDown (submitButtonName)) {
ClearSelection();
if (lookData.pointerCurrentRaycast.gameObject != null) {
GameObject go = lookData.pointerCurrentRaycast.gameObject;
GameObject newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.submitHandler);
if (newPressed == null) {
// try select handler instead
newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.selectHandler);
if (newPressed != null) {
Select(newPressed);
}
} else {
InputField infield = newPressed.GetComponent<InputField>();
if (infield != null) {
Select(newPressed);
}
}
if (newPressed != null) {
_buttonUsed = true;
}
}
}
_controlAxisUsed = false;
if (eventSystem.currentSelectedGameObject && controlAxisName != null && controlAxisName != "") {
float newVal = Input.GetAxis (controlAxisName);
if (newVal > 0.01f || newVal < -0.01f) {
if (useSmoothAxis) {
Slider sl = eventSystem.currentSelectedGameObject.GetComponent<Slider>();
if (sl != null) {
sl.value += newVal*smoothAxisMultiplier;
_controlAxisUsed = true;
} else {
Scrollbar sb = eventSystem.currentSelectedGameObject.GetComponent<Scrollbar>();
if (sb != null) {
sb.value += newVal*smoothAxisMultiplier;
_controlAxisUsed = true;
}
}
} else {
_controlAxisUsed = true;
float time = Time.unscaledTime;
if (time > nextAxisActionTime) {
nextAxisActionTime = time + 1f/steppedAxisStepsPerSecond;
AxisEventData axisData = GetAxisEventData(newVal,0.0f,0.0f);
if (!ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, axisData, ExecuteEvents.moveHandler)) {
_controlAxisUsed = false;
}
}
}
}
}
}
}
If this gets any longer, I'll post over on github. :)
edit: made some minor fixes/improvements to the code above since original post:
1. only things that respond to axis input will get the SelectColor
2. have option to enable/diable SelectColor
3. removed the color option that wasn't used
