Unity 4.6 UI system - Render to world space

That's great news indeed. It still comes down to how you can actually interact with them. We can be reasonably sure they will support interaction via a screenspace mouse cursor, but will it be friendly to working with the cursor being stereoscopic, or a world-space cursor? Or other forms of input like head-tracking gaze?

I asked exactly that in the offical Unity forum thread, didn't get any answer.

Yeah this looks great. I'm looking forward to getting my hands on it.

I adapted NGUI to work with head-tracking gaze and it works great. That worked easily because NGUI elements use regular collider objects so it just involved using the physics raycaster to detect what GUI element you are looking at. I probably added 50-100 lines of code to NGUI to get it to work for VR. I'm hoping for something similar here. I didn't see the use of colliders in the video, but I suspect there will be a way to add this fairly easily even if it isn't included.

CCS, could you elaborate more on how you prepared NGUI for Oculus use? What code did you insert?

"smx" wrote:
CCS, could you elaborate more on how you prepared NGUI for Oculus use? What code did you insert?

I'm happy to share. Feel free to use this code (I can only post my snippets and not the whole file). Note that I didn't try to get mouse interaction working so I disable that in the UICamera checkbox for mouse. The UICamera Event Type should be set to '3D World'. You will probably also want to use a specifc layer for all your GUI elements, and then set UICamera Event Mask to that layer.

I went with head + controller. I changed several things (all in UICamera.cs) to get basic usage working:

1. UICamera.cs: added a select button name parameter (you'll have to set the value based on your project Input control project settings):

	public string selectButtonName = "ButtonA";

2. UICamera.cs: modified Raycast function to add a useLook parameter and some code to use camera forward as the ray direction when enabled:

	//static public bool Raycast (Vector3 inPos)
	static public bool Raycast (Vector3 inPos, bool useLook = false)
...
			Ray ray;
			if (useLook) {
				Transform camtrans = Camera.main.transform;
				ray = new Ray(camtrans.position, camtrans.forward);
				currentCamera = Camera.main;
			} else {
...original code up to next line shown here...
				// Cast a ray into the screen
				ray = currentCamera.ScreenPointToRay(inPos);
			}

3. UICamera.cs added ProcessLook() function for calling the look raycast and managing what to do with the selectButtonName param that was added:

	public void ProcessLook ()
	{
		if (Camera.main) {
			currentScheme = ControlScheme.Controller;
			
			GameObject lastHoveredObject = hoveredObject;
			if (Raycast(Vector3.zero, true)) {
				if (hoveredObject != lastHoveredObject) {
					Notify (lastHoveredObject, "OnHover", false);
					Notify (hoveredObject, "OnHover", true);
				}
				if (Input.GetButtonDown(selectButtonName)) {
					Notify (hoveredObject,"OnPress", true);
					SetSelection (hoveredObject,currentScheme);
				}
				if (Input.GetButtonUp (selectButtonName)) {
					Notify (hoveredObject,"OnClick", true);
					//SetSelection (hoveredObject,currentScheme);
				}
			} else { 
				hoveredObject = fallThrough;
				if (hoveredObject != lastHoveredObject) {
					Notify (lastHoveredObject, "OnHover", false);
				}
				if (Input.GetButtonDown(selectButtonName)) {
					SetSelection (null,currentScheme);
				}
			}
			if (hoveredObject == null) hoveredObject = genericEventHandler;
		}
	}

4. UICamera.cs: call new ProcessLook function right after ProcessMouse call:

		if (useTouch) ProcessTouches ();
		else if (useMouse) ProcessMouse();
		else ProcessLook ();

Note that once you have a GUI element selected with head look + select button, all the other built-in NGUI controller support just works natively without modification. The above code doesn't support drag as mouse and touch do. I'm not sure you would want to use head drag though. Better to scroll/drag with the controller button.