OculusPlayerControllers controlling each other (Unity3d) C#

Hi guys,
So Im trying to use the Oculus Rift with the Photon Server. I want to be able to have 2 or more clients connect to a server and move around a simple plane using the rift. I was able to do this using a 3rd person controller and a first person controller, but Im having some issues with the OVRPlayerController.

So players can connect to the server but they are then controlling each other. They do not control each others rotation. That seems to work fine. However player 1 controls where player 2 moves, and vice versa. Here is the code I use to instantiate a player after they have joined.

GameObject player = PhotonNetwork.Instantiate("OVRPlayerController", Vector3.zero, Quaternion.identity,0);

		OVRGamepadController gamepadController = player.GetComponent<OVRGamepadController>();
		gamepadController.enabled = true;

		/**
		 * OVRPlayerController playerController = player.GetComponent<OVRPlayerController>();
		playerController.enabled = true;
		*/

		OVRMainMenu mainMenu = player.GetComponent<OVRMainMenu>();
		mainMenu.enabled = true;

		foreach(Transform t in player.transform)
		{
			if(t.name != "OVRCameraController")continue;
			OVRCameraController cam = t.GetComponent<OVRCameraController>();
			if(cam ==null)continue;
			cam.enabled = true;
			break;
		}

		foreach(Transform t in player.transform)
		{
			if(t.name != "OVRCameraController")continue;
			OVRDevice device = t.GetComponent<OVRDevice>();
			if(device ==null)continue;
			device.enabled = true;
			break;
		}

When doing the same work with a First Person Controller or Third Person Controller, I had to deactivate components such as the camera, and only enable them once a player became instantiated. By default I have disabled the following componenets:

OVRGamepad Controller
OVRMainMenu
OVRCameraController
OVRDevice

These are all enabled once the player is instantiated. However, the issue remains where players control each other instead of just their own instance of the player.

Does it have anything to do with the "ForwardDirection" child object of the OVRPlayerController? I really dont know how to get past this issue. Any help would be greatly appreciated.