Can't disable Xbox controller from interacting with my headset

When a controller is connected to my PC while using (wired) Oculus Link and I am viewing my monitors from within the headset, there will be a dot on the screen that follows my head movements. If I pr...