Controllers not functioning properly when grabbing objects that cant be moved freely.

I use the d-link Air Link with the quest 2.

Whenever inside a game I grab a lever or a dial, the controllers start acting differently.

If it's a dial then it won't spin as much as my hand, or it'll spin out of control.

If it's a lever it'll either almost not move at all or just jump between on and of.