I’m also having this same issue while playing stuff on my Quest 3 cable linked to my PC. Windows 11, Ryzen 9 3900x, 2080 Super and 32GB ram. It happens anytime the game stutters.
Try turning off hardware accelerated graphics scheduling (HAGS) it's only beneficial if you are not GPU limited anyway which is very rare in VR I'd say. I think it helps but I haven't played on PC much due to this and airlink / steam link poor performance. I reluctantly purchased Virtual Desktop the other day too after giving up with the terrible performance and quality of both of the previous solutions.
Double tapping to go to pass through and back is the quickest workaround.
I'll give that a try today and report back.
