That's correct. An RDP session is isolated from the normal interactive user session that sees the real video card. Since OVRServer can't see an interactive session, it dies and you get a message in the Windows Application event log.
You may need to use a remote access application that runs within the interactive session, e.g. VNC. You might also need to find a way to activate your headset if that is needed for what you're trying to do.