I was looking at the native samples in the mobile SDK, specifically the 360 videos sample, hoping to see how video frames are decoded and uploaded to GL.
I have an idea of how I would approach this, perhaps with libav and a few PBOs on the GL side, but I wanted to see how Oculus does it. Does anyone know where in the code is this happening? There are the turbojpeg and OVR_Turbojpeg classes, but they don't seem (at least at first sight) to be related to the actual video frame decoding.
The 'Oculus360VideosSDK' sample app for Android uses 'MediaPlayer' for decoding video. The main activity (MainActivity.java) registers a callback (onFrameAvailable) that is invoked every time a frame has been decoded. The decoded frame (the entire equirectangular projection) is passed to the Oculus engine via a texture ('movieTexture' and 'movieSurface'). Internally, the engine determines head pose and selects the appropriate viewport for rendering.
You may replace 'MediaPlayer' with your own media engine (e.g., libav). You are right: turbojpeg is not related to video decoding (at least not in this sample app).
For the record, 'MediaPlayer' in Android only supports local file playback and HTTP progressive download. It does not support HLS and DASH streaming. So other media engine (e.g., ExoPlayer) is needed to support streaming in Android.
The 'Oculus360VideosSDK' sample app for Android uses 'MediaPlayer' for decoding video. The main activity (MainActivity.java) registers a callback (onFrameAvailable) that is invoked every time a frame has been decoded. The decoded frame (the entire equirectangular projection) is passed to the Oculus engine via a texture ('movieTexture' and 'movieSurface'). Internally, the engine determines head pose and selects the appropriate viewport for rendering.
You may replace 'MediaPlayer' with your own media engine (e.g., libav). You are right: turbojpeg is not related to video decoding (at least not in this sample app).
For the record, 'MediaPlayer' in Android only supports local file playback and HTTP progressive download. It does not support HLS and DASH streaming. So other media engine (e.g., ExoPlayer) is needed to support streaming in Android.
