On the PC side: Hello, we are looking into making some experiences with a mixture of 360 stero live footage and realtime rendering. This is working as long as the framesize is about full HD. Bigger sizes of our videos always come out black. Full HD in a spherical projection looks kind of bad. Anyone a solution for this dilemma?
On Mobile: Really interesting is the question on how to get a spherial projection in Unity running on mobile devices. Even the Jaunt applications run pretty unstable and tend to crash even on high end mobile devices.
Unfortuantly with the current version of Unity you will not be able to achieve video resolution over 1080p if your lucky, 720p is more consistant, this is because of a couple of factors, 1) unity does video decoding on the CPU not the GPU and the unity supported codec .ogv isn't the best. 2) you quickly run out of memory when playing video any longer then just a few seconds. You can look into 3rd party plugins but your mobile support will be limited and you will still be far from achieving 4k playback.
I've been in talks with Unity and they've told me that in Unity 5.1 they have optimized the .ogv codec and it will be able to play a single 4k stream or up to 3 1080p streams, its still being decoded on the CPU though. You can look into the Bink codec system, they have a sdk integration for unity but you need to special request if from Bink, and even then its only on the PC and they've informed me that they are currently reworking the package and will be ready early next year.
