Forum Discussion
5.1.2p2 -> 5.2.1f1 - Large Performance Loss
Long post short (i.e. tl;dr;): Avoid 5.2.1 for now. Major performance loss. I'm sure nobody will read the rest of this since Oculus Connect is happening now, but if you want more details, read on!
Just a heads up for those who are tempted to try 5.2.1 that came out 2 days ago. I'm seeing a large performance drop, similar to what was seen when moving to 0.1.0 utils and away from legacy integration. That performance drop was fixed in 5.1.1p3 -> 5.1.2p2, but now something worse seems to have happened 5.1.2p2 -> 5.2.1f1.
I had previously posted these "benchmark" results in the "Oculus Utilities for Unity 0.1.0 Beta released" topic when I was seeing issues with 5.1.1p3 + 0.1.0 utils. This benchmark is a measure in ms of how much CPU time is left for your own scripts outside of rendering a fairly complex scene. I put this together so I could track how Unity + Oculus runtimes improved or degraded over time. It involves a script that acts as fake CPU load, that allows adjusting ms of CPU load until frame skips and judder kick in. This essentially calculates the performance overhead you have. Here were my previous results from the other post:
5.1.1p3 + 0.1.0 Utils: 6.0ms
5.1.1p3 + 0.5.0.1 Legacy Integration: 9.0ms
5.1.2p2 + 0.1.0 Utils: 7.9ms
5.1.2p2 + 0.5.0.1 Legacy Integration: 9.0ms
New data for 5.2.1:
5.2.1f1 + 0.1.0 Utils: 4.0ms
Not good. Can't do much with 4ms of CPU time. I'm rolling back to 5.1.2p2. Unfortunately this means the 0.7 features can't be used as that is what 5.2.1 offered up:
http://unity3d.com/unity/whats-new/unity-5.2.1
VR: Oculus: Upgraded to 0.7 dependencies.
VR: For Windows Oculus Development, the Oculus 0.7.0.0 runtime is required in order to run in VR mode. Future releases will also require this runtime going forward.
I did verify the performance monitor HUD is working now in 5.2.1f1, but unfortunately it just tells me the performance sucks. :)
I'm starting to cringe every time a new SDK version and/or Unity version comes out.
ccs
Just a heads up for those who are tempted to try 5.2.1 that came out 2 days ago. I'm seeing a large performance drop, similar to what was seen when moving to 0.1.0 utils and away from legacy integration. That performance drop was fixed in 5.1.1p3 -> 5.1.2p2, but now something worse seems to have happened 5.1.2p2 -> 5.2.1f1.
I had previously posted these "benchmark" results in the "Oculus Utilities for Unity 0.1.0 Beta released" topic when I was seeing issues with 5.1.1p3 + 0.1.0 utils. This benchmark is a measure in ms of how much CPU time is left for your own scripts outside of rendering a fairly complex scene. I put this together so I could track how Unity + Oculus runtimes improved or degraded over time. It involves a script that acts as fake CPU load, that allows adjusting ms of CPU load until frame skips and judder kick in. This essentially calculates the performance overhead you have. Here were my previous results from the other post:
5.1.1p3 + 0.1.0 Utils: 6.0ms
5.1.1p3 + 0.5.0.1 Legacy Integration: 9.0ms
5.1.2p2 + 0.1.0 Utils: 7.9ms
5.1.2p2 + 0.5.0.1 Legacy Integration: 9.0ms
New data for 5.2.1:
5.2.1f1 + 0.1.0 Utils: 4.0ms
Not good. Can't do much with 4ms of CPU time. I'm rolling back to 5.1.2p2. Unfortunately this means the 0.7 features can't be used as that is what 5.2.1 offered up:
http://unity3d.com/unity/whats-new/unity-5.2.1
VR: Oculus: Upgraded to 0.7 dependencies.
VR: For Windows Oculus Development, the Oculus 0.7.0.0 runtime is required in order to run in VR mode. Future releases will also require this runtime going forward.
I did verify the performance monitor HUD is working now in 5.2.1f1, but unfortunately it just tells me the performance sucks. :)
I'm starting to cringe every time a new SDK version and/or Unity version comes out.
ccs
58 Replies
Replies have been turned off for this discussion
- This echoes my own observations.
I've become very frustrated over the last 3 months as every new release either performs worse or has broken/missing functionality. (Still no field-of-view adjustment possible, can't disable time-warp anymore, requires a specific version of Nvidia drivers, lightmap baking is broken for point-lights in 5.1.3p2, etc.)
These, plus the the lack of a roadmap or even a "we know; we're working on it" from Oculus has shaken my confidence in their ability to deliver a decent product. I know software can be hard, but this has gotten ridiculous.
Can we get the old Oculus back? The one that was open about its efforts and where it's headed? The devkit from pre-FB buyout was a lot more useful and functional than what we have today. As if to pour salt on the wound, this is the second time I've had to type this post, as midway through the first a "Facebook has encountered an error" message removed everything I typed. (WTF is authentication doing while I type? SRSLY.) That small company was outperforming this one by a factor of 10.
Anyway, css, do you happen to know which version combination performs the best with 0.7 runtime currently? I'd like to release my project, but can't seem to find a stable and performant version reliable enough for commercial release. - 5.2.1 is a mess all around. Tons of really obvious bugs have been cropping up. I've been working on a 2D mobile game project recently, and upgrading to 5.2.1 completely wrecked my project. Well, not wrecked... dropped a bunch of inspector references, broke all UI input, and had random texture artifacting. Supposedly there is a patch coming out today or tomorrow to address some of these... but I haven't heard about VR fixes.
- I have tried a lot of combinations, and for now 5.1.2p2 + 0.1.0 Utils + 0.7.0.0 runtime + latest Nvidia drivers (released a few days ago) seems to perform the best (lowest latency + minimal CPU/GPU overhead) and works with both DK1 and DK2.
If you have a really CPU-intensive app, you might also go back to 5.1.2p2 + 0.5.0.1 Legacy Integration + 0.6.0.2 runtime, but in my mind this is maybe setting yourself up for more work later as it is clear the built-in VR integration in Unity is the future and legacy integration is a dead-end.
ccs - Looks like 5.2.1p1 was just released. I'm curious to see how it performs. (I won't be able to try it for a couple of hours.)
- Unfortunately no mention of any VR improvements or anything that might solve this issue with 5.2.1p1. I was going to go post this issue over on Unity forums, but I see you already did this. Thanks for doing that! I think it is important Oculus and Unity are aware of issues as they come up. It would be good if they had their own in-house performance regression that would get run prior to releases. 1.0 SDK can't come soon enough! I hope they put a lot of effort into testing and performance optimization before release of that.
ccs - Thanks for the heads up ccs. As always the development community is in debt to your attention to detail! The last Rift app I released (and is running really well) was with Unity 5.1.2p3, so perhaps it's close to the number you saw for 5.1.2p2.
Hard to say how much code or pipeline is shared between Rift and Gear VR for Unity's native VR functionality, but I have not yet seen acceptable performance on the Gear VR side for any Unity version so I have just continued to use the legacy Unity 4 integration on Unity 5 and that's been working out fine so far. That's obviously not sustainable long-term, so I would like to echo the pleas for a focus on stabilizing the performance of the native VR functionality in Unity. - So basically any Unity 5 based game will be much slower than Unity 4 based game on Gear VR ?
"ccs" wrote:
5.1.1p3 + 0.1.0 Utils: 6.0ms
5.1.1p3 + 0.5.0.1 Legacy Integration: 9.0ms
5.1.2p2 + 0.1.0 Utils: 7.9ms
5.1.2p2 + 0.5.0.1 Legacy Integration: 9.0ms
5.2.1f1 + 0.1.0 Utils: 4.0ms
Thanks for compiling this. Have you tried the latest recommended version from https://forums.oculus.com/viewtopic.php?t=25882, which is currently 5.1.2f1? Since then, there have been known issues affecting CPU overhead and stability on PC. They may be fixed in 5.2.1p1, but we need to test it before we can recommend it. We are working out a better QA process to ensure stability and performance only improve over time. In the meantime, please stick to recommended versions instead of using the latest release."kideternal" wrote:
The devkit from pre-FB buyout was a lot more useful and functional than what we have today.
You mean DK1? That was a conventional display and USB HID device with no need for special driver support. Although it took advantage of well-tested codepaths in Unity and the OS, it was horrible VR for your users. To help you deliver a low-latency, high-quality user experience, we've had to make a lot of changes at many layers in the stack. Your feedback is a huge help. This is still beta software, but we are converging on partner-supported display drivers, engine-level VR, and performance optimizations that "just work". We aren't there yet, but we expect to be within the next few months."kideternal" wrote:
this is the second time I've had to type this post, as midway through the first a "Facebook has encountered an error" message removed everything I typed.
This is the same forum software we've been using for over a year, but we know it leaves a lot to be desired. We're looking at alternatives, but there isn't an ETA on an upgrade. I'll see what we can do about the issue you reported.- Dave B - thanks for responding. I'm glad you guys are on it. Hopefully some of this data will help track down what the issue is. 5.1.2f1 (and 5.1.1p3) actually have performance issues as well that I reported about a while back that were fixed in 5.1.2p2. I have included more data on this below. Here is the bug fix related to that performance fix (seen in 5.1.2p2 release notes):
(none) - VR: Improved MSAA performance by only antialiasing eye textures, not the composited back buffer.
In addition 5.1.2p2 included a ton of other VR fixes. Based on that, I'm surprised you guys still recommend 5.1.2f1.
I have improved my benchmark a bit by trying to make it more consistent and I now run it in built application instead of in editor. Then I spent more time retesting most of the latest versions of Unity to track down what works and what doesn't. Here are the results (higher ms is better, indicating more CPU time for your application):
5.1.2f1 7.1ms
5.1.2p2 9.2ms (What I have been using for almost 2 months now as I found it most stable)
5.1.2p3 9.2ms
5.1.3f1 crash every time - can't benchmark
5.1.3p2 9.4ms (BEST! I will probably move to this version)
5.1.3p3 6.8ms
5.2.0p1 6.5ms
5.2.1p1 6.7ms
Based on this, it looks like a major rendering performance issue was introduced in 5.1.3p3 and beyond. I don't see it noted in 5.1.3p3 release notes, but it appears this is when Unity moved to 0.7.0.0 dependencies because the performance monitor started tracking app CPU/GPU time in this version. In previous versions, this data was still N/A. Hopefully the performance drop is fixable on Unity side and not some indication of performance loss with 0.7.0.0 API and runtime.
edit: note all of the tests above were run with 0.7.0.0 runtime, 0.1.0 utils, and Nvidia 355.98 drivers. Only variance was Unity version.
ccs - Thanks for the numbers, css! They echo my own experience precisely and your efforts are most appreciated. :D
5.1.3p2 is the version I'm working with because it performs the best, but it does still need some work. (The move to v0.7 in 5.1.3p3 does seem to be the culprit for recent performance loss, but I'm confident there's still something else going on with the way the driver is integrated into Unity.)
I spent 18 hours optimizing my project yesterday and am unfortunately still frustrated by the occasional judder. It seems that as soon as you drop to what would be ~68 FPS for a single frame, that frame is delayed until the next pass: the dreaded "OculusWaitForGPU". It would be great if there was a way for this to degrade gracefully instead of causing the jarring stutter for the user. I suspect this also is what causes Unity to lose sync with the Rift's positional information, thus requiring a "recenter" after a few too many of them occur.
I realize maintaining 75+ FPS (and soon 90+) at all times is essential, but it's very difficult to build a rich experience that never stalls below ~68 FPS, for even an instant, while loading resources. (Remember, this is the equivalent of sustaining 200+ FPS @1080p with the Rift turned-off. No small feat!)
@vrdaveb, thanks for the response. I'm glad you're looking into a better performance QA process, but we're getting so short on time it's probably worth embedding an engineer at Unity's office for a week.
