Skip to main content
DOODLYDOO
New Participant
December 14, 2018

VRC validator gives inconsistent results for TestSubmitFramesWhenNotVisible, but passes manual test

When running the VRC validator using the arguments "--test TestSubmitFramesWhenNotVisible --test TestSubmitFramesWhenVisible" using version 1.33 of the validator, the TestSubmitFramesWhenNotVisible fails with 2 frames submitted.

The strange thing is this same build used to pass both tests until a few days ago, before the latest Oculus Update on the 11th. We've also been working on the project, but removing the changes we made gives us the same results.

Strangely, I can trick the 1.33 validator into passing TestSubmitFramesWhenNotVisible by holding my finger over the HMD's sensor when the test starts, and removing it before the 5 second wait has elapsed. As far as we can tell, the manual version of this test passes just fine, the Oculus menu appears and gameplay pauses, no longer accepting input.

 Here is the log showing results from both the 1.33 validator and 1.32 validator (both using 1.33 software however). 1.32 fails TestSubmitFramesWhenVisible since no frames are ever submitted at all apparently, but it used to pass before the update (holding my finger over the sensor in this case gives the same results as 1.33) . Any insight would be appreciated. This is the only issue holding up our submission.

============================= 2018-12-14 10:38:41 ==============================
INFO: Starting TestSubmitFramesWhenNotVisible
INFO: Waiting for the application to run for 5 seconds before testing begins...
INFO: Starting test...
INFO: Requesting the void...
INFO: Number of texture swap chains committed when visible 34
INFO: Number of texture swap chains committed when not visible 2
ERROR: Committed a texture swap chain (called ovr_CommitTextureSwapChain) when application not visible
INFO: Please refer to VRC Guidelines: https://developer.oculus.com/distribute/latest/concepts/vrc-pc-input-1/
INFO: Cleaning up...
INFO: Test FAILED

INFO: Starting TestSubmitFramesWhenVisible
INFO: Waiting for the application to run for 5 seconds before testing begins...
INFO: Starting test...
INFO: Number of texture swap chains committed when visible 902
INFO: Number of texture swap chains committed when not visible 0
INFO: Cleaning up...
INFO: Test PASSED

INFO: Summary:
INFO: TestSubmitFramesWhenNotVisible: Failed
INFO: TestSubmitFramesWhenVisible: Passed
INFO: Generated by OculusVRCValidator version 1.33.0.0
============================= 2018-12-14 11:14:16 ==============================
INFO: Starting TestSubmitFramesWhenNotVisible
INFO: Waiting for the application to run for 5 seconds before testing begins...
INFO: Starting test...
INFO: Requesting the void...
INFO: Number of texture swap chains committed when visible 0
INFO: Number of texture swap chains committed when not visible 0
INFO: Cleaning up...
INFO: Test PASSED

INFO: Starting TestSubmitFramesWhenVisible
INFO: Waiting for the application to run for 5 seconds before testing begins...
INFO: Starting test...
INFO: Number of texture swap chains committed when visible 0
INFO: Number of texture swap chains committed when not visible 0
ERROR: The application did not commit any texture swap chains
INFO: Please refer to VRC Guidelines: https://developer.oculus.com/distribute/latest/concepts/publish-rift-app-submission/
INFO: Cleaning up...
INFO: Test FAILED

INFO: Summary:
INFO: TestSubmitFramesWhenNotVisible: Passed
INFO: TestSubmitFramesWhenVisible: Failed
INFO: Generated by OculusVRCValidator version 1.32.0.0




    3 replies

    DOODLYDOOAuthor
    New Participant
    December 14, 2018
    Some further info. If the HMD has been idle for a while before running the test, it shows no frames submitted for either test (so it passes NotVisible, but fails Visible). If the HMD had been active recently, I see the same results as the first test indicated above.
    Stendor
    New Participant
    December 28, 2018
    Hi, I have this issue too. A trick with finger fix it and if you put finger again before "TestSubmitFramesWhenVisible" starts, you pass this test too.

    I use Unreal Engine and decreases this "Number of texture swap chains committed when not visible 2"  to 1 using guideline https://developer.oculus.com/documentation/unreal/latest/concepts/unreal-dash/

    For tests, I use this project with UE4 v4.21.1 from Oculus Git
    DOODLYDOOAuthor
    New Participant
    January 4, 2019
    For anyone else having this specific  issue, we did submit and managed to pass, so it would appear that the manual test should be considered over the validator's result.

    Terms & ConditionsAccessibility statement