Up and Coming

Solved

Unity + Quest: repeated VK_ERROR_INITIALIZATION_FAILED error

Forum|Forum|3 years ago
May 18, 2023
16 replies
6084 views

I am updating an existing Quest game to use the latest Oculus integration etc. The game runs fine for a minute or so and then the visuals of the game freeze on one frame and the following is repeated on the log. It appears that the game is still running fine in the background because I can hear sounds that suggest that it is running fine, besides the visuals.

Warn Adreno-GSL <log_gpu_snapshot:471>: panel.gpuSnapshotPath is not set.not generating user snapshot
Warn Adreno-GSL <ioctl_kgsl_syncobj_create:5290>: (45, 33, 2329) fail 22 Invalid argument
Warn Vulkan_Utils Vulkan error: vulkanLoader.vkQueueSubmit(VulkanContext.queue, 0, NULL, fenceToUse): 'VK_ERROR_INITIALIZATION_FAILED'(-3) at arvr/projects/xrruntime/mobile/VrRuntime/Client/VrRuntimeClient/VrRuntimeClient.cpp:2625
Warn Adreno-GSL <log_gpu_snapshot:471>: panel.gpuSnapshotPath is not set.not generating user snapshot
Warn Adreno-GSL <ioctl_kgsl_syncobj_create:5290>: (45, 33, 2330) fail 22 Invalid argument
Warn Vulkan_Utils Vulkan error: vulkanLoader.vkQueueSubmit(VulkanContext.queue, 0, NULL, fenceToUse): 'VK_ERROR_INITIALIZATION_FAILED'(-3) at arvr/projects/xrruntime/mobile/VrRuntime/Client/VrRuntimeClient/VrRuntimeClient.cpp:2625
Warn FenceChecker FenceChecker: wait timed out after 100 msec
Warn Adreno-GSL <log_gpu_snapshot:471>: panel.gpuSnapshotPath is not set.not generating user snapshot
Warn Adreno-GSL <ioctl_kgsl_syncobj_create:5290>: (45, 33, 2331) fail 22 Invalid argument
Warn Vulkan_Utils Vulkan error: vulkanLoader.vkQueueSubmit(VulkanContext.queue, 0, NULL, fenceToUse): 'VK_ERROR_INITIALIZATION_FAILED'(-3) at arvr/projects/xrruntime/mobile/VrRuntime/Client/VrRuntimeClient/VrRuntimeClient.cpp:2625
Warn FenceChecker FenceChecker: wait timed out after 100 msec
Warn Adreno-GSL <log_gpu_snapshot:471>: panel.gpuSnapshotPath is not set.not generating user snapshot
Warn Adreno-GSL <ioctl_kgsl_syncobj_create:5290>: (45, 33, 2332) fail 22 Invalid argument
Warn Vulkan_Utils Vulkan error: vulkanLoader.vkQueueSubmit(VulkanContext.queue, 0, NULL, fenceToUse): 'VK_ERROR_INITIALIZATION_FAILED'(-3) at arvr/projects/xrruntime/mobile/VrRuntime/Client/VrRuntimeClient/VrRuntimeClient.cpp:2625

I am about to start the trial and error process to determine what the parameters are that cause the issue and what devices are affected by it.

Here is some of my build information:

Unity 2022.2.15f1
Oculus XR Plugin 3.3.0
Oculus Integration 53.1 (1.85.0)
Device: Quest Pro
- System Version 50000390062500330
- Version 53.0.0.147.132.475553041
- Runtime Version 53.0.0.147.132.475553298
- OS Version SQ3A.220605.009.A1

Please let me know if you have any information or if it might be a known issue with my software versions.

Best answer by EnvisionVRmaster

I don't know how to solve this issue proprly, but I have the same error. It never happens until I load into a specific scene that has some extra UI in it. I think the issue could be something to do with the always on top shader that I have for the UI. Or TextMeshPro maybe.

I didn't really have time to figure it out, but I unticked "Symmetric Projection (Vulkan)" in Project Settings > XR Plug-in Management > Oculus > Android. And I stopped getting the error and my app stopped crashing.

A

AllenPestalukyAuthor

Up and Coming

I can also reproduce this exact same issue with a Quest 2:

System Version 50543050028800150
Version 54.0.0.107.344.474388434
Runtime Version 54.0.0.110.348.475023458
OS Version SQ3A.220605.009.A1

A

AllenPestalukyAuthor

Up and Coming

This problem also happens with Unity 2022.2.21f1:

Unity 2022.2.21f1
Oculus XR Plugin 4.4.0
Oculus Integration 53.1 (1.85.0)

A

AllenPestalukyAuthor

Up and Coming

Turns out this problem also happens with Unity 2021.3.26f1 LTS.

And I realized that the actual error messages happen on a 0 process ID, so my logcat filter was removing them from my view. These are the actual errors that I'm seeing:

2023-05-26 15:50:34.502 0 0 Fatal kgsl-3d0 CP opcode error interrupt | opcode=0x00000000
2023-05-26 15:50:34.513 0 0 Fatal kgsl-3d0 MISC: GPU hang detected
2023-05-26 15:50:34.513 0 0 Error kgsl-3d0 dnobodyexplodes[8095]: gpu fault ctx 36 ctx_type VK ts 1235 status 00800005 rb 1124/1124 ib1 0000004003AE6000/0220 ib2 0000004003CE4000/0000
2023-05-26 15:50:34.513 0 0 Error kgsl-3d0 dnobodyexplodes[8095]: gpu fault rb 2 rb sw r/w 1124/1205
2023-05-26 15:50:34.513 0 0 Error platform 3d6a000.qcom,gmu Suspended GMU
2023-05-26 15:50:34.527 0 0 Fatal kgsl-3d0 CP opcode error interrupt | opcode=0x00000000
2023-05-26 15:50:34.536 0 0 Fatal kgsl-3d0 MISC: GPU hang detected
2023-05-26 15:50:34.537 0 0 Error platform 3d6a000.qcom,gmu Suspended GMU
2023-05-26 15:50:34.537 0 0 Warn healthd battery l=97 v=4272 t=30.2 h=2 st=4 c=-257324 fc=3519000 cc=57 chg=u
2023-05-26 15:50:34.546 8095 8168 Warn Adreno-GSL <log_gpu_snapshot:471>: panel.gpuSnapshotPath is not set.not generating user snapshot
2023-05-26 15:50:34.547 8095 8168 Warn Adreno-GSL <ioctl_kgsl_syncobj_create:5290>: (44, 24, 1238) fail 22 Invalid argument
2023-05-26 15:50:34.547 8095 8168 Warn Vulkan_Utils Vulkan error: vulkanLoader.vkQueueSubmit(VulkanContext.queue, 0, NULL, fenceUsed): 'VK_ERROR_INITIALIZATION_FAILED'(-3) at arvr/projects/xrruntime/mobile/VrRuntime/Client/VrRuntimeClient/VrRuntimeClient.cpp:2603
2023-05-26 15:50:34.541 0 0 Error kgsl-3d0 dnobodyexplodes[8095]: gpu failed ctx 36 ts 1236
...

S

seanb22ct

New Participant

Hi there, Allen. I think we're seeing the same issue. We think the problem may be related to discarding fragments in the fragment shader.
Is it possible to see if removing discard or clip in your fragment shaders makes this problem go away?
I'm not saying that's a solution but it may help our Meta friends provide a real fix.

A

AllenPestalukyAuthor

Up and Coming

Hey seanb.22ct, would you mind trying with Vulkan Validation Layers? I just found out about this. It seems that I am also getting these four validation errors that might be related to my issue:

VULKAN DEBUG: ERROR:  [Validation, 0]: Validation Error: [ VUID-vkDestroyFence-fence-01120 ] Object 0: handle = 0x96fbe2000000005e, type = VK_OBJECT_TYPE_FENCE; | MessageID = 0x5d296248 | VkFence 0x96fbe2000000005e[] is in use. The Vulkan spec states: All queue submission commands that refer to fence must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkDestroyFence-fence-01120)

VULKAN DEBUG: ERROR:  [Validation, 0]: Validation Error: [ VUID-vkDestroyImage-image-01000 ] Object 0: handle = 0xb40000788c201e40, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xf2d29b5a | Cannot call vkDestroyImage on VkImage 0xb206020000000343[] that is currently in use by a command buffer. The Vulkan spec states: All submitted commands that refer to image, either directly or via a VkImageView, must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkDestroyImage-image-01000)

VULKAN DEBUG: ERROR:  [Validation, 0]: Validation Error: [ VUID-VkImageViewCreateInfo-image-01762 ] Object 0: handle = 0xda6ce50000000156, type = VK_OBJECT_TYPE_IMAGE; | MessageID = 0x6516b437 | vkCreateImageView() format VK_FORMAT_R8G8B8A8_UNORM differs from VkImage 0xda6ce50000000156[] format VK_FORMAT_R8G8B8A8_SRGB.  Formats MUST be IDENTICAL unless VK_IMAGE_CREATE_MUTABLE_FORMAT BIT was set on image creation. The Vulkan spec states: If image was not created with the VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT flag, or if the format of the image is a multi-planar format and if subresourceRange.aspectMask is VK_IMAGE_ASPECT_COLOR_BIT, format must be identical to the format used to create image (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-VkImageViewCreateInfo-image-01762)

VULKAN DEBUG: ERROR:  [Validation, 0]: Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0xb4000077fec106c0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[2] command buffer VkCommandBuffer 0xb4000077fec106c0[] expects VkImage 0x22bb760000000ff5[] (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_UNDEFINED.

I think the only discard or clip fragment shaders would be in an old version of TextMesh Pro that we're using (the last full source code version that was published by the original author of the plugin)... I'm trying to reproduce it in a simple project with that version of TextMesh Pro, but I haven't had success just yet...

A

AllenPestalukyAuthor

Up and Coming

Hmm. I tried in a test project with my old TextMesh Pro and didn't find any validation errors with a single simple text mesh, so there's more at play here.

Here are the validation errors that I'm seeing in Unity 2022.2.21f1. There is some overlap with my previous errors, but it seems that not all of them are specific to the GPU hang:

VULKAN: ERROR:  [Validation, 0]: Validation Error: [ VUID-vkCmdClearAttachments-rect-02682 ] Object 0: handle = 0xb4000078c9af0e20, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0xadbd476f | CmdClearAttachments(): pRects[1].rect.extent.width is zero. The Vulkan spec states: The rect member of each element of pRects must have an extent.width greater than 0 (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkCmdClearAttachments-rect-02682)

VULKAN: ERROR:  [Validation, 0]: Validation Error: [ VUID-VkImageViewCreateInfo-image-01762 ] Object 0: handle = 0x50c1c40000000176, type = VK_OBJECT_TYPE_IMAGE; | MessageID = 0x6516b437 | vkCreateImageView() format VK_FORMAT_R8G8B8A8_UNORM differs from VkImage 0x50c1c40000000176[] format VK_FORMAT_R8G8B8A8_SRGB.  Formats MUST be IDENTICAL unless VK_IMAGE_CREATE_MUTABLE_FORMAT BIT was set on image creation. The Vulkan spec states: If image was not created with the VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT flag, or if the format of the image is a multi-planar format and if subresourceRange.aspectMask is VK_IMAGE_ASPECT_COLOR_BIT, format must be identical to the format used to create image (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-VkImageViewCreateInfo-image-01762)

VULKAN: ERROR:  [Validation, 0]: Validation Error: [ VUID-vkResetFences-pFences-01123 ] Object 0: handle = 0x1d4e2e0000000062, type = VK_OBJECT_TYPE_FENCE; | MessageID = 0x68a5074e | VkFence 0x1d4e2e0000000062[] is in use. The Vulkan spec states: Each element of pFences must not be currently associated with any queue command that has not yet completed execution on that queue (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkResetFences-pFences-01123)

It seems that the Validation Error: [ VUID-vkResetFences-pFences-01123 ] is the only error that happened exclusively after the hang was observed, so my guess is that this is the problem error.

S

seanb22ct

New Participant

Hi again, Allen. I'm seeing a lot of validation spam with that turned on but once things go bad with the GPU hang I do see the same kind of fence error in the log that you're reporting:

VULKAN DEBUG: ERROR:  [Validation, 0]: Validation Error: [ VUID-vkResetFences-pFences-01123 ] Object 0: handle = 0x27138a00000000c2, type = VK_OBJECT_TYPE_FENCE; | MessageID = 0x68a5074e | VkFence 0x27138a00000000c2[] is in use. The Vulkan spec states: Each element of pFences must not be currently associated with any queue command that has not yet completed execution on that queue (https://www.khronos.org/registry/vulkan/specs/1.3-extensions/html/vkspec.html#VUID-vkResetFences-pFences-01123)

A

AllenPestalukyAuthor

Up and Coming

Cool, thanks for sharing. I'll post back if and when I'm able to find a solution.

E

EnvisionVRmasterAnswer

New Participant

I don't know how to solve this issue proprly, but I have the same error. It never happens until I load into a specific scene that has some extra UI in it. I think the issue could be something to do with the always on top shader that I have for the UI. Or TextMeshPro maybe.

I didn't really have time to figure it out, but I unticked "Symmetric Projection (Vulkan)" in Project Settings > XR Plug-in Management > Oculus > Android. And I stopped getting the error and my app stopped crashing.

E

EnvisionVRmaster

New Participant

I also think its an issue that doesn't exist in Unity 2021

S

seanb22ct

New Participant

FWIW: We're seeing this error with Unity 2021.3.22

W

WilliamIH

Participant

A question for those who found that disabling Symmetric Projection fixed the issue -- what option do you have set for Stereo Rendering Mode?

A

AllenPestalukyAuthor

Up and Coming

Multiview.

W

WilliamIH

Participant

Thanks, that makes sense. We're using Multi Pass -- for anyone else who's using Multi Pass, it sounds like disabling Symmetric Projection probably doesn't fix the issue if you're using Multi Pass unfortunately (based on the docs at https://docs.unity3d.com/Packages/com.unity.xr.oculus@2.0/manual/index.html it sounds like Symmetric Projection only has an effect when using Multiview).

J

jsmith2004

New Participant

I've seen that VK_ERROR_INITIALIZATION_FAILED issue before on Quest Pro when moving a project to newer Unity + Oculus XR versions. In my experience it was mainly Vulkan unstable -- switching to OpenGLES3 in the Player Settings verify that immediately. It is also possible to clean out the shader cache, and then testing using Oculus XR plugin 3.2.x as 3.3.0 has been known to cause issues in conjunction with Vulkan on certain devices.

While I was working on troubleshooting my issues, I realized that I was having to go through endless trial-and-error loops and I began to track the duration using a basic work time calculator in order to keep an eye on the length of each build process was taking. It was a great way to stay in order when trying to figure out the cause.

Regards
John Smith

Get Help

Sign up

Login with SSO

Login to the community

Login with SSO