Forum Discussion

jGhost's avatar
jGhost
Honored Guest
2 years ago

Feature Request: Custom passthrough styling with kernels, textures, and blending functions

Hi, it would be really nice if there was an API where you can supply a convolution kernel and/or texture with some blending function to apply to passthrough as a custom filter.

Since there is no access to camera feed, the ability to apply custom kernels to passthrough image could be really useful. I am guessing this is what the SDK already does with "passthrough styles" (Passthrough Styles Sample | Oculus Developers). However, there's only edge detection currently.

With custom kernels you can achieve a variety of other effects like gaussian blurs, image sharpening, etc. Kernel (image processing - Wikipedia

With textures, you can apply these kernels at some strength sampled by values in some supplied stereo textures. This will allow for cool effects like applying blur or some custom lighting effect in a particular area, not necessarily the whole passthrough view.

With blending functions, blend mode along with the supplied texture, you can achieve a wider variety of image color effects. Blend modes - Wikipedia

This can all be done without access to the camera feed, only using scene information and I think this is somewhat of an essential feature if there isn't a plan to allow access to the camera.

It would be really cool and maybe simpler if you could instead just upload some shader to the headset to be applied to passthrough, but I am guessing this would be difficult or it would have been done already.

If anybody agrees or have other suggestions or alternatives for implementing cool passthrough effects, post a comment.

3 Replies

  • This would be very interesting. Especially gaussian blur can be useful for many use cases. Anyone have succesfully implemented a blur shader in passthrough?

  • Has anyone been able to apply shaders to passthrough? I'm surprised this is not already a feature.

  • I agree.  Would be nice to be able to do more effects in passthrough than posterize or edge detections.  I happen to have a commercial corner case use for a training app where I need mild warping, slight motion blur, and perhaps other effects to give the user a visual experience of being drunk at various intoxication levels.  I can achieve all this using post processing effects in VR mode.  However, the request is to do it in AR/passthrough for use in the field.