Forum Discussion
jGhost
2 years agoHonored Guest
Feature Request: Custom passthrough styling with kernels, textures, and blending functions
Hi, it would be really nice if there was an API where you can supply a convolution kernel and/or texture with some blending function to apply to passthrough as a custom filter.
Since there is no access to camera feed, the ability to apply custom kernels to passthrough image could be really useful. I am guessing this is what the SDK already does with "passthrough styles" (Passthrough Styles Sample | Oculus Developers). However, there's only edge detection currently.
With custom kernels you can achieve a variety of other effects like gaussian blurs, image sharpening, etc. Kernel (image processing - Wikipedia
With textures, you can apply these kernels at some strength sampled by values in some supplied stereo textures. This will allow for cool effects like applying blur or some custom lighting effect in a particular area, not necessarily the whole passthrough view.
With blending functions, blend mode along with the supplied texture, you can achieve a wider variety of image color effects. Blend modes - Wikipedia
This can all be done without access to the camera feed, only using scene information and I think this is somewhat of an essential feature if there isn't a plan to allow access to the camera.
It would be really cool and maybe simpler if you could instead just upload some shader to the headset to be applied to passthrough, but I am guessing this would be difficult or it would have been done already.
If anybody agrees or have other suggestions or alternatives for implementing cool passthrough effects, post a comment.
3 Replies
Replies have been turned off for this discussion
- JoitsExplorer
This would be very interesting. Especially gaussian blur can be useful for many use cases. Anyone have succesfully implemented a blur shader in passthrough?
- watermelon.sunriseHonored Guest
Has anyone been able to apply shaders to passthrough? I'm surprised this is not already a feature.
- MetaDevXRProtege
I agree. Would be nice to be able to do more effects in passthrough than posterize or edge detections. I happen to have a commercial corner case use for a training app where I need mild warping, slight motion blur, and perhaps other effects to give the user a visual experience of being drunk at various intoxication levels. I can achieve all this using post processing effects in VR mode. However, the request is to do it in AR/passthrough for use in the field.
Quick Links
- Horizon Developer Support
- Quest User Forums
- Troubleshooting Forum for problems with a game or app
- Quest Support for problems with your device
Other Meta Support
Related Content
- 16 days ago
- 12 months ago
- 4 months ago
- 2 months ago
- 9 months ago