I am working on a Unity project here using version 2022.3.17f1 running, all of the the Meta all-in-one SDK, and the Meta starter samples github were imported. Everything is working fine.
I have the project all setup and passthrough works well. The issue is I am working with AI images and they have a green background around the object, the mono to 3D converter I am using makes them work great in passthrough and they have a full 3D effect, but I need a way to chrome key out the green background for passthrough.
Example:
Above is original ComfyUI generated AI image with background removal node (also I tried to have it remove background completely with transparent background and saved as a png, but that didn't work either and was black in Unity in passthrough mode)
Above is a through the lense photo of the image after imported to Unity and it does a realtime mono to depth estimation. I slightly rotated the object to show the 3D. The background with the green color is still there but I can't do anything to cut it out or get rid out it with the Meta passthrough settings. I want to set the Green Alpha channel to 0 on this specific texture/mesh.
Everything works great when using Virtual Desktop of course and doing his passthrough mode as a workaround, but I need to do this via Unity and the official Meta sdk stuff so when I export to standalone eventually this will work correctly. I have no idea how he was able to achieve that.
I have tried all of the passthrough scripts and checked all the camera components and there is no option I see for choosing a specific chroma key color on a texture, there is stylizing setting I see but that's not what I need.
I feel like this is a pretty important feature to have. It would also allow me to record videos of a subject (with a green screen background) and put them in the scene doing realtime mono to 3d using a lightweight MIDAS model, and it could give 3D depth to video character in a scene for added immersion. As it is now though the green is all around the model.
Does anyone know a way to fix this, or if I am missing a section that allows for this?
