Unity: switch VR to Passthrough and keeping UI visible

Hello, I am developing a simple application that simulates walking in a virtual forest, and lets the user change to passthrough ( and back) with a simple gesture of the hand. The character has a canvas following the camera using this script:

using UnityEngine;

public class CanvasFollower : MonoBehaviour
{
    public Camera cameraRig;
    public float distance = 2.0f;
    public float smoothTime = 0.3f;
    private Vector3 velocity = Vector3.zero;
    private Transform target;

    void Awake()
    {
        target = cameraRig.transform;
    }

    void Update()
    {
        // Calculate the target position at the specified distance in front of the camera
        Vector3 targetPosition = cameraRig.transform.position + cameraRig.transform.forward * distance;

        // Smoothly move the canvas to the target position
        transform.position = Vector3.SmoothDamp(transform.position, targetPosition, ref velocity, smoothTime);

        // Make the canvas face the camera by rotating it
        transform.LookAt(cameraRig.transform);
        
        // Correct the canvas orientation by rotating it 180 degrees around the y-axis
        transform.Rotate(0, 180, 0);
    }
}

This is the settings of the canvas:

This is the function to toggle the passthrough in-game, where virtualWorld is a GameObject father of all the virtual elements that I want to hide behind the passthrough layer:

    
public void TogglePassthrough()
{
    if (passthroughLayer)
    {
        passthroughLayer.hidden = !passthroughLayer.hidden;

    }

    // Toggle renderers in the virtual world
    foreach (Renderer renderer in virtualWorld.GetComponentsInChildren<Renderer>())
    {
        renderer.enabled = !renderer.enabled;
    }
}

}

And these are the passthrough layer settings:

Now, the passthrough layer is correctly enabled when requested by the user, but the UI canva disappear as well, even if is not included in the Virtual World object, does somebody know how to fix that?