MR Colocation Mesh

I am currently setting up building blocks on a Unity project using Photon/Fusion, on both a Quest 3 and a Quest 2.

Despite the few available tutorials that all seem to work, I can't manage to add the 'Mesh Effect' to a scene with colocation. The session is successfully joined automatically, we can hear each other, and the test cubes move in real-time. However, each device loads its own mesh instead of sharing it, and the meshes (usually the one from the second headset to join) end up drifting away from the actual walls.

I've tried multiple Unity versions, mainly 2022.3.X and 2023.X, as well as several versions of the Meta All-In-One package after reading on this forum that version 64 was more stable. Still, I can't seem to achieve anything clean.

Am I the only one facing this issue? Is there a way to fix it ?