Not seeing each other in my world - why?

So, I took Build Your First Game sample, added a few things there, added a trigger with simple script to change material on an entity. It's not a public world. I published the world, added a few testers and sent invites to them.

When I join the world in Quest 3, everything works for me. However, when another tester joins it - I don't see them at all. They don't see my avatar, but they see white label where I spawned (which doesn't move when I move). Trigger and script don't work for them either. We can be on voice call talking , but that's about it.

What am I missing ?

P.S. I had my Desktop Editor open with my world loaded. I saw one of the testers joining in and I saw his avatar. Then I pushed play button and joined him in the world. Everything worked as expected. However, when we both join from HMDs, we don't see each other (I join via pressing "i" icon, then 3 dots icon on the right and choosing Preview World).