Real-world objects impact virtual world

I want real-world objects impact virtual world.

Example - I want to have data/events when the real-world ball crosses the virtual-world score lines, so I can count the score:

Any ideas of how to achieve this?

My thoughts:

1. Capture camera feed with scrcpy and do some computer vision magic on PC.
   • Do you see if it's possible and worth to dig in it?
2. The same CV magic but with capturing video from external camera like a smartphone pointed to my play area.
   
   • In this case there is a benefit - there is no need to look at the ball all the time to have visual data so players could keep their eyes up which is good skill to develop for hockey (and other) players.
3. Track real-world hockey stick by attached controller and detect collisions of virtual hockey stick with score lines.
   • I don't like this approach due to lack of movement freedom - player can't score by pushing the ball of the hockey stick blade. As a player I want to feel that it's the real-world ball crossing the score lines.