Getting raycast collisions using PointerPose ray without using Interaction SDK

Seeing as the PointerPose ray for hand tracking is very smooth, it would be useful for development time-saving to use it with our own UI systems (handling our own raycast physics collisions). However it seems to be locked into Interaction SDK's UI system and only easily taps into raycast hits on Meta interactables. 

Is anyone aware of a way to take advantage of this feature while sticking to their own systems? 

Thanks