Using Controller as a real world anchor and hand tracking separately

I am using MetaXR Unity SDK with passthrough and hand tracking, now I need to have the controller set as well so that I can use it as a real world anchor, however when i enable both SDK fails to differentiate transforms, It detects controller when I pick it up in hand but that's not  as its own.