Virtual finger physics, applytorque, parenting problem

Hi All,

My question is based on this setup:

I created a simple
Finger model that has 3 digits in Unity. It opens and closes based on the bend of a flex sensor that follows my real finger in a data glove.

In Unity I named the the objects that make up the finger pointer1, pointer2, pointer3 from base to tip. I created 3 empty game objects named pPivot1, pPivot2, pPivot3 that I used as parents of the pointer digits respectively. The hierarchy is: pPivot1>pointer1>pPivot2>pointer2>pPivot3>pointer3
I rotated each pivot through their transforms like this:

pPivot1.transform.localEulerAngles = new Vector3(x-Bend,0f,0f);
pPivot2.transform.localEulerAngles = new Vector3(x-Bend/2,0f,0f);
pPivot3.transform.localEulerAngles = new Vector3(x-Bend/2,0f,0f);

This worked perfectly. The virtual finger bent as my real finger bent the flex sensor with almost imperceptible latency.

However, since I wanted the finger to
be able to interactive with the scene objects through physics, I tried another method by using rigidbodies, colliders and applytorque instead of rotating the transforms directly.

I created code that affected 1 digit of the finger by using
physics as mentioned above.
This worked well in that I could move the 1 digit in response to my finger and it interacted with other dynamic rigidbodies.

However, the problem is that the parenting structure shown above that I used when rotating via transforms doesn't work when rotating via applytorque. I have experimented with every combination of hinge, kinematics on/off, parenting etc., but can't make it work.
I have searched online and read the Unity docs on physics but I still have not found a way to replicate through physics what I can do affecting transforms directly.

I hope someone can point out
a way to do this or something I'm missing.

Also, I did find this website: https://serrarens.nl/en/tutorial-grabbing-virtual-hands-in-unity/ but
What he is doing it not the same thing that I'm trying to do. I don't want to simply parent an object to the virtual hand. I want the hand to pick up objects with physics alone.

Thanks so much in advance.