Forum Discussion

dynamaxz's avatar
dynamaxz
Honored Guest
5 days ago

Record and replay real hand pose at runtime (Meta Interaction SDK – Unreal)

Hi everyone,
I’m currently working with the Meta Interaction SDK in Unreal Engine (UE 5.6) and using hand tracking only (no controllers). I’m using the ISDK Hand Rig Component for both hands.

What I’m trying to achieve is:
I want to capture the user’s real hand pose at runtime, save that pose, and then reapply (replay) that exact pose later on command.

So, basically my requirement is,
Is there a built-in way in Meta Interaction SDK to record and replay hand poses ?
What’s the best way to temporarily override hand tracking and apply a custom pose?

Any guidance, suggestions, or pointers would be really helpful.
help in any way you can. Thanks.

1 Reply

  • Degly's avatar
    Degly
    Start Partner
    2 days ago

    As far as I know there’s no built-in record/replay for hand poses in Meta Interaction SDK (Unreal).

    What I would do:

    Record part

    • Read joint data each frame from the hand (e.g. OVR/ISDK hand skeleton joints)
    • Store per-joint transforms (position + rotation)

    After record replay

    • Disable or override hand tracking input
    • Apply your saved transforms back to the hand skeletal mesh/bones

    Key points:

    • You need a custom “input override” layer (tracking OFF → playback ON)
    • Smooth/blend when switching (tracking ↔ replay) to avoid snapping
    • Make sure you’re writing in the same space (local vs world)

    So: capture joints → store → override skeleton → reapply

→ Find helpful resources to begin your development journey in Getting Started

→ Get the latest information about HorizonOS development in News & Announcements.

→ Access Start program mentor videos and share knowledge, tutorials, and videos in Community Resources.

→ Get support or provide help in Questions & Discussions.

→ Show off your work in What I’m Building to get feedback and find playtesters.

→ Looking for documentation?  Developer Docs

→ Looking for account support?  Support Center

→ Looking for the previous forum?  Forum Archive

→ Looking to join the Start program? Apply here.

 

Recent Discussions