Forum Discussion

debarjo's avatar
debarjo
Honored Guest
21 days ago

[Spatial SDK, Bug] 2D HitInfo from PointerEvents is wrong when interacting with 2D curved panels

Hi,

I am using the Meta Spatial SDK to interact with Webviews and UIs within curved panels.
While the cursor is correctly rendered by the IsdkDefaultCursorSystem, UI and web interactions do not intersect correctly leading to not being able to interact with curved panels since the HitInfo.textureCoordinate does not take in account the curvature of the panel clamping values erroneously.

How to test and validate the issue:
Use the AnimationsSample of the Meta SDK Samples,  place UI buttons on corners of the main panel, switch to a curve panel and try to use the buttons. 

Expected behaviour:
Independently of  the curved panel curvature, reachable buttons should be usable when the cursor overlays it.

Current behaviour:
UI elements are non interactable when the curvature of the panels differs from flatness significantly.

Source of the problem
PointerEvent.HitInfo.textureCoordinate provided by the IsdkSystemNative.tick function are incorrect leading to misbehaviour for any subcomponent that rely on  2D coordinates such as the ones  providing input to the Android View system and any UI element (i.e. IsdkSystem, IsdkInputListenerSystem). 

Note: SDK Systems such as the IsdkDefaultCursorSystem that rely directly on PointerEvent.HitInfo.position and  PointerEvent.HitInfo.normal will present 2D information such as the Cursor correctly.

Current workaround
Override IsdkSystemNativeApi tick method and fix hitInfo.textureCoordinate computation when curved panel entities are detected

interaction sdk
Spatial SDK

3 Replies

  • Degly's avatar
    Degly
    Start Partner
    13 days ago

    Yes! I have seen this and connects with a known class of issues: ray hit is correct in 3D (position/normal), but the reported 2D UV (textureCoordinate) is still computed as if the panel were flat, so it clamps/warps on curved surfaces. That’s why the default cursor looks fine, but UI/WebView input (which depends on UV) misses.

    Until that gets patched, fixing textureCoordinate for curved panels is the most production-safe approach.

  • leweaver's avatar
    leweaver
    Meta Employee
    8 days ago

    Thanks for the detailed bug report on this issue!

    We've been able to reproduce this and have a fix in the works for an upcoming release. Once patched, the default IsdkSystemNativeApi tick will return the correct texture coordinate values for curved panels, which should resolve curved panel view behavior.

    - Lewis

    • debarjo's avatar
      debarjo
      Honored Guest
      8 days ago

      Thanks, looking forward to see the fix.

      In the meantime, the tick output can be corrected as following:
      newTextureCoordinate.x = asin((textureCoordinate.x-0.5) * angle) / angle + 0.5
      where angle is the field of view of the curved panel in radians.

→ Find helpful resources to begin your development journey in Getting Started

→ Get the latest information about HorizonOS development in News & Announcements.

→ Access Start program mentor videos and share knowledge, tutorials, and videos in Community Resources.

→ Get support or provide help in Questions & Discussions.

→ Show off your work in What I’m Building to get feedback and find playtesters.

→ Looking for documentation?  Developer Docs

→ Looking for account support?  Support Center

→ Looking for the previous forum?  Forum Archive

→ Looking to join the Start program? Apply here.

 

Recent Discussions