Forum Discussion

Thormj's avatar
Thormj
Protege
8 days ago

Bug Report MRUK 83.0.4 -> 85.0.0 _cameraAccess.GetCameraPose() bad

It looks like you shifted from "headpos" to "camera.main" in everything.
But... _cameraAccess.GetCameraPose()  is still using HeadPos which... was somewhere near my chest?

To reproduce:
Take CameraKit demonstration for QR codes.  Run with 83.0.4 in an unmapped space.
Works fine, the green quad covers/is close to the QR code.

Now copy/paste the code/objects into a newly set up 6003.9f1 project with MRUK85.0.0.  Run again in an unmapped space.  The QR is off.  The green quad is someplace else.  if you put tracers and debug flavor in, you will find that -- at least Y is ~34 cm off (look at CP)...
Debug Spice:

    private static Ray BuildWorldRay(QrCodeResult result, Vector2 uv)
    {
        var viewport = ToViewport(uv);
        var intrinsics = result.Intrinsics;
        var sensorResolution = (Vector2)intrinsics.SensorResolution;
        var currentResolution = (Vector2)result.captureResolution;
        if (currentResolution == Vector2.zero)
        {
            currentResolution = sensorResolution;
        }

        var crop = ComputeSensorCrop(sensorResolution, currentResolution);
        Debug.Log($"V {viewport} ");
        Debug.Log($"I -{intrinsics.PrincipalPoint} - {intrinsics.FocalLength} ");
        Debug.Log($"C {crop}");

        var sensorPoint = new Vector2(
            crop.x + crop.width * viewport.x,
            crop.y + crop.height * viewport.y);
        Debug.Log($"S {sensorPoint}");
        var localDirection = new Vector3(
            (sensorPoint.x - intrinsics.PrincipalPoint.x) / intrinsics.FocalLength.x,
            (sensorPoint.y - intrinsics.PrincipalPoint.y) / intrinsics.FocalLength.y,
            1f).normalized;
        Debug.Log($"CP {result.cameraPose.position} R{result.cameraPose.rotation}");

        var worldDirection = result.cameraPose.rotation * localDirection;
        return new Ray(result.cameraPose.position, worldDirection);
    }

To Fix:

    private async Task<CaptureFrame?> AcquireFrameAsync()
    {
        while (true)
        {
            if (_cameraAccess && _cameraAccess.IsPlaying)
            {
                var texture = _cameraAccess.GetTexture();
                if (texture)
                {
                    return new CaptureFrame
                    {
                        Texture = texture,
                        Pose = TMJGetCameraPose(), //_cameraAccess.GetCameraPose() is off...
                        Intrinsics = _cameraAccess.Intrinsics,
                        Resolution = _cameraAccess.CurrentResolution
                    };
                }
            }
            await Task.Delay(1000/fps);   //16=60fps
        }
    }

    private Pose TMJGetCameraPose()
    {
        var headPose = new Pose(_cmain.transform.position, _cmain.transform.rotation);

        // Apply VisionKit lens offset in head-local space
        Vector3 cameraPos = headPose.position + headPose.rotation * _cameraAccess.Intrinsics.LensOffset.position;
        Quaternion cameraRot = headPose.rotation * _cameraAccess.Intrinsics.LensOffset.rotation;

        // 4. Compose final camera pose
        return new Pose(cameraPos, cameraRot);
    }

And in Start(), _cmain = Camera.main();

Result:

Now the green quad is in the same place as it was in the Example code that uses 83.0.4

Yay!

2 Replies

  • Degly's avatar
    Degly
    Start Partner

    Nice! There could be a head pose vs camera sensor offset (LensOffset) mismatch. Did you submit the bug report?

    Recommendation for others hitting this:

    • Don’t rely only on GetCameraPose() for CV alignment in 85+
    • Use Camera.main + intrinsics.LensOffset instead (like your TMJGetCameraPose)

    Until Meta updates it, treat GetCameraPose() as not 100% reliable for image-space → world-space mapping.

  • steve_40's avatar
    steve_40
    Honored Visionary

    Meta has increasingly centralized feedback into private channels (in‑app reporting, developer programs, internal analytics) rather than open forums where users can see and upvote ideas. You should submit that as an issue to the Developer feedback Center.

→ Find helpful resources to begin your development journey in Getting Started

→ Get the latest information about HorizonOS development in News & Announcements.

→ Access Start program mentor videos and share knowledge, tutorials, and videos in Community Resources.

→ Get support or provide help in Questions & Discussions.

→ Show off your work in What I’m Building to get feedback and find playtesters.

→ Looking for documentation?  Developer Docs

→ Looking for account support?  Support Center

→ Looking for the previous forum?  Forum Archive

→ Looking to join the Start program? Apply here.

 

Recent Discussions