Meta Spatial SDK Development

Can I get a passthrough camera stream open while running the Spatial SDK? I modified the Starter Template (https://github.com/meta-quest/Meta-Spatial-SDK-Templates/tree/main/StarterTemplate/app) just to open the camera but it seems to cause the camera to disconnected immediately.I added a startCamera method to run when one of the panels is selected (instead of running the video): private fun startCamera() { if (checkSelfPermission(Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) { Log.i("MainActivity", "Camera permission not granted") } if (checkSelfPermission("horizonos.permission.HEADSET_CAMERA") != PackageManager.PERMISSION_GRANTED) { Log.i("MainActivity", "horizonos.permission.HEADSET_CAMERA permission not granted") } val cameraManager: CameraManager = getSystemService(Context.CAMERA_SERVICE) as CameraManager val cameraIds: Array = cameraManager.cameraIdList var cameraId: String = "" for (id in c

Hi Team,I am struggling to perform screen navigation from a VR screen to another VR screen.I checked the samples but could not get any clue.Is it even supported or do we have to survive with loading different scenes in the launcher activity itself?Yes, I have added the activity in the manifest file and it works when I make that 2nd activity as launcher. However when I try to launch it from the 1st activity it simply crashes.This is the error I get ...FATAL EXCEPTION: main                                                                                                    Process: com.mixed.reality.factory, PID: 26659                                    &n

I was curious if there is a way to retrieve the distance the system ui appears from the headset when spawn. it would be nice if we could initially place our ui the same distance away for consistency

Not much detail here:https://developers.meta.com/horizon/reference/spatial-sdk/v0.7.0/root/com.meta.spatial.toolkit/body-joint/h-e-a-dIs it directly above where the spine meets the skull and behind the eyes? i.e. the "head atlas"? Is this a good pivot point for billboarding view screens?

I'm developing an Immersive App for Meta Quest using the Meta Spatial SDK in Android Studio with R8 minification enabled. When building a signed APK, I encounter the following error:Missing classes detected while running R8. Please add the missing classes or apply additional keep rules that are generated in D:\AndroidProject\[package-name]\app\build\outputs\mapping\questRelease\missing_rules.txt.Missing class horizonos.app.container.TaskContainer (referenced from: horizonos.app.container.TaskContainer com.meta.spatial.runtime.PanelDisplay.taskContainer and 4 other contexts)Missing class horizonos.app.container.TaskContainerManager$TaskContainerCallback (referenced from: horizonos.app.container.TaskContainer com.meta.spatial.runtime.PanelDisplay.tryGetTaskContainer(android.content.Context, android.content.Context) and 2 other contexts)Missing class horizonos.app.container.TaskContainerManager (referenced from: horizonos.app.container.TaskContainer com.meta.spatial.runtime.PanelDisplay.t

Hello,I’m fairly new to the Meta Spatial SDK, and while reviewing the Controller Input documentation, I couldn’t find any interface that allows retrieving the joystick’s travel range.What I came across were examples treating joystick directions as “Buttons” (like ButtonThumbRL). However, what I’m trying to do is map the joystick’s travel to an attribute like “velocity.” Is there any way to access this information?If I’ve missed something, my apologies, thank you!

Is there a way to run a method in an activity once every frame instead of every tick? I'm aware that Systems exist, but they seem to be tied to the activity tick rate. Side note, my system also seems to not consistently run at 90 ticks per second even though the contents of its execute() method take less than 11ms consistently. I'm running a simple project with this system in it and nothing else-- the tick rate jumps between 45 and 90 sporadically.

Im a beginner in 3D development. I import a glb object into environment.env with predefined textures(baseColorTexture, normalTexture, occlusionTexture), and I try to change the appearance(runtime) when clicking buttons. Here are my code,  but It didn't work to change the texture (baseColor or baseColorTexture). Is there anyone could help me? Thank you very much.private fun changeMaterial(node : SceneObject) {        val material = node.mesh?.materials?.get(0)        var mesh = node.mesh        var meshMaterial = node.mesh?.getMaterial(0) as SceneMaterial        var test_texture = SceneTexture(activity.getDrawable(R.drawable.test_texture))       //var texture_name = "baseColor" //--> exception: baseColor not found        var texture_name = "occlusion" //--> worked and texture is changed        meshMaterial.setTexture(t

I only see the alphaMode in the SceneMaterial constructor, but not in the SceneMaterial.custom function? Not is there a settable property for this? I want objects with some of my custom materials to use the TRANSLUCENT_POST option.

Referencing this post:there doesn't seem to be a way to set CPU/CPU perf levels in SpatialSDK. Can this be done?

It seems the pointer max distance is limited. Is there a way to increase this value, in order to be able to select ui items from a panel that is very large or far away for example ? Thx

Hi, I'm new to development, so maybe my problem is too simple. But still, thanks in advance for any recommendations! I'm trying to create a very simple AR app for Meta Quest 3, following the official instructions on Meta's website. However, the result so far leaves much to be desired. My goal is to fix an object from a prefab onto a table in the real room. Later, I also plan to add UI buttons to the object to enable interactions — but that comes later. I'm using Unity version 6000.1.6f1. I’ve installed the Meta All-in-One SDK (v77) and decided to start by using the provided Building Blocks. I also installed the OpenXR plugin, as recommended. In Player Settings, under Active Input Handling, I selected Input System Package. In XR Plug-in Management, I selected OpenXR > Meta XR Feature Group.Under OpenXR settings, in Enable Interaction Profiles, I selected: Oculus Touch Controller, Meta Quest Touch Pro Controller and Plus Controller Profile. In OpenXR Feat

Hey, I am trying to make a shooting range application, and I wish for the weapon in the game to be in the place of the controller. I don't have a beefy computer and meet the bare minimum, and this was the only way I code for Quest 3 at the moment, which is through the Meta Spatial SDK. I see in the spatial Editor there is an AvatarBody Component that allows you to attach things to it, but I am reading that it allows you to use it in the sense of making your own Playable Avatar and using that to attach things to your avatar. there are no samples out there yet anyone figured out how to replace the controller model with another object yet 

We are trying to implement a custom tracking system along with Quest's tracking. We want the headset to be able to detect custom gloves that we put in our hands, is it possible to train the current ml models or the Spatial sdk currently only uses the pretrained model? 

Hi, I see a paradox here... Apparently, devices enrolled on HMS cannot rely on Space Setup.What does this mean? No mixed Reality?Thank you.GL

I am trying to add to the batting tutorial and I am having issues spawning the ball above the players head. I have implemented a couple different ways but they dont work. They will compile but the collision detection doesnt work. I have recieved errors such as failed to call method 'start' as well as having the code work with error detection but break once the player moves. This is my current code: import * as hz from 'horizon/core';type State = {  originalPosition: hz.Vec3}class BallScript extends hz.Component<typeof BallScript> {  static propsDefinition = {};  originalPosition!: hz.Vec3;  start() {    // Listen for ball collisions with the ground.    this.connectCodeBlockEvent(this.entity, hz.CodeBlockEvents.OnEntityCollision,      (collidedWith: hz.Entity, collisionAt: hz.Vec3, normal: hz.Vec3,        relativeVelocity: hz.Vec3, localColliderName: string, otherColliderName: string) => {  &nb

The trySetMapValue is using the generic key type for the value:public final inline fun <reified T, reified V> tryGetMapValue(attribute: kotlin.Int, key: T): V? { /* compiled code */ }public final inline fun <reified T, reified V> trySetMapValue(attribute: kotlin.Int, key: T, value: T): kotlin.Boolean { /* compiled code */ }Is this the intended behavior? Also, what's the purpose of the Entity map? I'm trying to use it to save the internal state of the entity, but I don't know if this is ideal.

How do I turn off the default controller cursor (the annular shade marked off on surfaces by the incident beam of one of the controllers) for a given mesh or material?

Hi at all!I'm trying to start another activity with a different panel. From starter template I take the button and prepare the intent.I want to re-use the original environment and register a new panel for the new activity, but if I try I have this: java.lang.Exception: Native Assert. FILE=Utils/AndroidUtils.cpp:102 MSG=Global asset manager can only be set once at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3724)  This is my target activity: import com.meta.spatial.castinputforward.CastInputForwardFeature import com.meta.spatial.core.SpatialFeature import com.meta.spatial.debugtools.HotReloadFeature import com.meta.spatial.runtime.LayerConfig import com.meta.spatial.toolkit.AppSystemActivity import com.meta.spatial.toolkit.PanelRegistration import com.meta.spatial.vr.VRFeature class TestActivity : AppSystemActivity() { override fun regis

I have no problem creating SceneTexture objects using ARGB_8888 bitmaps, but when I try to use RGBA_F16 the resulting textures are stretched like it is only reading 8-bit data. Here is an example of how I am trying to create an RGBA_F16 bitmap from a W*H buffer (named sdf) of Float's (I am treating these a "gray" values -- i.e. using the same value for RGB and using 1 for alpha):val sdfBitmap = Bitmap.createBitmap(W, H, Bitmap.Config.RGBA_F16) val buffer = ByteBuffer.allocate(W*H * 8).order(ByteOrder.nativeOrder()) for (i in sdf.indices) { var g16 : Short = Half.toHalf(sdf[i]) var a16 : Short = Half.toHalf(1.0f) buffer.putShort(g16) buffer.putShort(g16) buffer.putShort(g16) buffer.putShort(a16) } buffer.rewind() sdfBitmap.copyPixelsFromBuffer(buffer) I am create the texture as followsval sdfTexture = SceneTexture(sdfBitmap, samplerConfig)which is later used for SceneMaterial with a custom shader.Are RGBA_F16 textures supported or am I doing something w

I don't see a SceneTexture constructor that takes a bitmap as an argument AND disables or enables mipmapping.val samplerConfig = SamplerConfig( mipmapMode = Filter.NEAREST, minFilter = Filter.NEAREST, magFilter = Filter.NEAREST, addressModeU = AddressMode.CLAMP_TO_EDGE, addressModeV = AddressMode.CLAMP_TO_EDGE, lodBias = 0.0f ) val myTexture = SceneTexture(myBitmap, samplerConfig)Are mipmaps auto always generated here? Is there anyway to control this (e.g., what if I don't want mipmapping)?Other SceneTexture constructors have a `numberOfMips` argument, but none of those take a Bitmap as an argument. Perhaps this is controlled at the SceneMaterial level, but I don't see anything concerning mipmaps there.

I would like to build a feature that would let you use your vr headset in "partial" pass thru. In other words, I would like the option to be able to see my real feet and a little of the surrounding area when in vr. The app Fluid has this feature where you can toggle pass thru, vr or a partial pass thru and see your feet. It is similar to some that let you see your real keyboard in vr. Anyone have any thoughts of this?

There are a number of "setter methods" for SceneObject, but I am not finding where the corresponding "getter methods" are.e.g. getName(), getPosition(), getRotation(), isVisible(), etc...

On the host we can query the position and orientation of the head as follows:fun getHeadPose(): Pose {    val head = Query.where { has(AvatarAttachment.id) }          .eval()          .filter { it.isLocal() && it.getComponent<AvatarAttachment>().type == "head" }         .first()    return head.getComponent<Transform>().transform }In a material shader we can get the position of the left and right eye via the uniforms:g_ViewUniform.eyeCenter0 and g_ViewUniform.eyeCenter1But is there either...* ....a way to get the position of the head in the shader (short of passing it as a uniform attribute every frame)?or* ... a way to get the position (or relative position) of the left and right eye on the host?  

I see in the Custom Component Starter Sample the execute() method fetches the head pose as follows:var headPose: Pose = getScene()!!.getViewPose()I assume this returns the position of the head sensor on the device in the current world coordinate system. Is there a way to fetch the positions/poses of the left and right eye relative to the head position?Follow up: is it possible to have different transforms in the ECS components depending on which eye is being rendered?