Which AR glasses to buy for research with LLMs and Raw data
Hello there, I'm writing you because, together with a group of PhD and Master's students at my university, we are exploring the development of new mobile or web-based applications that can interface with the Meta Ray-Ban or Aria smart glasses via SDK. Our goal is to test our own vision-language models (VLLMs) accessing directly the raw data streams—specifically video and audio—from the glasses and providing contextualized responses through the device’s built-in speakers using our own LLMs. We are particularly interested in whether it is possible to develop a mobile app or even getting access through a web browser that can: Collect and transmit raw sensor data (video/audio) Send processed responses back to the glasses Use Bluetooth or an Android mobile app (possibly via XRCore or Unity) as the communication bridge If this is feasible, could you kindly advise: Which smart glasses model(s) you recommend for this type of development. We want to buy some few glasses to start with What plugins, SDKs, or frameworks would be most suitable We appreciate your guidance and thank you in advance for your support. Best regards, Luis F.
Oculus Audio SDK and Meta XR Audio SDK Geometry based acoustics
I have been using Oculus Spatializer Unity (v47.0) in Unity (2021.3.29f1) to create reverberation based on the geometry of the game environment. It seems that there are a number of inconsistencies and behaviors I cannot explain: 1) When a source changes from visible to non visible (i.e the direct sound becomes occluded) the direct sound remains. Toggling Spatialization enabled resets it to as expected, after a while though it returns to the direct component being included again. 2) A different attenuation curve when reflections are enabled compared to when they are disabled. The former begins as a 1 / r releationship but shows a sharp roll off after around 2.3m. The first point in particular means I cannot use the Oculus Spatializer for my project. Therefore, I had a look at using the Meta XR Audio SDK (v55.0) instead as it is newer and seems to be a replacement for the Oculus Spatializer. However, as far as I can find it does not include the same geometry based acoustic rendering. It only seems to include a shoebox room approximation (described here: https://developer.oculus.com/documentation/unity/meta-xr-audio-sdk-unity-room-acoustics/) and is unable to account for occlusion of the direct sound by the scene geometry. Am I missing something? Or are there any plans to add any sort of geometry based acoustics to the Meta XR Audio SDK in the future? Otherwise, are you able to suggest why this behavior may be happening in the Oculus Spatializer?
quest 3 audio input latency ?
We have an app that works fine on quest 2. On quest 3 we have sync issues. The featuer is recording audio along with existing audio (and it syncs it that playback plays back same as recording) what could be different about quest 3 in this regard? Thank for any clue , cheersAUDIO MIDI AND ANALOG TO THE QUEST
We have found an open source called pd-for-android, written in JAVA, that allows to transform analog and MIDI signals into music. Do you know if it is possible to incorporate it into a Quest VR software and be able to play musical notes with an iRig2 or USB-C connected to the glasses? If it is not possible, is there any other way to send the sound of instruments to the glasses other than through the Quest's microphone?
Audio output problems in Wwise, sound disappears.
I am working on a Quest build, using Unity and Wwise. A two specific points in the Quest build, all audio disappears. The first time it comes back (again at a more or less certain point) the second time it newer comes back. When profiling in Wwise, the Output peak is hitting 774 dB, when the audio dissappears (see picture). None of the sounds playing at the given time, has unregular volume output. The CPU is not doing anything unusual, there is no voice starvations or anything else that would "justify" the behavior. This is ONLY a problem on the Quest. When build for Rift this never happens. I am using Resonance Audio as spatializer, i tried removing it. It didn't help. If I'm removing all ambience, then the problem doesn't happen. Even though the ambience is not doing anything out of the usual in the instances. Does anyone have experience with something similar to this? or maybe a solution? I appricated anything you can throw at me. Questions, tips or solutions. Thank you in advance.Hz level of speakers in Oculus Quest 2?
I'm curious as to how low of a Hz level the headphones goes down to. Does it go down to only 50Hz? Does it go down to 16Hz? I'm curious because I'm planning on doing frequency work with others using the headset, and I'd rather not people have to have headphones for plugging in, if possible.About music permission
We have a number of songs to play into a VR experience in development and we have questions of legal nature. In addition to having permission from the author to use these songs, is there any other process required by Oculus? Are there any other requirements to meet in the Store and prove that we have that permission?
