Which AR glasses to buy for research with LLMs and Raw data
Hello there, I'm writing you because, together with a group of PhD and Master's students at my university, we are exploring the development of new mobile or web-based applications that can interface with the Meta Ray-Ban or Aria smart glasses via SDK. Our goal is to test our own vision-language models (VLLMs) accessing directly the raw data streams—specifically video and audio—from the glasses and providing contextualized responses through the device’s built-in speakers using our own LLMs. We are particularly interested in whether it is possible to develop a mobile app or even getting access through a web browser that can: Collect and transmit raw sensor data (video/audio) Send processed responses back to the glasses Use Bluetooth or an Android mobile app (possibly via XRCore or Unity) as the communication bridge If this is feasible, could you kindly advise: Which smart glasses model(s) you recommend for this type of development. We want to buy some few glasses to start with What plugins, SDKs, or frameworks would be most suitable We appreciate your guidance and thank you in advance for your support. Best regards, Luis F.Meta Quest Browser & BabylonJS = Slow FPS
I'm trying learn about WebXR and I'm finding that my library of choice isn't playing well in the Meta Browswer, locking at about 30 FPS. Now, I see 60 FPS on the desktop with Chrome and I know that's because Chrome is locked in a vsync. I can pass Chrome some parameters to unlock the FPS limiting. At that point, I'm seeing around 500 fps. Chrome works great with Link, but I'm looking to work with the native Meta Browser inside the Quest. I've found other engines, such as the Wonderland Engine, demonstrates 72 FPS in the Quest with their app Escape Artists (https://esc.art). A-Frame applications show up in the browser at 90 FPS. So my question is, is there some sort of parameter or command the WebGL lib should be passing to the browser to speed it up? Or, is this likely to be some sort of bug in BabylonJS that is locking it to 30 FPS and the browser doesn't have any sort of artificial rate limiting?
Oculus Browser - Problem receiving/posting web requests
Hi, we have an vr app that is embed on a web app that loads data from a node-express API. Since the start of this week, more precisely on tuesday, the default quest 2 browser stopped to handle the data from http requests to our API. We can't login, load courses list on website or fetch any data from API inside the WebXR application embedded on the site. Monday was working well, the next day (in a very bad lucky day that we presented the product to the client), nothing works. Works well on alternative web browser, like Wolvic.
PIPELINE_ERROR_DECODE in Meta Quest Browser
I'm very new to Meta Quest development, so please let me know if this is the wrong place. I'm trying to stream video data to my Meta Quest 3, but the Meta Quest Browser keeps throwing a PIPELINE_ERROR_DECODE error whenever I feed it data via SourceBuffer.appendBuffer(). I know this is an issue specifically with Meta Quest Browser as I've tested this code in Chrome/Firefox/Edge on Linux and Windows and they all correctly display the video. It is only when I open this HTML file in the Meta Quest Browser that there is an issue displaying the video. The code below reproduces the error described above: https://gist.github.com/BFergerson/808cc4c9c5eba9e0efc05033a431ced2 <!DOCTYPE html> <html> <head> <title>IE delay repro</title> </head> <body> <video id="app" width="160" height="90"></video><br> <button onclick="video.play()">Play!</button> <div id="time"></div> <script type="text/javascript"> var codecs = ['video/mp4; codecs="avc1.42E01E"']; codecs.forEach(function(codec) { if (MediaSource.isTypeSupported(codec)) { console.log(codec + ' is supported'); } else { console.log(codec + ' is not supported'); } }); var video = document.getElementById('app'); video.addEventListener('error',function(e){ console.error(e); }); window.MediaSource = window.MediaSource || window.WebKitMediaSource; if (!window.MediaSource) { console.log('MediaSource API is not available'); } var startTime = 0; var stopTimer = function () { document.getElementById('time').innerHTML = '' + ((new Date()).getTime() - startTime) + ' ms'; video.removeEventListener('playing', stopTimer); }; var mediaSource = new window.MediaSource(); video.src=window.URL.createObjectURL(mediaSource); video.addEventListener('playing', stopTimer); var sourceBuffer; var buffer = []; //Frames are appended here when they 'arrive' via setInterval var readFromBuffer = function () { // Returns if the buffer is empty or has state updating if (buffer.length === 0 || !sourceBuffer || sourceBuffer.updating) { return; } try { var data = buffer.shift(); sourceBuffer.appendBuffer(data); } catch (e) { console.log(e); } }; mediaSource.addEventListener('sourceopen', function () { console.log('source open'); sourceBuffer = mediaSource.addSourceBuffer('video/mp4; codecs="avc1.42E01E"'); mediaSource.duration = Infinity; sourceBuffer.addEventListener('updateend', readFromBuffer); }); //jscs:disable /* eslint-disable */ var frames = ['...remove for brevity...'].map( function(str) { var raw = window.atob(str); var rawLength = raw.length; var buf = new ArrayBuffer(rawLength); var array = new Uint8Array(buf); for(i = 0; i < rawLength; i++) { array[i] = raw.charCodeAt(i); } return buf; }); /* eslint-enable */ video.addEventListener('play', function () { console.log('play clicked, starting streaming!'); var i = 0; // Adds a frame to the buffer every 100 ms var interval = setInterval(function () { if(i === 0) { startTime = (new Date()).getTime(); } if (i < frames.length) { buffer.push(frames[i]); i++; console.log('added frame to buffer'); readFromBuffer(); } else { clearInterval(interval); } }, 100); }, false); </script> </body> </html>
