Integrating Noesis UI Gizmo with data/event roundtrip using typescript in Horizon Worlds
(Also posted on Noesis forum)
Meta's documentation for using a Noesis Gizmo and driving UI changes via typescript are pretty sparse. I managed to get the basics set up, and I can make and see the UI in the World, but that's about it, no interaction. I got to that point by following this video: https://youtu.be/cu-u8cTYomg?si=Zr-JKNH5hJAt3WD5
Meta's documentation only has snippets of code here and there and I'm not able to follow it. I have got my player interacting with the Gizmo, which brings up the focused interaction mode and I can click buttons on the Noesis UI panel, but I'm stuck where I don't know how to drive UI data from the typescript nor respond in the typescript from UI events. I'm trying to get tab navigation going and buttons that affect game state/various meters, and I want buttons that need to be held down for a specified duration before their command is executed, so I'll need to intercept button click/down/up events.
I learn better from videos but this documentation here is hard for me to follow:
https://developers.meta.com/horizon-wor ... and-events
Any guidance is appreciated.
I have got some interaction working but have hit some blockers:
In terms of data flow, I understand how to assign the data context in order to send values to Noesis but cannot for the life of me find a way to read values set in Noesis. My example that I've been trying to solve is to get a slider working - the slider is draggable and can change values within Noesis that bind to the same bit of data BUT I can't seem to get this data to flow from Noesis into Horizon, data only seems to flow Horizon -> Noesis.The only way in which I can get information out of the UI is from (parameterless) events, e.g. from button presses.
I'm crossing all fingers that this is because they are yet to release this SharedMode that will resolve the data transfer issue, it would seem like such an own goal for them to do all the work to support a great tool like Noesis but limit interaction to button presses. Alternatively, hoping that I'm being an idiot.
In case you're not at this point yet:
Using this script, I can see that data I set to "Value" can inform the Noesis UI, e.g. move some rectangle around. Also events bound to HelloWorld get triggered.
import * as hz from 'horizon/core'; import { NoesisGizmo } from 'horizon/noesis'; class NoesisTest extends hz.Component<typeof NoesisTest> { static propsDefinition = {}; private time = 0; start() { if (this.entity.owner.get() == this.world.getServerPlayer()) return; var noesis = this.entity.as(NoesisGizmo); var dataContext = { HelloWorld: () => { console.log("Received noesis event"); }, Value: 0.5, }; noesis.dataContext = dataContext; this.connectLocalBroadcastEvent(hz.World.onUpdate, (payload: {deltaTime: number}) => { this.time += payload.deltaTime; dataContext.Value = Math.sin(this.time) * 100; console.log(dataContext.Value); }); } } hz.Component.register(NoesisTest);
I believe dataContext is all the values in all datasets in one object, so if you have 3 structures then all their aggregated properties flow through dataContext.Setting up the Value binding is just a case of selecting this in the Context tab of the Bindings:
Setting up the button listener tripped me up for a while, you set the Binding in the Command input BUT you have to do it as a context binding, I stupidly keep writing the event name in the text box:
I've found the EventTriggers to also be pretty useful for getting data out of Noesis:
You add these to UI components, then you can specify what UI event (e.g. MouseUp) and can invoke commands from this to get events fired within Horizon:
but do note that mouse up != touch up which will be the mobile device interaction..
Hope this braindump is somewhat helpful! Add me on Discord if you want to muddle through this further @JonkWongle