Voice SDK (Feedback/Issues)

Participant

So the problem is the NLU is coming back with a confidence < .6. This means it isn't confident that the intent it matched is the correct one. You can crank that confidence slider down lower, but what you really need to do is to go into the Wit.ai website and train a few more utterances for those intents.

^ confidence must be > this value (0.6) before the returned intent is considered an actual match.

I

Inpu

Up and Coming

Hi,

Thanks for coming back to me !

So I did another test :

I created the intent "Repeat sequence" and added the following utterances :

> Do it again

> Repeat

> Again

> Start again

> Play it again

If I'm using the Curl commands with the utterance "Start again", I'm getting a confidence of 0.996.

On Unity, if I say "Start again" (which the app understand as I'm doing a Debug.log on the Transcription.events), it still says : "Repeat_sequence matched, but confidence (0,00) was below threshold (0,60)"

I'm not sure what I'm doing wrong at this point ?

Thanks

R

ralphVR

Up and Coming

Voice SDK does not seem to work with Unity 2021.2.2f1. Stack trace below:

Object reference not set to an instance of an object
Request Stack Trace:
at System.Environment.get_StackTrace () [0x00000] in <c183552e59b24622ab6b4273bc2ea328>:0
at Facebook.WitAi.WitRequest.Request () [0x00038] in C:\RalphVR\Easely\Easely\Assets\Oculus\Voice\Lib\Wit.ai\Scripts\Runtime\WitRequest.cs:212
at Facebook.WitAi.Wit.ActivateImmediately (Facebook.WitAi.Configuration.WitRequestOptions requestOptions) [0x000dd] in C:\RalphVR\Easely\Easely\Assets\Oculus\Voice\Lib\Wit.ai\Scripts\Runtime\Wit.cs:374
at Facebook.WitAi.Wit.OnSampleReady (System.Int32 sampleCount, System.Single[] sample, System.Single levelMax) [0x00260] in C:\RalphVR\Easely\Easely\Assets\Oculus\Voice\Lib\Wit.ai\Scripts\Runtime\Wit.cs:225
at Facebook.WitAi.Lib.Mic+<ReadRawAudio>d__55.MoveNext () [0x001cd] in C:\RalphVR\Easely\Easely\Assets\Oculus\Voice\Lib\Wit.ai\Scripts\Runtime\Lib\Mic.cs:301
at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00000] in <809a92a718b54e21a55192c70e75634b>:0
Response Stack Trace:
at Facebook.WitAi.WitRequest.HandleResponse (System.IAsyncResult ar) [0x000cd] in C:\RalphVR\Easely\Easely\Assets\Oculus\Voice\Lib\Wit.ai\Scripts\Runtime\WitRequest.cs:335
UnityEngine.Debug:LogError (object)
Facebook.WitAi.WitRequest:HandleResponse (System.IAsyncResult) (at Assets/Oculus/Voice/Lib/Wit.ai/Scripts/Runtime/WitRequest.cs:363)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

Same project in 2019 worked fine with voice sdk. Is there a recommended version to use?

EDIT: Rolled back to 2020.3.22f1 and the sdk works.

Participant

Yes, this is a known issue with 2021.2.2. We're looking into it. 2021.2.0 works if you want to use the new 2021.2.x releases.

Up and Coming

Where can I find the documentation or instruction to display and activate the microphone button?

I want to try the Hand pinch click on UI button in this page.

hand pinch

Participant

We don't provide any prebuilt microphone button or activation methods with the Voice SDK. The general process:
1. Build your UI and add a mic button to click on with hand pinch click.
2. In the button's on click call your AppVoiceExperience component's Activate() method

Up and Coming

Thank you for the reply yolan ! I have another question about the voice SDK.

If I want to trigger only one word, what would be the intent and entity would be?

I am trying to do something like if a user finds a word and says the word in a game, the user will get a score.

So my utterance will be like apple, banana, coin, Santa or etc. What would be the intents and entity? I am thinking intents are apple, banana, coin... and the entity is the word.

Honored Contributor

I'm finding that utterances are parsed in Unity with a trailing full-stop (punctuation). This makes new utterances appear in Wit.ai, even though the exact same text has already been added and trained. Seems redundant. Is there a reason for the punctuation?

Participant

The period has to do with the asr model being used. The redundancy will just help train the nlu model. You can ignore it or annotate it to improve the model.

Up and Coming

Is there a way to activate voice command without click button and the using the Understanding Viewer?

Honored Contributor

This is a Unity thing, not a Voice SDK issue. You just need to call the Activate() method on your

AppVoiceExperience reference. Doesn't really matter how you do it, doesn't have to be a button. For example, you can call _appVoiceExperience.Activate(); when a UI panel is opened (to enable voice activation of buttons) or you can do the same thing when the player enters a trigger and is facing an NPC. It's entirely up to you.

Up and Coming

I see! Now I was able to activate it when the button click. Thank you so much.

Do you know how long will it listen after _appVoiceExperience.Activate(); called?

D

devangini9

New Participant

I understand that there is no wake word currently in voice SDK. Is there a workaround for that?

Participant

There isn't a simple workaround for it. You could potentially use an asset from the asset store that handles transcriptions locally on device and trigger wit activation from that. The only catch is this technique impacts perf on Quest so depending on your budget may not be viable for your project.

Up and Coming

Hi, it seems simple thing but I cannot call method from AppVoiceExperience class to another class. I get an error says

Assets\Apartment_Door\Scripts\DoorController.cs(19,12): error CS0246: The type or namespace name 'AppVoiceExperience' could not be found (are you missing a using directive or an assembly reference?)

-----------------------------------------

private Animation doorAnim;
private BoxCollider doorCollider;
public AppVoiceExperience AppVoiceExperience; //this line is getting the error

if (AppVoiceExperience.Activate() && playerInZone)

What I am trying to do is to call an activate method from the AppVoiceExperience class.

Thank you!

Honored Contributor

Posting this as feedback, and for comments...

So I found that going through the lengthy process of manual training and relying on the responses from Wit.ai was making things in my app very unpredictable.

For example, asking the user "What's your name?" and wanting to get the reply back as a custom 'name_response' Intent with an Entity ' name_of_person' was impossible without training Wit.ai with a whole lot of first names (which of course is impossible). These single word responses just came back as 'out of scope' until I actually taught the ai that 'John' is a name, 'Jill' is a name, 'Rajesh' is a name, etc. etc.

So I had to cheat and created a wildcard option is WitResponseMatcher.cs:

        private bool IntentMatches(WitResponseNode response)
        {
            var intentNode = response.GetFirstIntent();
            if (string.IsNullOrEmpty(intent))
            {
                return true;
            }

//JDP add '*' wildcard to ignore intent
            if (intent == intentNode["name"].Value || intent == "*")
            {
                var actualConfidence = intentNode["confidence"].AsFloat;
                if (actualConfidence >= confidenceThreshold || intent == "*")
                {
                    return true;
                }

                Debug.Log($"{intent} matched, but confidence ({actualConfidence.ToString("F")}) was below threshold ({confidenceThreshold.ToString("F")})");
            }

            return false;
        }
    }

Then using the path "text" I was able to get back whatever the user says

Seems very hacky! (I especially hate to change SDK scripts...) So fine for a Hackathon, but not sure how else I could have done it?

Participant

There's actually already the equivalent of a wildcard for intents. Just provide an empty value for intent. We could probably make that UI a little clearer on that. When you're training your app make sure you use the wit/contact builtin entity. It is already trained to recognize names. In the screenshot below I didn't have to train Rajesh when using the wit/contact entity it recognized it out of the box.