Mind controller, speech recognition, and front facing camera

Yep.

All of this has been covered and people are working on projects that involve these accessories (Although I'm not sure I've seen anything about voice recognition).

It's certainly exciting, I can't wait to see what people come up with. ( I probably just typed the most used sentence on the forums :D)

I personally can't wait until EEG headsets or other Neural control schemes are good enough for precise game controls. The possibilities then...Just wow.

Yes agreed, exciting is defiantly the word :D I believe the oculus is the next big name in gaming, implications will go much further im sure!
Anyone know their stock symbol I wanna buy shares!? Lol

http://www.mtbs3d.com/phpbb/viewtopic.php?f=140&t=16324

Link to video I found on EEG controlled gaming whilst in a VR (ish) world (video at the bottom of the page)

I think speech recognition for NPC interaction can really go a long way in terms of immersion. Basing interactions around a well-designed but constrained dialog can make it seem quite natural and real. This could take games (especially adventure) to the next level, further blurring the lines between movies and games.

Speech recognition can also have benefits for non-game applications. Imagine a VR 3D modelling program - instead of clicking on menus, you could literally call them up instead, increasing efficiency.

Fortunately, the dialogue may not have to be completely constrained to one game at a time. =)

I have been thinking about this for some time now and I have also done a bit of research. - At a number of large universities there is a research going on into Virtual Humans. Virtual Human research appears to be very ambitious. Even my casual observation has me convinced I cannot summarize it in a meaningful way since it deals with the many aspects of what it means to be human.
Of more immediate relevance to us is that the work should yield a 'grammar' that can be adapted for NPC speech recognition in games. This grammar is a collection of words that an underlying engine or wizard can recognize in various combinations.

Right now there exists grammars which are mainly developed to let people interact with telephone services. These telephone services usually require you to answer a rather small set of simple questions with a small set of possible answers. Technically what you do when interacting with them is triggering a collection of scripts to exclude all options until you arrive at a single pre-determined option which is then sent to output. This output is only relevant to the service provided by the number that you called. This isn't interesting for emulating interaction with humans. These services are called Expert Systems since they only deal with one very limited subject per system.

It should be possible to define an 'NPC' as a tiny and unrealistic subset of a Virtual Human, who is concerned with things like fulfilling their role in the game while in a believable way avoiding the collateral damage that usually follows in the the player's wake. - For good, human-like reason the player could want to know how the NPC survived and the same excuse might not work twice. :twisted:

A generalized, speech-aware NPC should probably be an expert on game systems and the perils which faces it there. - It should emulate gamers, but with more controlled homicidal tendencies and longer attention spans.
The grammar and corresponding logic in the game engine would talk about topics of e.g. danger, bartering, navigation, revenge, loyalty, teamwork... perhaps even construction and demolition. The logic for those discrete components are already well-known to game devs so mapping a grammar to those components appears at least to me to be 'trivial'. I also wrote about speech-based story content here: viewtopic.php?f=38&t=235

Making the NPC utilize these components together in a semblance of intelligence may require some very deep thought. Fortunately, a single guy working over a few weekends with some known, fancy AI algorithms can produce stuff like this:



I think this game-aware AI could be a useful stepping-stone in the quest for Strong AI, such as pursued by http://opencog.org/. It would be extremely cool if one day you could install their latest research into your favourite game as a mod package. :mrgreen:

I agree with this.

And it doesn't need to be as complicated as some people would imagine.

For example, I was thinking about this in regards to Doom 3 BFG edition. Currently, in Doom 3, the NPC interaction consists of: cutscenes, NPCs that say a predefined line when you walk up to them and press the USE or ATTACK button, NPCs stopping what they were doing and looking at you when you get their attention, and monsters that wake up and attack you when you do anything that makes a sound.

So I want to change it to add a microphone input that detects when you start speaking (without caring what you say). When you start speaking it should activate the talk sequence of the nearest NPC within earshot, and also wake up any monsters within earshot.