Forum Discussion
Phone Controller
Well, this is just an idea, but solves many problems I have been having with input. As an android developer, I realize there is so much you can do with a phone, and that's where the idea came. I linked my phone into my computer, and in turn to the oculus rift. The phone is set up to monitor its accelerometer, for sudden movement in certain directions. Then, after hooking this up to skyrim, I was able to walk forward by touching the screen of the phone (simulated w press), and strike by swinging the phone as if it was a sword (and double tap phone for menu in skyrim etc). This was actually surprisingly good, compared to the typical controller, and I am adding haptic feedback to the phone app along with other things. I know this is similiar to Wii, but the convenience of it just being an app that you can download cannot be overlooked. Does anyone have any suggestions, really, throw everything you can at me please, I really want to make this work well?
(Note, just updated it then, you have a bluetooth microphone headest connected to phone, only mic is modded, so just monitors for pressure (trimmed the microphone and put the two wires between a spring), then I have this on my shoe, so by jogging on the spot, it moves in game))
(Note, just updated it then, you have a bluetooth microphone headest connected to phone, only mic is modded, so just monitors for pressure (trimmed the microphone and put the two wires between a spring), then I have this on my shoe, so by jogging on the spot, it moves in game))
13 Replies
- petty cool idea with the microphone^^!
the availability of phones is definitely a super interesting factor and I very much like the idea of using them as controllers. I was surprised how nicely the controls worked in 'the gate'. having direct sensory-motor coupling between moving your finger and seeing the cursor on the screen enables you a really precise and easy to learn interaction. combining that with the ton of other sensors available in phones could lead to some really interesting interaction paradigms - im writing a bachelor thesis on interaction design and VR and im doing...this..well this...actually using a tablet. Going to put up some video of it later this week and i'm going to share/publish my results more in depth in the end of may. Also working at some demos for people to try out the interaction and navigation.
The application i've been testing out is more a 1 to 1 application than they have in the gate. im using a Full body avatar, trying to see how you can emulate and activate the kinesthetic senses of the user.
im not a programmer, only a designer so i have been banging my head against technical issues for weeks and it's only now that i have been able to explore interaction techniques. It's sad that not more tablets have haptic feedback, and most of the models out there (im using a galaxy note 8.0) have pretty poor feedback. But the range of sensors that you have available to you are amazing!
and this https://www.youtube.com/watch?v=bX4HE7prH6U
and this https://www.youtube.com/watch?v=WJFwHuCs3Yw
could be amazing with VR. - what a cool idea and a good application of the technologies. i'm sure it has the potential to be applied in many situations.
- There is application called FreePIE that allows you to easily translate some of the sensor feeds on Android phones and iPhones into key, mouse, or joystick commands. It uses a scriptable language so it's fairly flexible in terms of the types of commands that you can emulate. If nothing else it's really great for prototyping this type of thing.
http://andersmalmgren.github.io/FreePIE/ - i've been using unity on both the PC and the mobile device. Made it really easy to send things back and forth between the devices since unity don't differentiate between mobile or pc when doing network codes. The it's basicly just sending over bools, floats and int:s.
I have to finish up alitle more of the written part of my thesis work but then i'l post some video of it - Would this idea work? :
- stick QR codes to the inside of a large cardboard box
- place the box on its side on your desk such that you can move your arm(s) around in it.
- grab your smartphone (along with some sort of custom app to read the qr codes and accelerometer data)
- voila: positional hand-tracking, using your phone a la the Valve VR prototype.
Anyone with the knowhow capable of knocking this up as a prototype? - Anyone working on a phone controller for an Unreal Engine 4 projects?
(and willing to share the initial code?) - I was quite excited when I saw the original Crystal Cove prototype that I'd be able to use the front-facing camera on my phone to track the oculus. I'd have the phone deduce its position relative to the headset and send it over to the computer; this would mean that you could show the phone within the rift, and therefore the user's hand.
However, given that the lights on the DK2 shine only in infrared (or perhaps the visible light is filtered out by the plastic - not sure which) I suspect I will not be able to use the phone camera to track the rift. I think most phone cameras will actually pick up IR, but they tend to have filters built in to prevent that. Enough IR might get through if I covered the phone camera with a visible light filter, but that kind of defeats the purpose of having a motion controller that most people will have available anyway. - Unfortunately this type of thing won't really be possible with any other external camera. The LED's are modulated and the camera shutter synchronized with the flash frequency of the LEDs. Unless your camera is using the same sync signal it will miss the flashing of the LED's on most frames.
- Actually Brant if you use a high enough exposure time (10-15 ms), a camera without an IR filter can pick up the IR LEDs (tested today).
