Pointless ponderings about mouse movement

for an OS control enviroment ,the 3rd dimension is probably not useful, not with a conventional design at least. Simply having a single layer to use a mouse with would be sufficient, not need to worry about complex control schemes to mouse in 3 dimensions

I was thinking about mouse input for a VR OS a while ago. I don't think using the mousewheel to control the 3rd dimension is the best idea, but I do agree that there is some requirement to be able to interact with a 2D mouse in a 3D virtual space.
(What I would really like is some device that you slot your mouse into that allows you to freely move it up or down without having to lift its weight, but we're not talking about new devices here are we ;) )

A way that you could do it is use a mousebutton (say middle-click) to lock the mouse cursors z-position to the frame that the HMD is currently pointed at.
Alternatively, you have the mouse mapped to the X and Y axes just like how it is right now (that is to say, mouseX -> virtualX and mouseY -> virtualY), and while holding both mouse buttons (or perhaps middle mouse), the mouseY maps to virtualZ, so you can very naturally push and pull the cursor forwards and backwards.

"mystify" wrote:
for an OS control enviroment ,the 3rd dimension is probably not useful, not with a conventional design at least. Simply having a single layer to use a mouse with would be sufficient, not need to worry about complex control schemes to mouse in 3 dimensions

Ah, I beg to differ! Imagine if you had a Windows shell for VR where you were able to have windows behind other windows, and the ability to really naturally move them forward or backwards by dragging them exactly how you currently do, only with the 3rd dimensional capabilities of some novel way of interpreting mouse input!

But yeah, I agree that it would be super awesome if some VR productivity software had the ability to use a regular mouse for 3D input, since they're everywhere! :D

"mptp" wrote:

"mystify" wrote:
for an OS control enviroment ,the 3rd dimension is probably not useful, not with a conventional design at least. Simply having a single layer to use a mouse with would be sufficient, not need to worry about complex control schemes to mouse in 3 dimensions

Ah, I beg to differ! Imagine if you had a Windows shell for VR where you were able to have windows behind other windows, and the ability to really naturally move them forward or backwards by dragging them exactly how you currently do, only with the 3rd dimensional capabilities of some novel way of interpreting mouse input!

But yeah, I agree that it would be super awesome if some VR productivity software had the ability to use a regular mouse for 3D input, since they're everywhere! :D

That sounds like a significant increase in interface complexity with negligible actual advantage.

i think i've said it before, but i really feel that gesture detect is the way forward.

It wouldn't be any more complex than the current typical desktop environment. The only difference is rather than having the equivalent of a bunch of pieces of paper on top of one-another, to be sifted through to find the one you want, you have a series of pieces of paper floating in front of you that you can look behind and move around to find the one you want.

And regarding gesture detection - it's cool if you have a trackpad or some other device that can output more information than a mouse can. But trying to do gesture detection with a desktop mouse is a path littered with obstacles. It would be too easy to accidentally perform some gesture during normal operation and perform some unwanted action.
Although now that you say that - a trackpad would open the door to a whole bunch of really awesome ways to interact with a 3D virtual desktop...

"mptp" wrote:
It wouldn't be any more complex than the current typical desktop environment. The only difference is rather than having the equivalent of a bunch of pieces of paper on top of one-another, to be sifted through to find the one you want, you have a series of pieces of paper floating in front of you that you can look behind and move around to find the one you want.

As it is, there are already many ways in place to find that window. You have the entire taskbar showing you which things are open so you can directly click on it. If you have multiple windows of a type open, it presents you with a nice list of them. If they are in 3d, they would still be blocking each other visually, and moving your head to peer behind them would be far more involved, and making them transparent wouldn't work much better than making them transparent on a 2d screen. Additionally, with this scheme, once you do manage to spot the window, selecting it is more complex, as you have to manuever your cursor in 3 dimensions. This is far, far more cumbersome than a normal OS interface.

Necrobump, although this isn't an OS it's mouse related and probably not worthy of a new thread:

I created a small game that does map a desktop mouse into the 3D scene.
It does a 3D ray cast each frame to determine the X,Y,Z surface contact point of the cursor.
The hand avatar model is then translated such that it's fingertip is on that point.
It's a fairly intuitive and natural way to interact with 3D GUI panels.
It has the added advantage of mapping to 2D playing (on a regular monitor) perfectly without any code change.
And as OP says "everyone has one", a mouse that is.

Here's a small DK1 compatible gambling game demo:

Windows EXE:
http://www.game4coins.com/dev/shiva/bitbot.exe

Screenshots:
http://www.game4coins.com/dev/gfx/oculus.jpg
http://www.game4coins.com/dev/gfx/bitbot.jpg

"g4c" wrote:
Necrobump, although this isn't an OS it's mouse related and probably not worthy of a new thread:

I created a small game that does map a desktop mouse into the 3D scene.
It does a 3D ray cast each frame to determine the X,Y,Z surface contact point of the cursor.
The hand avatar model is then translated such that it's fingertip is on that point.
It's a fairly intuitive and natural way to interact with 3D GUI panels.
It has the added advantage of mapping to 2D playing (on a regular monitor) perfectly without any code change.
And as OP says "everyone has one", a mouse that is.

Here's a small DK1 compatible gambling game demo:

Windows EXE:
http://www.game4coins.com/dev/shiva/bitbot.exe

Screenshots:
http://www.game4coins.com/dev/gfx/oculus.jpg
http://www.game4coins.com/dev/gfx/bitbot.jpg

Awesome, I think you are on the right track, I have also experimented this way and maybe the short-term solution is indeed this, but I would rather use an air/gyro mouse than a real one. They are dirt-cheap nowadays and some of those are even Android compatible, so can be used with GearVR for basic motion control in the absense of any input currently.

"mystify" wrote:

"mptp" wrote:

"mystify" wrote:
for an OS control enviroment ,the 3rd dimension is probably not useful, not with a conventional design at least. Simply having a single layer to use a mouse with would be sufficient, not need to worry about complex control schemes to mouse in 3 dimensions

Ah, I beg to differ! Imagine if you had a Windows shell for VR where you were able to have windows behind other windows, and the ability to really naturally move them forward or backwards by dragging them exactly how you currently do, only with the 3rd dimensional capabilities of some novel way of interpreting mouse input!

But yeah, I agree that it would be super awesome if some VR productivity software had the ability to use a regular mouse for 3D input, since they're everywhere! :D

That sounds like a significant increase in interface complexity with negligible actual advantage.

I tend to disagree about the actual advantage. In my workplace, it's the norm to have 4-6 screens piled around your desk. That doesn't include the 3 wall-mounted displays of about 60" diagonal. There's a major need to consume information simultaneously, and it has to be SIMULTANEOUS -- not nearly so.

I don't see how what was suggested would accomplish that.