Skip to main content
bullale
Up and Coming
March 15, 2015

Working PS Move plugin for UE4; need help coreg with DK2

Update:

The plugin is not done but it's ready for public testing. I made a new thread here.


Hello all,

I recently made a PSMove plugin for UE4 that works in both Windows and OSX. It should work in Linux too with a simple compile of the psmoveapi binaries. The changes I had to make to the psmoveapi and PS3EYEDriver should also help anyone that wants to use the PSMove controllers in Windows or OSX (it's always worked in Linux) outside of UE4.

I need help co-registering the coordinates I get from the psmove (in cm from the camera) with the DK2's coordinate space.

For orientation, the roll and pitch are easy. The PSMove has a magnetometer so it's possible to get its true yaw but to make that useful in game I would also need access the DK2's magnetometer (I've read mixed reports that it even has one). Does anyone have any ideas here? Does it truly have a magnetometer? If so, does anyone know how to access it from within UE4?

If it does not then I'll need to somehow coregister the orientations by relying on the user to somehow point the controller in a specific orientation and pressing a button. I'm not too excited about that option.

For position, I would love it if I could access the DK2's camera's image feed. I could have the users run a quick calibration app that asked them to draw in the air then for each frame I would estimate the PSMove controller's position in 3D space relative to each camera, then create a transformation matrix from PS3Eye coordinates -> DK2 camera coordinates. This matrix could be stored in a data file somewhere (or registry entry) and as long as the cameras don't move relative to each other then it should stay the same. Any ideas on accessing the DK2's images, even outside of a game engine or the SDK?

If I can't access the DK2's camera feed then I'll probably need to touch the PSMove at specific locations on the surface of the DK2 and sample >=4 fiducial locations. I would need accurate dimensions of the DK2 body relative to whatever origin
it uses. Anyone have any experience with this?

Accuracy is important to me. We've all read that accuracy goes a long way to helping with immersion, so I hope that it's important to you too, but it's especially important to me because I'm using the data for research. So I can't really accept calibration methods that simply ask the user to put the controller near where they think it should be in the world and press a button.

    180 replies

    Show previous replies
    bullaleAuthor
    Up and Coming
    March 19, 2015
    "Fredz" wrote:


    "bullale" wrote:
    Does the red LED come on, or just the blue?

    Only the small red LED comes on which indicates it's connected, the orb doesn't light up with test_tracker.exe, only with example.exe.


    Sorry, I meant the camera. The camera has two lights on the front. One blue, which comes on whenever it has power, and one red, which comes on when it is opened. If the red one never comes on then that suggests the tracker program is not identifying your camera. You must install libusb drivers for the camera; this part is not done by the api. I use Zadig to do it. Zadig provides three driver options. They all work for me, but the libusb0.sys option seems to work more generally. Just be sure you uninstall the camera from device manager before doing so.

    If the red light comes on then it is failing after opening the camera. libusb might be crashing, in which case I'd suggest switching libusb drivers (uninstall before and reboot after).
    Fredz
    Up and Coming
    March 19, 2015
    "bullale" wrote:
    Sorry, I meant the camera.

    No the red light of the camera doesn't come. When I had the CL Eye driver installed it did with the test_tracker.exe compiled from your sources.

    "bullale" wrote:
    You must install libusb drivers for the camera

    Ok I'll do that, thanks.
    Fredz
    Up and Coming
    March 19, 2015
    I've installed Zadig and now the camera turns red when I launch test_tracker.exe. The PS Move doesn't light up with the WinUSB driver and I have this message, like when the CL Eye driver was installed :

    ### Found 1 controllers.
    Trying to init PSMoveTracker...Warning: No lens calibration files found.
    C:\Users\Fred\AppData\Roaming\.psmoveapi\colormapping.dat is too old - not restoring colors.

    But with the libusb-win32 driver it does work, the orb lights up in magenta and a window is opened that shows it being tracked. The tracking works flawlessly. Many thanks !

    Now I'll see if it does work inside Unity in 32 bit and 64 bit mode and then I'll have a look at calibrating the PS Move using the DK2 positional tracking. If you want me to investigate something in particular don't hesitate to ask.

    Thanks again, great job !
    Fredz
    Up and Coming
    March 20, 2015
    The positional tracking works very well in Unity, it's better than with the previous driver. It doesn't jitter and I can update the position each frame instead of one out of three frames. I'm really impressed by your implementation.

    I guess the API has changed since I don't have rotational tracking working with my old Unity project, but I guess it can be quite easily corrected.
    bullaleAuthor
    Up and Coming
    March 20, 2015
    "Fredz" wrote:
    If you want me to investigate something in particular don't hesitate to ask.


    Heh. There's a lot. I couldn't post all the links here so I put it on the psmove google group instead.

    For these types of conversations, I'd rather use github's issue tracker because it has some nice features for conversing about code, but we can't use mine because it's a fork. We can use the upstream one (thp's) and we'll get input from other people that develop for the psmoveapi, including a couple that use it in Windows (previously with CL driver), but I think most code changes will have to be done in/against my repo because the upstream repo is unlikely to merge these major changes until they've been stable for a while. I'll leave it up to you whether you want to keep the conversation here, on google groups, or move to github.
    Fredz
    Up and Coming
    March 21, 2015
    I got the rotational tracking working. The API didn't change but I lost the calibration so the orientation couldn't be enabled, maybe because of changes in the PS Move API or the switch to 64 bit. I had to remove the pairing and the .psmoveapi folder in AppData/Roaming, then I re-ran psmovepair.exe.

    The rotational tracking is drifting badly now, maybe related to the magnetometer calibration. I couldn't go to 100% after several tries, I'll try again tomorrow in different places. Does it drift a lot for you as well ?

    When I get this resolved I'll have a look at what I can do. I don't know enough low-level Windows programming to dive into your proposed architecture so I'll probably try with optical calibration or co-registering with the DK2 first.
    bullaleAuthor
    Up and Coming
    March 21, 2015
    "Fredz" wrote:

    The rotational tracking is drifting badly now, maybe related to the magnetometer calibration. I couldn't go to 100% after several tries, I'll try again tomorrow in different places. Does it drift a lot for you as well ?

    No, but admittedly I don't test the orientation all that often.
    This reminds me of something else I want to change. Currently the magnetometer_calibration program uses a constant to determine 'how much' data to get. See here. Try changing that to a lower value then recompile and try again. Obviously there needs to be a better dynamic solution than relying on people to modify this constant.

    "Fredz" wrote:

    When I get this resolved I'll have a look at what I can do. I don't know enough low-level Windows programming to dive into your proposed architecture so I'll probably try with optical calibration or co-registering with the DK2 first.
    Sure, whatever interests you.
    Fredz
    Up and Coming
    March 29, 2015
    I got the magnetometer calibration resolved by allowing to not go to 100% as I said on the PS Move API Google Groups, so everything is working correctly now. I've started working on a Unity app to co-register the PS Move with the DK2.

    I couldn't make both work at the same time at first, but by switching USB ports the problem has been resolved. No idea if it's related to the DK2, the PS Eye or my machine, I hope it's only the latter.

    I've built a small app that allows to see both the DK2 and PS Move tracking in the same scene that'll use for co-registration later. If you want to test it to see if it does work on your machine, you can download it here (64 bit only) : PS Move Setup.

    Screenshot :
    PS Move Setup 1.png
    DK2 model from MannyLectro.

    EDIT: now I display the coordinates of both the PS Move and DK2 and the difference between them and I store these values. Next step will be to press a button to start the registration, press again to stop the registration, calculate a transform between the PS Move and the DK2 and display the PS Move at the corrected position.
    SethVR
    New Participant
    March 30, 2015
    I wish there was a way to make this work with the gearvr using the note 4 camera to track the move controller connected by bluetooth
    bullaleAuthor
    Up and Coming
    March 30, 2015
    "SethVR" wrote:
    I wish there was a way to make this work with the gearvr using the note 4 camera to track the move controller connected by bluetooth


    It should be possible. However, is the Note 4 "strong" enough to do the image processing while running GearVR? Below is sort of an intellectual exercise in what it would take to get the psmoveapi working in the Note4, but I think a better solution would be to write code optimized specifically for Android including custom image processing.

    The psmoveapi is designed for use on Linux first, Mac second, and Windows a distant third. I suspect it would be possible to modify it to work on Android too. The only external libraries that are absolutely necessary are hidapi, opencv, and the Madgwick IMU sensor fusion algorithm. The former two definitely have Android implementations, and I think the latter does too.

    The Note4's camera is actually quite good, so I think you would get good tracking with it. And there wouldn't be a co-registration problem because you'd always know exactly where the camera is relative to the headset.

    I don't have a Note 4 and I'm not about to buy one. But if there are any Android developers with a Note 4 and a PSMove then I can point out the parts of the psmoveapi they'd have to test and modify. Off the top of my head:

    Make sure the correct camera is found. Android has linux v4l2 so it should just be a matter of enumerating the devices and selecting the correct one automatically.

    Make a new pairing process (relies on USB connection to the host device, but most users won't have the correct mini-USB (PSMove) <--> micro?USB (Note4), so it'll be better to use a computer to send the Note4's bluetooth address to the PSMove).

    That's it for the psmoveapi.

    Then you'd have to make the Unity/UE4 plugin understand the different coordinate system of the Note4 camera when in landscape mode, and you'd have to hardcode the position offset relative to the GearVR. But these are trivial.
    Show more replies
    Terms & ConditionsAccessibility statement