Skip to main content
bullale
Up and Coming
March 15, 2015

Working PS Move plugin for UE4; need help coreg with DK2

Update:

The plugin is not done but it's ready for public testing. I made a new thread here.


Hello all,

I recently made a PSMove plugin for UE4 that works in both Windows and OSX. It should work in Linux too with a simple compile of the psmoveapi binaries. The changes I had to make to the psmoveapi and PS3EYEDriver should also help anyone that wants to use the PSMove controllers in Windows or OSX (it's always worked in Linux) outside of UE4.

I need help co-registering the coordinates I get from the psmove (in cm from the camera) with the DK2's coordinate space.

For orientation, the roll and pitch are easy. The PSMove has a magnetometer so it's possible to get its true yaw but to make that useful in game I would also need access the DK2's magnetometer (I've read mixed reports that it even has one). Does anyone have any ideas here? Does it truly have a magnetometer? If so, does anyone know how to access it from within UE4?

If it does not then I'll need to somehow coregister the orientations by relying on the user to somehow point the controller in a specific orientation and pressing a button. I'm not too excited about that option.

For position, I would love it if I could access the DK2's camera's image feed. I could have the users run a quick calibration app that asked them to draw in the air then for each frame I would estimate the PSMove controller's position in 3D space relative to each camera, then create a transformation matrix from PS3Eye coordinates -> DK2 camera coordinates. This matrix could be stored in a data file somewhere (or registry entry) and as long as the cameras don't move relative to each other then it should stay the same. Any ideas on accessing the DK2's images, even outside of a game engine or the SDK?

If I can't access the DK2's camera feed then I'll probably need to touch the PSMove at specific locations on the surface of the DK2 and sample >=4 fiducial locations. I would need accurate dimensions of the DK2 body relative to whatever origin
it uses. Anyone have any experience with this?

Accuracy is important to me. We've all read that accuracy goes a long way to helping with immersion, so I hope that it's important to you too, but it's especially important to me because I'm using the data for research. So I can't really accept calibration methods that simply ask the user to put the controller near where they think it should be in the world and press a button.

    180 replies

    Show previous replies
    bullaleAuthor
    Up and Coming
    April 2, 2015
    "cpldave" wrote:

    Queries:

    • Is USB always required?

    • Is windows supposed to constantly connect/disconnect the move until something using the psmoveapi tries to take control of the device?

    • Should repeatedly pressing the PS button be required to pair/use the move controllers?



    When plugged in by USB, the slowly flashing light indicates that it's charging, that's all. When it is fully charged it won't flash at all. If you press the PS button and it flashes faster that indicates that it is trying to connect to whatever machine it is paired to. If you are successfully connected then the LED will be on continuously, but the glowing orb will not be on unless the controller is being tracked.

    USB is only required for pairing. It should never be needed after pairing. In fact, as far as I know, the IMU data only transmits via bluetooth. When connected by USB the tracker gives junk results. Unfortunately, it is difficult to distinguish between the USB-connected controller and BT-connected controller and most of the applications don't bother to check that. The pairing apps are the only exception.

    After the controller is paired that first time (and in Windows 8.1 there are some extra steps, see below), connecting should just be a matter of pressing the PS button, when it is not connected by USB. That's how it works on my system.

    In Windows 8.1 there is an extra step after pairing. If you used the version of psmove-pair-win that I compiled (i.e., in my plugin's Binaries/Win64 folder) then it takes care of the extra steps for you... at least it should. If not then you can try reading here.

    If that, and FredzL's suggestion doesn't work, then I think you should try a different Bluetooth dongle. Belkin ones supposedly work well. I have an ASUS-BT400 that works great for me.
    bullaleAuthor
    Up and Coming
    April 2, 2015
    "Fredz" wrote:
    @cpldave : I would remove the .psmoveapi directory in C:\Users\<yourname>\AppData\Roaming and try again with psmove-pair-win.exe.


    How did you come to that? If that's true then I think I'll have to modify my instructions to not use psmovepair.
    Anonymous
    April 2, 2015
    "Fredz" wrote:
    @cpldave : I would remove the .psmoveapi directory in C:\Users\<yourname>\AppData\Roaming and try again with psmove-pair-win.exe.


    Tried removing profile data and re-running psmove-pair-win.exe and it paired the device 3 times, then I Ctrl-C'd out of the loop. Still having the same issues though. The controller won't stay connected for more than 5 seconds. I've got the registry settings for "VirtuallyCabled=1" configured properly and the HID checkbox in the "Devices and Printers" properties for "Motion Controller" is definitely checked.

    I'm going to call it a night and try again tomorrow. I'll start by replacing the USB cable I'm using :P
    bullaleAuthor
    Up and Coming
    April 2, 2015
    "cpldave" wrote:

    Tried removing profile data and re-running psmove-pair-win.exe and it paired the device 3 times, then I Ctrl-C'd out of the loop. Still having the same issues though. The controller won't stay connected for more than 5 seconds. I've got the registry settings for "VirtuallyCabled=1" configured properly and the HID checkbox in the "Devices and Printers" properties for "Motion Controller" is definitely checked.


    It sounds like it is setup correctly. It could be a problem with the bluetooth dongle, but this is the first I've heard of it being a problem in Windows 8.1 since the registry hacks. I don't have any experience or expectation of the symptoms to expect, but it might also be that the batteries are shot? Try charging it over night to see if you can get a few more seconds out of it.
    Fredz
    Up and Coming
    April 2, 2015
    "bullale" wrote:
    How did you come to that? If that's true then I think I'll have to modify my instructions to not use psmovepair.

    Although previously I didn't have any problem pairing with my Belkin dongle, when I wanted to try your tracker I found out that my PS Move was no longer paired. No idea where that came from, and I couldn't pair it with the old 3.0.0 psmovepair or the new psmovepairwin.

    I removed this directory as a last resort and the pairing did work, but that may be a coincidence with other things I may have tried at the same time, can't remember.
    Fredz
    Up and Coming
    April 2, 2015
    First implementation of a co-registration utility with a very naive and non-robust technique as a proof of concept :




    Download 64 bit MS Windows version.

    Basically I read 250 positions of the DK2 and PS Move when the DK2 is oriented toward the DK2 camera (with a 5° tolerance). Then I calculate the mean distance in each axis (x, y, z) between the two lists of positions and I use these values to shift the PS Move displayed position.

    The result is quite good in the central FOV (especially if you compare with previous screenshots) but diverges a bit when outside this FOV. It's not robust to either rotation since I didn't calculate a transform matrix yet and I didn't take rotation into account.

    I'll need to implement an outlier detection mechanism so that spikes in the tracking don't affect the calculations. The simple method would be to use variance/standard deviation to only keep points that are close to the mean (most probably the one I'll try first). A more complicated method would be to use RANSAC or LMedS.

    Then I'll need to implement what was proposed by AD7GD in the reddit thread (nonlinear least-squares solving of an over-constrained system P_Rift.T_Rift2Move = T_Eye2Cam.P_Move).

    I'll need to come up with a procedure for users that is both simple and robust, probably telling them to move the DK2 + PS Move in each corner of the cone of vision of the DK2 camera to not have too much manipulations to make and have meaningful points (coverage of the whole FOV) and at the same time try to calculate the distortion of the PS Eye, if that's possible. Implementing a reliable ellipse-based tracking algorithm would need to be done before that though.

    All this may well be beyond my abilities, but I'll give it a try anyway.
    Fredz
    Up and Coming
    April 3, 2015
    I've read a bit about outlier detection mechanisms and there seems to be a reasonably simple technique that is more robust than the often used "mean ± 3 standard deviations", it's the "median absolute deviation". Some papers about this technique for people interested :

    - Outlier on Wikipedia
    - Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median
    - An improved framework for periodic outlier patern detection in time series sequences

    If anyone knows something better, don't hesitate to tell.
    Fredz
    Up and Coming
    April 4, 2015
    I've implemented the absolute deviation from the median technique and it helps greatly with outliers.

    Some results :
    - Euclidean distance with outliers (min/max/avg) : 0.330301588486093 - 0.421048349487938 - 0.336294840562034
    - Euclidean distance without outliers (min/max/avg) : 0.330301588486093 - 0.336985212564737 - 0.332969764429639
    - Distances with outliers (x/y/z): 0.0549731780216098 - 0.0463917214637622 - 0.328481661558151
    - Distances without outliers (x/y/z): 0.0547513312173892 - 0.046946846389632 - 0.325064052497985

    You can see that there are some spikes in the readings (max distance of .42 although the mean is ~0.336). With the MAD calculation the mean is 0.332, so it's a bit more precise.
    bullaleAuthor
    Up and Coming
    April 4, 2015
    That's great. I'm loving reading the progress.
    If you want to put this up on a repo somewhere then I will try to contribute if I can.
    How do you propose we work together on the psmoveapi tracker? I don't know if using the upstream repo is a good idea.
    jllabelle
    New Participant
    April 4, 2015
    Thanks for all your work, it's really cool to read on your progress!

    Are you still planning to release a version for Unity integration?

    Thanks
    Show more replies
    Terms & ConditionsAccessibility statement