Skip to main content
bullale
Up and Coming
March 15, 2015

Working PS Move plugin for UE4; need help coreg with DK2

Update:

The plugin is not done but it's ready for public testing. I made a new thread here.


Hello all,

I recently made a PSMove plugin for UE4 that works in both Windows and OSX. It should work in Linux too with a simple compile of the psmoveapi binaries. The changes I had to make to the psmoveapi and PS3EYEDriver should also help anyone that wants to use the PSMove controllers in Windows or OSX (it's always worked in Linux) outside of UE4.

I need help co-registering the coordinates I get from the psmove (in cm from the camera) with the DK2's coordinate space.

For orientation, the roll and pitch are easy. The PSMove has a magnetometer so it's possible to get its true yaw but to make that useful in game I would also need access the DK2's magnetometer (I've read mixed reports that it even has one). Does anyone have any ideas here? Does it truly have a magnetometer? If so, does anyone know how to access it from within UE4?

If it does not then I'll need to somehow coregister the orientations by relying on the user to somehow point the controller in a specific orientation and pressing a button. I'm not too excited about that option.

For position, I would love it if I could access the DK2's camera's image feed. I could have the users run a quick calibration app that asked them to draw in the air then for each frame I would estimate the PSMove controller's position in 3D space relative to each camera, then create a transformation matrix from PS3Eye coordinates -> DK2 camera coordinates. This matrix could be stored in a data file somewhere (or registry entry) and as long as the cameras don't move relative to each other then it should stay the same. Any ideas on accessing the DK2's images, even outside of a game engine or the SDK?

If I can't access the DK2's camera feed then I'll probably need to touch the PSMove at specific locations on the surface of the DK2 and sample >=4 fiducial locations. I would need accurate dimensions of the DK2 body relative to whatever origin
it uses. Anyone have any experience with this?

Accuracy is important to me. We've all read that accuracy goes a long way to helping with immersion, so I hope that it's important to you too, but it's especially important to me because I'm using the data for research. So I can't really accept calibration methods that simply ask the user to put the controller near where they think it should be in the world and press a button.

    180 replies

    Show previous replies
    Fredz
    Up and Coming
    April 28, 2015
    "SputnikKaputtnik" wrote:
    Oh, btw: Seen this? Or is it you, FredZ?

    Nope not me, I haven't seen this before.

    But he did use the MoveFramework which is Windows only and relies on the CL Eye driver which is not free, unstable, doesn't work with 64 bit executables and AFAIK is no longer maintained.

    This kind of application can already be done with the PS Move API and the UniMove plugin, but it'll suffer from tracking distortion until there is a co-registration with the DK2. That's why we're working on this.

    Maybe I should release an updated version of the UniMove plugin with bullale's modifications to the PS Move API, without waiting for the finalization of the co-registration implementation.
    cubedparadox
    Participant
    April 30, 2015
    Hey all, I've followed cboulay's ps move configuration & pairing guide, and have hit a snag. I can see my camera feed with ps3eye_sdl.exe, and the move controller is paired (windows sees it as a Bluetooth device, which I think is correct).

    That said, I'm getting an error when running either magnetometer_calibration or test_tracker: [PSMOVE CRITICAL] Assertion fail in psmove_connection_type: move != NULL
    Ignoring non-Bluetooth PS Move #0

    I can clearly see that the move is connected with Bluetooth, and not usb, so I'm unsure of the issue.
    Any help would be appreciated; thanks and keep up the great work.
    bullaleAuthor
    Up and Coming
    April 30, 2015
    First, make sure there are no PSMove controllers plugged into the computer by USB at the time you use it. Charge them somewhere else if necessary.

    Second, Windows does not always report Bluetooth disconnections properly. Is the red LED still on? Sometimes the controller will disconnect but Windows will think it is still connected (e.g., by holding the PS button for ~8 seconds), and then the psmoveapi gets confused by the presence of a controller that it cannot communicate with.

    Finally, what OS are you using? Windows 8.0 does not work well but Windows 8.1 does.
    cubedparadox
    Participant
    April 30, 2015
    "bullale" wrote:
    First, make sure there are no PSMove controllers plugged into the computer by USB at the time you use it. Charge them somewhere else if necessary.

    Second, Windows does not always report Bluetooth disconnections properly. Is the red LED still on? Sometimes the controller will disconnect but Windows will think it is still connected (e.g., by holding the PS button for ~8 seconds), and then the psmoveapi gets confused by the presence of a controller that it cannot communicate with.

    Finally, what OS are you using? Windows 8.0 does not work well but Windows 8.1 does.


    I'm charging with my pc, but unplugging before running any programs. There is a solid red light on the move controller, which I assume indicates that windows isn't lying about the connection. I'm on windows 8.1.
    Thanks for responding, I'd love to get this working!
    bullaleAuthor
    Up and Coming
    April 30, 2015
    "cubedparadox" wrote:

    I'm charging with my pc, but unplugging before running any programs. There is a solid red light on the move controller, which I assume indicates that windows isn't lying about the connection. I'm on windows 8.1.
    Thanks for responding, I'd love to get this working!


    Maybe Windows isn't registering them being disconnected by USB. How many controllers does test_tracker report that it found? In fact, can you copy-paste the text here?

    There is another possibility. There was some code in the psmoveapi that I changed that enabled this to work reliably in Windows. However, someone else has reported some oddities that I guess might be caused by the code that I changed, and it might cause your problems too. Briefly, hidapi enumerates 3 devices for each BT controller, and each of the 3 devices has different features. There might be hardware differences in how the devices are enumerated depending on controller firmware or the BT adapter. If your devices are not enumerated as mine are, then the psmoveapi will not be able to find the "serial number" of the controller (just the BT address) and it will assume it is connected by USB. What BT dongle are you using?

    The correct solution to this problem would be for the psmoveapi to use hidapi for OSX & Linux but to use Windows' bluetooth API for Windows. Unfortunately, this kind of change will take a long time; time I don't have right now.

    There might be a quick-fix if I can figure out exactly what's going wrong on your setup. Do you know how to program? Can you edit code and build? If so then maybe we can figure out if it's that your 'devices' are being enumerated differently. If so, then we will need another way to know which 'device' to use for communication and which 'device' to use to get the BT address.
    cubedparadox
    Participant
    April 30, 2015
    "bullale" wrote:

    Maybe Windows isn't registering them being disconnected by USB. How many controllers does test_tracker report that it found? In fact, can you copy-paste the text here?


    test_tracker reports one controller
    test_tracker.exe
    ### Found 1 controllers.
    Trying to init PSMoveTracker...libusb: error [init_device] device '\\.\USB#VID_1415&PID_2000&MI_01#6&29F017CD&0&0001' is no longer connected!
    Warning: No lens calibration files found
    Calibrating controller 0...[PSMOVE CRITICAL} Assertion fail in psmove_tracker_enable: move != NULL

    (the last line repeats until I ^C)

    And here is the magnetometer_calibration
    magnetometer_calibration.exe
    [PSMOVE CRITICAL] Assertion fail in psmove_connection_type: move != NULL
    Ignoring non-Bluetooth PS Move #0

    Finished PS Move #0
    [PSMOVE CRITICAL] Assertion fail in psmove_connectoin_type: move != NULL




    What BT dongle are you using?


    I'm using an Abe model UB22S dongle, FCC ID NV6-CS8078, Serial Number 0905005483


    There might be a quick-fix if I can figure out exactly what's going wrong on your setup. Do you know how to program? Can you edit code and build? If so then maybe we can figure out if it's that your 'devices' are being enumerated differently. If so, then we will need another way to know which 'device' to use for communication and which 'device' to use to get the BT address.


    I mainly program in python, but I've played with visual studio enough to be able to build and edit code, as well as read any debug messages that are there. I'd love to help fix this, I'm sure I'm not the only one who has this problem.
    bullaleAuthor
    Up and Coming
    April 30, 2015
    OK let's take the conversation to pms to leave this thread for tracker & coregistration talk.
    Fredz
    Up and Coming
    May 3, 2015
    I've found a paper that seems to correspond to the problem that needs to be solved :
    Simultaneous Robot-World and Hand-Eye Calibration

    Basically it shows three techniques (linear, closed-form, non-linear) to solve the A.X=Z.B problem. It considers 6dof values for both sets, but I guess it can be used with sets composed of 3dof and 6dof values as well (by setting rotations to zero for one set).

    The closed-form solution is similar in principle to the Horn method that solves the A.X=B problem, so hopefully I'll be able to reuse my existing code for this. It also slightly outperforms the linear method and can be used as an initialization for the non-linear method, so it looks like a good technique to start with.
    Fredz
    Up and Coming
    May 9, 2015
    Another paper that presents a more reliable algorithm (less translational error) which is simultaneous (computes rotation and translation at the same time) and doesn't suffer from the incorrect peaks with the previous method (due to the nonuniqueness of the quaternion representation) :



    This algorithm is also designed for 6dof data like the previous one, but it's not clear if it can work on partial data (3dof). If the orientation data from the PS Move is not reliable enough I'll probably need another algorithm, the one presented in this paper (QR15 algorithm) is able to work with partial data :

    Fredz
    Up and Coming
    May 10, 2015
    The matlab code for the Shah algorithm is a bit too complicated for me to translate into C# so I'm using this algorithm instead :
    Simultaneous robot-world and hand-eye calibration using dual-quaternions and Kronecker product (Li et al. 2010).

    The corresponding matlab code is here : http://math.loyola.edu/~mili/Calibration/AXYB/li.m

    It's similar to the Shah algorithm and according to the previous paper (Shah 2013) the X rotational errors are similar, the Y rotational errors are slightly better and the positional errors are larger.

    I've converted it to C#, now I'm running the matlab code in Octave (a free implementation of matlab) with test data to verify that I've the same output in Octave and Unity.
    Show more replies
    Terms & ConditionsAccessibility statement