Skip to main content
bullale
Up and Coming
March 15, 2015

Working PS Move plugin for UE4; need help coreg with DK2

Update:

The plugin is not done but it's ready for public testing. I made a new thread here.


Hello all,

I recently made a PSMove plugin for UE4 that works in both Windows and OSX. It should work in Linux too with a simple compile of the psmoveapi binaries. The changes I had to make to the psmoveapi and PS3EYEDriver should also help anyone that wants to use the PSMove controllers in Windows or OSX (it's always worked in Linux) outside of UE4.

I need help co-registering the coordinates I get from the psmove (in cm from the camera) with the DK2's coordinate space.

For orientation, the roll and pitch are easy. The PSMove has a magnetometer so it's possible to get its true yaw but to make that useful in game I would also need access the DK2's magnetometer (I've read mixed reports that it even has one). Does anyone have any ideas here? Does it truly have a magnetometer? If so, does anyone know how to access it from within UE4?

If it does not then I'll need to somehow coregister the orientations by relying on the user to somehow point the controller in a specific orientation and pressing a button. I'm not too excited about that option.

For position, I would love it if I could access the DK2's camera's image feed. I could have the users run a quick calibration app that asked them to draw in the air then for each frame I would estimate the PSMove controller's position in 3D space relative to each camera, then create a transformation matrix from PS3Eye coordinates -> DK2 camera coordinates. This matrix could be stored in a data file somewhere (or registry entry) and as long as the cameras don't move relative to each other then it should stay the same. Any ideas on accessing the DK2's images, even outside of a game engine or the SDK?

If I can't access the DK2's camera feed then I'll probably need to touch the PSMove at specific locations on the surface of the DK2 and sample >=4 fiducial locations. I would need accurate dimensions of the DK2 body relative to whatever origin
it uses. Anyone have any experience with this?

Accuracy is important to me. We've all read that accuracy goes a long way to helping with immersion, so I hope that it's important to you too, but it's especially important to me because I'm using the data for research. So I can't really accept calibration methods that simply ask the user to put the controller near where they think it should be in the world and press a button.

    180 replies

    Show previous replies
    SputnikKaputtni
    Participant
    April 21, 2015
    "bullale" wrote:
    "SputnikKaputtnik" wrote:
    Would you mind also uploading a simple unreal project including the necessary code/blueprint mentioned in the wiki? As I said, I dont' have a programming background and this would speed up the process considerably. I do have UE4 4.7.5 installed though. I'm very interested in helping you test this!


    I'm afraid UE4 projects aren't really meant to be shared like that. Even the most basic project I could come up with is 1.38 GB. On top of that, the directories that the Visual Studio project point to will be all wrong. There's really no good way other than creating the project yourself.

    I promise it's not that difficult. You really don't need to know any programming. You will never have to edit a line of code. I will expand the instructions to give more details about creating a project in UE4, adding the plugin, and building the project. In the meantime, you should download and install Visual Studio Community 2013.


    Ok, thanks for your patience! :)
    i7-4790K @4.00 GHz, EVGA GTX 980 SC, 16 GB RAM, Win 8.1 64 Bit
    bullaleAuthor
    Up and Coming
    April 22, 2015
    "Fredz" wrote:
    I've put the latest version of my setup utility with the sources here : http://www.mediafire.com/download/07crp ... 282%29.zip


    I finally got a chance to try this and I fell flat on my face. I don't use Unity so I have no idea what to do. The provided executables don't work (something about PSMoveSetup_Data folder is missing). If I try to build, I get errors about 'Assets\Src\###.cs' could not be found (this folder is empty for me).
    bullaleAuthor
    Up and Coming
    April 22, 2015
    "SputnikKaputtnik" wrote:
    Ok, thanks for your patience! :)

    To you, too. I touched up the instructions a little bit but I need to do more. If you can give it a try and let me know if/where you fail then I'll know where to start making screenshots. Start here.
    Fredz
    Up and Coming
    April 22, 2015
    "bullale" wrote:
    The provided executables don't work (something about PSMoveSetup_Data folder is missing).

    Ah sorry, I've probably not built an executable recently so it may not recognize the folders correctly. I'm working on it tonight, I'll upload a new version when I'm finished.

    "bullale" wrote:
    If I try to build, I get errors about 'Assets\Src\###.cs' could not be found (this folder is empty for me).

    Weird, it's not empty in the zip I kept on my machine and neither in the file I just downloaded from mediafire. I think you have had a problem while downloading.

    "bullale" wrote:
    I finally got a chance to try this and I fell flat on my face. I don't use Unity so I have no idea what to do.

    Basically you'll only have to open the project in Unity and click on the play button. If you want to play with the sources (C#) they're all in PS Move Setup\Assets\Src. The coregistration algorithms are in this folder and the code to handle the PS Move and the DK2 are in their respective subfolders.

    PS Move\MoveController.cs is where everything happens, but be warned, the code is an awful mess.

    I'm developing with the 0.5.0.1 Oculus SDK so you'll probably need the corresponding runtime.
    Fredz
    Up and Coming
    April 23, 2015
    I won't make a release tonight, too late (it's 3AM here) and I'm a bit tired, but I've made some progress.

    I've heavily modified my code that was riddled with bugs and I've validated that the coregistration algorithm does work when I add a random rotation and translation to the DK2 position. I'm able to get back the correct values using the Horn method.

    It still doesn't work correctly when using the PS Move coordinates but there is maybe only a little bug left somewhere.

    Also I think I've found a way to handle the rotation of the DK2 while registering the positions. Instead of storing the correlations only when the DK2 orientation is close to zero, which is clearly not a good approach, I canceled the DK2 rotation in the PS Move position by applying the inverse rotation to the DK2-PS Move vector by adding the DK2 position. Basically I rotate the PS Move position with the opposite angle of the DK2 so that it stays in the same space referential.

    That makes the problem much easier to resolve since I had no idea about how to implement non-linear least squares for two unknown transform matrices, basically the technique AD7GD proposed on reddit. I asked him for more information about how he thought this could be implemented but it looked very complicated to me (basically using the bundle adjustment feature of the Ceres C++ library).
    Fredz
    Up and Coming
    April 24, 2015
    I've cleaned up the sources and pushed to Github : https://github.com/Fredz66/PS-Move-Setup

    I've corrected the bug, I had the translation calculation code inverted, but the coregistration doesn't work correctly most of the time and I still don't know why.

    Since it does work with perfect test data I guess this might be related to the PS Move tracking code that is less accurate than the DK2 tracking and produces too much noise for the algorithm to give a good result. I guess a non-linear method could provide better results than this closed-form linear one, but I need to try your last tracking implementation to see if there is an improvement.

    That's enraging because at one time I oriented the PS Eye with a very different angle from the DK2 camera and the coregistration was good, the PS Move model was displayed between the lenses of the DK2 model when I moved the DK2/PS Move. But subsequent tests didn't give the same result.

    I'll continue to play a bit with this. When I get something acceptable I'll add some code to write the transform matrix in a file in the AppData\Roaming\.psmoveapi directory for others to use in their apps.
    Fredz
    Up and Coming
    April 25, 2015
    I wanted to find out if the registration didn't work because of noise/distortion in the PS Move tracking, so I've added random noise to my use case with test data. But I've found out that I didn't correctly understand the problem, I didn't need to use the inverse of the rotation and translation to display the corrected position, but the inverse of the transform matrix.

    So now the registration works correctly with my test data, I can see that the rotation and translation are correctly calculated, and when I display the PS Move based on this test data it stays firmly in the DK2 origin position, with a zero error distance in the three axes. When I add noise it's still very good (even with 1 cm random noise), so now I can see that the problem was not the noise in the PS Move tracking.

    I've also added scale calculation but it does look mostly correct (~1.0) so I've kept it out for now, but it can be enabled with a #define if needed.

    I've disabled the cancellation of the DK2 rotation for the PS Move position calculation and now the registration works correctly in principle. Even when I put the PS Eye upside down the orb stays at the position of the DK2 origin and the error distance seems quite minimal (mostly less than 1 cm when I keep the distance fixed) which is good news.

    But now I can see that I wrongly thought that it was enough to get a correct registration. I need to show the PS Move at the correct position relative to its attachment to the DK2, so that it isn't shown at the DK2 origin. I'll think a bit more about it and I'll hopefully find the solution.
    bullaleAuthor
    Up and Coming
    April 26, 2015
    That's great progress. Based on your description, it sounds like you're getting closer, and learning a lot in the process. I wish I could contribute but I am quite busy for the next 7-10 days so I probably won't have time to do anything except try it out. When I get to the other side of this busy week then I'll look into the code and try to port it to OSX. Keep up the good work!
    Fredz
    Up and Coming
    April 26, 2015
    "bullale" wrote:
    I wish I could contribute but I am quite busy for the next 7-10 days so I probably won't have time to do anything except try it out.

    Don't hesitate to try it and report back, I'm curious to see if someone else can get the same results on his machine. You should be able to used the .exe from Github directly on Windows.

    I didn't use your new tracker code yet so it would be interesting to see if it gives even better accuracy. You only should have to replace the psmoveapi.dll and psmoveapi_tracker.dll files in the Assets\Plugins\x86_64 directory if you kept the same name for the functions.

    The calculation of the transform between the controller and the HMD would be needed to have meaningful results, but if you keep the DK2 perpendicular to the camera after the registration it should give a good idea already.

    "bullale" wrote:
    That's great progress. Based on your description, it sounds like you're getting closer, and learning a lot in the process.

    Yes I'm having a lot of fun with this, I feel like when I started programming some decades ago. :)

    "bullale" wrote:
    When I get to the other side of this busy week then I'll look into the code and try to port it to OSX. Keep up the good work!

    If you mean the code from PS Move Setup, it should work on OS X I guess since Unity is multi-platform. If you're using OS X exclusively I can built an .exe for it if you want, that can be done from Windows. That is, if PS3EYEDriver still works on OS X.
    SputnikKaputtni
    Participant
    April 28, 2015
    Phew, guess it'll take a while before I can give you any useable test results. Dusted off my PS Moves today and tried to pair them with my PC. The first one couldn't get a steady connection and kept disconnecting. I let the second one charge a little longer before I tried and finally got it properly paired and magnetometer-calibrated using the EXEs from bullale's GitHub.

    Unfortunately I don't have more time today to get the PSEye installed, maybe I'll try next weekend.

    Oh, btw: Seen this? Or is it you, FredZ? :) https://www.youtube.com/watch?v=gsSzy0aKV4Q

    If not, maybe you should get in touch with that guy. Posted his twitter account with the video.
    i7-4790K @4.00 GHz, EVGA GTX 980 SC, 16 GB RAM, Win 8.1 64 Bit
    Show more replies
    Terms & ConditionsAccessibility statement