Why there is a heavy latency in grab interaction

Hi,
I have an issue with the grab interaction as in this video. 
I replace the mesh of the hands and now they are working fine in when it comes to the animations but when it comes to the grab interaction it becomes slow and heavy.. the frame rate isn't affected but there's a lag. 

*I'll put the video link in the comments*