Human scans in Unity. Unity dev help needed.

I've decided that if you want, and don't mind sending the files for the frames, I'll go ahead and try to get something working, but I can't promise anything. The buffering is good in principle, but depending on the file size, it may not be possible. Not due to ram, but more due to just the loading speed of hard drives.

If they are 12mb/frame, that's a lot of data. At 24f/s, that's 288mb/s. Most hard drives (non-ssd) max out around 200mb/s, so it may not be possible at all (that's also with decently high end disk hdd's). I'm willing to try it, but I can't promise that I'll do it fast, or if I'll get anything more then preloaded frames.

Anyway, let me know if you want me to go ahead and try.

"CaliberMengsk" wrote:
To be honest, I have no clue if I could program something like that. O-o Would probably take some time to do, and unfortunately, my life is fairly busy at the moment. (programming a moba game takes quite a bit of time, with all the networking and such)

I'd try it out if I had more time. The concept should be easy enough to follow though for anyone to try.

OK thanks.

"Slopey" wrote:
You probably don't need to buffer them unless you're talking about thousands of frames.

But if you are, you'd be more efficient rigging the model in the first place and animating it, depending what you'd be trying to achieve.

It would likely be enough to just load the frames as an array of gameObjects and use those, although I'm not sure how you'd handle positioning in the best manner - you'd likely need an empty game object to map those onto to keep the position orientation correct, or hold the data in code.

Probably easier just swapping the meshes out on a single gameobject as you move it around.

This is really just for a proof of concept to see how far it can be pushed. Something to test with. 10 frames at 100,000k? 100 frames at 10k? etc, to see what the boundaries are.

"Stardog" wrote:
Did you ever try this? I would like to see these "animated". I'll post another script if you will try it.

Not yet I've been waiting for the right opportunity as this thread went quiet for a bit, as did the post on the other forum. I have some code from drash that I can test with but it wont buffer or anything like that.

I will try and prepare some data that I can share here.

"CaliberMengsk" wrote:
I've decided that if you want, and don't mind sending the files for the frames, I'll go ahead and try to get something working, but I can't promise anything. The buffering is good in principle, but depending on the file size, it may not be possible. Not due to ram, but more due to just the loading speed of hard drives.

If they are 12mb/frame, that's a lot of data. At 24f/s, that's 288mb/s. Most hard drives (non-ssd) max out around 200mb/s, so it may not be possible at all (that's also with decently high end disk hdd's). I'm willing to try it, but I can't promise that I'll do it fast, or if I'll get anything more then preloaded frames.

Anyway, let me know if you want me to go ahead and try.

This sounds good. I will share something here soon. Please see the attached chart to get an idea of the sort of files sizes I would imagine we could be dealing with, and the impact on quality it would have.



Please view the image in another window. This forum doesn't seem to re-size images.

Of course I don't have full body data to test with, as the system isn't built yet but I do have some face and hand 4D data, low quality but something to test with. I will upload shortly.

Looks great!

Thanks Cyber. Just an example but imagine it moving at 60-120fps, for a few minutes of play back. A dream at the moment but it opens up all kinds of interesting doors.

Here is some very basic data to test with, just a hand opening. 6 frames, 50k each, no texture for now.

http://www.ir-ltd.net/uploads/Hand-Frames.zip



They will need to be scaled to 0.01 in Unity. It would be good to be able to set playback rate at 0 / + fps, as well as looping the frames from open to close i.e Ping-Pong to extend playback time in this instance. If something can be achieved with this simple data set, I can organize a 4D capture of my face to test with that is longer and eventually share a build demo of it, if a script can be achieved to do playback.

Well, without loading it in real time (loading it into the level in the unity editor) I got some pretty good results. I'll have to try the resources.load when I get more spare time. Here's a webplayer.

http://calibermengsk.com/games/unity/3dframes/

You can change the framerate with the slider bar. It also rotates at around 50 degrees per second (around 7 seconds for a full rotation.) and is independent of the framerate. You can also type into the text box what framerate you want. The slider is limited to a range of 0 to 30, but if you want faster, you can just type it in. 6 frames though isn't much to work with.

EDIT:
Oh, here's the commented code if you want to try it out. Just make an empty game object and put this script on it, then import

using UnityEngine;
using System.Collections;
using System.Collections.Generic;

public class Cycle : MonoBehaviour {
	public float fps = 24f;
	public float rotateSpeed = 5f;
	
	private float curFrameID = 0f;
	private float thisFrame = 0f;
	private float defaultFPS;
	public List<GameObject> children;
	
	void Start () 
	{
		//Sets the default framerate.
		defaultFPS = fps;

		//Put all of the "frames" in order (by name) in a game object and place this script on it.
		foreach (Transform child in transform) 
		{
			children.Add(child.gameObject);
		}
	}
	
	void Update () 
	{
		//Rotates by the rotation speed per second.
		transform.Rotate (new Vector3(0,rotateSpeed*Time.deltaTime,0));
		
		//Check to see if the current frame has been displaying longer then the framerate.
		if(thisFrame >= 1 / fps)
		{
			//adds one to the current frame.
			curFrameID += 1;
			
			//Resets frames to 0 when we get to the end
			if(curFrameID > children.Count-1)
			{
				curFrameID = 0;	
			}
			
			//if it's 0, we turn of the last frame, otherwise, we turn off the frame before this one.
			if(curFrameID == 0)
			{
				children[children.Count-1].transform.GetChild(0).gameObject.renderer.enabled = false;	
			}else{
				children[int.Parse((curFrameID-1).ToString())].transform.GetChild(0).gameObject.renderer.enabled = false;
			}
			
			//turn on the current frame.
			children[int.Parse(curFrameID.ToString())].transform.GetChild(0).renderer.enabled = true;
			
			//We subtract based on the target framerate instead of setting to zero.
			//This is to remove inconsistancies, and keeps the frame accurate even if we go over by .01 milliseconds.
			thisFrame -= 1/fps;
			
		}
		
		//Add the delta so we know how long we've been playing this frame.
		thisFrame += Time.deltaTime;
	}
	
	void OnGUI()
	{
		//Displays default fps button and sets the framerate on click
		if(GUI.Button(new Rect(0,Screen.height - 60, 100,40),"Default FPS"))
		{
			fps = defaultFPS;	
		}
		
		//Displays the frames per second slider.
		fps = GUI.HorizontalSlider(new Rect(0, Screen.height-20, Screen.width, 20),fps,0,30);	
		
		//Displays the frames per second text area. This can also be typed in to change the value.
		fps = float.Parse(GUI.TextArea(new Rect(Screen.width-100,Screen.height-40, 100,20),fps.ToString()));
	}
}

"CaliberMengsk" wrote:
Well, without loading it in real time (loading it into the level in the unity editor) I got some pretty good results. I'll have to try the resources.load when I get more spare time. Here's a webplayer.

http://calibermengsk.com/games/unity/3dframes/

You can change the framerate with the slider bar. It also rotates at around 50 degrees per second (around 7 seconds for a full rotation.) and is independent of the framerate. You can also type into the text box what framerate you want. The slider is limited to a range of 0 to 30, but if you want faster, you can just type it in. 6 frames though isn't much to work with.

EDIT:
Oh, here's the commented code if you want to try it out. Just make an empty game object and put this script on it, then import

using UnityEngine;
using System.Collections;
using System.Collections.Generic;

public class Cycle : MonoBehaviour {
	public float fps = 24f;
	public float rotateSpeed = 5f;
	
	private float curFrameID = 0f;
	private float thisFrame = 0f;
	private float defaultFPS;
	public List<GameObject> children;
	
	void Start () 
	{
		//Sets the default framerate.
		defaultFPS = fps;

		//Put all of the "frames" in order (by name) in a game object and place this script on it.
		foreach (Transform child in transform) 
		{
			children.Add(child.gameObject);
		}
	}
	
	void Update () 
	{
		//Rotates by the rotation speed per second.
		transform.Rotate (new Vector3(0,rotateSpeed*Time.deltaTime,0));
		
		//Check to see if the current frame has been displaying longer then the framerate.
		if(thisFrame >= 1 / fps)
		{
			//adds one to the current frame.
			curFrameID += 1;
			
			//Resets frames to 0 when we get to the end
			if(curFrameID > children.Count-1)
			{
				curFrameID = 0;	
			}
			
			//if it's 0, we turn of the last frame, otherwise, we turn off the frame before this one.
			if(curFrameID == 0)
			{
				children[children.Count-1].transform.GetChild(0).gameObject.renderer.enabled = false;	
			}else{
				children[int.Parse((curFrameID-1).ToString())].transform.GetChild(0).gameObject.renderer.enabled = false;
			}
			
			//turn on the current frame.
			children[int.Parse(curFrameID.ToString())].transform.GetChild(0).renderer.enabled = true;
			
			//We subtract based on the target framerate instead of setting to zero.
			//This is to remove inconsistancies, and keeps the frame accurate even if we go over by .01 milliseconds.
			thisFrame -= 1/fps;
			
		}
		
		//Add the delta so we know how long we've been playing this frame.
		thisFrame += Time.deltaTime;
	}
	
	void OnGUI()
	{
		//Displays default fps button and sets the framerate on click
		if(GUI.Button(new Rect(0,Screen.height - 60, 100,40),"Default FPS"))
		{
			fps = defaultFPS;	
		}
		
		//Displays the frames per second slider.
		fps = GUI.HorizontalSlider(new Rect(0, Screen.height-20, Screen.width, 20),fps,0,30);	
		
		//Displays the frames per second text area. This can also be typed in to change the value.
		fps = float.Parse(GUI.TextArea(new Rect(Screen.width-100,Screen.height-40, 100,20),fps.ToString()));
	}
}

Many thanks CaliberMengsk this is cool. I will check out the code.

I've also had some help from drash on this as well, so I think I have something working for now. I plan to post some results here soon. I will also test out your script today. Thank you kindly for putting this together. Very cool. 8-)

It's very interesting to experiment with different frame rates of playback, whilst viewing at 60+fps with the Rift. I think playback of 24-48fps should be acceptable for 4D playback to look smooth.

Motion Blur:

I've heard many mention (I think even Carmack and Abrash) that motion blur is bad. I agree it is on everything, i.e. Camera movement but I think motion blur on objects is necessary. We do after all see this in real life, you can see it if you wave your hand in front of your face. We are also perfectly happy to watch film at 24fps with motion blur.

The next trick is figuring out how to do motion blur across obj frames (using textures as reference?) as these scan meshes don't have any correspondence at all!

I think there must be a way to do it, using per-pixel texture colour locations or geometry distance calculations? as they are a fixed set of motions, that will always repeat. Whether 10 frames or 10,000 frames! there must be a way to predetermine or precalculate motion and add blurring as a post process somehow?

Does anyone have experience on this? or are there any technical papers out there about it?

Thanks to drash and CaliberMengsk we got some early prototype 4D footage working.

We are playing back about 10,000,000 polygons here, on the fly in Unity. 50 frames so far. 50x 125k polygons per scan. 50x 2048x2048 .jpg textures and there is really no hit in performance at all, off a standard slave drive (not SSD) Early prototype test data.

Is that preloaded (like my script)? or is that using a resource.load and buffering?

"CaliberMengsk" wrote:
Is that preloaded (like my script)? or is that using a resource.load and buffering?

Honestly, I'm not sure. drash could answer that. What ever the method is, it's very smooth. But I would imagine there will come a point when it will top out or FPS will drop vastly.

I will try your script this afternoon. Waiting on an email ping from you so I can send you a link to test with.

"Infinite" wrote:

"CaliberMengsk" wrote:
Is that preloaded (like my script)? or is that using a resource.load and buffering?

Honestly, I'm not sure. drash could answer that. What ever the method is, it's very smooth. But I would imagine there will come a point when it will top out or FPS will drop vastly.

Yep, this one is all preloaded for now, like your script. Modern computers are sweet. :D