Beyond 1080p screens - Path to 4K uncertain for Oculus?

exponential growth in cpu and gpu is coming to an end......when was the last time processing speed doubled? On the up side price is going down. thanks to multy-threaded aps we can get around the problem but I think there has to be a limit. Some of the work will have to be done on the optimization side.

"raidho36" wrote:
Actually the eye moves really slow. What to you appear as an instantanous eye twitch, appear as a smoothed out slow movement to an external observer. This is due to cassadic masking. So provided optical tracking can do at least 30 fps scanning, that wouldn't be an issue.

Umm, the human eye is the fastest moving part in the human body. At peak speed, it can do 900 degrees per second. If you go by Abrash's requirement for motion tracking of 0.25 degrees, at maximum speed your eye would take about a quarter of a millisecond to move that much.

Of course, that ignores acceleration and deceleration, but the fastest saccades can be as fast as 20ms from start to finish. To have any amount of accuracy in predicting an event that only lasts 20ms in its entirety, your would need to be sampling at a much higher rate than every 20ms. Even just to notice the saccade beginning and measure the acceleration rate would require four samples* in what is presumably less than half the total length of the event, so you're looking at something like at least a 500hz sample rate to get meaningful data out of the movement...

Yeah, you can do eye tracking at lower rates and just poll where the eye is pointing at any given point in time, but to do it well for VR you need more than just knowing where its pointing.

*: One sample would be right before the movement, one sample would be right after (confirming movement has started), then you need three samples while in movement to calculate acceleration. You can use the second sample overall as the first sample for calculating acceleration, so four total. And that's just to notice and measure the initial acceleration...

"guspaz" wrote:

"raidho36" wrote:
Actually the eye moves really slow. What to you appear as an instantanous eye twitch, appear as a smoothed out slow movement to an external observer. This is due to cassadic masking. So provided optical tracking can do at least 30 fps scanning, that wouldn't be an issue.

Umm, the human eye is the fastest moving part in the human body. At peak speed, it can do 900 degrees per second. If you go by Abrash's requirement for motion tracking of 0.25 degrees, at maximum speed your eye would take about a quarter of a millisecond to move that much.

Of course, that ignores acceleration and deceleration, but the fastest saccades can be as fast as 20ms from start to finish. To have any amount of accuracy in predicting an event that only lasts 20ms in its entirety, your would need to be sampling at a much higher rate than every 20ms. Even just to notice the saccade beginning and measure the acceleration rate would require four samples* in what is presumably less than half the total length of the event, so you're looking at something like at least a 500hz sample rate to get meaningful data out of the movement...

Yeah, you can do eye tracking at lower rates and just poll where the eye is pointing at any given point in time, but to do it well for VR you need more than just knowing where its pointing.

*: One sample would be right before the movement, one sample would be right after (confirming movement has started), then you need three samples while in movement to calculate acceleration. You can use the second sample overall as the first sample for calculating acceleration, so four total. And that's just to notice and measure the initial acceleration...

You don't need to predict any faster than your framerate. So at 120 hz, at worst it's 5-7ms. You only need two frames: before and direction/location. You don't need to know EXACTLY where the eye is pointing by pixel, just a floating target for foveated rendering.

It would even work for doing eye contact. Otherwise, it doesn't need to be as precise as, say, head tracking.

"MrAdun" wrote:
You don't need to predict any faster than your framerate. So at 120 hz, at worst it's 5-7ms. You only need two frames: before and direction/location. You don't need to know EXACTLY where the eye is pointing by pixel, just a floating target for foveated rendering.

It would even work for doing eye contact. Otherwise, it doesn't need to be as precise as, say, head tracking.

On the contrary, direction isn't enough. With only two sample points, you basically know nothing about the speed. If you're sampling every 8ms (~120Hz), you don't know if the movement started 0ms after you first sampled or 8ms after you sampled, really. All you know is the direction the eye is moving. You can guess the speed, but that means that you'll probably be off by a few hundred degrees per second. Say the error in your guess is 400 degrees per second, and you're trying to predict 8ms into the future (one single frame at 120Hz)...

Over 8ms the eye in reality will have moved up to 7 degrees, but you'll predict that it moved only 3 degrees... an error of 4 degrees. That's a pretty big error.

A 16k screen would only be maximum if the optics were the limiting factor. I would agree though that 16k would be acceptable with advanced rendering techniques and downsampling for most things in VR for now.

20/20 vision is conservatively estimated to be 60 pixels per degree. That's for low contrast vision. For high contrast the ppd goes up.

60 ppd x 90 FOV = 5400 pixels horizontal per eye. That's for low contrast though.

For high contrast, even 85 ppd isn't enough. 85 x 90 = 7650 pixels horizontal per eye.

According to a Sony 4k PDF I read, NHK said some people can even see over 100 ppd.

Without the lenses, current pre-warp correction, and considering the pixel layout, a person with normal vision would still see aliasing even with 8k x 8k per eye with 90 degree FOV. Rendering at higher resolutions and downsampling will blur and get rid of aliasing, but also limit maximum visual acuity.

Increasing FOV will increase the need for an even higher resolution panel in a Rift type HMD.

I read some NHK studies a couple of years back myself (the same as you read?), and wasn't the high contrast stuff ANSI checkerboards? If so, then the very highest contrast will very seldom be relevant, because pretty much nobody will be doing 100% white and 100% black next to each other with curved interaction.

I might be mixing it up with the John Galt interview at Creative Cow, though (which is the best!).

In any case, normal contrast, i.e. any typical scene, is probably where the standard should be centered around, even though there are outliers (better than 20/20 vision). It also depends on brightness and contrast performance of the panel. When you have nothing else, in terms of light, entering your eyes, you won't be needing ridiculous brightness and contrast. As long as black is black, and white is perceived as white, you're set.

Hi, i'm not developper but i studied optics
it seems the limit for the best human eyes acuity is about 20/10 (but many people have more than 10/10 in France in my remember for be fighter pilot you must have 14/10 or 15/10 can't remember exactly)
The Eagles seem have about 25/10


About powerfull of 3d cards and framerate, I have question (ps: don't forget i'm not developper):
Isn't it possible to use the power of 3d cards better?
Actually before buy a 3d card we check benchmark about framerate the most is better but logically 25 or 30 fps must be usefull for human eyes (retinian persistence effect), in cinema it's enough.

In game it seems it's not enough, so my question is why we need more in games?
Correct me if I'm wrong, i suppose it's a periodicity problem.
In cinema each 1/25 of second (=0.04) exactly a new frame is displaying
In game when you have 25fps it's not great, i suppose it's because 25fps in game doesn't mean a new frame exactly each 0.04s, the displaying isn't periodic, so maybe for 0.25s you got only 2 frames, the 0.25s after 15 frames, the 0.25s after only 3frames and 5 for the last 0.25s

Isn't it possible to ask 3d card to calculate in priority the periodically frames and not lost time and power with unnecessary frame?

Best Regards

PS: sorry for my bad english

a 200 billion dollar phone industry has spurred RnD to get to where it needs to get up until this point.

But wheres the incentive to push past 2k, for high volume panel production after that?

I know many of you will disagree, but I doubt VR will spur the volume, revenue and RnD motivation to continue screen tech development in the same fashion.

"Phil06" wrote:
Hi, i'm not developper but i studied optics
it seems the limit for the best human eyes acuity is about 20/10 (but many people have more than 10/10 in France in my remember for be fighter pilot you must have 14/10 or 15/10 can't remember exactly)
The Eagles seem have about 25/10

About powerfull of 3d cards and framerate, I have question (ps: don't forget i'm not developper):
Isn't it possible to use the power of 3d cards better?
Actually before buy a 3d card we check benchmark about framerate the most is better but logically 25 or 30 fps must be usefull for human eyes (retinian persistence effect), in cinema it's enough.

I thought eagles and hawks were closer to 20/4? (At 20 ft, can clearly see something that would normally be for 4 ft away.) So 10/10 is the same as 20/20, assuming you didn't have a medical reason to switching to 10 for your base chart...

Anyway, no, 25-30 FPS is not good and it never has been. Even in movie theaters this is bad. I recently saw The Hobbit 2 in non-digital form (24fps) after having saw it in digital form (48fps) when it first came out. Holy cow is there a ton of flicker. If you read up on movie frame rates, you'll see that the 24fps is a carry over from very old days in film making as has no basis in science of the eyes. They have to add motion blur to movies during action scenes to limit the amount of flicker you perceive from your eyes tracking across the screen. But you're also stationary and your brain isn't expecting full immersion.

For VR, dropping below 60 (theoretically, dropping below where our perception stops ~56fps) is bad. It's very noticable. To account for eye tracking across the screen, it needs to be somewhere around 90-100fps.

"guspaz" wrote:

"MrAdun" wrote:
You don't need to predict any faster than your framerate. So at 120 hz, at worst it's 5-7ms. You only need two frames: before and direction/location. You don't need to know EXACTLY where the eye is pointing by pixel, just a floating target for foveated rendering.

It would even work for doing eye contact. Otherwise, it doesn't need to be as precise as, say, head tracking.

On the contrary, direction isn't enough. With only two sample points, you basically know nothing about the speed. If you're sampling every 8ms (~120Hz), you don't know if the movement started 0ms after you first sampled or 8ms after you sampled, really. All you know is the direction the eye is moving. You can guess the speed, but that means that you'll probably be off by a few hundred degrees per second. Say the error in your guess is 400 degrees per second, and you're trying to predict 8ms into the future (one single frame at 120Hz)...

Over 8ms the eye in reality will have moved up to 7 degrees, but you'll predict that it moved only 3 degrees... an error of 4 degrees. That's a pretty big error.

I really don't see a big deal about 60hz tracking.

The first guy described a method of determining where you where looking and giving low priority to peripheral vision. You are arguing that this is unacceptable because it could result in a 7 degree error. I really don't see how that prevents this from being used at all, simply give enhanced rendering to 14 degrees and you will have reduced your overhead substantially.

Below I describe two additional methods to selectively reduce gpu load further.

Firstly by tracking both eyes a you can tell what is being looked at, you know it's distance and you can create a focal plane. You simply drop the render quality and issue a blur effect on things outside of the plane of focus this would both reduce gpu requirements and improve reality. We are accustomed to delays in clarity as we adjust focus naturally so the this could be adjusted over several frames with out any issues.

Stare at your keyboard and wave your hand back and forth in front of it, you will notice that your hand is blurry, perhaps even somewhat translucent. Now track your hand as you move it rapidly across the keyboard, you will notice the keyboard is blurry but your hand is not, finally move your hand slowly over your keyboard, you will notice that both your hand the keys are relatively clear. It takes your cones and rods some time of persistent input to resolve details. In VR your eyes will still do this automatically you don't NEED to program it in, however if you know the vector of movement the eye is flowing you can safely drop render quality and blur things moving rapidly in relation to the eyes.


The focal plane method I think could be implemented successfully at just 15hz. The second method would work better at higher rates but I don't see why it wouldn't work reasonably well at 60hz.

When you are just staring blankly at unmoving things you can get quite a bit of visual information but when things are rapidly active it takes visual concentration and the amount of things you can observe drops. GPU's are taxed most heavily during high action moments, these are precisely the moments that if we what your eyes where doing we could safely drop the most detail from much of what is being displayed.

At least it seems that Samsung won't stop developing higher res screens after 1440p (which is used in GS5 that'll launch this month):

"Samsung will not stop at 560 PPI. The company is actually planning an even higher density panel - a UHD panel that will feature 860 PPI. It will be very hard to justify such a high density on a mobile device, but I guess it's a marketing race that is hard to stop. At any case, it seems that Samsung will be adopting the Diamond Pixel architecture for those high resolution displays."

Source: http://www.oled-info.com/samsung-display-confirms-developing-qhd-5-amoled-panel

Although Oculus most likely won't be using their screens, I still feel that we're going to the right direction :)