Is there a way to turn on anti aliasing MSAA for GearVR in Unreal Engine 4.14?

[saldavonschwart]

I'm working with Unreal 4.14 which supposedly even has a new forward rendering path for VR, though not sure if it is meant just for desktop vr or for mobile too.

I'm attempting to get antialiasing working on a Galaxy S6 with GearVR, but I've spent the past 2 days trying out all sorts of things and nothing makes a difference.

Basically, I start with a blank project set to mobile + scalable graphics quality. Then I've tried the following. Always packing to Android ETC2:

. default project settings (theoretically deferred path by default).
. With the new forward rendering option.
. Setting any combinations of the following console variables:
  . r.MSAACount
  . r.MSAACompositingSampleCount
  . r.DefaultFeature.Antialiasing
  . r.MobileOnChipMSAA
  . r.MobileMSAA
  . r.MobileContentScaleFactor
  . gearvr.EnableMSAA

Nothing seems to enable anti aliasing. I also posted this on SO, gamedev SE, Unreal AnwerHub and the Unreal forums. and no one has replied anything other than "MSAA should be on by default for GearVR."

Has anyone experienced this? Does anyone have any thoughts?

[rpalandri]

Hello!
Lots of questions in there, that I'll try to explain. UE4 Mobile VR (and mobile VR in general) have a looot of hacks in it, which makes it not very straightforward to understand. Will try to help.

b. The adb log spits out: "LogHMD: Enabling r.MobileOnChipMSAA, previous value 0" 
That is where the MobileOnChipMSAA variable is set, which actually enables the 2x MSAA. Depending on device, you can probaby run this on 4x MSAA. We currently have MSAA set to 2x only everywhere, but what we should do (and are planning to do for 4.15 or 16) is activate MSAA 4x on the GPUs which have a cheap 4x path like the Mali T880. Right now, you can only change this in code, in OpenGLRenderTarget,

                        const uint32 SampleCount = 2; to 4.

About the OpenGL version, you are correct that adb will tell you when running ES2 that the actual OpenGL context is ES3. Although the shaders generated in the .usf -> GLSL shadercompilation pass are GLES2 compliant (they do not use any ES3/3.1 features), the #version tag at the beginning of the shader is set on the device at runtime, and for every GearVR phone, that will be a #version 300 tag. The OpenGL context will also be set to 3, even if all the calls are compatible with ES2.

About my stereolayer project, sorry for hiding the actor containing the layer :neutral: . Right now I know the quad stereo layers are also supported on vive, and I don't know about daydream. I'm fairly sure right now only GearVR (not even rift) support the cylinder layers though.

About the aliasing you're seeing on the edges, can you try / did you try to put alpha borders on the textures you choose? On my stereolayer app when you're running it, do you also have AA issues?

[motorsep]

Is MSAA x4 cheaper than MSAA x2 on Snapdragon devices too? If so, why not just make MSAA x4 default for Gear VR in 4.15 already? 🙂 

P.S. I am still on S6 since I consider this a bottom line hardware for mobile VR, so I hope MSAA x4 is cheaper on it too 😉

[saldavonschwart]

@motorsep
@rpalandri


Ok I made a little project to test everything, maybe you want to take a look.

The project is a simple scene with 4 cards. Starting from the right, these are my observations:

1. A quad widget with a texture used for rounded edges. Notice how the aliasing on the edges is pretty bad. At this point I'm realizing that the issue is because of using the texture. But I'm not sure how would this be solved. Higher res texture? Some specific filtering or mipmapping setting in the texture editor?

2. The card I was originally working on when I started this question. It uses the same texture for the rounded edges and you'll notice the aliasing once again. Also, this card has text and another image, both of which show some aliasing too. I'd assume the way to mitigate the image aliasing would be the same as for the black rounded border image (which as stated in 1, I'm not sure what it would be -- I'd assume mipmapping it). As for the text, no idea how it would be made to alias less.

3. The stereo layer approach. This actually looks pretty good. But:

a. Why does it not show the rounded border if it's rendering the middle card which has rounded borders? Maybe I need to tweak some resolution or total render area / viewport parameter?

b. I actually need gaze input for this whole project. i.e.: gaze at a button in the card and stuff happens. It seems like it would be a pain to raycast onto the texture and then somehow extrapolate the hit point to the original card widget to activate whatever real control the original card would have.

c. What about performance? Now I have all the machinery of the original widget plus the extra RTT + stereo layer overhead.

4. Plain widget (orange colored). For some reason this time around it does not show aliasing on the edges like it did in the image I posted earlier. So I think this shows that MSAA is actually enabled. Though if I added elements to it I'd be back at 2.

Let me know if you have any thoughts. Thanks again for the help.

[saldavonschwart]

@rpalandri btw, when you say to try turning MSAA x4 on by changing:

const uint32 SampleCount = 2; to 4.

Would this require recompiling the engine source or can it be done with the Epic Launcher engine versions? I ask because I don't know if you read a previous comment where I explained that I downloaded the engine source and recompiled adding logs to see if multisampling was being set for the FBO, and while I was able to compile the engine and run the editor, attempting to package the sample project for Android would fail. It would say there were a lot of missing files which I have no idea why would be missing (since compiling the engine and running inside the editor worked fine).

[saldavonschwart]

Ok I closed the question as answered because MSAA itself is enabled and this was answered. But I still have pending questions:

@rpalandri

About MSAA:

1. Does changing the sample count from the default of 2 to 4 require a full engine source recompilation?

About UMG:

1. Is there a way to control antialiasing of text? That is, of the textures generated from the text? Maybe something like filtering options or maybe something like Unity's canvas scaler settings?


2.What would the optimal settings be as far as mipmapping and whatnot for the UMG images in the widget?


About stereo layers:

3. Why do they not render the rounded borders of the original UMG widget?


4. What should I do with the original widget? Rotating it 180deg so that it does not show seems not optimal.

5. What about raycasting and interacting with controls in the original widget? Do I need to compute the diff between the widget pos / rot and the stereo layer's and mimic any interaction onto the original widget? 

[rpalandri]

Alrighty I'll try to answers the questions I know the answer to 🙂

@motorsep 4x MSAA is not free on Snapdragon/Adreno (especially on older snapdragon chips), unlike on Mali, which is why we defaulted it to 2x. A better way to do this, coming normally around 4.16, would be to do 4x Mali, 2x Snapdragon by default.

For the other questions:
1) yeah changing it from 2 to 4 requires entire engine recompilation. Also, pressing the "Compile" button in the editor does NOT recompile the engine! (it'd have a hard time doing that, since the editor is running the engine 😉 ). It recompiles your game's specific C++ classes, but not the underlying engine. 

UMG.1) I'm not sure about UMG's rendering of text, I haven't seen antialiasing options but I'm far from a UMG expert. You can try rendering to a bigger UMG texture, then mipmapping it down to levels you want.

UMG.2) Depends of how far you're gonna be from the widget. You want to have a mipmap whose size is approximatively equivalent to the number of pixels the layer will take on the screen. There are 2 or 3 carmack posts out there on layers and mipmapping for text, I'd try to read those.

Layers.3) A layer only renders the texture you give it to. It's really straightforward. In your app (I opened it), you gave it a quad texture, so it rendered a quad. If you want rounded borders, you need to put a transparency-based rounded border on your texture. That is not, if you look, how the widget does its round borders though! The widget does it by creating a mesh whose borders are round, not by modifying the texture, therefore that will have no effect on the texture, and therefore no effect on the layer. I'm sure however that you can do your own UMG widget class that creates a rounded texture, which will work with stereo layers.

Layers.4) In this https://forums.oculus.com/developer/discussion/44130/gear-vr-rendering-umg-widget-in-stereolayer-com... thread, I showed 2 ways of doing widget to texture render, one using blueprint (that I don't like at all), one using code. The blueprint one uses render to material, and then extracts the texture from the material (which therefore needs the material to be rendered to, which will only happen if the widget is in view). In that case, the best way I know of like you said, is to turn the widget 180 degrees. If you use the code version though (which uses a function library that can be trivially added to any blueprint project), you can ask the widget to render to a texture manually, which shouldn't necessitate the widget to be in view. 

Layers.5) In order to use gazetracking, you want to use this: https://docs.unrealengine.com/latest/INT/Engine/UMG/HowTo/InWorldWidgetInteraction/ to basically emulate mouse input with gaze. It should be pretty straightforward, but to be honest I haven't used it yet.

You also had a question on performance. StereoLayers, due to the number of copies it requires (one more than non-stereolayers), will be a bit more expensive than non-stereolayers. In the parallel universe where I have free time to work on this, I'd like to have a way to output the layer's swapchain directly into the UMG renderer to avoid that extra copy. I'll try to get on that some day, promise! However, since Stereolayers are used for UI, I don't think you should have performance issues since those are the moments where the game doesn't require too much extra perf from physics, stuff like that. 

To finish, I'd like to reiterate that if you want to do quads without stereolayers, you 1000000% need to use alpha blending on the side of your textures. I REPEAT. use alpha blending. If you have to remember anything from those long posts, it's that. The only reason why I say "without stereolayers" here is that we already manually do alpha blending for you guys when you do stereolayers, so it's not necessary to do it twice.

[saldavonschwart]

@rpalandri

just an update, which unfortunately I did not get to do before you replied.

About MSAA:

I already figured out how to recompile the engine, etc. But the line you had suggested to change MSAA sample count (OpenGLRenderTarget.cpp: 152) does not actually change the sampleCount. That whole code path inside the #ifdef android does not execute when launching the game in the device. Further, there is a comment above line 152 which implies the sampleCount is hardcoded in other places. 

And the FBO after that line implies that code path is only for multi view which I am not using since it is experimental and @motorsep said was kind of buggy.

So I wasn't able to increase MSAA from x2 to x4. I'd need to know where else in the engine code this needs to be changed.

About Stereo Layers:

I'm using the C++ version of RTT btw (where you exposed the C++ static method as a BP function). But I'm not sure I follow how to get the stereo layer to render the rounded corners.

From what I know about UMG so far, it provides no out of the box (vertex or texture based) way of doing rounded corners for its elements. So The 2 ways I was able to do rounded corners were:

1. Creating an UMG border element with a "rounded rect texture" as the brush.
2. Creating a copy of the material the whole widget uses to render in the world and:
  a. setting the blend mode to mask.
  b. adding a sampler to the opacity mask value, where I sample a "rounded rect" texture.

The disadvantage of approach 1 is that even if I make the UMG border use the rounded rect texture, since I place an UMG image element as a child of it (on top of it), this image covers the top rounded corners of the border. So in this sense approach 2 works best because it literally masks the otherwise sharp square UMG widget.

If I use approach 2 and then do a RTT I believe I lose the rounded corners because these are a masking step that takes place AFTER the UMG widget has rendered, so I think it is not captured by rendering the widget to another texture. At least that's my assumption on why I was not getting rounded corners when doing the stereo layer.

If I use approach 1, I do get the rounded borders (only on the bottom of the card because of what I explained earlier), but the resulting render target that eventually becomes the texture of the stereo layer has a black background. So there is no transparency.

Does this make sense? I mean that in UMG, I assign a texture that is a black rounded rect with transparency (for the corner areas visible since the inner black rounded rect has rounded corners) and while the UMG widget in world space respects the transparency in the borders, the resulting render target instead has black in the borders instead of transparency. 

So I'm not sure how I would add a transparent mask to the texture after rendering and before adding it to the stereo layer, if I understood correctly your previous message.

The thing that nags me the most about the stereo layer is having to still have the widget around... seems clunky and a performance waste. But maybe I'm wrong and it's not that big of a deal; I honestly just don't know enough about UE yet.

And yes, I'll take your advice on Alpha blending.

[rpalandri]

I'm not sure how to achieve that, but on the sample I gave motorsep, I had transparency working in the UMG-rendered stereolayer (it was a cat picture with text underneath it with transparent background), so the same kinda technique should work there. Haven't played much with UMG. good luck.

[saldavonschwart]

@rpalandri Sorry but I'm not sure I understood what you meant by alpha blending in this context. Alpha blending to me just means using the alpha channel for transparency. From this, I'd understand that you mean, that I should make a square texture with rounded borders within the texture and the area difference on each edge of the square texture (square edge - rounded rect) be transparent. If this is what you mean, I was already doing that; otherwise I would not have gotten rounded borders. Or do you mean something else?

[saldavonschwart]

@rpalandri
Hey btw, I went back to your UILayer example and the stereo layer does not show transparency. In the original widget, you have the text "this is why they don't let me do UI anymore" over a transparent background (with the cat image ending just above this text). But the rendered target fills the transparent background with black. This is consistent with what I experienced: that there seems to be no transparency when rendering to texture.