Imgui: Primitive rendering on High DPI/Retina

[DoctorGester]

The default SDL + OpenGL3 example provides a minimal way of handling the framebuffer size/screen size difference. In this example DisplaySize is the screen size, let's say 800x600 and underlying framebuffer on retina is x2 so 1600x1200. That's all good, I load my fonts 2x sized and do FontGlobalScale = 0.5f.

However, lines and primitives produced by things like Separator or frames with corner rounding are blurry. Turning off antialiasing in imgui helps a little bit, but I was able to reach a much much better result with crisp looking lines and frames (with antialiasing turned on) if I set my DisplaySize to my framebuffer size and adjust the render code accordingly.

The 'solution' obviously produces a multitude of issues such as having to rely on relative sizes everywhere and having to scale all absolute values. Is there a better alternative?

[ocornut]

Hello,

such as having to rely on relative sizes everywhere and having to scale all absolute values

I don't have those answers yet for not having worked on a full DPI aware application yet. This direction you suggested is what I believe would be the simplest one, even if it puts bits of a burden on the user code.

The other possible direction is to keep the same coordinates from imgui point of view (so, 800x600 in your example) but the rendered elements are aware of the pixel size (e.g. 0.5), and elements such as lines would be rendered accordingly. The problem is that a lots of code in imgui relies on integer rounding to have a pixel-perfect control of the rendering, and with non-integer DPI scale (windows support 125%, 150%, 175% etc.) this could incur some complication and bloat in the code in many places. Haven't investigated this idea further yet but I presume I should.

Some pointers:

• You can use ImGuiStyle::ScaleAllSizes() to create a scaled size from a source style.

• We might want to add a standard "dpi scale" floating point value that's easy to access e.g. float ImGui::GetDpiScale() to scale distance/sizes when users needs to provide absolute sizes. This needs to be standardized (as in stored in a variable known by imgui) because multi-viewports (see #1542, #1676) makes it possible that each viewported window have a unique dpi scale.

• We might want to add a specific mechanism to scale the width and thickness of objects? Not really a problem, but worth nothing that lines that are larger than 1 px wide are more costly in term of shading/vertices due to how we generate the anti-aliasing geometry.

I believe this discussion could be moved to #1676.

[DoctorGester]

@ocornut after implementing a couple of custom components with this scheme, multiplying by a pixel ratio everywhere is a total nightmare. Particularly because Imgui-owned stuff like GetCursorScreenPos and CalcTextSize already receive screen coordinates and mixing them up with my own absolute coordinates is very inconvenient, I either need to scale the GetCursorScreenPos etc down, then decide on the coordinates and then scale everything back up or scale my own vectors up as I go.

An example of how I had to do things is there:
https://github.com/DoctorGester/wrike-imgui/blob/master/src/task_view.cpp#L122-L201

[ocornut]

I'm confused by why you are doing so much scaling.
The only thing you should scale are hard-coded sizes and position (like checkbox_size padding left_line_width in your code). Everything else will end up being relative to your FontSize and style sizes.
And therefore you can call e.g. draw_list->AddRectFilled() without any wrapping or modifications.

[DoctorGester]

Oh snap, you are right. It was way tougher when I was iterating on it before. That makes it a bit easier.
I still think there is a more 'automatic' solution of sorts though? Could blurring be related purely to antialiasing?

[rokups]

I am experimenting with free HDPI scaling for imgui. This is my approach:

1. Render imgui with io.DisplaySize = {resolution_x, resolution_y} / io.DisplayFramebufferScale;. This essentially renders imgui at 96 DPI.
2. When rendering - set up projection matrix at full resolution ({resolution_x, resolution_y}).
3. data->ScaleClipRects(data->FramebufferScale); obviously.
4. And this is meat and bones of the trick. Scale positions in vertex buffer up:

        // Scale buffers up.
        for (int i = 0; i < cmdList->VtxBuffer.Size; i++)
        {
            ImDrawVert& v = cmdList->VtxBuffer.Data[i];
            v.pos.x *= data->FramebufferScale.x;
            v.pos.y *= data->FramebufferScale.y;
        }

Now we have a "free" scaling up without blur. However fonts still need fixing, and if you are rendering textures with ImGui::Image() - make sure they are rendered at full resolution instead of half. When we use io.DisplaySize we get half of our real resolution when io.DisplayFramebufferScale = {2, 2};.

Results

Reference image, rendered at 1x resolution. Text and triangle icon are sharp, no blur anywhere.

Same image scaled 2x in gimp. Everything is blurry (obviously). Especially blurry triangle icon.

Now same area rendered with io.DisplayFramebufferScale = {2, 2};. Note that manually rendered triangle icon is 2x bigger, but there is no blurring around it. No blurring on rounded corners either. Original font size is used in this image therefore it is still blurry.

Now same area rendered with io.DisplayFramebufferScale = {2, 2}; + 2x font size + FontGlobalScale = 0.5f. For some reason fonts are still 2x larger than they should be, but at least they are crisp:

And same area rendered with io.DisplayFramebufferScale = {2, 2}; + 2x font size + FontGlobalScale = 0.25f. Now there is too much padding in the buttons, icons/test look of appropriate size though:

This is a hack, yep. But it gives us same look on all display scales. Developer who works on 96 DPI screen can do padding of fixed amount in pixels and it will look correct on all other screens with different DPI. So maybe it is worthwhile to investigate fleshing this out? I will keep looking into properly fixing fonts, but if anyone has any pointers - please let me know.

Edit:
I guess this illustrates best what i aim to achieve here:

Line pointed by red arrow is of hardcoded thickness and becomes very thin on hdpi screens. As you can see in the last image we get a free scaling up.

Edit2:
Or am i walking the wrong direction, and that 1px line with 2x font scale is a bug? But in that case user should multiply all hardcoded sizes by font scale. Not exactly convenient.

[ocornut]

Thanks for the report.

• Instead of (4) you should be able to modify the projection matrix so you don’t pay any cost.

• Would be worth investigating this with non-integer scales (1.25, 1.50, 1.75) and the effect on primitives and pixel alignment. Those are factors exposed by Windows settings and it is a big part of the problem there, would be interest in what you try/find.

[rokups]

Fractional scaling works. This is fragment rendered with 1.75 scale:

Both rounding and triangle icon are crisp.

[rokups]

Recently i was working on bringing up a proper HDPI support in viewports branch. sdl_opengl3 example works therefore we can now review it and figure out if other adjustments are needed.

Key points of HDPI support

• Units used for everything (sizes, positions, etc) are 96-DPI pixels.
• Everything is upscaled for the screen UI is displayed on.
• Windows can transition between monitors of different DPI, new scale is applied when user stops dragging window.
• Mouse coordinates are adjusted to compensate for different scaling.
• All fonts are rendered into atlas texture multiple times with different scale applied. Custom rects (icons) are rendered once with highest DPI scale applied.
• Font rendered with proper upscaling is automatically selected.

Things to pay attention to

While i hid input dpi scaling from the user, io.DisplaySize is still expected to be scaled by user. Is user expected to always set io.DisplaySize before calling ImGui::NewFrame()? If so - scaling could be done unconditionally in ImGui::NewFrame(). Current implementation in imgui_impl_sdl.cpp does this:

    float dpi = ImGui::GetPlatformIO().MainViewport->DpiScale;
    io.DisplaySize = ImVec2((float)w / dpi, (float)h / dpi);

I chose to prevent window changing it's DPI while window is being dragged. This solves the issue where window goes crazy during crossing of border between screens of different DPIs. Window is rescaled to the DPI of monitor it is on when dragging stops. Unfortunately this has one side-effect - ImGui does it's own compositing and docking guides visible through transparent dragged window are in the wrong position. Video for clarity: https://streamable.com/v770t (backup link) (you can also see window rescaling happen when dragging stops on a different monitor). I need your advice how to fix this compositing discrepancy. However ideally i would suggest viewport windows to not perform any compositing but instead they should become transparent native windows and allow desktop to do compositing instead. this is paves a way for wayland support too.

And last but most important - we need some kind of policy for picking DPI scaling we want to apply. For example my monitors report DPI scales of 0.993 and 1.134. While such scaling with "natural" fonts is fine, default font is totally ruined by fractional scaling and there is no way to fix that because font is meant to fit pixels precisely. So what are we supposed to do in this case? Use dpi scaling as reported? Clamp to 1..N range? Clamp to steps 1.0..1.25..1.5..1.75..2.0? Not sure what to do here..

---

A bonus image of how windows of exact same size look on monitors with different DPIs. Monitor on the left has 1.134 DPI scale and monitor on the right has 0.993 DPI scale. Without any patches sizes of those windows should have about 20% difference.

---

I also tested this on 4k monitor with 150% upscaling and Roboto-Medium.ttf font. It looks visually pretty much same as on 1440p monitor. It works :)

Code: https://github.com/rokups/imgui/tree/make-hdpi-great-again (be aware that i will be wrecking this branch so use github to view changes and download zip if you wish to test code).

[ocornut]

Thank you Rokas for your continued investigation. Great stuff here, I'm glad we are pushing for this variant on solving the multi-DPI issue. Not answering on the smaller details just yet, I'll focus on the big issue first.

My main observation is that with non-integer scale all the shapes are "instable" in the sense that their precise alignment and pixel surface vary as you move the window. If you move any window slowly and looks at the shapes you'll see this wobbling going on.

This is particularly noticeable if you enable e.g. WindowBorder=1 FrameBorder=1 BorderCol=ImVec4(1,1,01) but you can see it in every single shapes.

Somehow I think we ought to be alter a few more things to ensure consistent scaling (we can follow up in our private channel as I'm not yet sure what's the best course of operation). This is probably the hard thing to solve for allowing this approach, hence my early mention of borders. Definitively worth pursuing as this is a super healthy approach to DPI we have here.

I also noticed that the app doesn't react immediately if a monitor changes DPI scale (which is possible under Windows, certainly unusual, but nice for debugging), it would be nice if in the example app we could allow quickly override the scale somewhere.

(Let's not worry too much about default font now, thought we could have specific settings/imfontconfig stuff to sort of minimize the ugly mess with it)

(PS: Funnily as I am typing this talking about those instable shapes, the web form in Firebox is moving by a pixel up and down with most of my keystrokes..).

[alanjfs]

I also noticed that the app doesn't react immediately if a monitor changes DPI scale (which is possible under Windows, certainly unusual, but nice for debugging), it would be nice if in the example app we could allow quickly override the scale somewhere.

Just wanted to chime in with a common (daily) use-case for this; laptop screen connected to a monitor, one of which is non-HDPi. Whenever you disconnect for the evening, and plant yourself in the sofa, any app left scaled stands out like a sour thumb. 👍