Merge Conflicts for Non-Coders

I was discussing my day at work with my family, and I had spent much of the day resolving conflicts after doing a merge of two divergent branches.  Of course, neither my wife nor my son understood what I was talking about, so I came up with a very simple analogy to explain what I was doing.

Imagine a teacher wrote out a plot outline for a book, and then assigned each student to write one chapter.  The student writing chapter 5 decides that the heroes need to use a shovel, so he make a small change in the draft of chapter 3 so that they find a shovel at the beach.  Meanwhile the student writing chapter 3 decides she doesn't like the beach scene, so she changes it so that they go for a hike in the woods instead.

Eventually the students have to put all their work together.  The conflicting changes demonstrate two types of conflicts:

First, in adding in the shovel in chapter 3, the first student made changes to a paragraph that the other student also modified in changing the scene from the beach to the woods.  When using an automated merge tool, this sort of conflict is automatically detected, and the person doing the merge is shown the conflict.

Second, if in resolving the direct conflict, it's possible that the shovel is removed from chapter 3.  The merge had been completed, but no automated tools will tell you that the shovel used in chapter 5 now comes out of nowhere.  If this were computer code, this would either show up at compile time ("Error: Reference to undefined object "shovel" in chapter 5, paragraph 12."), or worse, would introduce a bug that only shows up when the code runs under certain circumstances.  If the book merge left the shovel out, hopefully it would be found by the editor (testing by the quality assurance team for a software project), not left for readers to be confused by (a bug in the field).

The right analogy can often make it much easier to explain things.  If you like this one, or have questions about other aspects of software engineering that you would like me to find similar analogies for, please comment.

Linux With a 4K Laptop Display

I have a new laptop with a 4K display.  This is double the resolution in each direction of a typical laptop, but with the same physical dimensions.  That means each pixel takes up a quarter of the space of a single pixel on a traditional display.  By default, many things will run using the same number of pixels as on a smaller display, resulting in text too tiny to read.

Making the problem more complicated, I want to be able to connect my laptop to an external display running at a lower resolution, and be able to disconnect it, walk away, and still have things work on the built-in screen.  I also want things look right if I plug in a much larger 4K display.  In other words, I want everything to work based on the DPI of the screen, not the pixels.

The simple solution is to run the display at a lower resolution.  This quick-and-dirty hack is good enough for most people, but you lose the crispness of the 4K display, and lose the ability to play 4K video at full resolution (which I may someday care about).  It also just feels like the wrong approach.

So I set my goal to make everything work nicely regardless of whether I'm using an external display or not, and handle changing between external and internal displays on the fly as seamlessly as possible.

Many people discussing this use the term "HiDPI" for high dots-per-inch displays, so if you're searching for solutions that I don't cover here, that's one term to search on.

I'm running Gentoo Linux, so some of my comments are going to be specific to that distribution, but for the most part it's simply a matter of finding the different locations for the equivalent files in your distribution.

Another good resource that I found in researching this is the Arch Linux Wiki page on the same topic:  https://wiki.archlinux.org/index.php/HiDPI

HiDPI Linux Console

First, there are two different things to worry about.  Almost everything I've seen online focuses on X, but I also want to have the console working correctly.  Here I have three issues to get working correctly:

1. Setting up the Grub boot menu to look right.
2. Setting the console font to a reasonable size.
3. Changing console fonts and resolution when changing displays.

Grub Boot Menu

After some searching, I found that I can install fonts for Grub to use.  The command I used was:

mount /boot
SIZE=48
grub-mkfont --output=/boot/grub/fonts/DejaVuSansMono-${SIZE}.pf2 \
   --size=${SIZE} /usr/share/fonts/dejavu/DejaVuSansMono.ttf
umount /boot

In Gentoo, you can select the font in /etc/default/grub, so when building the configuration file, it will use the selected font:

GRUB_FONT=/boot/grub/fonts/DejaVuSansMono-48.pf2

The only problem with this font and size is the vertical lines have slight gaps.  From what I've found, this is a font issue due to using the wrong vertical line character (ASCII vertical bar instead of a unicode vertical line, or the other way around).  I would like to figure out what the offending character is and remove it from the character set or swap it with a good one, but I haven't figured out yet which character it's using and which character I want it to use.

Another problem is that while this makes the text perfectly readable on my built-in display, the text is huge if I boot with an external monitor.  I looked into rewriting the grub scripts to select the font based on the screen resolution (from the 'videoinfo' command).  Normally the grub.cfg is generated dynamically from a script provided as part of the grub installation, but I could write my own or have a post-processing tweak script that added my changes.  Unfortunately, the grub scripting language just isn't powerful enough to do this.  There is simply no mechanism to have the script behave differently based on the output of a command (like 'videoinfo').  There's no ability to pipe and grep to test for certain output or to put the output of a command into a variable.  If there were, I could check the resolution of the active display and select from the installed fonts to get an appropriate size.  Of course, if I could do that, I could also parse the output of a directory listing and dynamically generate the menu to select from the installed kernels, eliminating the need to update the grub config every time I install a new kernel.

So for now, I'm stuck with selecting a single predetermined font for grub regardless of what screen happens to be attached at boot time.  I'm sticking with the -48 font, which looks good on the laptop screen even though it is ridiculously huge on my external display, but still has room for my menu options.  I submitted a feature request for grub to add dynamic font selection support (bug 51914).

Also, I've noticed that interactions with grub are very slow.  This is bug 46133.  It's mostly unrelated to the topic at hand, but the bug only shows up at high screen resolutions, so don't be surprised if you hit it.

Linux Console Font

For the console, I needed to find a font that looked reasonable.  I use a for loop in bash to try every font in /usr/share/consolefonts, and the only one that looked right for me was latarcyrheb-sun32.psfu.gz.  As a quick fix, 'setfont -C /dev/tty1 /usr/share/consolefonts/latarcyrheb-sun32.psfu.gz' worked, repeating for each console.  That's still a bit small, resulting in a text console of 240x67.  (Note: I've now installed the Terminus font, but the largest there is the same size; ter-i32b.psf.)

For a real solution, I could use the consolefont service on Gentoo, but that still leaves the kernel boot messages too tiny to read.  I found a great writeup of a solution for putting a font into the kernel:  https://www.artembutusov.com/modify-linux-kernel-font/  That's the real solution for booting with a good console font.  I followed his instructions to install Terminus font as the kernel font, and now all the boot messages are readable.

Still, I'm limited to a maximum size of a -32 font, which is readable but small on my screen, and I can't make it dynamically select the boot font based on the screen resolution.  That's a fundamental limitation in Linux for now.

For the Linux console, the kernel is stuck with one font as far as I can tell, but I can use the consolefont service to select the font based on the screen resolution.  All I need is to set /etc/conf.d/consolefont to check the screen size and then adjust the font if needed.  But what I really want is a solution that detects and adjusts the font if I switch between screens.

Generating a console font from a TrueType font like DejaVu Sans Mono would probably be ideal, as then a script could generate the font for exactly the right size based on the DPI.  Everything I've seen online suggests that this is difficult at best.  Supposedly fontforge can do this, but I haven't figured out how to use it.  The grub-mkfont program shows that it can be done, but that only generates grub fonts, and I haven't found anything that converts out of that format.  Some research showed that there was a ttf2bdf program with older versions of Freetype, which has been supplanted by otf2bdf.  The page for this is down, but I was able to find an ebuild and the source elsewhere.  The output of that can be sent to bdf2psf (a package available in Gentoo).  Put this all together, and we have console fonts dynamically sized to the active display.  But for some reason the resulting fonts don't work, so something is broken.

The good news is that the kernel developers are aware of this problem, and one of the features of the 5.0 Linux kernel is a new console font for HiDPI displays.  I look forward to trying this, especially seeing if they have a solution for changing monitors.

Dynamically Changing Console Resolution

The command: 'fbset -a -xres 1980 -yres 1200' corrects the resolution (provided it's connected to a 1980x1200 monitor).  The command 'hwinfo --monitor' will provide information about connected monitors, allowing this to be scripted.  Now all that's needed is a trick to activate a script anytime a monitor is added or removed.

To detect video changes, it appears that I can't rely on any system events, which is unfortunate.  You would think that connecting or disconnecting a monitor would trigger something like udev, but no such luck.  My laptop uses an external USB-C dongle for most video connections, and it would probably work for detecting the connection of the dongle (since that changes the devices), but if the video cable is connected after the dongle is plugged in, that doesn't help.  It also doesn't help if I use the built-in HDMI port on the laptop.  I want a solution that will always work and isn't dependant on other coinciding changes.

What does work is monitoring changes in /sys.  In particular, watch for changes to the directory /sys/class/drm and for changes in any file /sys/class/drm/*/status.  Unfortunately, this is a virtual file system, so the kernel only updates the files when someone looks.  If something changes in sysfs, but nobody looks, did it really change?  No, it didn't.  That means you can't use inotify to find out about changes.

So getting this to work requires a script that actively polls files in /sys to watch for resolution changes.

Of course, there will be things I will want to tweak in X on monitor changes, too, so those can go in the same script.  If requested in the comments, I'll post the script.

Dynamically Changing Console Fonts

The same script that changes the console resolution can also change the console fonts.  After playing with this, though, I decided it was a better solution to simply leave everything at the -32 font, which is a bit small on the 4K display and a bit big on other displays, but usable everywhere.

HiDPI Linux X11 Display

Running X Windows on a high DPI display is challenging because there are several different ways that applications adjust.  A few settings will cover most applications, but then others either require application-specific settings or simply don't have scaling options.  Also, changing DPI on the fly is difficult; most applications will need to be restarted to adjust.  We're in the early days of 4K displays right now, so hopefully everything that is still actively maintained will automatically adjust within the next year or two.

I expect most desktop environments like KDE and Gnome have their own ways of configuring the DPI, just like they do with other configuration options.  I don't run a desktop environment, so I'm not going to talk about them.  Some of the settings I'll discuss are hopefully adjusted under the covers by your desktop settings, but you can fall back to setting them manually like I do if that doesn't work.

Before setting the DPI, I need a good way of determining what the DPI is.  Fortunately, the screen dimensions are reported by the monitor's EDID information, and it's usually accurate.  In some cases, the information is wrong, or you'll find that things look better with a different DPI.  Until recently, a DPI of 96 was considered typical, so that's what you probably get if you don't adjust anything.

The command I use to determine the DPI extracts the screen size (in mm) from xrandr and converts it to inches:

DPI=$(xrandr | egrep ' connected .* [1-9][0-9]*mm x [1-9][0-9]*mm|[*]' | \
   sed -e N -e 's/\n/ /' | \
   sed -e 's/^[^ ]*//' -e 's/([^)]*)//' -e 's/^[^0-9]*[^ ]*//' -e 's/[^0-9]/ /g' | \
   awk '{printf("%g %g\n",$3/($1 * 0.0393701),$4/($2 * 0.0393701))}' | \
   awk '{printf("%d\n",($1+$2+0.5)/2)}' | head -n 1)

General Settings

Ideally we should set some value on the X server, and then your applications will use that value, select the right fonts, and adjust the scaling of everything automatically.  The good news is that since most programs use a small number of toolkits like gtk+, qt, and the like, we're in reasonably good shape.  The toolkits are DPI-aware, so if you tell them what DPI to use, programs based on them will just work.

Much of the configuration of the X server can now be done dynamically with xrandr.  This includes setting the DPI.  This is the most important setting, and hopefully everything will move to use this one setting in the future:

xrandr --dpi ${DPI}

For GTK, it uses a X resource.  In most systems, your ~/.Xresources file is loaded when X starts, but if you want something dynamic, the line to use is:

xrdb -override <<< "Xft.dpi: ${DPI}"

That uses a Bash "here string."  It's roughly equivalent to:

echo "Xft.dpi: ${DPI}" | xrdb -override

But it doesn't create a subshell.

That takes care of applications like Thunderbird and emacs (for the menus if compiled with gtk).

Cursors / Pointers

I've never played with modifying the cursors in X before, so I learned a lot about how they work.  There's one cursor that is displayed in the root window, which defaults to a 'x'.  This is the only one set by X itself, and you change it with the command xsetroot.  All the other cursors are set by the programs you're running (which includes the window manager).  Some programs will automatically adjust the cursor that they use based on the DPI settings they use for other scaling.  The method used by other applications is to read the information from X resources.

In playing with setting cursors, I found that you can install a bunch of different cursor themes.  I installed and tried a bunch, and rather liked the whiteglass theme, which is part of the x11-themes/xcursor-themes package in Gentoo.

xrdb -override <<< "Xcursor.theme: whiteglass"

xrdb -override <<< "Xcursor.size: $(( DPI / 6 ))"

For the root window cursor, I picked a cursor from a different theme, but stuck with the default X_cursor.  This is set using the full path (which may be different on your system or even with different themes).  X sets this cursor as part of its startup before the resources are loaded, so it can't be controlled by the resources.

xsetroot -xcf /usr/share/cursors/xorg-x11/Adwaita/cursors/X_cursor $(( DPI / 6 ))

You can change the root cursor anytime you like, but the cursors inside applications, including those set by the window manager, require that you restart your application to pick up the changes.  This means restarting your window manager when you change monitors.

Application-Specific Settings

Xterm: I've never worried about fonts in xterm before, but it turns out to be a fairly simple problem to solve.  You just select a TrueType font for xterm and set a reasonable point size.  Then it will select the font size to be correct for the DPI of the display (as of the time the terminal launches).  To make this work, I set two X resources:

XTerm*faceName: Dejavu Sans Mono Bold

XTerm*faceSize: 8

Chromium: You have to specify a command-line option for a forced scale factor.  Hopefully they'll fix this in the future, but you'll still want this if you don't like the default:

 --force-device-scale-factor=$(bc <<< "scale=5;${DPI} / 96" )

That's a little on the large size, so use a higher base DPI than 96 if it doesn't look right to you.

VNC viewer: I've been using tightvnc viewer, but it's been pretty much abandoned, and it doesn't have any scaling option.  There's a fork of the project, though, that solves this problem: ssvnc.  This is mostly a set of wrappers to tunnel VNC though ssl or ssh, but I've been doing that with my own wrappers already, so I'm ignoring that.  What's really useful is that ssvncviewer is command-line compatible with TightVNC's vncviewer, and it adds a scaling factor.  Since I'm already using a wrapper script for ssh, I just modified it to add a scale of 2 if the DPI is over 200.

ssvncviewer -scale 2 ...

twm: Yes, I'm the last person on Earth still running twm, but the information here might apply elsewhere, too.

First, your window manager has the job of launching programs, and some of those require special parameters based on the DPI.  If you don't want to have to restart your window manager when you switch monitors, the solution is to use wrapper scripts for all of those programs.  Second, you'll need some trick to have a dynamic configuration for your window manager.  I use twm, but I patched it to use popen() instead of fopen() when reading the configuration file if it finds that it's executable, so my .twmrc is already generated by script.  (Perhaps I'll add a blog entry on my hacks of twm.)

It's not surprising that twm doesn't support TTF fonts, so you can't just specify a point size and let it scale (though I've seem some references to patches, so it's not an unreasonable approach; moving to vtwm or another window manager might also make sense).  Instead, the fonts are hard-coded in the .twmrc file.  Since I already have that file dynamically generated, I can set the fonts based on the DPI.  One simpler method would be to have two config files and have a script that detects monitor changes handle setting a symbolic link to the correct one.

Others: If you find other applications that don't just work, I would suggest finding their bug tracking system and opening a bug.  Someday everything should just work.

Future Work

What if you have multiple displays at different resolutions?  Unfortunately, you can only set one DPI with xrandr or X resources at a time.  Every time someone asks about setting per-monitor DPI, the answer seems to be to scale the higher DPI display.  Making this work correctly would also require that applications supported changing DPI on the fly, so that they would adjust automatically if dragged to a different screen.  For those moving from X to Wayland, apparently it's using per-monitor scaling as its solution, too.