Monday, February 25, 2019

Linux With a 4K Laptop Display

I have a new laptop with a 4K display.  This is double the resolution in each direction of a typical laptop, but with the same physical dimensions.  That means each pixel takes up a quarter of the space of a single pixel on a traditional display.  By default, many things will run using the same number of pixels as on a smaller display, resulting in text too tiny to read.

Making the problem more complicated, I want to be able to connect my laptop to an external display running at a lower resolution, and be able to disconnect it, walk away, and still have things work on the built-in screen.  I also want things look right if I plug in a much larger 4K display.  In other words, I want everything to work based on the DPI of the screen, not the pixels.

The simple solution is to run the display at a lower resolution.  This quick-and-dirty hack is good enough for most people, but you lose the crispness of the 4K display, and lose the ability to play 4K video at full resolution (which I may someday care about).  It also just feels like the wrong approach.

So I set my goal to make everything work nicely regardless of whether I'm using an external display or not, and handle changing between external and internal displays on the fly as seamlessly as possible.

Many people discussing this use the term "HiDPI" for high dots-per-inch displays, so if you're searching for solutions that I don't cover here, that's one term to search on.

I'm running Gentoo Linux, so some of my comments are going to be specific to that distribution, but for the most part it's simply a matter of finding the different locations for the equivalent files in your distribution.

Another good resource that I found in researching this is the Arch Linux Wiki page on the same topic:  https://wiki.archlinux.org/index.php/HiDPI

HiDPI Linux Console


First, there are two different things to worry about.  Almost everything I've seen online focuses on X, but I also want to have the console working correctly.  Here I have three issues to get working correctly:

  1. Setting up the Grub boot menu to look right.
  2. Setting the console font to a reasonable size.
  3. Changing console fonts and resolution when changing displays.

Grub Boot Menu

After some searching, I found that I can install fonts for Grub to use.  The command I used was:

mount /boot
SIZE=48
grub-mkfont --output=/boot/grub/fonts/DejaVuSansMono-${SIZE}.pf2 \
   --size=${SIZE} /usr/share/fonts/dejavu/DejaVuSansMono.ttf
umount /boot

In Gentoo, you can select the font in /etc/default/grub, so when building the configuration file, it will use the selected font:

GRUB_FONT=/boot/grub/fonts/DejaVuSansMono-48.pf2

The only problem with this font and size is the vertical lines have slight gaps.  From what I've found, this is a font issue due to using the wrong vertical line character (ASCII vertical bar instead of a unicode vertical line, or the other way around).  I would like to figure out what the offending character is and remove it from the character set or swap it with a good one, but I haven't figured out yet which character it's using and which character I want it to use.

Another problem is that while this makes the text perfectly readable on my built-in display, the text is huge if I boot with an external monitor.  I looked into rewriting the grub scripts to select the font based on the screen resolution (from the 'videoinfo' command).  Normally the grub.cfg is generated dynamically from a script provided as part of the grub installation, but I could write my own or have a post-processing tweak script that added my changes.  Unfortunately, the grub scripting language just isn't powerful enough to do this.  There is simply no mechanism to have the script behave differently based on the output of a command (like 'videoinfo').  There's no ability to pipe and grep to test for certain output or to put the output of a command into a variable.  If there were, I could check the resolution of the active display and select from the installed fonts to get an appropriate size.  Of course, if I could do that, I could also parse the output of a directory listing and dynamically generate the menu to select from the installed kernels, eliminating the need to update the grub config every time I install a new kernel.

So for now, I'm stuck with selecting a single predetermined font for grub regardless of what screen happens to be attached at boot time.  I'm sticking with the -48 font, which looks good on the laptop screen even though it is ridiculously huge on my external display, but still has room for my menu options.  I submitted a feature request for grub to add dynamic font selection support (bug 51914).

Also, I've noticed that interactions with grub are very slow.  This is bug 46133.  It's mostly unrelated to the topic at hand, but the bug only shows up at high screen resolutions, so don't be surprised if you hit it.

Linux Console Font

For the console, I needed to find a font that looked reasonable.  I use a for loop in bash to try every font in /usr/share/consolefonts, and the only one that looked right for me was latarcyrheb-sun32.psfu.gz.  As a quick fix, 'setfont -C /dev/tty1 /usr/share/consolefonts/latarcyrheb-sun32.psfu.gz' worked, repeating for each console.  That's still a bit small, resulting in a text console of 240x67.  (Note: I've now installed the Terminus font, but the largest there is the same size; ter-i32b.psf.)

For a real solution, I could use the consolefont service on Gentoo, but that still leaves the kernel boot messages too tiny to read.  I found a great writeup of a solution for putting a font into the kernel:  https://www.artembutusov.com/modify-linux-kernel-font/  That's the real solution for booting with a good console font.  I followed his instructions to install Terminus font as the kernel font, and now all the boot messages are readable.

Still, I'm limited to a maximum size of a -32 font, which is readable but small on my screen, and I can't make it dynamically select the boot font based on the screen resolution.  That's a fundamental limitation in Linux for now.

For the Linux console, the kernel is stuck with one font as far as I can tell, but I can use the consolefont service to select the font based on the screen resolution.  All I need is to set /etc/conf.d/consolefont to check the screen size and then adjust the font if needed.  But what I really want is a solution that detects and adjusts the font if I switch between screens.

Generating a console font from a TrueType font like DejaVu Sans Mono would probably be ideal, as then a script could generate the font for exactly the right size based on the DPI.  Everything I've seen online suggests that this is difficult at best.  Supposedly fontforge can do this, but I haven't figured out how to use it.  The grub-mkfont program shows that it can be done, but that only generates grub fonts, and I haven't found anything that converts out of that format.  Some research showed that there was a ttf2bdf program with older versions of Freetype, which has been supplanted by otf2bdf.  The page for this is down, but I was able to find an ebuild and the source elsewhere.  The output of that can be sent to bdf2psf (a package available in Gentoo).  Put this all together, and we have console fonts dynamically sized to the active display.  But for some reason the resulting fonts don't work, so something is broken.

The good news is that the kernel developers are aware of this problem, and one of the features of the 5.0 Linux kernel is a new console font for HiDPI displays.  I look forward to trying this, especially seeing if they have a solution for changing monitors.

Dynamically Changing Console Resolution

The command: 'fbset -a -xres 1980 -yres 1200' corrects the resolution (provided it's connected to a 1980x1200 monitor).  The command 'hwinfo --monitor' will provide information about connected monitors, allowing this to be scripted.  Now all that's needed is a trick to activate a script anytime a monitor is added or removed.

To detect video changes, it appears that I can't rely on any system events, which is unfortunate.  You would think that connecting or disconnecting a monitor would trigger something like udev, but no such luck.  My laptop uses an external USB-C dongle for most video connections, and it would probably work for detecting the connection of the dongle (since that changes the devices), but if the video cable is connected after the dongle is plugged in, that doesn't help.  It also doesn't help if I use the built-in HDMI port on the laptop.  I want a solution that will always work and isn't dependant on other coinciding changes.

What does work is monitoring changes in /sys.  In particular, watch for changes to the directory /sys/class/drm and for changes in any file /sys/class/drm/*/status.  Unfortunately, this is a virtual file system, so the kernel only updates the files when someone looks.  If something changes in sysfs, but nobody looks, did it really change?  No, it didn't.  That means you can't use inotify to find out about changes.

So getting this to work requires a script that actively polls files in /sys to watch for resolution changes.

Of course, there will be things I will want to tweak in X on monitor changes, too, so those can go in the same script.  If requested in the comments, I'll post the script.

Dynamically Changing Console Fonts

The same script that changes the console resolution can also change the console fonts.  After playing with this, though, I decided it was a better solution to simply leave everything at the -32 font, which is a bit small on the 4K display and a bit big on other displays, but usable everywhere.


HiDPI Linux X11 Display

Running X Windows on a high DPI display is challenging because there are several different ways that applications adjust.  A few settings will cover most applications, but then others either require application-specific settings or simply don't have scaling options.  Also, changing DPI on the fly is difficult; most applications will need to be restarted to adjust.  We're in the early days of 4K displays right now, so hopefully everything that is still actively maintained will automatically adjust within the next year or two.

I expect most desktop environments like KDE and Gnome have their own ways of configuring the DPI, just like they do with other configuration options.  I don't run a desktop environment, so I'm not going to talk about them.  Some of the settings I'll discuss are hopefully adjusted under the covers by your desktop settings, but you can fall back to setting them manually like I do if that doesn't work.

Before setting the DPI, I need a good way of determining what the DPI is.  Fortunately, the screen dimensions are reported by the monitor's EDID information, and it's usually accurate.  In some cases, the information is wrong, or you'll find that things look better with a different DPI.  Until recently, a DPI of 96 was considered typical, so that's what you probably get if you don't adjust anything.

The command I use to determine the DPI extracts the screen size (in mm) from xrandr and converts it to inches:

DPI=$(xrandr | egrep ' connected .* [1-9][0-9]*mm x [1-9][0-9]*mm|[*]' | \
   sed -e N -e 's/\n/ /' | \
   sed -e 's/^[^ ]*//' -e 's/([^)]*)//' -e 's/^[^0-9]*[^ ]*//' -e 's/[^0-9]/ /g' | \
   awk '{printf("%g %g\n",$3/($1 * 0.0393701),$4/($2 * 0.0393701))}' | \
   awk '{printf("%d\n",($1+$2+0.5)/2)}' | head -n 1)

General Settings

Ideally we should set some value on the X server, and then your applications will use that value, select the right fonts, and adjust the scaling of everything automatically.  The good news is that since most programs use a small number of toolkits like gtk+, qt, and the like, we're in reasonably good shape.  The toolkits are DPI-aware, so if you tell them what DPI to use, programs based on them will just work.

Much of the configuration of the X server can now be done dynamically with xrandr.  This includes setting the DPI.  This is the most important setting, and hopefully everything will move to use this one setting in the future:

xrandr --dpi ${DPI}

For GTK, it uses a X resource.  In most systems, your ~/.Xresources file is loaded when X starts, but if you want something dynamic, the line to use is:

xrdb -override <<< "Xft.dpi: ${DPI}"

That uses a Bash "here string."  It's roughly equivalent to:

echo "Xft.dpi: ${DPI}" | xrdb -override

But it doesn't create a subshell.

That takes care of applications like Thunderbird and emacs (for the menus if compiled with gtk).

Cursors / Pointers

I've never played with modifying the cursors in X before, so I learned a lot about how they work.  There's one cursor that is displayed in the root window, which defaults to a 'x'.  This is the only one set by X itself, and you change it with the command xsetroot.  All the other cursors are set by the programs you're running (which includes the window manager).  Some programs will automatically adjust the cursor that they use based on the DPI settings they use for other scaling.  The method used by other applications is to read the information from X resources.

In playing with setting cursors, I found that you can install a bunch of different cursor themes.  I installed and tried a bunch, and rather liked the whiteglass theme, which is part of the x11-themes/xcursor-themes package in Gentoo.

xrdb -override <<< "Xcursor.theme: whiteglass"
xrdb -override <<< "Xcursor.size: $(( DPI / 6 ))"

For the root window cursor, I picked a cursor from a different theme, but stuck with the default X_cursor.  This is set using the full path (which may be different on your system or even with different themes).  X sets this cursor as part of its startup before the resources are loaded, so it can't be controlled by the resources.

xsetroot -xcf /usr/share/cursors/xorg-x11/Adwaita/cursors/X_cursor $(( DPI / 6 ))

You can change the root cursor anytime you like, but the cursors inside applications, including those set by the window manager, require that you restart your application to pick up the changes.  This means restarting your window manager when you change monitors.

Application-Specific Settings

Xterm: I've never worried about fonts in xterm before, but it turns out to be a fairly simple problem to solve.  You just select a TrueType font for xterm and set a reasonable point size.  Then it will select the font size to be correct for the DPI of the display (as of the time the terminal launches).  To make this work, I set two X resources:

XTerm*faceName: Dejavu Sans Mono Bold
XTerm*faceSize: 8

Chromium: You have to specify a command-line option for a forced scale factor.  Hopefully they'll fix this in the future, but you'll still want this if you don't like the default:

 --force-device-scale-factor=$(bc <<< "scale=5;${DPI} / 96" )

That's a little on the large size, so use a higher base DPI than 96 if it doesn't look right to you.

VNC viewer: I've been using tightvnc viewer, but it's been pretty much abandoned, and it doesn't have any scaling option.  There's a fork of the project, though, that solves this problem: ssvnc.  This is mostly a set of wrappers to tunnel VNC though ssl or ssh, but I've been doing that with my own wrappers already, so I'm ignoring that.  What's really useful is that ssvncviewer is command-line compatible with TightVNC's vncviewer, and it adds a scaling factor.  Since I'm already using a wrapper script for ssh, I just modified it to add a scale of 2 if the DPI is over 200.

ssvncviewer -scale 2 ...

twm: Yes, I'm the last person on Earth still running twm, but the information here might apply elsewhere, too.

First, your window manager has the job of launching programs, and some of those require special parameters based on the DPI.  If you don't want to have to restart your window manager when you switch monitors, the solution is to use wrapper scripts for all of those programs.  Second, you'll need some trick to have a dynamic configuration for your window manager.  I use twm, but I patched it to use popen() instead of fopen() when reading the configuration file if it finds that it's executable, so my .twmrc is already generated by script.  (Perhaps I'll add a blog entry on my hacks of twm.)

It's not surprising that twm doesn't support TTF fonts, so you can't just specify a point size and let it scale (though I've seem some references to patches, so it's not an unreasonable approach; moving to vtwm or another window manager might also make sense).  Instead, the fonts are hard-coded in the .twmrc file.  Since I already have that file dynamically generated, I can set the fonts based on the DPI.  One simpler method would be to have two config files and have a script that detects monitor changes handle setting a symbolic link to the correct one.

Others: If you find other applications that don't just work, I would suggest finding their bug tracking system and opening a bug.  Someday everything should just work.

Future Work

What if you have multiple displays at different resolutions?  Unfortunately, you can only set one DPI with xrandr or X resources at a time.  Every time someone asks about setting per-monitor DPI, the answer seems to be to scale the higher DPI display.  Making this work correctly would also require that applications supported changing DPI on the fly, so that they would adjust automatically if dragged to a different screen.  For those moving from X to Wayland, apparently it's using per-monitor scaling as its solution, too.

Tuesday, August 15, 2017

A Subtle Difference between C and C++: String Literals

For the most part, C++ is a superset of C.  You can write C code, rename the file with a .cpp extension, and the compiler will compile it in C++ mode, generating essentially the same code.  In fact, the very first C++ compilers were actually C compilers with extra pre-processing.  But there are some subtle differences, and I recently ran into one that has some important implications.

In C, string literals are not constant, but in C++ they are.


For most practical purposes, string literals are constants in C.  The compiler puts them into a data segment that the code can't modify.  All the libc string functions take (const char *) parameters.  But if you assign a (char *) pointer to the start of a string literal, the compiler won't warn you about dropping the 'const' from the type, and you can write code that modifies the string (only to have it fault at run time).  In fact, the warning about dropping the 'const' is probably the reason that newer versions of C haven't changed string literals to be constant.

Why does anyone care?

I've found two specific instances where code works when compiled as C++, but not as C because of this.

You can't use string literals as the targets of case statements.


This may seem obvious, as a case statement takes an integer type, not a string.  But suppose you want to do a switch based on the first four characters of a string (after ensuring that it's at least four characters long).  Imagine the following macro:

#define STR2INT(s) ( ((s[0]) << 24) | ((s[1]) << 16) | ((s[2]) << 8) | (s[3]) )

Now you could write code like:

   switch(STR2INT(option)) {
      case STR2INT("help"):
         ...
      case STR2INT("read"):
         ...
   }

That works in C++.  But in C, the compiler complains that the case statements aren't constant expressions.  To make it work in C, you have to have a much uglier version of the macro for the case statements:

#define CHARS2INT(w, x, y, z) (((w) << 24) | ((x) << 16) | ((y) << 8) | (z))

Then the code looks like:

   switch(STR2INT(option)) {
      case CHARS2INT('h','e','l','p'):
         ...
      case CHARS2INT('r','e','a','d'):
         ...
   }

That works in both C and C++, but is a pain to write.  At least the STR2INT macro works fine in other situations where the compiler insists on constant values.

You can't write asserts based on macro names.


In large software projects, it's not unusual to have sets of macros for specific purposes.  These macros are by convention supposed to follow some project-specific format.  There even may be a defined correlation between the name of the macro and the value.  It would be nice to be able to write asserts based on the macro name to enforce those conventions.

A quick aside on asserts:

Both C and C++ now support compile-time asserts.  It used to be that you would write code that would generate a negative shift if the expression wasn't true or something like that.  When the assert failed, you would get a compile-time error that was rather confusing until you looked at the offending line.  With the new mechanism, the compiler displays whatever message you tell it.  You use static_assert(expression,"message");  In C, you have to include <assert.h> or use _Static_assert.  This was added in C11 and C++11.

So for a trivial example, suppose we have macros like:

#define VAL_7B 0x7b

Now somewhere we use those macros:

   process_value(VAL_7B);

Obviously real code would have other parameters, but this is enough for our purposes.

To have asserts based on the macro name, what appears to be a function call must also be a macro; presumably a macro wrapper around the real function call.  Consider this definition:

#define process_value(v) \
   do { \
      _process_value(v); \
   } while(0)

That's a basic wrapper, forcing a semicolon at the end of the do-while loop.  This lets us add in asserts using the '#' preprocessor operator to stringify the input parameter:

#define CHAR2HEX(c) ( (c) <= '9' ? (c) - '0' : (c) - 'A' + 10 ) // Assumes uppercase
#define process_value(v) \
   do { \
      static_assert( (#v)[0]=='V' && (#v)[1]=='A' && (#v)[2]=='L' && (#v)[3]=='_', "Must use a 'VAL_xx' macro here" ); \
      static_assert( CHAR2HEX((#v)[4]) == ((v)>>4)  , "'VAL_xx' macro doesn't match defined value" ); \
      static_assert( CHAR2HEX((#v)[5]) == ((v)&0x0f), "'VAL_xx' macro doesn't match defined value" ); \
      static_assert( (#v)[06]==0, "'VAL_xx' macro format wrong" ); \
      _process_value(v); \
   } while(0)

In C++, that works great.  In C, you just can't do that.

And here's something interesting:  Why not change the above example to look like:

#define CHAR2HEX(c) ( (c) <= '9' ? (c) - '0' : (c) - 'A' + 10 ) // Assumes uppercase
#define process_value(v) \
   do { \
      static_assert( (#v)[0]=='V' && (#v)[1]=='A' && (#v)[2]=='L' && (#v)[3]=='_', "Must use a 'VAL_xx' macro here" ); \
      static_assert( CHAR2HEX((#v)[4]) <= 0xf  , "'VAL_xx' macro with bad hex value" ); \
      static_assert( CHAR2HEX((#v)[5]) <= 0xf  , "'VAL_xx' macro with bad hex value" ); \
      static_assert( (#v)[06]==0, "'VAL_xx' macro format wrong" ); \
      _process_value( CHAR2HEX((#v)[4])<<4 | CHAR2HEX((#v)[5]) ); \
   } while(0)

This uses the value directly out of the macro name, so you can leave the value off entirely when defining the macro, right?  Yes.  But it goes further than that.  Since the above code only uses the stringification of the parameter, it never expands it.  That means it's perfectly happy if you never define the VAL_XX macros at all, which is probably not what you want.  Be sure that the wrapper macro expands the macro somewhere if you want to be sure it's actually a defined macro.

Conclusion


So if you've followed my other writing up to this point, you're probably expecting some clever hack to make this work in C.  Sorry, but not this time.  It would probably be relatively simple to add a compiler option or #pragma directive to make string constants literal in C, but gcc doesn't have this, and I'm not aware of any other compiler that does.  (Please comment if you know otherwise.)  There are plenty of tricks you could do if you're willing to use additional tools in your build process, like an additional step between the regular preprocessor and the compiler to look for extracting characters from string literals and convert them into character constants (and you could tell the compiler to use a wrapper script to do that as the preprocessor), but that's not likely to be an acceptable option.

You just can't do that in C.

Resources

This is the test file I used to be sure my above examples were correct: c_vs_cpp_example.c

Friday, August 11, 2017

Installing Gentoo Linux on a Dell Precision 5510

From time to time I realize just how slow and outdated my computer is.  Usually that's long after the corporate policy says I can order a new one, and this time was no different.  The engineering laptop is a Dell (no surprise if you know where I work), and the default model is the Precision 5510.  New computers always come with Windows installed, so my first task is to install Linux.  I figured other people with this or similar models will want to hear about what issues I encountered, so I'm documenting the process here.

The first thing I did was update the BIOS.  There have been some low-level security issues in the press recently, so I figured I should start with the latest base.  This was easy: I booted into the installed Windows, logged in, and went go to http://dell.com/support.  From there I entered my service tag, clicked on downloads, and selected the BIOS.  As I expected, the installed BIOS was a few months older than the latest one.  I'm glad I didn't try to install this under Linux, though I assume there is a way to do so.  It was a simple process of clicking the obvious choices, and then it rebooted and completed flashing.

Now the challenges began.

This is my first laptop without a CD/DVD drive (unless you go back to the floppy days).  So to boot Linux, I needed a bootable USB flash.  I had followed some very complicated instructions to set up a flash drive, but then found that I had much better results by simply taking the Gentoo live CD and using 'dd if=source_file.iso of=/dev/sdb bs=8192k' to copy it.

So it should be simple to boot from the flash drive, right?  Wrong!  Windows set up a "fast boot" where it bypasses most of the BIOS, including the part that watches for holding down F12 for the boot menu.  After some searching, I found that you need to enter "control panel" in the Windows search thing that replaces the Start menu (this is my first time to ever run Windows 10).  From there I used one of the control panels to disable fast boot.  Even with that, it took several tries to get it to recognize the F12 for the boot menu--it checks it very early in the process and moves past it very quickly.

So I booted the Gentoo USB, and started following the installation guide.  I've done this before, so I wasn't expecting too much trouble, but I did have some surprises.  First, I wiped the partition table and set it up for Gentoo.  I chose to use 1GB VFAT for the EFI partition that is also used as my /boot partition.  When everything is good, I'll only mount it to copy over new kernels when updating.  Most of the rest of the drive is my root partition, for which I selected ext4.  The last 64GB I used as a swap partition, which is excessive, but I hate running out of memory and having the OOM killer sacrifice the one thing I wanted to keep alive.

At this point, I found that I had no network.  This is my first modern laptop without an ethernet port.  It came with three different USB-C dongles that I could use, one is just ethernet, one is ethernet, video, and USB, and one is essentially a docking port with lots of ports on it.  What I quickly learned is that USB-C isn't like traditional USB.  USB-C is the connector, but it can use several different protocols over that connector, including PCI, in which case the marketing name is Thunderbolt.  None of that was working in Linux.  More on this in a bit...

So for the time being, I tried copying over stuff using a second flash stick, but that didn't get me very far.  I found an old regular USB ethernet dongle, and that worked fine.  Once I set up my kernel to boot from EFI, everything should have been mostly good, but of course it wasn't.  I had to go back and forth between booting from USB and failing with my kernel.  This was a challenge because I couldn't get the F12 menu to come up once I had wiped the Windows partition.  I ended up switching back and forth between legacy and EFI booting in the BIOS; legacy for USB and EFI for my attempt at my installed kernel.  This probably wasn't necessary, but it worked to get me past the problem.  (I needed to go into the BIOS anyway to tell it to boot the grub loader from EFI instead of Windows.)

Getting the system to boot natively was a huge pain.  Part of the problem is that after the boot completes, Gentoo clears the screen for the login prompt, so you lose the boot messages.  I decided to fix this by commenting out the getty command for tty1 in /etc/inittab.  Now I have to switch consoles to log in, but I can always see the last screenful of the boot.  I eventually got my installed kernel to work.  It turned out that I had failed to configure tmpfs, and Gentoo fails the boot process miserably if it can't put a tmpfs file system on /run.  This is what I get for configuring my own kernels--a bit of pain, but I always learn things.

So I finally dived into the problem of the Thunderbolt devices simply not working.  The main problem was again the BIOS.  It was configured with security options for Thunderbolt.  This is a good idea--without security, any device plugged into that port can do anything it wants with your computer.  But for setting up Linux, it's not a good idea, so I turned off all the security options.  I would like to turn them back on at some point, but I'll have to do some research to see if and how Linux handles Thunderbolt security.

Issues

Now I'm able to get the system up and running for the most part, but I'm still having some issues.  I'll document each one here:


Issue:

Switching from X back to a console works, but then moving back to X doesn't

Solution:

Need to load firmware as mentioned in https://wiki.gentoo.org/wiki/Intel
Switch acceleration from Sandybridge New Architecture (SNA) to UXA
Setting 'Option "DRI" "False"' works--switching is consistent, though there's an 8-10 second black screen when switching back to X.

Issue:

The trackpad works fine in the console as long as you don't want to do a right or middle click.  If you're used to using gpm, you can't do cut-and-paste operations or anything else requiring a middle or right mouse button.

Solutions:

I have not found any solution to this problem.

Issue:

X has no middle-click on the trackpad.

Solutions:

You can set the logical location of the middle button to be in the middle of the bottom of the trackpad with the synclient command.  Unfortunately, this doesn't help with gpm on the console.  I added the following script to my X startup script:

#!/bin/bash
eval $(synclient -l | egrep '(Left|Right)Edge|RightButtonAreaTop|MiddleButtonAreaRight' | sed -e 's/ //g')
if [ "${MiddleButtonAreaRight}" != 0 ]; then
    echo Middle mouse button already set up
    exit 0
fi
# The order here is critical or it will be rejected:
synclient RightButtonAreaLeft=$(( RightEdge - (RightEdge - LeftEdge) / 3 + 1 ))
synclient MiddleButtonAreaRight=$(( RightEdge - (RightEdge - LeftEdge) / 3 ))
synclient MiddleButtonAreaTop=${RightButtonAreaTop}
synclient MiddleButtonAreaLeft=$(( LeftEdge + (RightEdge - LeftEdge) / 3 ))

You can play with that to get the buttons to work however you like.  I chose to have three equal buttons, and I didn't play with the other settings, but you can look at the output of 'synclient -l' to see what all you can fiddle with.  Do note my comment about the order.  You can't have overlapping buttons, so I had to be sure to shrink the right button before defining the middle button (the left button is implicit, so you don't have to adjust it).

Issue:

The Thunderbolt devices work fine if they're connected when the system boots.  If you unplug and replug them, the system sometimes needs a kick to rescan the PCI bus:
echo 1 > /sys/bus/pci/rescan
Takes care of it usually.  However, after several repeated disconnect/connect cycles, the PCI scan fails, and it can't use the device.

Solution:

I don't do a lot of disconnecting/reconnecting, so I haven't dug any further into this.

Kernel Config

Getting the kernel config right for all the devices can be tricky.  I'm old fashioned enough that I configure my kernel with support for exactly the devices I use, avoiding modules as much as possible.  I'm noting here some of the drivers you'll want to be sure to include:

Trackpad: From dmesg, I see it's a SynPS/2 Synaptics TouchPad.

Ethernet: The dock uses a Realtek 8153 USB ethernet chip.

Sound: The WD-15 dock uses a 0bda:4014 Realtek USB audio device

Touch Screen: It has an Elan touchscreen.  Select the "HID Multitouch panels" driver under "Special HID drivers" in the kernel configuration.  It's working, but I haven't tuned it.  At some point I may want to investigate, and this looks like a promising guide:  https://wiki.archlinux.org/index.php/Calibrating_Touchscreen

WiFi: It has an Intel 8260 chip, which uses the iwlwifi driver  It also requires firmware.  I used this guide:  https://wiki.gentoo.org/wiki/Iwlwifi

Bluetooth: It has an Intel 8087:0a2b USB Bluetooth device.  I configured it so that the kernel recognizes it and is happy, but I don't have a wireless keyboard or mouse, so I may not use it anytime soon, but I'm thinking about using it for audio.  The only catch was that before it would connect to my Amazon Echo, I had to tell it, "Alexa, pair with my phone."  A volume control shows up with "alsamixer -D bluealsa."  The guide I used is: https://wiki.gentoo.org/wiki/Bluetooth

Video: It has both Intel and Nvidia graphics, but unlike some laptops, you can't just ignore the Intel and use the Nvidia graphics--you have to access the Nvidia through the Intel GPU.  So you can't install the Nvidia drivers without first installing the Intel drivers.

Flash Reader: It's a Realtek RTS525A PCI device.  It requires the kernel to be built with CONFIG_MFD_RTSX_PCI and CONFIG_MMC_REALTEK_PCI.  Unlike my old laptop, the card sticks out when inserted, so I can't leave an empty microSD adapter in the laptop.

WebCam: I haven't looked into using the webcam.

Monday, June 19, 2017

Go To Statement Considered Helpful (part 5: Fun with Switches)

For my final segment on the use of gotos, let us not forget one important hidden goto. The switch statement is essentially a goto with a variable label. This means that the if(0) trick can apply here, too.

Before I get into the meat, I want to point out that many of the examples in this section aren't great code.  In most cases the best way is to find a more traditional solution.  Unlike the previous segments I've written on using gotos, this is mostly fun hacking food for thought, not something you're likely to include in your code.

You're probably vaguely familiar with Duff's Device. Typically you run across it as a loop unrolling technique that a professor mentions in some computer science class, but you then forget about it. Anyway, here's an example.  Without Duff's Device, you have a standard loop as below:

for(i=0; i<n; ++i) do_it_once();


With Duff's Device, the loop is unrolled manually as below:

int loops = (n + 3) / 4;
switch(n%4) {
    do {
        case 0: do_it_once();
        case 3: do_it_once();
        case 2: do_it_once();
        case 1: do_it_once();
    } while ( --loops > 0 );
}


That takes a bit to make sense of. The trick is that first time in, the switch jumps to only do the odd cases, then it hits the do/while loop and goes through all the cases for as many times as is needed. The most common case cited for using that code is writing out a buffer to a register.  If you're writing to a serial port or something like that, then that's the right way to do it, but if you're doing something more like a buffer copy, a simpler way that is easier to parse would be to pull the loop outside the switch as in the code below. To keep this example parallel to the Duff's device example, the odd instances are done first, though there's no reason for doing it one way or the other.  (In fact, for a buffer copy, you might do odd bytes first and last to optimize for alignment, depending on the needs of the system.)

switch(n%4){
    case 3: do_it_once();
    case 2: do_it_once();
    case 1: do_it_once();
}
for(i=0;i<n/4;++i) do_it_four_times();


That's rarely used, but in very specific cases it can be a big win. For example, a memory copy routine may use something very similar to use large registers for the bulk of the data while still supporting odd numbers of bytes. Duff's Device (with the do loop inside the switch statement) has the advantage of eliminating redundant code, especially when the do_it_four_times() call needs to be the same statement four times, and when a larger number is used. If you're working at this level of manual optimization, they key is to try a number of implementations and run performance testing. When combining clever code with optimizing compilers and specific hardware, there are usually more factors than you're taking into account, and you may be surprised at the results.

Now for what I like to call Crow's Corollary to Duff's Device. I sometimes find that I have a series of cases that handle mostly the same thing, but need a slightly different setup. Other cases have completely different code. See the example below.

switch(var) {
    case A:
        buf_to_use = A_BUF;
        use_buf(buf_to_use);
        break;
    case B:
        buf_to_use = B_BUF;
        use_buf(buf_to_use);
        break;
    case C:
        buf_to_use = C_BUF;
        use_buf(buf_to_use);
        break;
    case D:
        // Completely different code from A, B, C
        ...
}


Now imagine that "use_buf()" is 15 or 20 lines of code instead of a simple function.  (Yes, I'll accept that the best solution is probably to refactor those 15 to 20 lines into a simple function, but I'm having fun looking at other options.)  In order to avoid repeating the code that is the same for the similar cases, you can put the case labels inside an if(0) just like with a goto! Here is the example without the repeated code.

switch(var) {
    case A:
        buf_to_use = A_BUF;
        // Fall through but skip over assignment
        if ( 0 ) {
    case B:
            buf_to_use = B_BUF;
        }
        // Fall through but skip over assignment
        if ( 0 ) {
    case C:
            buf_to_use = C_BUF;
        }
        // General code for cases: A, B, C
        use_buf(buf_to_use);
        break;
    case D:
        // Completely different code from A, B, C
        ...
}


The importance here is the if(0) construct allows you to avoid duplicating the code that is common to several of the cases without requiring that the initial portion of the cases be identical.  This could be implemented with a nested switch statement to set up A, B, and C, but programmers are lazy; I've never seen anyone implement the above example with a nested switch to avoid the use_buf() duplication, even if it's 20 lines of code instead of one. Rarely will someone create a function to factor out the duplicated code. Usually I just see the 20 lines of code duplicated. That's bad. That's what if(0) is there to solve.

Of course, this can be implemented more directly using goto:

switch(var) {
    case A:
        buf_to_use = A_BUF;
        goto use_buf;
    case B:
        buf_to_use = B_BUF;
        goto use_buf;
    case C:
        buf_to_use = C_BUF;
        goto use_buf;
    // General code for cases: A, B, C
    use_buf:
        use_buf(buf_to_use);
        break;
    case D:
        // Completely different code from A, B, C
        ...
}


The former version with if(0) blocks interleaved with the case statements is more fun, but it's certainly more confusing and hard to read than simply using a goto. The goto version is also much more friendly to your text editor's auto-indent configuration. (I'm not even sure what "proper" indention is for the interleaved if(0) construct!)

And here's what you would need to write if you were being pedantic about not using gotos:

switch(var) {
    case A:
    case B:
    case C:
        switch(var) {
            case A:
                buf_to_use = A_BUF;
                break;
            case B:
                buf_to_use = B_BUF;
                break;
            case C:
                buf_to_use = C_BUF;
                break;
            default:
                flag_error(); // Should be impossible to reach
                break; // avoid warning
        }
        // General code for cases: A, B, C
        use_buf(buf_to_use);
        break;
    case D:
        // Completely different code from A, B, C
        ...
}


That's clunkier to write, harder to read, and more difficult to maintain. If you change the list of cases that use_buf() applies to, you have to modify the case statements in both switch statements, which might as well be begging for bugs. The repeated case statements in the nested switch statement still constitute duplicated code--exactly what we're trying to avoid by factoring out the use_buf() code.  Keep your code clean and concise. Use the right tool for the job, and sometimes 'goto' is the right tool.

If you really want to write the same code without a goto or repeated cases, there is a way, but I don't recommend it. You can interleave another loop inside the switch statement so the breaks for the cases break the loop, not the switch.  It may be a fun hack, but it's not good code:

switch(var) {
    // do loop to change where break goes
        do {
    case A:
            buf_to_use = A_BUF;
            break; // exits do loop, not switch
    case B:
            buf_to_use = B_BUF;
            break; // exits do loop, not switch
    case C:
            buf_to_use = C_BUF;
            break; // exits do loop, not switch
        } while (0);
        // General code for cases: A, B, C
        use_buf(buf_to_use);
        break; // regular switch exit
    case D:
        // Completely different code from A, B, C
        ...
}


This is the closest corollary to Duff's Device, but again, don't do that. The above example should be viewed by the compiler as being identical to the goto version. Most C programmers aren't used to mixing up switch statements with other control structures like that, which also makes it harder to understand. Unless you have some arbitrary "no gotos" rule handed down from management, the goto version is the simplest to read and understand. Use the gotos.

Go To Statement Considered Helpful (part 4: Tail Recursion)

One other place where goto can be useful is in implementing tail recursion. As you should recall, tail recursion is when a function returns with a call to itself. For example:

foo(int a)
{
    ...
    if ( recurse )
    {
        return(foo(a+1)); // tail recursion
    }
}

Most compilers will optimize that into a goto to the top of the function. However, in some cases the compiler will fail you, and you have a runtime fault when the stack overflows. Perhaps you turned off optimization to aid in debugging. Perhaps the compiler isn't as smart as it should be. Who knows? If you're running with a limited stack (and stacks are always limited), don't trust your compiler. Replace the recursion with a goto as below:

foo(int a)
{
  tail_recursion:
    ...
    if ( recurse )
    {
        a=a+1;
        goto tail_recursion; // return(foo(a)) without recursion
    }
}

Now you won't blow away your stack if the optimization settings change.

Of course, you can implement that by putting the body of the function inside a while(1){...; break;} loop, and then replace the goto with a continue. However, doing that is just as much of a hack as a goto. It also adds an unnecessary indention level, and it is definitely harder to read than the simple goto.

Unlike the previous examples, I'm not hiding the goto here in a macro. I am using a label name that should tell any experienced programmer what I'm doing. If I were to hide this in a macro, it would be less clear what's going on.

Now you might object that previously I've said to trust the compiler to optimize your code and don't worry about whether it looks like you're adding a few extra instructions.  I still stand by that, but this isn't a matter of trusting optimization for a few instructions.  This is a matter of your program not crashing.  I ran into this exact situation in real code.  When the correctness of your code requires optimization, you need to force the code to always be optimized, which is exactly why we need the goto in this case.

Go To Statement Considered Helpful (part 3: Finding Matches in a Loop)

Another time I often find use for a goto is in a for loop that is scanning an array to find a match. If a match is found, a break terminates the loop. See the standard version of this below. It repeats the condition check for the loop to see if the loop terminated early. The other option would be to use a flag to indicate if a match was found. This requires initializing the variable and then setting it if a match is found. Both of these variations are very common.


for (i=0; i<ARRAY_SIZE(data); ++i)
{
    if ( data[i].key == value ) break;
}
if ( i<ARRAY_SIZE(data) )
{
    // data[i].key is our match
    match_found:
    ...
}
else
{
    // No match was found
    ...
}

I hate repeating code; that's a source of bugs, as years later someone might need to make a change and only change one of the two copies. Adding a tracking variable is clunky. Again, goto can save the day. Consider the code below:


for (i=0; i<ARRAY_SIZE(data); ++i)
{
    if ( data[i].key == value ) goto match_found;
}
if ( 0 )
{
    // data[i].key is our match
    match_found:
    ...
}
else
{
    // No match was found
    ...
}

This eliminates the repeated comparison, so if something changes, it only has to change once. A quick reaction that many will have is that this also will be faster than the alternative, but generally compilers are very good at eliminating redundant comparisons, and processors are fast enough that it's rarely worth worrying about saving a few machine instructions; in fact, it's usually best to be willing to sacrifice machine instructions for better code.

Without the goto, you can't scope the loop index to be just the loop, or you end up using an extra boolean tracking variable. Most importantly, without the goto, the code is longer and more complicated, and that's exactly what you don't want. Of course, in most cases, you'll still need the loop index to know which item matched, but sometimes you only need to know if there was a match.

Once again, the C preprocessor comes to the rescue with a clever pair of macros below:


#define EXIT_LOOP_WITH_MATCH(name) goto _loop_match_ ## _name;
#define IF_LOOP_EXITED_WITH_MATCH(_name) if(0) _loop_match_ ## _name:

The macro works just like a regular if statement. You can use it with or without braces, and you can use a regular else condition. The only catch is that the compiler will likely complain about an unused label if you have the IF macro without a matching EXIT macro. That may well be more of a benefit than a restriction.

With the macro, our previous example is nice and clean as seen below:


for (i=0; i<ARRAY_SIZE(data); ++i)
{
    if ( data[i].key == value ) EXIT_LOOP_WITH_MATCH(my_loop);
}
IF_LOOP_EXITED_WITH_MATCH(my_loop)
{
    // data[i].key is our match
    ...
}
else
{
    // No match was found
    ...
}

Those macros may look familiar. They should. They're exactly the same as the THROW and CATCH macros I discussed previously for exception handling in C.

It is interesting to note that this and the previous example for the use of goto (breaking out of a nested loop and avoiding extra code to handle exiting a loop early) are exactly the two cases cited in Kernighan and Ritchie's The C Programming Language book when it discusses the goto statement in section 3.8.

Go To Statement Considered Helpful (part 2: Nested Loops)

If you're in a nested loop in C code, and you want to issue a break or continue statement, but want it to apply to the outer loop, you're going to have some awkward code.

  for(j=0;i<j_end;++j)
  {
      for(i=0;i<end;++i)
      {
          ...
          if (condition) next_j; // break then continue?
          ...
      }
      ...
  }

Implementing that 'next_j' statement could be done with setting a flag, issuing a break, and then checking the flag after the inner loop terminates.  What we want is a "goto continue_outer_loop." And when you do this, it can be a perfect place for one of my favorite tricks, the use of "if (0)" for live code. At the end of the outer loop, put in the block of code, "if (0) { continue_outer_loop: continue; }" and you can then use your goto to implement the feature that the language is missing.  This looks like:

  for(j=0;i<j_end;++j)
  {
      for(i=0;i<end;++i)
      {
          ...
          if (condition) goto continue_outter_loop;
          ...
      }
      ...
      if(0) continue_outter_loop: continue;
  }

A nice thing to do would be to imagine how the language might be extended to eliminate this need for a goto. Loops could easily be named, allowing the name of the loop to be an optional parameter to a break or continue statement. What if instead of for(i=0;i<10;++i), you could write for(i=0;i<10;++i;index_loop). Then you could use "break index_loop;" inside an inner loop with no need for a goto. Good luck getting the language committees to agree to this anytime soon. On the bright side, Java recognized this shortcoming, so the language provides for named loops; in C and C++, goto lets you implement the missing feature.

In this case, all this can be included in macros, as shown below. The first pass at trying this can result in compiler warnings about unused labels, which is why I added the extra do-nothing gotos. With even the most minimal compiler optimizations, this should provide exactly the same execution as if the language provided for named loops natively. The only restriction is that you can't use the same name for more than one loop in the same function.


/*
 * Named loops:
 *
 * Name any loop, typically the first thing after the opening brace.
 * Then in a nested loop, break or continue from it with
 * BREAK_LOOP/CONTINUE_LOOP.
 *
 * Code Copyright 2015 by Preston Crow
 * used by permission
 * (You have permission to use this as long as you keep
 *  this comment block intact.)
 */
#define NAME_THIS_LOOP(_name)            \
if(0)                                    \
{                                        \
    goto _continue_loop_ ## _name;       \
    _continue_loop_ ## _name : continue; \
}                                        \
if(0)                                    \
{                                        \
    goto _break_loop_ ## _name;          \
    _break_loop_ ## _name : break;       \
}                                        \
do { ; } while (0) /* Force a semicolon */
#define CONTINUE_LOOP(_name) goto _continue_loop_ ## _name
#define BREAK_LOOP(_name) goto _break_loop_ ## _name


Note that the naming macro can go anywhere in the loop where a continue or break statement would work as expected. Using those macros, the code is quite readable:

  for(j=0;i<j_end;++j)
  {
      NAME_THIS_LOOP(outter_loop)
      for(i=0;i<end;++i)
      {
          ...
          if (condition) CONTINUE_LOOP(outter_loop);
          ...
      }
      ...
  }


I thought I was very clever the day I came up with that. I felt a little less clever when researching revealed that others have also suggested very similar macros for the same purpose. In any case, I highly recommend that you use these macros or the like in all C and C++ projects where appropriate.