Scripting ssh passwords

One of the most powerful communications tools available is ssh.  Pretty much the only version on Linux is OpenSSH, and most of the versions I've come across on other platforms are derived from it.  I assume you know that, and I assume you also know that the best way to use it is with pre-shared keys so that you don't have to worry about passwords.  Unfortunately, that isn't always an option.

Recently I was working on a script that needed to use ssh to connect to an embedded system.  There are no keys on the remote system, so you have to use a password.  You can look the password up in a database.  Asking the user to do that and type it in manually would be a pain, and in this case, would add no security.

Openssh does everything it can to make scripting passwords difficult, which is only reasonable from a security standpoint.  But for anyone who has been around Unix systems for a while knows, there's a program called "expect" that solves the problem.  Expect allocates a pseudo TTY device, spawns a target program, and controls it through the TTY.  It can watch for prompts and issue responses, and it can eventually return control to the parent TTY, allowing a user to interact manually.

So I set up a script using expect to enter the password for ssh, and all was good.  I invited everyone else in my department to use the same script.  Most people loved it.  Then I started getting complaints.  It seems that while I would expect expect to be installed everywhere, that expectation was flawed.  I could ask everyone to install expect (and I did), but it seems that everyone is managing their own Linux systems, and many developers don't really know much about doing so.

I needed a better solution.

I found a better solution.

Ssh has a feature where it can run a GUI program to ask for a password.  That is exactly where we'll get our scripted password inserted.  This requires two things:  Set the environment variable SSH_ASKPASS to an executable that will write the password to stdout, and have the ssh process not be connected to a tty.

The first part is easy.  Just create a script that echos the password:

   echo echo $PW > /tmp/pw.$$
   chmod 700 /tmp/pw.$$

The second part is a little more tricky.  Also, it means that we can't interact with ssh once it connects.  Well, maybe we could with some additional trickery, but fortunately, my use of ssh didn't require interacting with it once the connection was established--I was just forwarding ports.  The obvious solution of redirecting stdin from /dev/null doesn't work, and neither does outright closing stdin.  The tty is part of a process state independent of the file descriptors, so we have to actually detach it.  What we want here is a program called 'setsid.'  This is part of the util-linux package, and it seems to be on every system I've been able to find, including the ones that didn't have expect.

Now there's one problem with setsid.  It immediately runs the program in the background.  I wanted to check the exit status of ssh to verify that everything was good.  I thought I was in luck, seeing that there's an option to do just this:  --wait.  Then I found that most of the other developers have older systems from before this option was added (in 2013, apparently).  This required creating another temporary script and a temporary results file, with the parent script waiting in a loop for it to finish.  So in /tmp/dossh.$$, I put the ssh command, followed by saving of the status ($?) in /tmp/dossh.$$.result.  And to keep everything clean, the last line of the script removes the script itself from /tmp.

So the parent script runs 'setsid /tmp/dossh.$$' then sits in a loop:

   while ! [ -f /tmp/dossh.$$.result]; do sleep .1; done

Then the parent script can grab the result, remove the remaining temporary files, and make use of the ssh tunnel. No manual password entry required.

No expect required!  (And that's a good thing.  Expect is a pain to use, in large part due to TCL being a painful language, but also due to issues where subtle changes in software versions can break everything, such as if a prompt changes slightly.)

Everything works exactly as the user would expect it to.

Why loopback is 127.0.0.0/8

If you've played with networking at all, you should be familiar with 127.0.0.1 being localhost.  If you look closely at the network settings, you'll see that this is on a loopback device, and it's configured as a /8 network, not just that one IP address.  That means any IP address in the range of 127.0.0.0 through 127.255.255.255 goes to the loopback device.  Typically your hosts file only uses one of those addresses, namely the aforementioned localhost.

So what's the point of having 24-bits of local IP addressing?  Isn't that just a waste?  Not entirely.  It turns out there is at least one very clever trick you can use this for.

Suppose you have a service running that uses a given port, say port 2159.  Now suppose you have that service running on several systems, and you want to use that service locally.  Now suppose there's a firewall on those boxes that blocks remote access to port 2159.  The obvious solution is to use ssh to forward that port back to the local system.  But if you have the same service in use locally, you have to use a different port number.  Or if you want to work with more than one remote system at a time, you have to use a different port.  This may break your client software that also insists on using port 2159.  You can fix this by inserting iptables rules that direct anything going to port 2159 at the remote IP address to instead go to whatever port number you picked locally.  That works great, except you have to have root access to set iptables rules, and you need to have your kernel configured with the right iptables options, and you have to clean up all the entries when you close your ssh tunnel (which gets tricky someone just kills the process).  This is not a solution that you can deploy as a script for lots of people to use.

Fortunately, you can use port 2159 locally.  The trick is to bind it to a different local IP address.  Since we have millions to choose from, not just one, this is easy.  Say we wanted to use 10.4.5.6:2159.  We set up a ssh tunnel to make that port appear locally on 127.4.5.6:2159.  We can do the same for lots of other systems at the same time.  No need for root access or any other magic hacks (unless the port number is below 1024).

So to implement the above example:

   ssh -L 127.4.5.6:2159:127.0.0.1:2159 -o ExitOnForwardFailure=yes \

       -f -N user@10.4.5.6

Now we just tell our program to connect to 127.4.5.6, and it works just like 10.4.5.6 would, only without being blocked by the firewall.

Interestingly, the designers of IPv6 apparently thought using a /8 for loopback was a waste, so you only get the one IP address there.  I haven't worked much with IPv6, though, so there be similar tricks available.  In any case, you can still use the local IPv4 space even if the real network is IPv6-only.

Oh, and in case you're curious, 2159 is the remote GDB debugging port, so I figured it was a good example.  I've been using this trick at work with a bunch of similar ports for debugging, much of it using internally developed software that doesn't provide an option for using an alternative port number.  While you can make a good argument that all software should allow alternative port numbers (and I would tend to agree), reality is that this is not the case, and this one way of dealing with that.