Wifi roaming on the move redux

It has been nearly six years with a netbook and five since I last wrote about wifi roaming from the bus to stay on irc without a costly celluar link during the daily commute.  Since then, some readers have asked me to share my refinements to the method in a followup post. So here it is.

The software

On the server:

  • openssh-server
  • screen
  • irssi

On the client:

  • screen
  • wpasupplicant
  • isc-dhcp-client
  • openssh-client
  • openbox
  • sudo & gksudo (optional)
  • urxvt
  • wavemon (optional)
  • three shell scripts (provided below)

Putting it together: on the client

Make sure if you have a wireless manager installed (such as NetworkManager) it is configured to skip your wireless interface, disabled entirely, or if possible, removed. Set up /etc/wpa_supplicant/wpa_supplicant.conf and /etc/network/interfaces for roaming, as per the instructions in /usr/share/doc/wpasupplicant/README.modes.gz. Don’t forget to add yourself to the netdev group if you are not in it already.

In /etc/wpa_supplicant/wpa_supplicant.conf, list common names of open networks. Normally the catch-all network that associates with any essid, i.e. the first stanza below, works well. However, occasionally the strongest signal is neither one of the common networks nor an easily accessible network (e.g. web portals), so having a list of common open networks helps to quickly select from among those instead. The more you travel, the more of these will discover and add. Just use reconfigure from wpa_cli to reload your edited list each time you add a new one.

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
        key_mgmt=NONE
}
network={
        ssid="default"
        key_mgmt=NONE
}
network={
        ssid="linksys"
        key_mgmt=NONE
}
...

Since you’ll be using ssh repeatedly to connect and it has to be fast, make sure your server is set up to accept your key and use ssh-add so that you only have to enter your ssh key password once.

You can tweak isc-dhcp-client to make connections faster. In /etc/dhcp/dhclient.conf, use:

backoff-cutoff 1;
initial-interval 1;

Here are a few scripts I wrote to facilitate quick roaming from one open AP to another and reconnect to irssi running in screen, to break a connection and try the next one, and to recover from occasional lockups (more about that later).

~/bin/screen_reconnect

This is a script to reconnect continuously via ssh to a screen session:

#!/bin/sh
reset
while ! ssh -t 10.9.8.7 'screen -UDr' 2>/dev/null ; do echo -n "." ; sleep .1 ; done

Just substitute the IP of your own server here. Using an IP instead of domain name makes the connection faster because a DNS lookup is not required.

~/bin/wifi_reassociate

This script closes any open ssh sessions and informs wpa_supplicant to attempt to connect again.

#/bin/sh
/sbin/wpa_cli rea
killall ssh >/dev/null 2>&1

~/bin/wifi_killall

This optional, somewhat ugly script addresses an issue I hope you never have. On my ASUS Eee PC 1001PX, occasionally scanning stops. When this happens, and I have never figured out why, apparently ACPI events are blocked. At this point wifi becomes unusable and ACPI sleep is inhibited. By trial and error I found that if you bring down the interface, kill all network-related processes, and bring it up again, ACPI events are unblocked and wifi is usable once more (and any pending request to sleep will finally happen). The script requires sudo, and to use the openbox key binding, gksudo.

#/bin/sh
sudo ifdown wlan0
# in case any of these are hung
sudo killall dhclient3
sudo killall wpa_cli
sudo killall wpa_action
sudo killall wpa_supplicant
# in case any of these are *really* hung
sleep 1
sudo killall -9 dhclient3
sudo killall -9 wpa_cli
sudo killall -9 wpa_action
sudo killall -9 wpa_supplicant
sudo dhclient -r
sudo ifup wlan0

Openbox

Since certain actions need to be performed repeatedly and quickly, it is useful to have hotkeys bound in your window manager to the scripts. In ~/.config/openbox/rc.xml, key bindings for <alt>-r to reassociate and <alt>-d to disconnect a hung connection would look like:

  
<keyboard>
  <!-- My keybindings -->
  <keybind key="A-R">
    <action name="Execute">
        <execute>/home/synrg/bin/wifi_reassociate</execute>
    </action>
  </keybind>
  <keybind key="A-D">
    <action name="Execute">
        <execute>gksudo /home/synrg/bin/wifi_killall</execute>
    </action>
  </keybind>
</keyboard>

Putting it together: on the server

There is very little to do here. Just start screen, and start irssi in screen. Running screen on the client as well as the server means you should either bind the screen meta keys to a different key sequence on each system, or else learn to press meta twice to pass through meta to the server screen as needed. I use the latter approach. Alternatively, you could use a tabbed terminal on the client, or separate terminals per client process instead of screen. This is a matter of personal taste.

Ready to roam

Here is a typical setup for roaming on the bus:

In a terminal (I use urxvt), first ssh-add, then start screen with these three processes running in separate virtual terminals:

  • /sbin/wpa_cli
  • screen_reconnect
  • wavemon (optional)

March of the dots

Most of the commute, just enjoy watching the dots march by, waiting for a new connection. If you estimate a connection is unusable, press <alt>-r to reassociate immediately, giving the next network a chance. If the connection is already firmly established, this might not work on the first try. If the dots don’t resume immediately, wait a bit and press it again. This might take a few tries.

Changing selected networks on the fly

Use wpa_cli when you need to do some fine-tuning of network selections on the fly. While normally you can just watch the march of the dots until a connection is acquired, sometimes you can improve your chances of connecting to a good network by manually controlling the selected candidate networks here.

For example, by watching the speed of the bus relative to known “good” APs, you can predict which networks are more likely to succeed. Rather than connect to any arbitrary network, you might select a specific one by id, and then later when it goes out of range, revert to the original configuration, e.g.

> select_network 5
...
> reconfigure

You can use tab-completion in wpa_cli to type these commands quickly or else just abbreviate the commands.

Another common scenario is when you pass through a business area with many captive portal hotspots. These rarely make good choices because they either require a password not known to you or else you can’t click through “I agree” in time before the bus moves on. In this case, you might just disable the catch-all stanza and let the common open network stanzas you listed (“default”, “linksys”, etc.) do the work:

> disable_network 1

Become a type ahead wizard

While running, a continuous stream of periods fills the screen, which provides you with a highly visible cue that no available APs are in range. When the movement stops, you know a connection is being attempted.

While waiting to connect, you can type ahead any comments you want to make in the current irssi window (taking care to remember which one you are in!) While having periods interspersed in what you type may be disorienting at first, you get used to it.

There is a point when a connection is first established and ssh is accepting input, but anything you type can no longer be seen while you’re typing. Depending on whether the connection was completely successful or not, what you type now may or may not finally be sent. For best results, only type ahead before the dots stop moving.

Eventually you can become skilled enough at this to type ahead a comment in one channel, switch channels with /win # and continue typing ahead in the new channel, all buffered until the next few seconds (or even fraction of a second) of connection time.

Fine-tune antenna direction with wavemon

When the bus has come to a standstill, you may find wavemon useful to pull in a weak signal. Because wavemon has continuously updated signal level and link quality bars, you can use it to fine-tune the antenna position. Just turn your laptop until the bars are at their maximum.

Captive portals

I have not figured out how to do any automation for this, so it really is a crapshoot, as it is likely the bus has moved on by the time you’ve managed to manually navigate the login through a captive portal. But in rush hour, you may have the luxury of time to connect to these as you pass them. I have recently learned about the CoovaFX Firefox plugin which automates logins to captive portals. I’m going to give it a try to see if it helps. Update: I can’t recommend this plugin, as it is not compatible with Iceweasel >= 23.0. Also, the standard it is based on, WISPr, appears to have an uncertain future. That, coupled with the fact that the plugin appears to not be open source means I’m still looking for alternatives.

Summary

If all of this sounds a bit nuts to you, well, it probably is. But after half a decade enjoying free access to irc from the bus, it all seems perfectly natural to me! If you try this method and like it, please let me know in the comments. Likewise, if you have any improvements to the process or scripts, please share them!

Taskwarrior new blog, getting involved

The Taskwarrior Team has just started a blog, kicking it off with a series of articles about development of Taskwarrior itself. Go, team!

Almost from the moment I started using Taskwarrior (thanks to Jakub WIlk for an excellent job maintaining this) I knew I had finally found the todo system that I could love. Right away, I started hanging out with the Taskwarrior community at #taskwarrior @ irc.freenode.net and found out what an awesome bunch of people they are, both developers and users alike. I have plunged in with bug reports and feature requests, and am helping get more Taskwarrior-related things into Debian. In NEW right now I have uploaded several of the dependencies needed by my ITP of taskwarrior-web. It’s looking like I’ll have that finished later this month or early next month. Also, I have uploaded a wheezy backport of Taskwarrior itself (aka ‘task’) which, if all goes well, enters the archive next week.

Tutorial: wheezy live iso-hybrid with persistence on USB

As of wheezy, Debian no longer offers for download any prebuilt hdd (formerly known as usb-hdd) live images, which, in squeeze, were necessary if you wanted to use persistence. The good news is, since the new wheezy iso-hybrid images use xorisso, you can copy an image to a USB key and repartition it to add a persistence partition. Unfortunately, the filesystem itself is read-only, so you have to manually append the persistence boot parameter each time you boot the image. If that doesn’t bother you, you can skip this whole tutorial and just follow the instructions in live-manual for adding a persistence partition and you’re all set.

This tutorial explains how to extract the contents of a wheezy iso-hybrid live image to a USB key and modify it to boot with this parameter permanently enabled. I am writing it solely as a workaround and do not endorse this as a means to customize Debian Live images in general. I would love to help make a more convenient, supported way to accomplish the same thing for a future release. Meanwhile, the supported and recommended way to make any change to a Debian Live image is to build it from scratch using live-build, as described in live-manual. A simplified Debian Live Images Autobuilder web service is available that works well for many common customizations, if you prefer. Please use one of these methods if you want to do this, or other customizations in the supported way. Otherwise, read on.

Prerequisites

  • You need to download a Debian iso-hybrid live image. For this tutorial, I will use the Debian live wheezy amd64 LXDE image:
  • Note the size of the image. With ls -lh debian-live-7.0.0-amd64-lxde-desktop.iso you can see the image size for this example is 854M. Later, you will use this size plus 5% for the size of the partition to receive its contents.
  • You need a USB key large enough for both the image itself and your persistence partition. For this image, you could even use a 1G key, although that would leave very little space for persistence. In this tutorial, the test key is 8G, leaving lots of room on the key for growth.
  • I assume for this example you are preparing the key on a Debian or other Linux system that automounts media when they are plugged in, a fairly normal configuration.
  • This tutorial also uses the following tools, which you will need to install if you don’t have them already:
    • parted
    • mbr
    • dosfstools
    • p7zip
    • syslinux

Identify the USB key device

Caution: Always double-check the device you are writing to is the correct one to avoid losing precious data. There are two things to do to protect yourself. The first is, don’t write the USB key as root. The second is, first use ls -l /dev/disk/by-id to identify which device is the target USB key.

lrwxrwxrwx 1 root root  9 Jun 22 11:58 
usb-SanDisk_Cruzer_Contour_0000184CA87406BC-0:0 -> ../../sdb

As you can see, my SanDisk Cruzer Countour is /dev/sdb. For the rest of the tutorial, I’ll be referring to this device as /dev/sdX to avoid unfortunate cut-and-paste disasters. When you follow along, make sure you substitute the correct device for your USB key on your system.

Initialize the USB key

Start by plugging in the key and unmounting any automounted partitions to ensure nothing is accessing the device, e.g.

$ umount /dev/sdX1

Now make changes to the partition table. Use parted because it may be run as an ordinary user and has some advanced capabilities beyond more basic tools like fdisk.

$ /sbin/parted
WARNING: You are not superuser.  Watch out for permissions.
GNU Parted 2.3
Using /dev/sdX
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print devices
/dev/sdX (8221MB)
(parted) select                                                           
New device?  [/dev/sdX]?                                                  
Using /dev/sdX
(parted)

Note that when run as an ordinary user, if you only have one removable drive (which may include mp3 players, cameras, e-readers) plugged in, parted immediately selects that device. But just to be sure, use print devices to list the devices and select to if you need to select a different one, as shown above.

Repartition the USB key

Once you’ve verified you have the correct device, start repartitioning. Write a new partition table with mklabel msdos to wipe out all of the old partitions on it. You may skip this step and adapt the remaining instructions if there are partitions you need to keep. This is your last chance to bail if the device is wrong, so make sure it is the correct one before proceeding.

(parted) mklabel msdos
Warning: The existing disk label on /dev/sdX will be destroyed and all data
on this disk will be lost. Do you want to continue?
Yes/No? y
Error: Partition(s) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 on
/dev/sdX have been written, but we have been unable to inform the kernel of
the change, probably because it/they are in use.  As a result, the old
partition(s) will remain in use.  You should reboot now before making further
changes.
Ignore/Cancel? i
(parted)

The error is normal, and appears when you run parted as an ordinary user, so it is OK each time it appears to press “i” to ignore it. For the rest of this tutorial I will omit these errors from the output for brevity.

Make three partitions, a fat32 partition for the image itself that is 5% larger than the size of the ISO, a 1G ext4 partition for persistence, and an extra fat32 partition for the rest of the space. Put the extra partition first so that other OSes you might use the key with will be able to use it. Use relative positioning from the end of the device to make it easy to put all of the rest of the space in the first partition. Finally, flag the live image partition bootable.

(parted) mkpart primary fat32 1 -1897M
(parted) mkpart primary ext4 -1897M -897M
(parted) mkpart primary fat32 -897M -0
(parted) set 3 boot on
(parted) quit
Information: You may need to update /etc/fstab.

Unplug the USB key and plug it in again so that the kernel will re-read the updated partition table. If there were old filesystems on the key that were automounted, unmount them again. Now we’re ready to make the new filesystems.

$ umount /dev/sdX1
$ /sbin/mkdosfs -nEXTRA /dev/sdX1
mkdosfs 3.0.16 (01 Mar 2013)
$ /sbin/mkfs.ext4 -q -Lpersistence /dev/sdX2
$ /sbin/mkdosfs -nLXDE /dev/sdX3
mkdosfs 3.0.16 (01 Mar 2013)

Make the USB key bootable

Use install-mbr to install an MBR on the key and syslinux to install the bootloader to boot into your live image partition.

$ /sbin/install-mbr /dev/sdX
$ syslinux -i /dev/sdX3

Mount the partitions

Unplug the key and plug it in again so the new partitions will be automounted.

Observe where the partitions are mounted with df.

$ df
...
/dev/sdX1        6204568         4   6204564   1% /media/EXTRA
/dev/sdX3         831888         4    831884   1% /media/LXDE
/dev/sdX2         944120      1204    894124   1% /media/persistence

Extract the ISO contents to the key

To continue preparing this key as an unprivileged user, use p7zip to extract the ISO. You may alternatively mount the ISO loopback and extract it that way, but you must be root to do that.

$ cd /media/LXDE
$ 7z x ~/debian-live-7.0.0-amd64-lxde-desktop.iso
...
Extracting  live/filesystem.packages-remove
...

When live/filesystem.squashfs is extracted (right after live/filesystem.packages-remove) it will take the longest time of all of the files to extract, as it contains the whole live filesystem. Be patient and eventually it and all other files on the image will finish extracting.

...
Everything is Ok

Folders: 245
Files: 370
Size:       892183367
Compressed: 895483904

Modify the bootloader configuration files

The syslinux bootloader configuration directory and files within it are named isolinux when installed on an ISO. You need to rename the directory and two files changing isolinux to syslinux so the bootloader will find them on your fat32 live image partition.

$ mv isolinux syslinux
$ mv syslinux/isolinux.cfg syslinux/syslinux.cfg
$ mv syslinux/isolinux.bin syslinux/syslinux.bin

Enable full persistence

Next, append “ persistence” to the live boot parameters and turn on full persistence by putting “/ union” in a persistence.conf file in the persistence partition.

$ sed -i 's/\(append boot=.*\)$/\1 persistence/' syslinux/live.cfg
$ cd /media/persistence
$ echo / union > persistence.conf

Reboot into the live system

Now your USB key is ready to boot with full persistence enabled. Have fun!

Debugging the installer on Debian Live

I’ve been using this process for debugging the installer while launched from a Debian Live desktop. Having a full live environment at your disposal during install is a boon to debugging because you don’t have to work within the constraints normally imposed upon you by the limited debian-installer environment. Caveat: running debian-installer from the debian-installer-launcher still has one or two unresolved issues, so unless you’re specifically working on live integration issues like I am, compare with a conventional d-i image install to gain the added confidence the install is the same in both environments.

There are no longer active maintainers working on debian-installer-launcher and live-installer, and I think it’s really important for wheezy to release with solid support for live installs, so I’ve been using this setup to try to solve #702335. So far, progress is slow because I’m new to this. If anyone is interested in collaborating in this work, give me a shout. I am SynrG @ irc.oftc.net and can be found on #debian-live and #debian-boot if you would like to drop in.

Overview:

  1. Use images/lxde-desktop config from live-images.git and build your own image with live-build >=3.0.1-1. Just add “-b hdd” in auto/config before starting your build, as having a read/writable image is handy for this exercise.
  2. Boot the live medium. You may prefer to do this in a VM for convenience, having prepared a blank virtual disk image as your target in advance.
  3. Make any small changes you want to the installer before launching it.  A quick hack I’ve used is to “sudo vi /usr/sbin/debian-installer-launcher” and near the end on the line after “prepare” right before “run”, open a new line and add “bash” so an interactive shell will be opened after the installer is extracted but before it is launched. For larger changes (e.g. inclusion of updated udebs, etc.) add them to your live image configuration before building as described in live-manual.
  4. Open two root terminals and do “debian-installer-launcher -t debug” in one and “tail -f /lib/live/installer/var/log/syslog” in the other. If you’ve added the hack from step 3, don’t forget to “exit” the interactive shell after making changes. Now you can watch debug output as you go through the steps of the installer. If you like, you may append additional boot prompt parameters (e.g. preseeds) to the debian-installer-launcher command.
  5. During the install, you may modify other parts of the unpacked installer under /lib/live/installer (e.g. to add ‘set -x’ to some scripts) prior to executing steps that would call them.