Xautomation: visually grepping for gui elements

In my first article about testing tuxpaint with xautomation, I mentioned that I wasn’t happy with hardwiring button positions. But before I could proceed, I discovered a bug in xautomation that prevented me from using visgrep to search for the “Yes” button in tuxpaint screenshots. Fortunately, the fix was straightforward enough after reading the libpng man page.

At the same time, I also decided to borrow an idea from the example scripts in the xmacro package. Running the test script on the tester’s own display may be fine for developing the script, but it is nice to be able to run it on a virtual display after the kinks are worked out. For one thing, the test doesn’t get in the way of other things when you’re waiting for it to finish. For another, it is much easier to control the test environment. Xvfb is perfect for this.

The resulting script requires tuxpaint, Xvfb, xautomation with the above patch applied, xbase-clients and Imagemagick.

Video hub: Why stream at all?

In response to Home video hub on a shoestring, Justin asked, “Why not just stream the media over cifs/nfs/sshfs?”

Why indeed? Well, as my fellow amateur Internet broadcaster Justin Zeigler (coincidentally of the same first name) observed, “some people mistake the unix way as doing things the oldest way possible.” There are niceties streaming video solutions provide that are much better suited to solving my particular problem.

Streaming puts the burden of handling different codecs and providing material at a bitrate the client systems can handle on the server. Remember, I said my client system is underpowered and the network I’m delivering across is wireless B. These constraints make streaming video the ideal solution.

Now, if I only wanted to stream the music videos and other material designed to deliver over the Internet, I’d be fine with a remote filesystem. But DVDs and mythtv recordings are too demanding on my little laptop’s resources to work at all.

But, just to make sure, I tried both sshfs and nfs today. Sure enough, I could stream a Quicktime music video from sshfs just fine. However, neither sshfs nor nfs could deliver the material from a mythtv recording fast enough for the client to keep up. I tried both vlc and mplayer, and both were so stuttery they were unplayable. I gave mplayer a larger buffer (32K) which cleaned up the first few seconds of play while the player read from the buffer, but after that, it started stuttering again. And this was with only one floor of separation between the WAP and the laptop, to make sure I had a solid connection.

Clearly, Videolan’s ability to transcode on the fly and stream over the network is a great convenience to me. I suppose I could do a file-to-file transcode and then read the resulting file from sshfs, but I don’t find that nearly as handy as the streaming solution. So I’m sticking with Videolan, as I find it the easiest to use, most elegant solution to delivering a broad range of video material from our powerful hub system to our less powerful client system.

A pcsx mythgame player using xautomation

Scripting a gui tester for Tuxpaint with xautomation was fun and quite useful too, but really only appeals to developers. Today I found use for xautomation that will appeal to sys admins and users: hacking around a user interface that refuses to work the way it ought to.

I wanted to add another mythgame player, this time for the pcsx-df Playstation emulator. Unfortunately, I couldn’t pass it a CD image from the command line. It seems to know only how to do two things: run a game file, or run a real CD. My best guess is that it doesn’t know its plugins until the GUI starts, so it is unaware that it ought to be able to use the cdrmooby plugin to load the specified image when it is parsing the command line.

After a brief and fruitless grep through the source, it dawned on me that I could put xautomation to work for me to do the job in much less time. Here is the resulting startpcsx script, which sucessfully operates as my pcsx mythgame player.

#!/bin/bash
slowtype () {
   echo "$@" | sed -re 's/(.)/\"str \1\"\n\"usleep 15\"\n/g'
}

pcsx &
xte "sleep 2" "keydown Control_R" "key o" "keyup Control_R" "sleep 1"
slowtype "$@" | xargs xte
xte "key Return"

The hard part was that pcsx is afflicted by one of my pet peeves, a typeahead find that cannot be disabled. As a result, if you make xte type at full speed into the open file dialog, the filepath comes out all jumbled up. This happens because the filepath is constantly being redrawn as xte types characters, so the cursor isn’t always at the end of the line. The usleep 15 allows the cursor to reach the end of the line before typing the next character.

Of course, this is truly a hack, subject to variations in timing between systems, so if you use it, you may have to tweak the sleep and usleep values. But it works well enough as a workaround until pcsx-df is fixed to accept a CD image on the command line.

Home video hub on a shoestring

In our family, we have centralized all our best entertainment technology in a single system, a white box amd64 with PVR, 3D graphics, 1G, a 21” monitor, DVD burner and VCR. A number of older, far less capable systems have always surrounded this “hub”. A K6-2/400 with a less-than wonderful ATI Rage IIc graphics card presently serves as my wife’s system, and my PII/300 laptop roams the house on wireless B.

This has always been the model for our home network, dating back to the late 90’s when the hub in our cramped apartment was a P/100 delivering sound through a $30 pawn-shop stereo to two main rooms, and the satellite node in the family room was a DEC VT-420 on a long serial cable. So, house-wide delivery of sound has always been possible. But doing the same for video remained unsolved until just this weekend.

Kudos to the Videolan team for making it possible. A number of years ago, I recognized their vlc player was ideal for playing DVDs on my K6-2, handling the job with less CPU and less frame drop than other alternatives at the time. Not too long afterwards, though, we upgraded our hardware so saving CPU cycles was no longer an issue, and switched to other players to take advantage of features they offered over vlc.

But a recent email from a friend brought me back. This friend asked if I had seen a certain Linux documentary video he had found. I hadn’t, and was intrigued. But since the hub was tied up by the kids at that moment, I figured I’d try watching on the old PII laptop. I didn’t really think it would work, but remembering how well vlc did on my other old hardware, I gave it a try. It worked! No frame drop at all.

Buoyed by that victory, I recalled that I had downloaded the Debconf 6 DVD ISOs some days ago, but had not yet been able to watch them. You probably can guess why … yup, the single hub system, the only one really capable of doing a good job displaying video, was pretty much always tied up by various other family members.

Now, I knew there was no way I could get those DVDs onto the laptop, as it has no drive and only a 4G hdd. Nor was I willing to go to the bother of ripping individual tracks to transcode and send over one at a time. But I then recalled that vlc, as well as being an excellent standalone player, is primarily the client for a streaming video system. So I set out to learn how to stream video.

Back when I was first introduced to Videolan, I knew there was a server program, vls. I had a quick look at that and was overwhelmed by the wide variety of possible MRLs and codecs, and a bit mystified as to which magic spells were needed to make it stream to the laptop.

It then occurred to me I had probably plunged in too quickly, and should back off and go look for more docs than just the man pages. Sure enough, they have plenty of excellent docs, leading to the discovery that the vlc gui itself has easy to use Wizard and Open dialogs to set up streaming. With that sorted out, I was streaming in no time.

At this point, I was delighted to discover that I could successfully stream a 1024 kbps mp4 stream across wireless B form the basement to any other room of our two-story semi. And ever since then, I’ve been having great fun with the family, trying a wide variety of other video sources lying around: music videos, home videos made by friends, TV shows recorded from the PVR, DVDs direct from disc, and yes, finally, those Debconf 6 ISOs.

I tell you, it’s like Christmas come early! Thanks, Videolan. You’ve made my week.

Towards a Debian Jr. Live CD

Debian Jr. development revived

Recently there has been some lively discussion on the Debian Jr. list about how to arrange an account for a child of 1 to 3 years of age. Suggestions included using set-top box software like Freevo, tailoring DEs with panels and large buttons, using simpler WMs like fvwm, or using an “activity centre” app like Gcompris.

Now, I respect those parents who hold that a child this young should interact more with the “real world”. While I wouldn’t go so far as to outright prohibit my young ones from computer use, I can see the wisdom in keeping it to a minimum. But, for better or worse, our family is one of several who have some experience helping our youngest members use our Debian systems. We would like to share what we’ve learned through the Debian Jr. project.

Getting started with live.debian.net

To that end, last week I was inspired to follow the Debian Live ISO Howto to produce my first rough draft of a Debian Jr. live CD built from scratch on Etch.

For the final product, we’ll want to use a local partial Debian archive mirror, as it not only optimizes fetching packages for several build iterations, but also can be kept stable, which is important as we near release. But for this draft, I ran into trouble building the complete package list to populate the partial mirror, as cdebootstrap, which make-live uses, has no handy --print-debs switch like debootstrap does. So for now, I rely on approx to cache packages for optimization only.

A straightforward process to create a working live CD

The rest of the process was straightforward: using live-package, I created a package list1 containing Gnome and the Debian Jr. metapackages, I configured /etc/make-live.conf to point at my proxy, and then I ran the make-live script on the package list. The end result was ./debian-live/binary.iso, which successfully booted under qemu.

At this stage, there is not much to show. To be truly useful, the live CD needs to be set up so that children and their guides can immediately find and use the material intended for them. We will need not only one live CD user, but four, varying in age range and role. (More about this in a future article.) However, it is an encouraging start, and shows that we may be able to produce something usable by the time Etch releases.

1 For this exercise, I simply combined /usr/share/make-live/lists/gnome with the junior-* metapackages in Etch, and added mozilla-firefox-gnome-support to satisfy Gnome’s web browser dependency and cut down on redundancy, resulting in the following list.

junior-art
junior-doc
junior-games-card
junior-games-gl
junior-games-net
junior-games-sim
junior-games-text
junior-gnome
junior-internet
junior-kde
junior-math
junior-programming
junior-puzzle
junior-sound
junior-system
junior-toys
junior-typing
junior-writing
mozilla-firefox-gnome-support
eject sudo
console-common locales
gdm gdm-themes gnome-desktop-environment gnome-cups-manager gnome-screensaver
gnome-themes-extras
rhythmbox synaptic
x-window-system-core

Mythgame player to run native Linux games

I recently started to be aware of a problem in our family. Ever since we configured Mythgame to run Xmame games, we started to ignore native Linux games. This is a shame, because often the quality of such games is much higher, and besides, they are open source, which I strongly believe in.

I looked around for help configuring Mythgame to run Linux games, and was unable to find a way to make a game player that would play any Linux game. So I devised the following solution:

Step 1: Set up directories for pseudo-roms, command switches and snapshots


mkdir /usr/share/games/roms
mkdir /usr/share/games/switches
mkdir /usr/share/games/snaps

Step 2: Define a player in Mythgame


Player name: Linux
Type: OTHER
Command: %s `cat \`dirname %s\`/../switches/\`basename %s\``
Rom Path: /usr/share/games/roms
ScreenShots: /usr/share/games/snaps

Step 3: Populate the directories with data


cd /usr/share/games/roms
ln -s /usr/games/oolite
ln -s /usr/games/starvoyager
cd /usr/share/games/snaps
cp ~synrg/snaps/oolite.png .
cp ~synrg/snaps/starvoyager.png .
cd /usr/share/games/switches
touch oolite
echo "-f" >starvoyager

Step 4: In Mythgame, “Scan for Games”

Step 5: Play!

Tuxpaint GUI testing with xautomation

Tuxpaint could be a poster child for l10n. It has translations now for 68 different locales. The large number of locales poses somewhat of a problem for testing1.

Tuxpaint 0.9.16 just released. During the release preparation, the manual testing of all of these locales drove me nearly insane. So to help out next time, I created the following bash script using xautomation2, imagemagick and xprop (from xbase-clients):


#!/bin/bash
testname="tuxpaint"
root_geometry=`xprop -root _NET_DESKTOP_GEOMETRY |cut -d= -f2`
root_x=`echo $root_geometry |cut -d, -f1`
root_y=`echo $root_geometry |cut -d, -f2`
xc=$((root_x/2-1))
yc=$((root_y/2-1))

capture_tuxpaint () {
   tuxpaint --startblank --nolockfile --lang $1 --640x480 --windowed --nosound &
   sleep 1
   xte "mousemove $xc $yc"
   xte "mouseclick 1"
   sleep 1
   xte "key Escape"
   sleep 2
   import -silent -window "Tux Paint" $testname-$1.png
   xte "mousermove -120 -40"
   xte "mouseclick 1"
}

for lang in `tuxpaint --lang help |awk '$1~/^[[:lower:]]/ {print $1}'`; do
   capture_tuxpaint $lang
done

I’m not entirely happy with it, but it’s at least a starting point. Ideally, I’d make .pat files to use with visgrep to find coordinates for the button images instead of hardwiring the approximate button positions. Also, the script needs a companion script to configure the system for testing (i.e. perform the locales configuration and install all fonts).

Here are the resulting screenshots which show up a few problems with the translations. It looks like the quit dialog strings have been updated since they were first translated, so now we need retranslation for several languages. Also, there are some languages, like Swahili and Klingon, for which there are no locales in glibc in Debian. I’ll have to talk to the glibc maintainers about that. And for that matter, I don’t even know if a DFSG-free Klingon font exists.

In any event, I’m delighted with the results, and look forward to using xautomation more for GUI testing.

1 For one thing, several languages require special fonts. For another, there is no way to get tuxpaint to tell you what locales it supports. Granted, there is --lang help, but this is a crude approximation, as the mapping between language and locale is arbitrarily hardwired (which hmh on #debian-devel irc pointed out to me is just plain wrong). It doesn’t help me at all in the arduous task of configuring glibc locales so I can test them. For that, I need to look at i18n.c.

2 I tried using xmacro at first, but xmacrorec just made my pointer jiggle back and forth. It seemed like it was updating the position for a short distance, and then X was resetting it back to the former position of the pointer. Some strange interaction with my window manager, perhaps? But ultimately I am happier with the xautomation approach anyway.

Adequate mobile text editing device

Ideal mobile text editing device redux

As I read the responses1 to my search for the ideal mobile text editing device, reality began to set in. The heating season is now upon us, and even used technology meeting my criteria would likely exceed my stated budget. Furthermore, I’d risk replacing my current issues with a whole new set of issues and end up with a less-than-ideal mobile text editing device.

So I have rethought the problem and have decided it would be best to work on with what I have now while I wait for this ideal device to arrive on the market.

Keyboard

While I can’t change the keyboard of the ZR-5000, with a practiced lighter touch on the keys I can, in fact, touch-type passably well. It has served me well enough to draft several blog articles so far.

Software

Although not up to the high standard set by Vim, for quick note-taking, I don’t really need a fancy editor. My thoughts flow directly through my fingers into the device with only light editing. Cut-and-paste, motion keys and delete are the bare essentials. Anything more is a frill.

Synchronization

I do have an ACTiSYS IR-200L infrared dongle for my PC. The trouble is, although this is supported in Linux using IrDA, the ZR-5000 insists on speaking ASK-IR which is not supported.

There is a Windows-only program, ZRLink, which might work. I’ll try it either on an XP system at work, (… mixed reviews out there as to whether it will work on recent Windows,) or under Wine. I can get the dialogs to display properly on Wine, but I have my doubts about whether serial emulation works. My initial tests at home under Wine indicate it doesn’t. (Could #277618 have something to do with this?)

There is a 15-pin serial connector on the device, but I have no cable for it. And there’s a PCMCIA slot, but compatible devices are likely limited to older, harder-to-find components: either a 2M CF card or a modem card. While I have the latter, it exceeds the power available from batteries and I have no DC adaptor for the unit.

Perhaps I overrate synchronization anyway. My best articles are rewritten from scratch more than once.

Summary

Living in an embarrassingly technology-rich country, it is too easy for a geek like me to convince himself that his gadget cravings are needs whereas, in fact, there is still plenty of life left in the devices he already has. After all, my venerable ZR-5000 was given to me by a fellow who figured if any one of his friends would still be able to use it for something, it would be me. Well, I won’t let him down. I’ll stick with it and make it work.

There is one virtue of this device I doubt if any modern device of the same form factor can match. After a week of daily use, I still haven’t had to change the 2 ordinary AA NiMH batteries. Top that.

1 Thanks, everyone, for your responses. Although there were the inevitable few that mentioned modern devices way over my budget, and I’ve decided now to stick with the ZR-5000, your suggestions did help me bring the real issues into focus.

Apt verb

In the English language, it is so trivial to create new words. In no domain is this quite so evident as the world of computers.

Thus, informally amongst Debian users, you’ll often hear “apt-get it” or simply “apt it” which has the virtue of being equally apt a term for using aptitude. Now, I have no problem with people using these verbs. They’re precise, if a bit geeky. But really, shouldn’t there be a more general verb to cover this activity?

Back when apt was new, and nobody else was installing packages this way, the lack of a general term made sense. But these days, in certain user communities, you might hear “yum it” instead. How many variants does the English language need? One per apt-like installer?

Compare this with the evolution of instant messaging. Back when ICQ dominated the field, “ICQ” was synonymous with instant messaging. These days, although you might occasionally still hear people say “ICQ me” or “Jabber me” (where the protocol is important, because both participants must have access to that specific service) we can more generally say “message me,” encompassing in one fell swoop all possible protocols. Or if you prefer, the awkwardly geeky but more precise term, “IM me” (though arguably slightly less useful than “message me” because it seems to exclude irc, the king of all chat systems).

So, what’s the equivalent verb that means “conveniently installing software from one or more software repositories by merely asking for the package by name”?

My mind draws a blank.

Ideal mobile text-editing device

Dear Lazyweb,

I’m looking for a small, (easily fits on my lap on the cramped seat of a bus,) inexpensive, (around $100 CAD,) mobile text-editing device with a 90%-sized keyboard and decent battery life that runs an open source OS and Vim, and that I can sync to my desktop system.

My first mobile device was a NEC PC-8201A, circa 1984. It ran on 4 AA NiCad batteries which I hardly needed to change more than once a week. At the time, it was my ideal mobile notetaking device. Its near-full-sized keyboard, Wordstar-compatible text editor, terminal program and serial interface allowed me to take notes which I would sync to my father’s Mac at home or to the VAX at the university.

Today, my mobile computing needs have hardly changed: text editing is almost all I do. I don’t ask for much of a display, but as an 80 to 90 wpm touch-typist, I won’t settle for anything less than a near-full-sized keyboard. I’d prefer a system that runs Debian, and Vim is a must. Some means of easily synchronizing the device with my home system is necessary: PCMCIA wireless-B would be ideal, but is not a must-have.

At the moment, I’m using an old Zaurus (not Linux-compatible) ZR-5000 which has good battery life but a very cramped keyboard, poor software, and no functional means of synchronizing the device to my PC. OK for jotting down quick notes, but frankly, I’d rather have my PC-8201A back again. I could at least type on it and sync it across the serial cable.

I took a brief look at various Psion models, but they appear to be near-impossible to find. The HP Jornada 720 looks intriguing. And the NEC Mobilepro 780
appears to be another possibility. Have I overlooked anything?

So, suggestions please.