Xautomation: visually grepping for gui elements

In my first article about testing tuxpaint with xautomation, I mentioned that I wasn’t happy with hardwiring button positions. But before I could proceed, I discovered a bug in xautomation that prevented me from using visgrep to search for the “Yes” button in tuxpaint screenshots. Fortunately, the fix was straightforward enough after reading the libpng man page.

At the same time, I also decided to borrow an idea from the example scripts in the xmacro package. Running the test script on the tester’s own display may be fine for developing the script, but it is nice to be able to run it on a virtual display after the kinks are worked out. For one thing, the test doesn’t get in the way of other things when you’re waiting for it to finish. For another, it is much easier to control the test environment. Xvfb is perfect for this.

The resulting script requires tuxpaint, Xvfb, xautomation with the above patch applied, xbase-clients and Imagemagick.

A pcsx mythgame player using xautomation

Scripting a gui tester for Tuxpaint with xautomation was fun and quite useful too, but really only appeals to developers. Today I found use for xautomation that will appeal to sys admins and users: hacking around a user interface that refuses to work the way it ought to.

I wanted to add another mythgame player, this time for the pcsx-df Playstation emulator. Unfortunately, I couldn’t pass it a CD image from the command line. It seems to know only how to do two things: run a game file, or run a real CD. My best guess is that it doesn’t know its plugins until the GUI starts, so it is unaware that it ought to be able to use the cdrmooby plugin to load the specified image when it is parsing the command line.

After a brief and fruitless grep through the source, it dawned on me that I could put xautomation to work for me to do the job in much less time. Here is the resulting startpcsx script, which sucessfully operates as my pcsx mythgame player.

#!/bin/bash
slowtype () {
   echo "$@" | sed -re 's/(.)/\"str \1\"\n\"usleep 15\"\n/g'
}

pcsx &
xte "sleep 2" "keydown Control_R" "key o" "keyup Control_R" "sleep 1"
slowtype "$@" | xargs xte
xte "key Return"

The hard part was that pcsx is afflicted by one of my pet peeves, a typeahead find that cannot be disabled. As a result, if you make xte type at full speed into the open file dialog, the filepath comes out all jumbled up. This happens because the filepath is constantly being redrawn as xte types characters, so the cursor isn’t always at the end of the line. The usleep 15 allows the cursor to reach the end of the line before typing the next character.

Of course, this is truly a hack, subject to variations in timing between systems, so if you use it, you may have to tweak the sleep and usleep values. But it works well enough as a workaround until pcsx-df is fixed to accept a CD image on the command line.