Ansible As Scripting Language

Ansible is billed as a configuration manager similar to Puppet or cfengine. But it occurred to me recently that it’s really (at least) two things:

  1. A configuration manager.
  2. A scripting language for the machine room.

Mode 1 is the normal, expected one: here’s a description; now make the machine(s) look like the description. Same as Puppet.

Mode 2 is, I think, far more difficult to achieve in Puppet than it is in Ansible. This is where you make things happen in a particular order, not just on one machine (you’d use /bin/sh for that), but on multiple hosts.

For instance, adding a new user might involve:

  1. Generate a random password on localhost.
  2. Add a user on the Active Directory server.
  3. Create and populate a home directory on the home directory server.
  4. Add a stub web page on the web server.

This is something I’d rather write as an ansible play, than as a Puppet manifest or module.

Which brings me to my next point: it seems that for Mode 1, you want to think mainly in terms of roles, while for Mode 2, you’ll want to focus on playbooks. A role is designed to encapsulate the notion of “here’s a description of what a machine should look like, and the steps to take, if any, to make it match that description”, while a playbook is naturally organized as “step 1; step 2; step 3; …”.

These are, of course, just guidelines. And Mode 2 suffers from the fact that YAML is not a good way to express programming concepts.  But I find this to be a useful way of thinking about what I’m doing in Ansible.

Pseudo-Numeric Identifiers

Let’s say you’re a programmer, and your application uses Library of Congress Control Numbers for books, e.g., 2001012345, or ZIP codes, like 90210. What data types would you use to represent them? Or maybe something like the Dewey Decimal System, which uses 320 to classify a book as Political Science, 320.5 for Political Theory, and 320.973 for “Political institutions and public administration (United States)”?

If you said “integer”, “floating point”, or any kind of numeric type, then clearly you weren’t paying attention during the title.

The correct answer was “string” (or some kind of array of tokens), because although these entities consist of digits, they’re not numbers: they’re identifiers, same as “root” or “Jane Smith”. You can assign them, sort them, group them by common features, but you can’t meaningfully add or multiply them together. If you’re old enough, you may remember the TV series The Prisoner or Get Smart, where characters, most of them secret agents, refer to each other by their code numbers all the time; when agents 86 and 99 team up, they don’t become agent 185 all of a sudden.

If you keep in mind this distinction between numbers, which represent quantities, and strings that merely look like numbers because they happen to consist entirely of integers, you can save yourself a lot of grief. For instance, when your manager decides to store the phone number 18003569377 as “1-800-FLOWERS”, dashes and all. Or when you need to store a foreign phone number and have to put a plus sign in front of the country code.

Programming Tip: Open and Close at the Same Time

One useful programming habit I picked up at some point is: if you open or start something, immediately close it or end it. If you open a bracket, immediately write its closing bracket. If you open a file, immediately write the code to close it.

These days, development environments take care of the niggling little details like matching parentheses and brackets for you. That’s great, but that’s just syntax. The same principle extends further, and automatic tools can’t guess what it is you want to do.

There’s a problem in a lot of code called a resource leak. The classic example is memory leaks in C: the code asks for, and gets, a chunk of memory. But if you don’t free the memory when you’re done with it, then your program will get larger and larger — like a coffee table where a new magazine is added every month but none are ever taken away — until eventually the machine runs out of memory.

These days, languages keep track of memory for you, so it’s easier to avoid memory leaks than it used to be. But the best way I’ve found to manage them is: when you allocate memory (or some other resource), plan to release it when you’re done.

The same principle applies to any resource: if you read or write a file, you’ll need a file handle. If you never close them, they’ll keep lying around, and you’ll eventually run out. So plan ahead, and free the resource as soon as you’ve alocated it:

Once you’ve written

open INFILE, "<", "/path/to/myfile";

go ahead and immediately write the code to close that file:

open INFILE, "<", "/path/to/myfile";
close INFILE;

and only then write the code to do stuff with the file:

open INFILE, "<", "/path/to/myfile";
while ()
	print "hello\n" if /foo/;
close INFILE;

The corollary of this is, if you’ve written the open but aren’t sure where to put the close, then you may want to take a look at the structure of your code, and refactor it.

This same principle applies in many situations: when you open a connection to a remote web server, database server, etc., immediately write the code to close the connection. If you’re writing HTML, and you’ve written <foo>, immediately write the corresponding </foo>. If you’ve sent off an asynchronous AJAX request, figure out where you’re going to receive the reply. When you throw an exception, decide where you’re going to catch it.

And only then write the meat of the code, the stuff that goes between the opening and closing code.

As I said, I originally came across this as a tip for avoiding memory leaks. But I’ve found that doing things this way forces me to be mindful of the structure of my code, and avoid costly surprises down the line.

If You Use Unix, Use Version Control

If you’ve used Unix (or Linux. This applies to Linux, and MacOS X, and probably various flavors of Windows as well), you’ve no doubt found yourself editing configuration files with a text editor. This is especially true if you’ve been administering a machine, either professionally or because you got roped into doing it.

And if you’ve been doing it for more than a day or two, you’ve made a mistake, and wished you could undo it, or at least see what things looked like before you started messing with them.

This is something that programmers have been dealing with for a long time, so they’ve developed an impressive array of tools to allow you to keep track of how a file has changed over time. Most of them have too much overhead for someone who doesn’t do this stuff full-time. But I’m going to talk about RCS, which is simple enough that you can just start using it.

Most programmers will tell you that RCS has severe limitations, like only being able to work on one file at a time, rather than a collection of files, that makes it unsuitable for use in all but a few special circumstances. Thankfully, Unix system administration happens to be one of those circumstances!

What’s version control?

Basically, it allows you to track changes to a file, over time. You check a file in, meaning that you want to keep track of its changes. Periodically, you check it in again, which is a bit like bookmarking a particular version so you can come back to it later. And you can check a file out, that is, retrieve it from the history archive. Or you can compare the file as it is now to how it looked one, five, or a hundred versions ago.

Note that RCS doesn’t record any changes unless you tell it to. That means that you should get into the habit of checking in your changes when you’re done messing with a file.

Starting out

Let’s create a file:

# echo "first" > myfile

Now let’s check it in, to tell RCS that we want to track it:

# ci -u myfile
myfile,v <-- myfile
enter description, terminated with single '.' or end of file:
NOTE: This is NOT the log message!
>> This is a test file
>> .
initial revision: 1.1

ci stands for “check in”, and is RCS’s tool for checking files in. The -u option says to unlock it after checking in.

Locking is a feature of RCS that helps prevent two people from stepping on each other’s toes by editing a file at the same time. We’ll talk more about this later.

Note that I typed in This is a test file. I could have given a description on multiple lines if I wanted to, but usually you want to keep this short: “DNS config file” or “Login message”, or something similar.

End the description with a single dot on a line by itself.

You’ll notice that you now have a file called myfile,v. That’s the history file for myfile.

Since you probably don’t want ,v files lying around cluttering the place up, know that if there’s a directory called RCS, the RCS utilities will look for ,v history files in that directory. So before we get in too deep, let’s create an RCS directory:

# mkdir RCS

Now delete myfile and start from scratch, above.

Done? Good. By the way, you could also have cheated and just moved the ,v file into the RCS directory. Now you know for next time.

Making a change

All right, so now you want to make a change to your file. This happens in three steps:

  1. Check out the file and lock it.
  2. Make the change(s)
  3. Check in your changes and unlock the file.

Check out the file:

# co -l myfile
RCS/myfile,v --> myfile
revision 1.1 (locked)

co is RCS’s check-out utility. In this case, it pulls the latest version out of the history archive, if it’s not there already.

The -l (lower-case ell) flag says to lock the file. This helps to prevent other people from working on the file at the same time as you. It’s still possible for other people to step on your toes, especially if they’re working as root and can overwrite anything, but it makes it a little harder. Just remember that co is almost always followed by -l.

Now let’s change the file. Edit it with your favorite editor and replace the word “first” with the word “second”.

If you want to see what has changed between the last version in history and the current version of the file, use rcsdiff:

# rcsdiff -u myfile
RCS file: RCS/myfile,v
retrieving revision 1.1
diff -u -r1.1 myfile
--- myfile 2016/06/07 20:18:12 1.1
+++ myfile 2016/06/07 20:32:38
@@ -1 +1 @@

The -u option makes it print the difference in “unified-diff” format, which I find more readable than the other possibilities. Read the man page for more options.

In unified-diff format, lines that were deleted are preceded with a minus sign, and lines that were added are preceded by a plus sign. So the above is saying that the line “first” was removed, and “second” was added.

Finally, let’s check in your change:

# ci -u myfile
RCS/myfile,v <-- myfile
new revision: 1.2; previous revision: 1.1
enter log message, terminated with single '.' or end of file:
>> Updated to second version.
>> .

Again, we were prompted to list the changes we made to the file (with a dot on a line by itself to mark the end of our text). You’ll want to be concise yet descriptive in this text, because these are notes you’re making for your future self when you want to go back and find out when and why a change was made.

Viewing a file’s history

Use the rlog command to see a file’s history:

# rlog myfile

RCS file: RCS/myfile,v
Working file: myfile
head: 1.2
locks: strict
access list:
symbolic names:
keyword substitution: kv
total revisions: 2; selected revisions: 2
Test file.
revision 1.2
date: 2016/06/07 20:36:52; author: arensb; state: Exp; lines: +1 -1
Made a change.
revision 1.1
date: 2016/06/07 20:18:12; author: arensb; state: Exp;
Initial revision

In this case, there are two revisions: 1.1, with the log message “Initial revision”, and 1.2, with the log “Made a change.”.

Undoing a change

You’ve already see rlog, which shows you a file’s history. And you’ve seen one way to use rcsdiff.

You can also use either one or two -rrevision-number arguments, to see the difference between specific revisions:

# rcsdiff -u -r1.1 myfile

will show you the difference between revision 1.1 and what’s in the file right now, and

# rcsdiff -u -r1.1 -r1.2 myfile

will show you the difference between revisions 1.1 and 1.2.

(Yes, RCS will just increment the second number in the revision number, so after a while you’ll be editing revision 1.2486 of the file. Getting to revision 2.0 is an advanced topic that we won’t cover here.)

With the tools you already have, the simplest way to revert an unintended change to a file is simply to see what the file used to look like, and copy-paste that into a new revision.

Once you’re comfortable with that, you can read the manual and read up on things like deleting revisions with rcs -o1.2 myfile.

Checking in someone else’s changes

You will inevitably run into cases where someone changes your file without going through RCS. Either it’ll be a coworker managing the same system who didn’t notice the ,v file lying around, or else you’ll forget to check in your changes after making changes.

Here’s a simple way to see whether someone (possibly you) has made changes without your knowledge:

# co -l myfile
RCS/myfile,v --> myfile
revision 1.2 (locked)
writable myfile exists; remove it? [ny](n):

In this case, either you forgot to check in your changes, or else someone made the file writable with chmod, then (possibly) edited it.

In the former case, see what you did with rcsdiff, check in your changes, then check the file out again to do what you were going to do.

The latter case requires a bit more work, because you don’t want to lose your coworker’s changes, even though they bypassed version control.

  1. Make a copy of the file
  2. Check out the latest version of the file.
  3. Overwrite that file with your coworker’s version.
  4. Check those changes in.
  5. Check the file out and make your changes.
  6. Have a talk with your coworker about the benefits of using version control..

You already know, from the above, how to do all of this. But just to recap:

Move the file aside:

# mv myfile

Check out the latest version:

# co -l myfile

Overwrite it with your coworker’s changes:

# mv myfile

Check in those changes:

# ci -u myfile
RCS/myfile,v <-- myfile
new revision: 1.3; previous revision: 1.2
enter log message, terminated with single '.' or end of file:
>> Checking in Bob's changes:.
>> Route around Internet damage.
>> .

That should be enough to get you started. Play around with this, and I’m sure you’ll find that this is a huge improvement over what you’ve probably been using so far: a not-system of making innumerable copies of files when you remember to, with names like “file.bob”, “”, “file.new2”, “file.newer”, and other names that mean nothing to you a week later.

There Are Days When I Hate XML

…and days when I really hate XML.

In this case, I have an XML document like

<?xml version="1.0" ?>
<foo xmlns="">
  <thing id="first"/>
  <thing id="second"/>
  <thing id="third"/>

and I want to get things out of it with XPath.

But I couldn’t manage to select things exactly, such as the <foo> element at the top. You’d think “/foo” would do it, but it didn’t.

Eventually I found out that the problem is the xmlns=”...” attribute. It looks perfectly normal, saying that if you have <thing> without a prefix (like “<ns:thing>”), then it’s in the “” namespace.

However, in XPath, if you specify “ns:thing”, it means “a thing element in whichever namespace the ns prefix corresponds to”. BUT “thing” means “a thing element that’s not in a namespace”.

So how do you specify an element that’s the empty-string namespace, as above? The obvious way would be to select “:thing”, but that doesn’t work. Too simple, I suppose. Maybe that gets confused with CSS pseudo-selectors or something.

No, apparently the thing you need to do is to invent a prefix for the standard elements of the file you’re parsing. That is, add “ns” as another prefix that maps onto “” and then select “ns:thing”. There are different ways of doing this, depending which library you’re using to make XPath queries, but still, it seems like a giant pain in the ass.

DD-WRT Config Backups

So the other day I managed to hose my DD-WRT configuration at home, badly enough that I figured I really ought to back up my config so I don’t wind up trying to reconstruct my config from memory.

(If you just want to see the script, you can jump ahead.)

I have nightly backups of my desktop (and so do you, right? Right?!) so I figured the simplest thing is just to copy the router’s config to my desktop, and let it be swept up with the nightly backups. So then it’s just a question of getting the configuration and such to my desktop.

In DD-WRT, under “Administration → Backup”, you can do manual backups and restores. This is what we want, except that we want this to happen automatically.

The trivial way to get the config is

curl -O -u admin:$PASSWORD http://$ROUTER/nvrambak.bin


wget --http-user=admin --http-password=$PASSWORD http://$ROUTER/nvrambak.bin

where $ROUTER is your router’s name or IP address, and $PASSWORD is your admin password. But this sends your password over the network in the clear, so it’s not at all secure.

Instead, let’s invest some time into setting up ssh on the router; specifically, the public key method that’ll allow passwordless login.

Go follow the instructions there. When you come back, you should have working ssh, and a key file that we’ll use for backups.

Back? Good. Now you can log in to the router with

ssh root@$ROUTER

and entering your password. Or you can log in without a password with

ssh -i /path/to/key-file root@$ROUTER

Once on the router, you can save a copy of the configuration with nvram backup /tmp/nvrambak.bin. So let’s combine those last two parts with:

ssh -i /path/to/key-file root@$ROUTER "nvram backup /tmp/nvrambak.bin"

And finally, let’s copy that file from the router to the desktop:

scp -i /path/to/key-file root@$ROUTER:/tmp/nvrambak.bin /path/to/local/nvrambak.bin

So the simple script becomes:

ROUTER=router name or IP address

$SSH -4 -q ${RSA_ID} root@${ROUTER} "nvram backup /tmp/nbrambak.bin"
$SCP -4 -q ${RSA_ID} root@${ROUTER}:/tmp/nbrambak.bin ${LOCALFILE}

I’ve put this in the “run this before starting a backup” section of my backup utility. But you can also make it a daily cron job. Either way, you should wind up with backups of your router’s config.

I Don’t Want Flying Cars; I Just Want Working Bluetooth

I love Bluetooth. I love that it’s supported on all my various electronic gadgets, and lets them talk to each other and exchange information, be it streaming audio data, or a text note, or what have you.

Or at least I love the idea of Bluetooth. The unfortunate reality is that the implementations that I’ve seen never quite live up to the ideal.

For instances, it often takes several attempts to pair two devices, even when they’re two feet from each other. Sometimes devices disconnect for no obvious reason, or seem to become unpaired without me doing anything.

And then there’s the stuttering, which might be related. I have yet to find a Bluetooth headset, speaker, or other audio receiver that doesn’t stutter for five minutes until it finds it groove. In fairness, after the initial five minutes, things tend to stay pretty stable (at least until I, say, move my phone five feet further from the speaker, at which point, they need to resync). But if it’s a matter of the two devices negotiating, I don’t know, frequencies and data throttling rates and protocols, why don’t they do it at the beginning? Or is it a TCP thing, where the two start out using little bandwidth and ramping up over time?

Lastly, there are the tunnel-vision implementations. From what I’ve seen, the Bluetooth standard defines roles that each device can play, e.g., “I can play audio”, “I can dial a phone number”, “I can display an address card”, “I can store files”, and so forth. But in practice, that doesn’t always work: my cell phone sees my desk phone as an earpiece, and earpieces can’t handle address cards, don’t be silly, so I can’t copy my cell phone’s contact list to my desk phone.

In the age of the Internet of Things, my desk phone can store contacts, my TV can run a browser, and pretty soon my toaster will be able to share its 5G hotspot with the neighborhood. There’s no reason to be limited by a noun on the box it came in.

I understand that most of the above is likely caused by bad implementation of a fundamentally decent protocol. But Bluetooth has been around for, what, a decade or more? And I still regularly run into these problems. That points to something systemic in the software community.

Fun With Barcodes

If you have an Android phone, odds are that you have the Barcode Scanner app. And if you’ve looked in the settings, you may have noticed one called “Custom Search URL”.

This is, as the name says, a URL you can fill in. Once you do, you’ll get a “Custom search” button at the bottom of the screen when you scan a barcode. A “%s” in the URL will be replaced by the numeric UPC value, and “%f” with its format (which is displayed next to the code when you scan one).

It seems to me that this can be used as a poor man’s plugin API. You can use, and make barcode be a CGI/PHP/whatever script that looks at the format and code and decides what to do.

For instance, $EMPLOYER has barcoded asset tags on all inventory items. So today I was able to scan a machine’s code and be redirected to the inventory web page for that machine.

Likewise, if it’s an EAN-13 code that begins with 978 or 979, then presumably it’s an ISBN or ISMN, and you can look it up at Amazon, your library, or wherever.

As far as I know, you can’t recognize that a UPC corresponds to a CD or DVD, without having a table of every CD/DVD publisher, but there’s nothing that says your script has to only do redirection; you can present a list of links to the user. So anyway, for CDs, you can construct MusicBrainz or Discogs lookup URLs. Or perhaps you can parse the code, get the manufacturer, and based on the user’s choice, remember what sort of item that manufacturer corresponds to. Over time you can build up a “good enough” database of the things you scan most often.

I wouldn’t mind having a properly organized library of books, CDs, etc. Which is kind of the point of looking this data up on the net in the first place. But while a phone may make a serviceable barcode scanner, it’s no good for lengthy data input. So really, what I’d like would be for the script to remember what I’ve scanned, along with a quick and dirty readable reference (e.g., “ISBN such-and-such: The Wee Free Men by Terry Pratchett) and stash that someplace so that I can later go back and import the data into Koha or whatever I’m using.

Of course, since it’s a web page, I’m guessing you have access to the full range of goodies that people put in browsers these days. I’m thinking of geographical location, in particular. So the script could in principle behave differently based on where you’re located at the moment (e.g., at home or at work).

There are lots of possibilities. Must. Explore.

A Couple of Shell Quickies

Since I got asked several sh-related questions, I might as well get a post out of them.

One person asks:

I’m writing a portable shell script to download a file from the web, and then compare it to a locally-stored version of the same file, using diff.

My first version of the script used mktemp to download the web-file to temporary file, run the diff command, and then delete the temporary file afterwards. e.g.


However I later discovered that the BSD version of mktemp has an incompatible syntax to the GNU version of mktemp. So then I got rid of the usage of temporary files completely, by using input-redirection, e.g.

diff <(wget -q $ONLINEFILEURL -O -) $LOCALFILE

However while this works fine under bash and ksh, it fails under ash and sh with

Syntax error: "(" unexpected

to which I replied:

The first obvious problem here is that “(wget -q $ONLINEFILEURL -O -)” isn’t a filename, it’s a subprocess. So the shell sees “<” and expects a filename, but finds “(” instead.

It looks as though the way to get diff to read from stdin is the standard way: specify “-” as the filename, and give it input on stdin. Since you’re feeding it the output from a process, you want to use a pipe:

wget -q $ONLINEFILEURL -O - | diff - $LOCALFILE

I also suggested that he could try to figure out which version of mkfile he was using:

# Wrapper function for GNU mktemp
gnu_mktemp() {
	mktemp /tmp/tmpfile.XXXXXX "$@"

# Wrapper function for BSD mktemp
bsd_mktemp() {
	mktemp -t /tmp/tmpfile.XXXXXX "$@"

# Try to figure out which wrapper to use
if mktemp -V | grep version >/dev/null 2>&1; then


And, of course, if race conditions and security aren’t a big concern, there’s always


Another person wanted to write a bash script that would do one thing when run by him or root, and another thing if run by anyone else (basically, die with an error message about insufficient privileges and/or grooviness).

He asked whether the following two expressions were equivalent:

  • if [[ ( `whoami` != "root" ) || ( `whoami` != "coolguy" ) ]]
  • if [[ ! ( `whoami` = "root" ) || ( `whoami` = "coolguy" ) ]]

They’re not, but maybe not for obvious reasons, because propositional logic is a harsh mistress.

In the first expression,

if [[ ( `whoami` != "root" ) || ( `whoami` != "coolguy" ) ]]

let’s say that the user is joeblow. In this case, “`whoami` != "root"” is true, and so the shell can short-circuit the rest of the “||“, because the entire expression is true.

If the user is root, then the first part, “( `whoami` != "root" )” is false. However, the second part, “( `whoami` != "coolguy" )” is true (because rootcoolguy), and so the entire expression is “false || true”, which is true.

The second expression,

if [[ ! ( `whoami` = "root" ) || ( `whoami` = "coolguy" ) ]]

is closer to what he wanted, but doesn’t work because of operator precedence: “!” binds more tightly than “||“, so the expression is equivalen tto “(whoami ≠ root) || (whoami = coolguy)”.

In this case, if the user is joeblow, the first clause, “whoami ≠ root“, is true, and so the entire expression is true.

Worse yet, if the user is root, then neither the first nor second clause is true, so the entire clause is false.

What he really wanted was something like:

if [[ ( `whoami` = "root" ) || ( `whoami` = "coolguy" ) ]]; then
	# Do nothing
	# Do something
	echo "Go away"
	exit 1

Except, of course, that since the if-clause is empty, it can be cut out entirely. Then all we need to do is to negate the condition and only keep the code in the else-clause:

if [[ ! ((`whoami` = "root" ) || ( `whoami` = "coolguy" )) ]]

Note the extra pair of parentheses, to make sure that the “!” applies to the whole thing.

(Update, May 18: Fixed HTML entities.)

Countdown to Backpedaling Widget

Over on the right, in the sidebar, you should see a countdown clock entitled “Countdown to Backpedaling”. (If not, then something went wrong.)

If you’ve been listening to Ask an Atheist, then you should recognize this as a widget version of the Countdown to Backpedaling clock. And if not, then you should definitely be listening to them. Because they’re cool.

At any rate, it’s a clock that counts down to May 22, the day after Jesus’ return and the Day of Judgment, when the backpedaling and excuses begin.

So anyway, now you want to know a) where to download this, b) how to install it, and c) how to complain to me about all the problems you’ve had with (a) and (b).


The main download page is .

If you’re using WordPress, you can download , put it in your wp-content/plugins directory, and with any luck, a “Countdown to Backpedaling” widget should magically show up in your “Widgets” control panel. You can then drag it into position, and it should work.

If you’re using some other software, you’ll want . Installation depends on what you’re using, of course, but you should be able to insert it anywhere that takes HTML.


The main configuration option is the “show_secs” variable at the top. If you want to see seconds in the countdown, set it to true. If you find the seconds’ flashing annoying, set it to false.

You can also look through the CSS part, and edit as you please. You might need to change the width.

I might improve on this, if time permits and I don’t get raptured before getting around to it.

If you have any comments or complaints, leave a comment. Bug reports accompanied by a rum and Coke will get higher priority. Bug reports accompanied by a patch will get even higher priority.