Ansible: Running Commands in Dry-Run Mode in Check Mode

Say you have an Ansible playbook that invokes a command. Normally, that command executes when you run ansible normally, and doesn’t execute at all when you run ansible in check mode.

But a lot of commands, like rsync have a -n or --dry-run argument that shows what would be done, without actually making any changes. So it would be nice to combine the two.

Let’s start with a simple playbook that copies some files with rsync:

- name: Copy files
    - name: rsync the files
      command: >-
  hosts: localhost
  become: no
  gather_facts: no

When you execute this playboook with ansible-playbook foo.yml rsync runs, and when you run in check mode, with ansible-playbook -C foo.yml, rsync doesn’t run.

This is inconvenient, because we’d like to see what rsync would have done before we commit to doing it. So let’s force it to run even in check mode, with check_mode: no, but also run rsync in dry-run mode, so we don’t make changes while we’re still debugging the playbook:

- name: Copy files
    - name: rsync the files
      command: >-
      check_mode: no
  hosts: localhost
  become: no
  gather_facts: no

Now we just need to remember to remove the --dry-run argument when we’re ready to run it for real. And turn it back on again when we need to debug the playbook.

Or we could do the smart thing, and try to add that argument only when we’re running Ansible in check mode. Thankfully, there’s a variable for that: ansible_check_mode, so we can set the argument dynamically:

- name: Copy files
    - name: rsync the files
      command: >-
        {{ '--dry-run' if ansible_check_mode else '' }}
      check_mode: no
  hosts: localhost
  become: no
  gather_facts: no

You can check that this works with ansible-playbook -v -C foo.yml and ansible-playbook -v foo.yml.

City Mileage

I’ve known forever that city mileage for cars is worse than highway mileage, but I never knew why. But it’s a bit like riding a bike.

When you ride a bike, you have to put in a lot of work at the beginning, getting up to speed. And after that, you can mostly coast. Assuming you’re on flat ground, you have to pedal a bit because friction is slowing you down, but it’s nowhere near how hard you had to work getting up to speed.

And then you stop at a red light, and you throw away all the energy you had, so that when the light turns green, you have to put in another burst of work getting up to speed. And, of course, in the city you’re stopping like this all the time. Every few blocks, you throw away all your accumulated energy, and have to start over. This applies to cars the same way as to bikes, except that on your bike, you immediately sense this in your legs, not at the end of the week when the tank is empty.

So it’s not that highways somehow enhance your mileage. Rather, it’s cities that make for wasteful driving. It’s a bit embarrassing that it took me this long to figure it out, but better late than never.

Grand Juries: Some Suggestions on Note-Taking

In which I share some things I only figured out after over a month of grand-jurying.

If your grand jury is anything like mine, each session you have a pad of paper with your name on it, that you get back every week, and a docket list listing the cases for that week, and for each case, the case number, name of the accused, name of the police officer, list of charges, and some other details.

The docket has a lot of blank space on it, so you can take notes. The pad of paper is also good for taking notes. However, note that the pad stays, while the docket is different every week. So try to use the margins of the docket for things that only matter that week, and use the pad for notes that you’ll want to refer to in future weeks.

The latter category includes things like:

  • Legal definitions: what’s the difference between first-, second-, and third-degree burglary? How much heroin is considered “personal use”, and how much is possession with intent to distribute?
  • Anything to do with ongoing cases. In particular, if you hear from a witness one week, it’s likely that you’ll hear from another witness in the same case another week. You’ll be happy you wrote down names and places: it’ll help you get a better idea of what went on.
  • Anything else you might need to know in weeks to come, like the phone number of the courthouse, or the names of your co-jurors.

Now, the docket lists the cases you’ll be hearing that week. But if your case load is anything like mine, you’ll hear anywhere from twenty to forty cases per session, so by the time you’re ready to vote, you won’t be able to keep all of the different cases straight.

This means that as the police officer (or whoever) reads the police report for each case, you need to listen for the elements of the crime the person is charged with. Put a check mark next to each one as you hear it: if the report contains “… a search of the vehicle revealed a digital scale, forty baggies, and twelve rounds of ammunition”, you can check off “possession of drug paraphernalia” and “illegal possession of ammunition”.

If the report later says, “upon questioning, Smith said that the baggies were his, but the bullets belonged to his friend”, you can add an A (for “admitted”) next to “possession of drug paraphernalia”, in case that makes it easier to determine whether there’s probable cause.

During the reading of the report, pay attention to where the information came from: if it begins with “On April 17, defendant Smith threatened victim Jones with a pistol”, who is saying that? Witnesses? A police officer describing surveillance camera footage? As a grand juror, you’re not determining guilt or innocence; only probable cause. In practice, “no probable cause” means that a cop made it up, or something along those lines. It’s a low bar to hurdle, but make sure you do.

Don’t be afraid to ask where the information comes from. If you or someone you cared about were accused of a crime, would you want them indicted by someone who simply took a cop’s word for it, and didn’t ask any questions?


Programming Tip: Open and Close at the Same Time

One useful programming habit I picked up at some point is: if you open or start something, immediately close it or end it. If you open a bracket, immediately write its closing bracket. If you open a file, immediately write the code to close it.

These days, development environments take care of the niggling little details like matching parentheses and brackets for you. That’s great, but that’s just syntax. The same principle extends further, and automatic tools can’t guess what it is you want to do.

There’s a problem in a lot of code called a resource leak. The classic example is memory leaks in C: the code asks for, and gets, a chunk of memory. But if you don’t free the memory when you’re done with it, then your program will get larger and larger — like a coffee table where a new magazine is added every month but none are ever taken away — until eventually the machine runs out of memory.

These days, languages keep track of memory for you, so it’s easier to avoid memory leaks than it used to be. But the best way I’ve found to manage them is: when you allocate memory (or some other resource), plan to release it when you’re done.

The same principle applies to any resource: if you read or write a file, you’ll need a file handle. If you never close them, they’ll keep lying around, and you’ll eventually run out. So plan ahead, and free the resource as soon as you’ve alocated it:

Once you’ve written

open INFILE, "<", "/path/to/myfile";

go ahead and immediately write the code to close that file:

open INFILE, "<", "/path/to/myfile";
close INFILE;

and only then write the code to do stuff with the file:

open INFILE, "<", "/path/to/myfile";
while ()
	print "hello\n" if /foo/;
close INFILE;

The corollary of this is, if you’ve written the open but aren’t sure where to put the close, then you may want to take a look at the structure of your code, and refactor it.

This same principle applies in many situations: when you open a connection to a remote web server, database server, etc., immediately write the code to close the connection. If you’re writing HTML, and you’ve written <foo>, immediately write the corresponding </foo>. If you’ve sent off an asynchronous AJAX request, figure out where you’re going to receive the reply. When you throw an exception, decide where you’re going to catch it.

And only then write the meat of the code, the stuff that goes between the opening and closing code.

As I said, I originally came across this as a tip for avoiding memory leaks. But I’ve found that doing things this way forces me to be mindful of the structure of my code, and avoid costly surprises down the line.


One of the things I learned in math is that a polynomial of degree N can pass through N+1 arbitrary points. A straight line goes through any two points, a parabola goes through any three points, and so forth. The practical upshot of this is that if your equation is complex enough, you can fit it to any data set.

That’s basically what happened to the geocentric model: it started out simple, with planets going around the Earth in circles. Except that some of the planets wobbled a bit. So they added more terms to the equations to account for the wobbles. Then there turned out to be more wobbles on top of the first wobbles, and more terms had to be added to the equations to take those into account, and so on until the theory collapsed under its own weight. There wasn’t any physical mechanism or cause behind the epicycles (as these wobbles were called). They were just mathematical artifacts. And so, one could argue that the theory was simpler when it had fewer epicycles and didn’t explain all of the data, but also was less wrong.

Take another example (adapted from Russell Glasser, who got it from his CS instructor): let’s say you and I order a pizza, and it comes with olives. I hate olives and you love them, so we want to cut it up in such a way that we both get slices of the same size, but your slice has as many of the olives as possible, and mine have as few as possible. (And don’t tell me we could just order a half-olive pizza; I’m using this as another example.)

We could take a photo of the pizza, feed it into an algorithm that’ll find the position of each olive and come up with the best way to slice the pizza fairly, but with a maximum of olives on your slices.

The problem is, this tells us nothing about how to slice the next such pizza that we order. Unless there’s some reason to think that the olives on the next pizza will be laid out in some similar way on the next pizza, we can’t tell the pizza parlor how to slice it up when we place our next order.

In contrast, imagine if we’d looked at the pizza and said, “Hm. Looks like the cook is sloppy, and just tossed a handful of olives on the left side, without bothering to spread them around.” Then we could ask the parlor slice to slice it into wedges, and we have good odds of winding up with three slices with extra olives and three with minimal olives. Or if we’d found that the cook puts the olives in the middle and doesn’t spread them around. Then we could ask the parlor to slice the pizza into a grid; you take the middle pieces, and I’ll take the outside ones.

But our original super-optimal algorithm doesn’t allow us to do that: by trying to perfectly account for every single olive in that one pizza, it doesn’t help us at all in trying to predict the next pizza.

In The Signal and the Noise, Nate Silver calls this overfitting. It’s often tempting to overfit, because then you can say, “See! My theory of Economic Epicycles explains 29 of the last 30 recessions, as well as 85% of the changes in the Dow Jones Industrial Average!” But is this exciting new theory right? That is, does it help us figure out what the future holds; whether we’re looking at a slight economic dip, a recession, or a full-fledged depression?

We’ve probably all heard the one about how the Dow goes up and down along with skirt hems. Or that the performance of the Washington Redskins predicts the outcome of US presidential elections. Of course, there’s no reason to think that fashion designers control Wall Street, or that football players have special insight into politics. More importantly, it goes to show that if you dig long enough, you can find some data set that matches the one you’re looking at. And in this interconnected, online, googlable world, it’s easier than ever to find some data set that matches what you want to see.

These two examples are easy to see through, because there’s obviously no causal relationship between football and politics. But we humans are good at telling convincing stories. What if I told you that pizza sales (with or without olives) can help predict recessions? After all, when people have less spending money, they eat out less, and pizza sales suffer.

I just made this up, both the pizza example and the explanation. So it’s bogus, unless by some million-to-one chance I stumbled on something right. But it’s a lot more plausible than the skirt or football examples, and thus we need to be more careful before believing it.

Update: John Armstrong pointed out that the first paragraph should say “N+1”, not “N”.

Update 2: As if on cue, Wonkette helps demonstrate the problems with trying to explain too much in this post about Glenn Beck somehow managing to tie together John Kerry’s presence or absence on a boat, his wife’s seizure, and Hillary Clinton’s answering or not answering questions about Benghazi. Probably NSFW because hey, Wonkette. But also full of Glenn Beck-ey crazy.

Removing Missing Podcast Episodes from iTunes

So I figured something out today.

I add and remove podcasts in iTunes all the time. And every so often, iTunes loses track of, er, tracks. This usually shows up in a smart playlist, where a podcast episode exists in iTunes’s database, but there’s no corresponding MP3 file on disk. You can’t manually delete entries from smart playlists; and if I’ve deleted the podcast, then I can’t even go back to the podcast list to delete the bogus episode. Nor does it show up in the “Music” list, since it’s a podcast episode.

What I finally figured out is that if you change the Media type from “Podcast” to “Music”, that’ll move the episode to the Music list, where you can delete it.

You can change the media type with “Get info” (⌘-I) → Options → “Media Kind”.

Unfortunately, if the file is missing, iTunes won’t let you edit the media kind. So first you need to associate the podcast episode with an MP3 file. The easiest way is to copy an existing MP3 file to /tmp/foo.mp3 or some such. Then, when you ⌘-I and iTunes says no such file and “Do you want to locate it?”, say yes, and point it at /tmp/foo.mp3. Then you can edit the media kind, and delete the file from the Music list.

Da da da

Today is National Grammar Day, but rather than rail against common misuses of the English language like the insufferable language snob that I am, I thought I’d mention a peculiarity of language that I happened to notice.

The German word “da” means “there”, as in “Mein Bier ist da” — “My beer is right here”. In this sense, it refers to a location.

But in certain other combinations, it refers to a noun: “dagegen” means “against it” or “in contrast to it”. It literally means “that-against”. “Dafür” means “for it”.

So sometimes “da” refers to a location, and in at other times it refers to a “thing”. I put “thing” in scare-quotes because the object that a da-compound word refers to need not be an object made of atoms: one of the examples linked to above is “Haben Sie etwas dagegen, wenn ich rauche?” — “Do you mind if I smoke?”. Smoking is an activity, not an object, but our minds still treat it in many ways as an honorary object.

In fact, I can imagine an evolution of language in which “da” started out referring to a location, perhaps a location being pointed to, later came to also represent the thing in the location being pointed to, and eventually came to encompass honorary nouns.

But before we go pointing fingers at those silly Germans da, it’s worth pointing out that “there” — the English word for “da” — is similarly schizophrenic: it usually refers to a location, as in “I live in that house there”, but sometimes, in combinations, it refers to the same kind of “thing” as in German: “therefore”, “thereof”, “therewith”, and so forth.

In fact, the most common English example of this location/thing oddity is “wherefore”, in Romeo and Juliet, when Juliet says, “O Romeo, wherefore art thou Romeo?”.

Wherefore” has “where” in it, which makes people think Juliet’s wondering about Romeo’s location. But actually it means “why” or “for what reason”. She’s asking why Romeo is Romeo, as in “of all the guys I could have fallen for, why did it have to be Romeo?”

Okay, so I couldn’t help myself and snuck in some grammar-railing there at the end. I warned you I was a snob.

Atheist Contributions

James Osborne (1834-1881) was an early settler of Seattle, who opened the Gem Saloon (seen in the illustration to the left). His obituary said that “He maintained a reputable saloon patronized by the best citizens of the type that patronized saloons.”[1]

In addition, in his will, he left money to the city to build a civic building, provided that the city contributed matching funds. Unfortunately, he was so filthy rich (like much of the American west, Seattle was founded by a bunch of people trying to make a quick buck) that the city couldn’t afford to match the funds until 1927, when they built the Seattle Opera House (see also Speidel).

Speidel also tells us:

The will specified, for one thing, that his brother-in-law—who had thrown him out of the house once—was not to get “one thin dime.” It also stated flatly that he forbade anyone to take his body into—or even near—any church and there were to be no religious ceremonies conducted at the funeral.

Instead he asked that the funeral be held at Yesler’s Pavilion, that a competent brass band be employed to liven up the occasion, and that a good free-thinker like Judge I. R. W. Hall be paid one hundred dollars for the oration … and failing Judge Hall, that “either Thomas Burke or W. H. White will do.” The services were to end with burial in Lakeview Cemetery by the Ancient Order of United Workmen.

So that we can truthfully tell you that the “father” of our Opera House was an atheistic saloon keeper from the Skid Road.

[1]: William C. Speidel, Sons of the Profits, Nettle Creek Press, pp. 72-73.

Why Is Universalizability a Good Thing?

Back in 2010, Greta Christina wrote about liberal and conservative moral systems. At the core was a set of studies showing that while everyone shares the same core values — fairness, minimizing harm, authority, purity, loyalty, and a few others — that liberals and conservatives prioritize these values differently: liberals tend to put a higher value on fairness, for instance, while conservatives tend to put a higher value on authority.

She then argues that “liberal” core values like fairness and harm-reduction are better than “conservative” ones like purity and authority, because the liberal ones are universalizable: they aren’t parochial, and apply to every human being (and possibly animals and extraterrestrials) equally.

That explanation is okay, but I’m not quite satisfied with it. I kept asking why the fact that a value applies to everyone is a good core value. And that led me to the open marketplace of ideas.

And to do that, let me step back and look at the open marketplace of, well, markets.

Everyone in a capitalist society understands why, say, $3.79 is a fair price for a bag of chips: thousands of sellers pick prices at which to sell their goods, and millions of buyers make decisions as to whether to buy at that price or not. Of course I’d prefer to buy chips for a nickel, and of course the store would rather charge me twenty bucks. But I understand that that wouldn’t cover manufacturing costs, the store understands that if their price is too high, I won’t buy it, and out of many such interactions, of people either buying or not buying, a consensus emerges: $1.00 is too low, $10.00 is too high, and that something like $3.50 is a price that everyone can live with.

There are also times when prices can be tilted to favor or penalize some group of people or set of goods, such as “Buy American” campaigns or boycotts, or when a designer like Louis Vuitton convinces people to pay extra for goods that have a particular logo on them.

Over time, we will act as both buyer and seller, comparison shopper and haggler, and can appreciate at least the rudiments of everyone’s views.

Now, since morality is a way of regulating interactions between people (if it weren’t for the fact that we live together, we’d have no need for morality), I claim that a similar calculus takes place: that we are constantly negotiating The Rules in a corner of the marketplace of ideas.

Just as the store would love to charge me $20 for a bag of chips, I would like for everyone to call me “Your Highness” and let me skip ahead in line at the store. The problem is persuading people to treat me that way.

I also know that if someone else wanted to be treated that way, I’d resent and resist it. Nor can I come up with a convincing argument for why I should get special treatment, one that I would accept if the shoe were on the other foot. And so collectively we negotiate a compromise that we can all live with, in which nobody gets called “Your Highness” and we wait in line in first-come, first-served order.

And gosh, it sure looks as though this sort of free negotiation favors those rules and compromises that everyone can agree on. That is, universalizable values.

Now, unlike the economic marketplace, where I will by turns take the role of buyer or seller, in the marketplace of moral ideas, I will never be a woman, or Asian, or left-handed, or gay. But I do interact with people who are. Even if we ignore for a moment the effects of sympathy, and consider that everyone just wants the moral rules that most favor themselves, men will argue for rules that privilege men, and women will argue for rules that privilege women, and over time, they ought to compromise on something that isn’t what anyone wanted, but that everyone can live with, like equality.

In this analogy, asking why one group gets special privileges is like asking why one brand costs more than another. Sometimes there’s a good answer (“Brand L jeans are more durable than brand X”, “You should give up your subway seat to older people because they need it more”), and sometimes there isn’t (“Brand A costs more because we just redesigned the label”, “Men should be in positions of power because they have a Y chromosome”).

And yes, this process takes far longer than anyone would like, partly because (for the vast majority of people) it’s not a conscious process: we don’t set out to figure out what moral rules are best for us, for our loved ones, for the rest of society; we just sort of go along with what’s around us, and either complain when we don’t like something, or adapt when other people complain about our behavior. There are many other complicating factors as well.

But on the whole, this semi-conscious marketplace should favor those values that apply to everyone with a voice, or at least an advocate. That is, things like fairness and harm reduction.

Natural Selection in the Fossil Record

table.figure {
background-color: #f0f0f0;
table.figure caption {
caption-side: bottom;
font-size: 90%;
padding: 1em;
background-color: #f0f0f0;

For quite some time now, I’ve had a question:

We can see evolution in the present.
And we can see natural selection in the present.
And we can see lots of evolution in the fossil record.
But can we see natural selection in the fossil record?

Continue reading “Natural Selection in the Fossil Record”