Exploded Clown

Other the past week I’ve been writing some code for work that has to happen periodically and then re-schedule itself so it can happen again. We never want two of this process running, and we want the process to automagically expand its time window if it takes longer than we think it should, because arbitrary magic numbers are a Bad Thing™.

So I keep track of each time we run and keep an average of how long the job takes every time. But I don’t want to keep around every run time, so all I do is keep around the average, and how many times the job has actually run. From those two numbers plus our current run time I can approximate a new average and use that to schedule the next run. I came up with a nice little algorithm that pushes the average around whenever a new number comes in. It works great:

def find_new_avg(old_avg, old_count, new_number)
  weight = (new_number.to_f - old_avg.to_f)/(old_count+1)
  old_avg + weight
end

But in my tests, I noticed something; this doesn’t just approximate the average. It actually calculates the average. It’s right all the time.

I did few quick Google searches and checked Wikipedia couldn’t find anything. I looked at it again. I pared down the code to the above (it used to be a bit more complicated). I tried more tests. It still worked.

I asked my friend Nick if he had ever heard of this algorithm. He hadn’t. He pushed some numbers at it and decided it did work. We did a video conference over Skype and worked out the algebra, and we proved that it did work. I said: “It just seems insane that I could have invented–by accident– a new way to average numbers that uses less space than any other way I’ve ever seen.”

I searched online again, and finally found it. It’s from The Art of Computer Science, Volume 2, by Donald Knuth.

Knowing that I’m not the first one to discover this algorithm restores my faith in how much research has been done in the past (although my faith in Google’s capabilities was somewhat shaken), but the fact remains that, not knowing about it beforehand, I discovered a reliable algorithm that does what I need in a fraction of the space it could take. And that makes me happy, even if I did reinvent the wheel.

I’m a big fan of Trac, and I use it for project management when I’m given the choice. It has a few issues, but nothing a little plugin magic can’t fix. On of my favorite plugins is TracRedirect, which allows me to make one page point to another and make the browser redirect there, so “Customers” points to “Customer”, and I can be lazy when I’m typing other pages and use whichever one makes the most sense.

But TracRedirect does not play nice with the latest version of Trac, 0.11. In fact, it makes 0.11 die. This is bad.

I don’t really know Python. I mean, I can get by, but I don’t really know it. So the prospect of patching some code for a plugin in a language I don’t know for a project whose code I’ve never seen is at once intimidating and exciting. The reason I’m a programmer is because the exciting won, and I fumbled around until I got it working. The diff between the latest svn code and my own is after the jump; it’s mostly just updating IWikiMacroProvider to take the new args 0.11 passes its way, but there’s another little fix to include the redirect stylesheet using the new method, as the old method didn’t seem to work reliably.

Read the rest of this article »

The Get Out Clause is (apparently; I’d never heard of them before) a band in Manchester. They have no contract, and no money to put together a music video. But what they do have is time and a pretty good song. So they sung their song in front of lots of CCTVs and then requested the footage via the Data Protection Act (or maybe the Freedom of Information Act; reports vary). I’m sure it ate up a lot of time in the playing and the requesting and the editing, but it got all kinds of publicity, including getting me–half way around the globe– to write this about them, so I think that counts as a pay off.

There are already approximately seventeen million reviews of Matthew Yglesias’ new book, Heads in the Sand, so I figured one more wouldn’t hurt.

Yglesias is one of my must-read bloggers; when I’m behind a few days, I’ll plow through his feed first because he makes great points, ties it into the larger picture, and doesn’t belabor the issue. All of these things are true in his book, as well.

It has been said elsewhere, but the transition from blog posts to book chapters went very well for Yglesias’ style. In large part this is attributable to the decision to tie the book into a narrative following the last decade of American foreign policy, and make the points along the way. This allows Yglesias to follow his normal method of commenting on events “as they happen,” which is why reading the book feels a lot like reading the blog or his articles.

But it’s a two-edged sword. Too often the points Yglesias is making are directly related to the background they are surrounded by, but the background tends to bury the commentary by dint of leading rather than following. This makes the principals Yglesias puts forward somewhat harder to find in the text.

The exception to this pattern is the final chapter “In With the Old,” which dispenses with the narrative and instead takes a more forward-looking view to the next administration and how they should go about foreign policy. This chapter benefits greatly from this freedom, and makes some of the best points in the book.

But enough about the structure of the book; the content is the important bit. The book as a whole advocates not new ideas, but a return to the “liberal internationalism” that Yglesias says the US followed from World War II until the current administration. Early on, the yardstick we are given to measure foreign policy decisions is:

The question, then, that must be asked of any proposed policy is… whether it brings us closer to or further from the dream of a peaceful, rule-governed liberal world order. p8

Those four attributes: peaceful, rule-governed, liberal, and world order are the keys. The first is the goal, and the rest are how to achieve it in a sustainable way. If you abandon the latter three, you will inevitably lose the first.

Yglesias establishes the heredity of the idea: Wilson as grandfather, FDR as father, Truman as mentor. The first Iraq War proves that it works; bringing an international coalition together to enforce the rules and keep the peace is a success. But then we immediately trip over Kosovo.

Western leaders of the period were, and are, often accused of a selective approach to humanitarianism, acting forcefully in Kosovo, while being less concerned with more serious humanitarian problems in Africa and elsewhere. The charge is essentially accurate but largely misses the point: that Kosovo presented a mixture of humanitarian and interest-based reasons for intervention was precisely what strengthened the case for playing fast and loose with the UN rules, making intervention a reasonable option. p17

As Yglesias has conceded, Kosovo is somewhat of an edge case, but how much of one is debatable. The problem is that Kosovo is rather obviously a use of force outside of the approval of the UN Security Counsel, but it seems right anyway. Seeming right isn’t rules governed, though.

This highlights one of the shortcomings I found with the book; in its desire to hew closely to the chronology, it spends its time on what happened and why it worked or didn’t, and misses opportunities to talk about what should have happened. In the case of Kosovo, what is the US to do if the UN won’t act? (We chose asking NATO instead, and Kevin Drum makes the point that that was good enough). But in other cases this just left me wanting a few more pages.

One such case was the (excellent) discussion of the 2004 Presidential Race. Yglesias makes a good argument that Dean fell in the primary because he “was, in various respects, a less-than-ideal candidate” (p88), and that Kerry fell in the general because he couldn’t properly distance himself from his previous pro-war stances. But none of the other Democratic candidates could have done so, either. So what should have been done?

This oversight is especially sad because it ties in with Yglesias’ view that being the opposition party means being unafraid to oppose the other party. Bush was for the war, but for political necessity, so was most of the Democratic Party. As Yglesias puts it:

The 2004 primary offered the best possible opportunity for a Democratic course correction: a change to reject the errors of the congressional leadership in favor of the greater wisdom demonstrated by the bulk of the party’s rank and file, both in and out of Congress. But that opportunity was squandered, and the Democrats would up deeper in a political quagmire of their own making. p89

This comes close to getting to what should have been done, but misses the mark by being too abstract. Who was the candidate who could have done this? How would they have avoided being painted with the Dean smears of being “a throwback to post-Vietnam quasi-isolationism and left-wing radicalism” (p98)?

Another thing that seemed to be alluded to but missing is a real strategy of how to fight al-Qaeda. Specifically, is Beinart’s idea of “democrati[zing] the Muslim world” p121 a key to combating al-Qaeda, even if it isn’t the key, and Iraq wasn’t a useful means of obtaining that goal? Yglesias comes close to an answer:

The relevant sort of resentment, however, is not resentment at the absence of democracy, but resentment at the absence of democracy’s logical precursor–self determination. p135

And with this great quote from Michael Lind:

Woodrow Wilson said “we must make the world safe for democracy.” He did not say we must make the world democratic. p143

But going against Bush’s narrative that “Democracy is on the March” is too easily painted as being against democracy or– worse– being of the opinion that Muslims can’t handle democracy. Granted that we shouldn’t accept this framing, but how does one reject the argument without tripping into it?

The beginning of an answer is there, and I think it lies in the line that “Democracy is fundamentally incompatible with the idea of empire” p144. This idea– more fully explored by Dave Meyer here– is interesting, but stands without any support in the book.

Another glimpse at the larger context for this question is elsewhere in the book:

It was fashionable for a long time for liberals… to argue that the Bush administration was right about the need to promote democracy in the Middle East but wrong in the way to go about it. What was needed, perhaps, was a new national commitment to a new method of promoting democracy. I myself wrote some articles along these lines, and there is some truth to that way of looking at things. p194

Yglesias outlines some of the ways to go about democracy promotion, and acknowledges that they’re not riveting, but still important. (And what does Yglesias think of Thomas Barnett’s rather riveting TED Talk? Too much like Max Boot’s proposal?)

One of the things that Yglesias does incredibly well, though, is present the current situation and show how untenable it is:

The Bush administration’s embrace of militaristic nationalism has not brought democracy to the Middle East and has not frightened Iran or North Korea out of conducting nuclear research, nor has it intimidated Iran or Syria out of supporting Hezbollah, spooked Pakistan into ending its support for Kashmiri radicals or into clamping down on al-Qaeda sympathizers in its border areas, owerawed China out of efforts to become a grat power, or frightened Russia out of reasserting itself. p187

Ultimately, I liked the book rather a lot. It was a quick run-down of most of the important foreign policy decisions in the last decade with good interpretations of the meanings and repercussions behind them. And I look forward to the next book, where Yglesias can answer his own challenge:

There is no panacea ere (just look at Singapore), but it is in many ways the best thing we can do. The subject lies largely outside the scope of this book, but one of the real challenges of liberal economic policy in the coming years will be rebuilding te domestic social contract in a way that once again makes further expansions of global trade acceptable– and, indeed, beneficial– to the American working class. p196