Archive for August, 2006

Slicing and Dicing the Senate

Wednesday, August 9th, 2006

Below are two syncronized animated charts. The file containing them is large, a half a Meg (sorry about that).

Each point on the charts represents one US Senator; they don’t move. The idea of this technique, “Optimal Classification Estimates”, is to reduce each legislator to two numbers and them pin them onto that graph. It’s extremely reductionist, but it works. The lines slicing thru the chart illustrate how voting proceeds on various bills. A line running from top to bottom reveals that a bill was decided largely on economic issues; while a line running left to right is a bill that was decided on social issues. The democrats on the left are economically liberal, i.e. they tend to look out for the weaker and more numerous economic actors. The Republicans on the right are economically conservative; look after the economically large, but few.

The chart on the right shows how well the model works. The hand full of points shown are Senators who votes didn’t fit the model.

I grab’d that chart from here.

The model is extremely accurate; around 95% these days. Amazingly you don’t actually need two axis; you will get 90% accuracy with a single axis that runs almost top to bottom, but slices slightly at an angle. You can see the entire Senate sorted into that ordering here. For example Joe Liberman isn’t the most conservative democratic Senator, there are a handful who are more to the right than he is.

I’ve writen about this model before; and I keep coming back to it because it totally changed the way I think about politics. It’s all economic; all the noise about social issues never actually flows thru into the legislative agenda.

If you download the chart and stick it into the right program (Quicktime player works for me) you can single step thru the votes. You can then go look up particular votes. That facinating because the topic of a bill may appear to be along one dimension but the vote shows clearly that it was entirely decided by another dimension. For example, the votes around now failed plan to create a guest worker program are a good example of social and economic conservative issues playing off against each other.

Unlinkablity

Wednesday, August 9th, 2006

I see that the New York Times found a reporter with sufficent wit to actually track down one of the users who’s searchs are revealed in the AOL search data. It’s a pretty good article.

Meanwhile Ben Laurie is curious about creating an anonymous package delivery system by mimicing the ideas found in Tor. It’s an amusing idea Anonymous Package System, or APS; I’m sure that Fedex is scared! If two of you out there want to give this a try you can forward a package thru me; send me a pgp email and I’ll send you back a token to include in the package (think of it as postage) to provide you “bought” the one unit of forwarding. I’ll can then take pictures of the various artifacts for and do blog posting to report on how it works out.

Today, rather than anonymous physical package delivery, I find my self yearning for Tor like functionality that I can target at particular URLs; because today I’m much more interested in hiding my search data from Google than I was a week ago.

Decay, Rust, Terror

Tuesday, August 8th, 2006

Actual gorrilla mask worn by global guerrillas!Why do I keep expecting John Robb to point out that rust is yet another example that validates his global guerrillas model of where the world is headed? What distructive force is further out on the long tail of bad actors? The ultimate in fire power rust coordinates it’s actions thru a swarm based attack. Does not metal’s primary loyality for oxygen illustrate the inevitability of states losing their monopoly on power? Surely corrosion’s vile attack on the Prudhoe Bay pipeline is yet another sign: our economic system is just a long line of fat sitting ducks? And, yet again we see our over muscular military totally unprepared to deal with the real threats we now face in this post super-power globalized economy!

This just in: rust now teaming with sludge!

Search Frequencies/Person & A Public Service Announcement

Monday, August 7th, 2006

AOL recently released a huge sample of search engine queries. In a highly questionable move they tied these queries to reasonably anonomous user identifiers; for example we know that user known as #724 searched for “how to install a glue down floor”, as well as “carbol tunnel” etc. He did 366 searchs between March 1st and May 5 2006.

Unsuprisingly the distribution of search generators is power-law distributed. This is a log-log chart. Each dot on the chart represents one AOL users. The vertical axis is how many searches they did; for example the highest dot, aka user 2263543, did 8695 searches. This is only the most active 20 thousand users in this data set, the least active of whom did 313 searches. The complete data set has 657 thousand users, 57 thousand of whom only did one search.

Actually I dropped the most prolific searcher, user #71845, who made over a quarter million searchs; and totally messes up my nice straight line.

Today is national mental health day. I think the most disturbing thing I’ve noticed as I browse this data is the number of people searching for information on how to commit sucide. There are effective treatments for depression.

Crazy

Monday, August 7th, 2006

I installed ubuntu on an G3 blueberry iMac, and it works ok.


bhyde@ubuntu:~$ set | wc -l
4251

Golly.

FYI - installing via the Live CD doesn’t work, while the alternative CD did - but that requires time and expertise.