Ascription is an Anathema to any Enthusiasm

These are cool clever animated maps showing bicycle trips. A beautiful example of innovation on open substrate. The maps like these are possible because the US census created maps for the entire nation back in 1980 and then publicly released them. Systems like Map Quest are built on that public good. Get them here, for example. The code stands on a mess of Perl modules for manipulation such things. Just a guy having fun with the oportunities all that created.

Back on the topic of cartograms: source code for making those using Gastner and Newman's technique can be downloaded here (zip file). I’ve not used it yet, but the command line API appears quite straight forward. Input regions and population counts for each region; run the software; cartogram regions are output. In addition you get two postscript files so you can check the input/output. Regions are described by the polygon(s).

There is really no end to things you could use that for. Some examples include cartograms of where members, employees, customers reside. Organization charts scaled by various metrics: cost, income, depreciation, education level, etc. Transaction volumes by classification in your web site, your firewall. In many of these cases the topology of the input regions doesn’t necessarily need to reflex something, but if you have something at hand you can use that.

I’d love to see a cartogram of income over the shelf space of a grocery store, or the gates of an airline, or turn over rates.

Or how about a cartogram showing the click thru numbers for various links on a web page! If you project an image thru a mapping like this you get a homunculus.

As an advocate for the power-law distribution one frustration is how difficult it is to visualize. You can use examples like Mark Twain’s delightful description of evolution: “Man has been here 32,000 years. That it took a hundred million years to prepare the world for him is proof that that is what it was done for. I suppose it is. I dunno. If the Eiffel tower were now representing the world’s age, the skin of paint on the pinnacle-knob at its summit would represent man’s share of that age; & anybody would perceive that that skin was what the tower was built for. I reckon they would. I dunno.” I.e. those at the top of the power-law wealth distribution it makes no sense to bend over and pick up a thousand dollar bill.

We generally assume that things are distributed uniformly. The rain, for example, falls reasonably uniformly overy my back yard. These systems with power-law distributions aren’t like that. They are more like some freak storm passing by and dropping all it’s rain into a bucket in my yard.

One of the classic examples of a power-law distribution is the size of cities and towns. Stop and think what that means. It means that every time you look at a geographic map of which displays some human activity your being horribly mislead. Because humans are spread out evenly like the rain in my backyard. Instead they are all concentrated in a handful of buckets. In New York, Paris, Hong Kong, etc. etc. large regions of the city have from a quarter to a million people per square mile. It is as hard to think about that as it is to think about men who’s income is so high they ought not reach down and pick up a thousand dollars.

It’s no help trying to address this problem by telling outlandish stories like the ones above. The listener’s mind just locks up and switches over into denial. For example, if I say that social networks are power-law distributed what should you immediately think? You should expect to find people who’s place in the social network is like the lower east side of Manhatten; who are like the skim of paint on top of the Eiffel tower. Can you think about that?

If people can not think about these distributions what is one to do? If people can not get into their head any reasonable model of rich the rich are, or how dense real cities are, or how long evolution takes how can we expect to work thru the consequences of these facts?

Visualization of data is one possiblity. One possible technique is the cartograph. For example here’s a nice short article by the Mark Newman that tries to give a more accurate visualization of the last election.

A drawing like that is just barely accessible. Notice that how you can see San Fransisco bay on that map. Notice the scale of Long Island.

That map was drawn using technique outlined in this paper. That paper includes some wonderful examples of how hard this kind of visualization is. For example here are two maps of the 2000 election. The more distorted, and hence somewhat more accurate, map reflects a more accurate model of were the population lives. New York, New Jersy, Massachusetts, and Pennsilvania all become distorted to show where they have urban centers. But even maps like this can’t quite capture how dense the population really is in places like Manhattan.

That paper also has a very nice example of how important it is to get this right. In this example we look at cancer cases. In the first drawing we can see that if your looking to find somebody with cancer you’d best go to NYC. But that is an entirely different from saying that NYC is a place where your likely to get cancer. In the second drawing the incidence of cancer looks almost entirely independent of population density; just the slightest sign that Love Canal near Buffalo and the poor districts of upper manhatten and the south Bronx might be problem areas.

But again second map certainly doesn’t look at all like New York State. In fact it’s proably a good rule of thumb that when visualizing data about human society if the drawing bears any resemblance to something in the physical world your probably about to be mislead.

This is the problem. How can we get people to think constructively about systems that are inherently hard to think about?

Phil Greenspun wrote a satirical posting on why it’s ok not to care about other people troubles. His self amused argument is that in the Modern world such work can be left to specialists.

Modern, aka Urban, life is all about residing in numerous communities. If you buy into my two part model of communities (That they stand on three legs and generally have a limited liablity for their extent.) Then Phil’s strawman can be reframed as one’s duty to other members of a community is limited to that which is within the liablity scope of that particular community.

Phil’s posting is amusing because of it’s exagerated nature. Few communities are very formality about their limited liablity contract. In fact it may well be a sign of a scam in progress when a community becomes exacting about it.

Why exactly am I expected to be sympathetic to a coworker who’s not feeling well? Is that work part of the community social contract of the workplace? Possibly not, more likely it’s part of the social contract of some larger community we both inhabit.

Sometime ago a friend of mine suffered a horrible event. It laid waste to a major aspect of their life. Discussing it, a long after the fact one aspect of the story struck me. People in other spheres of this person’s life had no idea.

It was just like the sailor with two wives, two families, in two ports recieving word of the distruction of his other home. People would have idea what to make of his mood. If he told them their first reaction would be “Gee! I never knew you had another life.” In part their reaction included an element of “Oh, I thought your loyalities lay elsewhere.”

Modern life’s advantage is that it provides a diversified portfolio. Diversifited portofolios only work well if your willing to dump parts of the portfolio.

I’m sure there is some executive at Apple who’s bonus is based on how many spare power adaptors they sell. The damn things are about as reliable as an American car back when Ralph Nader wrote “Unsafe at any Speed.” These things have a half life of about 4 months and they cost a bit under a $100 each to replace. What a scam.

Once you have a loyal customer the devil is always tempting you to insist that he buy some add-ons, spare parts, etc. The more captive he is the more you can pile on this kind of abuse.

The hotel I checked into at the end of a long day recently had a “manditory resort fee.” No, it’s not a tax from the local municipality it’s just an extra charge they throw onto everybodies bill. Now isn’t that hospitible of them? Of course the room had trays of food layed out with a prices list next to them. The lobby had a booth, just in case you wanted to buy one of their beds.

At the dentist’s office yesterday they attempted to sell me a hundred dollar electric tooth brush. Three extra brushs for $30 dollars. This was urged on me by the dental hygienist as something they strongly recomend. It’s not like I can do my own dentistry. So I go and place myself into the hands of a proffesional. Having handed the responsiblity over to this professional his agent then attempts to upsell me some accessories. I wonder if dental hygienist school now includes additional training on how to upsell the customer.

How long before all the professionals do this? Doctors could sell bedpads. Surely we can find something for lawyers to sell you that will help with your day to day legal hygiene! What’s wrong with my plumber, he hasn’t even tried to sell me a powder that I must sprinkle into my drains once a week. I do love my pipes!

Shifting gears, a bit, these customer loyality/lock-in senarios are very similar to the variation on a public good; the club-good. Club goods are a scheme for addressing some of the failure modes of public-goods; for example overcrowding or freeriding. Your public swimming pool might become over crowded; so you create a club and use that to both limit access and to assure all the users are paid-up club members. In a sense what your buying when you purchase a hotel room is a shortterm membership in the hotel-club. Once inside the club the facilities behave just like a public-good - abundant and collegial.

The fence around the club helps to assure that we avoid the organizational problems of the public-goods - freeriding and overcrowding for example. But, if we are running the club - and we are feeling a bit evil - we can use that same fence to hold the members hostage and abuse them. Substituting the appearance of abundance for the reality, subtituting the collegial for the salesman’s bonhomie.

On the plane they charged $2 for a headset to watch the movie, and then called upon our solidarity with our our fellow members of the flight club to draw down the shades. They packed us in like cattle. So much for using the club idea to resolve the overcrowding problem.

I should probably mention that I had managed to get that hotel room mentioned above for about 30% of list price. Which only brings us around to taking note that one of the things the club owner can do if he manages the fence to his advantage is discrimitory pricing.

Isn’t there is something unnatural about a venture capitalist who’s obviously a hard-core geek?

A long time back Tim Orton posted a list of links on systems that allow delegation of rights to other parties. I’ve been meaning to get around to writing about this kind of think because while lots of people tend to know about private/public key encryption and other approaches to the identity problem these methods seem to have been largely forgotten.

These are such a nostalgia trip for me. Back in the early mid 70s at CMU I had some light involvement in the development of a multiprocessor system known as C.mmp. One unique feature of the operating system of that toy was that it managed permissions using “capabilities” rather that “access control lists”. Capabilities are a much better design but nobody has ever really managed to make them very practical.

Access control lists, or ACL, protect things by marking objects with a list of who is allowed to access. For example if the question arises is Bob allowed to update the payroll data the ACL system works by checking a list associated with that data and then checking that the person about to make the change has proved he is Bob. ACL’s are like the guest list at an exclusive party, each time another guest shows up the doorman checks their ID against the guest list before letting them in. Maintaining the guest lists is a pain.

Capabilities allows Bob to delegate the task of updating the payroll to one of his acquaintances.

In a capability system Bob carries around a set of capabilities, think of them as like one of those huge key rings that building maintenance guys carry around. When Bob wants to modify the payroll data he pulls out the right key/capability presents it and the system checks it. If Bob want’s to delegate, he can hand the key to his assistant. Keeping all these keys under control is a pain.

Real world system work with a hybrid of keys and guest lists. In some situations you present an ID card (for example to withdraw books from the public library) and in other scenarios your provided with a key (for example when you rent a car).

One of the systems Tim points to is really just too cute: i.e. a programming language called E.

Most programming languages consist of a handful of tricks. For example one of Lisp’s trick is to bring the syntax and the parse tree so very close to each other so that at program build time you can do powerful transformations of your program. One of Python’s tricks is to leverage the source text’s indenting for syntax. One of Perl’s tricks is to weave a number of powerful micro-languages together into one dense rich stew. One of Erlang’s clever tricks is designing extremely light weight distributed tasks into the heart of the language.

E seems to have three key tricks. The most minor of these is that it stands on top of the Java VM. The second trick is to fold the idea of capabilities into the language. While in a traditional programming language you can manipulate an object if you can get a pointer to it in E you need to get the capability not the pointer. All kinds of clever cryptography is built in so that these capabilities enabling them to do clever things, like only work for an interval of time.

The last clever thing, the fun one, is an approach to multitasking based on “promises”. Rather than the usual collection of processes, micro-tasks, thread, locks, semaphores, queues, event handling, and messages that most languages cobble together in various combinations to create a multitasking system we get one construct; the promise.

Here is a very amusing description of why this might be “a better way(tm)”:

Let us look at a conventional human distributed computation. Alice, the CEO of Evernet Computing, needs a new version of the budget including R&D numbers from the VP of Engineering, Bob. Alice calls Bob: “Could you get me those numbers?”

Bob jots Alice’s request on his to-do list. “Sure thing, Alice, I promise I’ll get them for you after I solve this engineering problem.”

Bob has handed Alice a promise for the answer. He has not handed her the answer. But neither Bob nor Alice sits on their hands, blocked, waiting for the resolution.

Rather, Bob continues to work his current problem. And Alice goes to Carol, the CFO: “Carol, when Bob gets those numbers, plug ‘em into the spreadsheet and give me the new budget,okay?”

Carol: “No problem.” Carol writes Alice’s request on her own to-do list, but does not put it either first or last in the list. Rather, she puts it in the conditional part of the list, to be done when the condition is met–in this case, when Bob fulfills his promise.

Conceptually, Alice has handed to Carol a copy of Bob’s promise for numbers, and Carol has handed to Alice a promise for a new integrated spreadsheet. Once again, no one waits around, blocked. Carol ambles down the hall for a contract negotiation, Alice goes back to preparing for the IPO.

When Bob finishes his calculations, he signals that his promise has been fulfilled; when Carol receives the signal, she uses Bob’s fulfilled promise to fulfill her own promise; when Carol fulfills her promise, Alice gets her spreadsheet. A sophisticated distributed computation has been completed so simply that no one realizes an advanced degree in computer science should have been required.

Very cute.

While I presume that Tim’s interest in all of this is so he can make, break, keep, and delegate responsiblity for promises with greatly increased efficency I suspect that it’s really just about money.