A GARLAND OF PRECEPTS
by Phyllis McGinley

Though a seeker since my birth,
Here is all I’ve learned on earth,
This is the gist of what I know:
Give advice and buy a foe.
Random truths are all I find
Stuck like burs about my mind.
Salve a blister. Burn a letter.
Do not wash a cashmere sweater.
Tell a tale but seldom twice.
Give a stone before advice.

Pressed for rules and verities,
All I recollect are these:
Feed a cold to starve a fever.
Argue with no true believer.
Think-too-long is never-act.
Scratch a myth to find a fact.
Stich in time saves twenty stitches.
Give the rich, to please them, riches.
Give to love your hearth and hall.
But do not give advice at all.

On Fridays I usually go shopping downtown in Boston where we have one of the very few remaining vegtable markets. That shouldn’t be confused with farmers market, the stalls at Haymarket are run by people who’s only goal is to sell you vegtables; cheap, fast, and often crummy. It’s aggressively competitive down there.

My house is a first ring suburb. Such suburbs were made possibly by the steam train late in the 19th century. Since steam trains took a while to get up to speed and a while to bring to a stop these suburbs where, originally, surrounded by farms. The farms were driven out, progressivly, by the street car and then the automobile.

It’s a 15 mile drive from my house to Haymarket, or about $4.00 in gas round trip at current prices. It costs me another dollar to park at Haymarket, since there’s a deal. I easily save that much by buying my vegtables, sausage, cheese, etc. inside the competitive market. (Notice that rising gas prices should allow Walmart to raise prices. )

But I mostly do it because I love the dense, crowded, diverse, urban experiance. I like buying my sausage at the butcher with a cat. I like buying my pita bread from the eight year old son of the halal butcher who’s shop is in a basement. I like buying my green pepers from the amazingly old Italian lady, who unlike other vendors tends to sell only 2 or three things each week. And I always enjoy watching the tourists.

I’m a huge fan of cities. I really dislike the American enthusiasm for the rural. I agree with Steven Johnson’s comment: “I think the long tail premise has a tacit anti-urban bias to it, since it used to require big city scale to find obscure long tail books or albums that are now readily available to anyone with an Internet connection.” I’d go further and argue that all the vast majority of the happy long tail stories involve the emergence of a commercial entity that substitutes for an urban or civic institution; but the story tellers are carefully to remain blind to the risk that such private entities abuse the power that results.

Is the Internet good or bad for cities? Now there’s a question that merits some further work! I think it’s clear that cars, for example, were very bad for cities. Is the Internet better or worse than cars?

For example I find it facinating how the Internet can be a huge help in finding information about vendors and services in low density areas but if you look for similar information in the Bronx, Brooklyn, or down town Boston you find hardly anything. My hypothisis about that has been that vendors seek ways to connect with their customers and that if your issolated then the Internet provides a welcome way to create connections; but if your embedded in a dense urban area then the Internet is only yet another way to create connections.

Steve has a short article in Discover magazine that takes a look at this question. He talks a bit about how cities enable people to rondevous. So like my hypothisis it’s about connections. But he talks about things that benefit from physical connections. And he talks about how we might be able to build Internet systems, ala meet-up or dodge-ball, which have are synergistic with the city. What I found neat in the essay was the thought that there may well be a class of systems waiting to be built that strengthen and leverage urban density; systems that have little if any value for rural and suburban densities.

If such systems exist it would be a strong argument that the net can be good for cities.

Meanwhile the real mystery is what in the world was the guy in the suit buying six pounds of ginger root for?

This is a long rambling post about an authentication trick I’ve not observed used in the wild. But it’s analagous to two tricks often observed in the wild. This trick is a way to do authentication. It is a hybrid of the web bugs, used by firms to build models of user behavior, and the trick of creating personalized Ads. Like Amazon does for it’s donation buttons.

Here’s the scheme. All authentication schemes sooner or later work by having some third party vouch for the user. At that point there are always, at least, three parties in the game. The user who wishes to be authenticated. The site that wants to get to know him. And finally the third party that already knows the user and who the curious site also trusts.

Lots of third parties get used for this. Paypal has a trick where they satisfy their curiosity by depositing some pennies into your bank account and then you prove that it’s your account by telling them how many pennies they deposited. They also get you to reveal your bank account data as a bonus.

Google recently adopted the trick of sending a SMS message to your cell phone. As an added bonus they get you to reveal you cell phone #.

The most typical technique is to have you reveal your email address and then the curious site sends you an email and you prove that is in fact your email address.

The bank, the cell phone company, the email address provider are filling the role of third party that can vouch for you. Of course these are more or less trustworthy. Any third party with a ongoing relationship might fill this role. You library, you government, your ISP, Amazon, Yahoo, Google, your OS vendor.

So here’s the trick. Any of these could offer a service to curious sites. When you go to set up a new account the curious site could place a web bug or larger image on account set up page.

The trick involves what we put into that image. What if we put a one time pin into that image? The user then copies the pin from the image into the account sign up page. The site he’s signing up with then takes that data and queries the 3rd party site to setup the account.

Of course a firm like DoubleClick can offer fraud protection services without the bother of getting permission from the user, and they could use web bugs to do that. But the key thing here is that the user is explicitly in the loop, he is implicitly granting permission for the trusted third party to help with authenticating him as he sets up the account.

Notice one key thing. While in the examples above the users bank account, cell phone #, or email address was revealed to the curious site. In fact the higher the level of trust the 3rd party enjoys the more serious the bit of information revealed. This scheme breaks that pattern. The 3rd party doesn’t need to reveal anything beyond the fact that they know the user. They don’t have to give up any account data. The user can remain pretty close to anonymous. Of course if more information needs to be revealed that can be arranged.

This scheme is slightly analagous to OpenID. In that system users are prompted by the curious site for their OpenID url. The site then uses that to fetch a page of info about them, and on that page is a pointer to a 3rd party that can vouch for them (well vouch that they control the page in quesiton). But actually this is quite different because the OpenID design forces the user to reveal a universal identifier, i.e. his OpenID url. While this system requires only that the user admit he has a relationship with the the trusted third party.

This is also analagous to the common scarab systems where a site places a branded scarab on their page and the user is encouraged to click on it to authenticate. These scarabs don’t need to be web bugs and usually aren’t. So unlike the Amazon donation scheme only the third party’s brand appear on them and nothing showing how the third party recognizes this users.

Scarab schemes didn’t gain traction in the market. The curious sites hated them because the threatened their customer relationships. The scarab vendors, like Passport, looked like they would stick their nose into the middle of the relationship. One term used for that entanglement is “account linking” the authentication site and the curious site both have account relationships with the user and part of the design for most of these systems involved linking these accounts. Another way to describe the fear that the scarab vendors would intrude on the the relationship of the curious sites is to say that they feared that one account would become subordinate to the other one. For example that before the user could get to his eBay account he would need to pass thru his dominate Passport acount.

The scheme outline here involves no account linking at all. The in this scheme the trusted third party X is only providing a single service – a means for the user to prove to the curious site that he has a relationship with X. That’s it. That’s less likely to threaten the curious sites.

The point of all that is that we reduce the threat to the user and the curious site.

This is also analagous to the capcha schemes. They present a puzzle to the user that by solving increases the site’s confidence that the user is a human. In this case we are asking the user to prove he has a sufficently high quality relationship with a third party site. Since such relationships are, presumably, difficult to obtain – i.e. they take time.

While there are two things I like about this scheme – very little is revealed about the user and no long term account linking is done – it is tempting to do a modicum of durable linking.

After the user enters the pin presented to him the curious site then queries the trusted site to see if that pin is valid. The trusted site can reply yes, no, or it might send back something more complex. Anything more complex implies either more revealing or more linking.

If the third party site hands back a token representing the user that allows further transactions about that user. For example if the curious site uses this to prevent spam his blog that token could be used later to report a spam event back to the trust site. that seems like a fine use. Of course it could also be used to send back more private or slanderous info about the user.

Tokens like the one in that example are common in account linking designs. They denote the linking.

Meanwhile if you suffered thru this entire thing I’m amazed! But here’s an amusing variation on this idea. How about a scheme were you can only comment on a blog if you make a small donation to one of the set of charities selected by the blog’s operator.

BlogDay posting instructions:

• Find 5 new Blogs that you find interesting
• Notify the 5 bloggers that you are recommending on them on BlogDay 2005
• Write a short description of the Blogs and place a a link to the recommended Blogs
• Post the BlogDay Post (on August 31st) and
• Add the BlogDay tag using this link: http://technorati.com/tag/BlogDay2005 and a link to BlogDay web site at http://www.blogday.org

Artist trading cards are the size of a baseball card. Artists make them to trade with each other. Some are just amazing. The one at right is one of my wife‘s. It was sent to the other side of the planet. One of these, is coming back.

You know, this Internet thing is pretty neat.

update: HA! These are just like QSL cards. I bet old ham radio guys grumble “been there, done that” just as much as old Lisp guys.

This is a very cool example of how peer to peer systems might displace centralized hubs.

Most email today is stored on centralized servers; i.e. hotmail, gmail, the firm’s email servers, etc. Mail, i.e. the post office, is one of the oldest centralized services. The network effects are strong and they tend toward creating a monopoly.

ePOST is a serverless peer to peer email system. Download it, backup your copy, fire it up, setup your email client, exchange mail. Later blow up your computer (this step is optional). Pull the backup, fire it up, and magic! All your mail reappears.

Your copy is part of a peer to peer swarm that is collaborating to exchange encrypted mail and store all the mail redundently amoung the members.

The hub is unnecessary. Hotmail? gmail? AOL? Uneccessary!

Now to be honest this system is clearly a first generation; a proof that these hub killers are possible.

Christenson’s last book on innovation put a model into my mind about how innovation proceeds. It’s kind of systolic. On one hand innovation proceeds by cobbling together out of parts at hand solutions to problems at hand. These solutions are, well, Rube Goldberg like. But they are cool because the solve problems that weren’t solved before; which makes them valuable. Time passes and these solutions are refined. And then, enough knowledge is accumulated that the modular boundries in the solutions become apparent. These modules are then broken out and fall out as peices. The peices can then engender another round of problem solving.

A system like ePOST feels like one of the highly integrated systems of the first phase. At the same time it’s designers are part of a storm of activity going on in the peer to peer community to find the modular boundries so the component parts can be distilled out.

If you pop the lid on ePost you find first, second, third drafts of a lot of these modules.

• Peers self assign a place around a ring of integers; that ring is the swarm.
• FreePastry – allows one peer to send messages the peer nearest any point on the ring.
• PAST – allows distributed, reliable, key/value hash table lookups in the swarm.
• POST – allows encrypted objects storage in the swarm along with encrypted user to user messaging.
• ePOST – build mail on POST
• Glacier – provides durable storage so that huge percentages of the swarm die you can still recover all your data.

There is the most marvalous amount of research going on around all these modules these days. ePOST is a beautiful example of how what is becoming possible. This work on peer to peer DNS lookups is another.

I got an Onion of the day calendar for Christmas and this gem from 2002 came up recently.

U.S. Middlemen Demand Protection From Being Cut Out

WASHINGTON, DC-Some 20,000 members of the Association of American Middlemen marched on the National Mall Monday, demanding protection from such out-cutting shopping options as online purchasing, factory-direct catalogs, and outlet malls. “Each year in this country, thousands of hard-working middlemen are cut out,” said Pete Hume, a Euclid, OH, waterbed retailer. “No one seems to care that our livelihood is being taken away from us.” Hume said the AAM is eager to work with legislators to find alternate means of passing the savings on to you.

The classic paper by Saltzerz, Reed, and Clark End-to-End Arguments in System Design which gave rise to the stupid network is principally about how to expend your design resources. It argues that your communication subsystem needn’t address a range of seemly necessary functions: bit recovery, encryption, message duplication, system crash recovery, delivery confirmation, etc. etc. This is a relief for the system designer, he can ship earlier.

The end-to-end principle drove a lot of design thinking for the Internet. For example DNS, the mapping of names to IP addresses, is layered above UDP, which is above IP. The end-to-end principle drove DNS up the stack like that.

The designer in the thrall of the end-to-end principle strives to leave problems unsolved. That makes it a kind of lazy evaluation technique. Leaving problems for later increases the chances the will get solved by the end users rather than by the system designer. It pushes the locus of problem solving toward the periphery. It creates option spaces for third party search, innovation, etc.

It is possible to look on the design principle as shifting risk. By leaving the problem resolution to later the design is relieved of the risk that he will screw it up. While users might prefer to have their problems solved by some central authority they do get a bundle of benefits if the problem is handed off to them. These benefits are otherside of the coin of agency risks.

The end-to-end principle is always about managing the risk associated with agency. The Internet’s designers were well aware that they were attempting to create a communication subsystem that would remain open, robust, and hard to capture. Those goals were complementary with designing a system that could survive in battle.

When ever you clear the fog around one of these communication or distribution networks you find a power-law distribution. I.e. you find hubs. I.e. you find middlemen. I.e. you discover the risks of agency.

I don’t think that the original designers of the Internet expected to see the concentration of power we see in Internet traffic, domain name service, email, instant messaging, etc. etc. Nor do I suspect they expected to see the concentration of power that the internet has triggered in the industries that are moving on top of it; i.e. a single auction hub, a handful of payment hubs, a single world wide VOIP hub, an handful of book distributors, a handful of music distributors, one browser, one server, etc. etc.

Nothing in the end to end principle actually frustrates that outcome. It argues that there are a collection of reasons why a middleman, i.e. designer of a distribution/communication cloud, might find it advantagous to limit what functions he preforms in his role as intermediary. It doesn’t argue that intermediaries shouldn’t exist. The middlemen in the Onion piece are not being displaced by other middlemen. Middlemen rarely disappear completely.

This is why it is an ongoing effort to keep the network open. While we have a bag of tricks for shifting problems toward the edges and out of the center we seem largely at sea about how to control the degree to which hubs condense on the layers above us.

Network Applications of Bloom Filters: A Survey (pdf) by Broder and Mitzenmacher is a fun read.

Bloom filters were first invented to solve a problem near and dear to my heart, spelling correction. While entire dictionary of every known word is a very large beast the bloom filter of a dictionary isn’t. It allow you to ask the question “Is this in the dictionary?” using a cleverly arranged bitmap. The down side is that occationally says yes when the answer is no.

Bloom filters are simple. You start with a bit map, all zeros, and then for each symbol in your dictionary you hash it and set a bit in the map. We aren’t done yet, but clearly you can hash your spelling word and see if it’s bit is set. Instead of one bit you actually hash the symbol a dozen times different ways, and set a the dozen bits. If your bit map is large enough then it becomes quite unlikely that a misspelled word will have all 12 bits set. The math to get the right size bit map and number of hashs isn’t too terrible, you can look it up.

The paper mentions a cool varient of a Bloom filter that solves the problem of how to find hot-spots. While this problem comes up a lot but I’ve never seen a satisfactory method before. The hot-spot problem comes up a lot, it is the problem of finding the elite in your power-law distribution. It’s hard because you can’t afford anything that’s the scale of the whole population. Here three example applications. You want to notice that a particular sensor was noticing a large number of small earthquakes. You want to dynamically shift a page into a caching network when it suddenly gets slash dotted. You want dynamicly limit the bandwidth of a client that starts downloading your entire site. Let’s generalize the problem: given a stream of symbols how can you pluck out the ones that are appearing in the stream a lot.

The trick mentioned in the paper was, I gather, outlined in this other paper, New Directions in Traffic Measurement and Accounting (citeseer). Instead of a bit map use counters. Each time a symbol arrives we increment all it’s associated counters. As we do that we assemble the minimum over the counters; that’s a good estimate of how many times we have seen that symbol.

Tada! Symbols with high counters can then be put on a list of likely to be hot. You can then start filtering, caching, watching more carefully, etc. that guy.

This could be used to add a possibly-hot? operator to lots of server data: pages, ip addresses, users, etc. which could then be fed into the server’s operating logic. You could use it to find newly emerging high volume pinger’s in feedmesh. You could use it to find phrases hot new phrases in netnews, newswire, stock trading forums, or to notice sensors in your sensor net that seem to be all a quiver.

ps. There is a cute trick for avoiding the need to have a dozen hash functions. If you only have two h1(x) and h2(x) you can make hi(x) by combining the output of h1 and h2, say by doing h1(x) + i*h1(x). (pdf).