Category Archives: standards

Newton’s immovable installed base

Here’s a fine example of the tensions between an irresistible force v.s. an immovable object.

At some point in my childhood my father, presumably in an attempt to keep me from wasting a summer in idle pleasures, got me an unpaid job working with a locksmith. I really enjoyed it, though I never did get the hang of picking locks. One thing I loved about the job was all the paraphernalia. One of the principle artifacts in every lock smiths tool kit is a box of pins. These pins are tiny bits of brass all of various lengths. They were color coded so you could put them back in the case.

These pins are packed into the lock so that when the right key is slide in they align just right and the lock will turn. Sweet little springs push the pins back into place when the key is removed. Each spring sits in a hole and the hole has two pins whose length sum up to fill its column just right. A lock with a master key will have three pins in one or more of the columns.

Locks of varing sophistication modify this design by having the columns oriented in various patterns. The typical lock just has the pin-columns in a straight line. If you look at your key ring you’ll probably find at least one key who’s bumpy bits are set up in some tricky way. Complex topology makes it harder to pick the lock; or at least that was the idea.

The design patterns for key-and-pin locks form a the plaform for a huge installed base of locks and keys. So it’s a great standards story and like all standards used for security things get messy when a security flaw is revealed. The usual exemplar of that is Microsoft Windows, which was never really designed to be secure and now sustains the vast cyber-crime industry (said to be larger than the drug trade).

You can’t ‘just fix’ a system like this because the installed base is very slow to move. As Bill Gates is rumored to have said back in the 1990s, “My biggest competitors is old versions of Windows operating system.” Users don’t upgrade quickly.

Over the last year or two knowledge of a huge security flaw in the key-and-pin lock design pattern has been revealed. There is a fun video (with subtitles) from a Dutch TV show you can watch (WMV) and a paper about it (pdf).

It’s easy to understand though. The common name for the technique is bump key. You make a key that bumps the pins. Well, actually, it taps the pins sharply. The sharp tap is then transmitted thru the stack of pins until it reaches the top most pin. That pin then floats up and way from the rest of the stack. At the moment the gap appears you turn the lock. All you need is a good bump key, a sharp tap, and to time the turn to the right moment.

You have seen this dynamics in one of those executive desk top toys (these are known as Newton’s cradle) where a group of balls hang in a line and you drop one ball one end and ball on the other end floats up.

Designing around this problem is, I presume not too hard. For example, since only the top most pin will float up when tapped you need to assure it’s movement won’t open the lock. That’s not too hard since you can arrange to have the top pin above point where the lock turns. In some cases you might even be able to repin an existing lock to prevent the problem. In other cases you probably have to redesign the locks.

There are techniques for moving a large installed base. Firms, like Microsoft, that depend on upgrade revenue are very practiced at these. Moving an installed base can be very profitable. Rekeying the entire planet, changing every lock in every door, replacing the keys on everybody’s key ring – wow! The lock industry ought to be very excited about this. I bet there is quite a backlog of key-and-pin patents piling up at the patent office right now.

Of course, the profits to be made from migrating the installed base are not the first thing most people think of when they hear this story. But then, most people don’t tend to think of Microsoft’s security problems as an upgrade driver either.

Stick-up — Part II

Philip Jacob asks a question about a recent spec out of Microsoft. In particular what are his rights and responsibilities if he decides to use or implement it.

Let me take a stab at that.

There is a scenario in standardization I call the stick-up. The standard is written. The standard becomes widely adopted. Then an IP-rights holder shows up and announces he is going to be collecting a tax on the entire installed base. The stick-up artist can be anybody from one of the original advocates of the standard thru to complete strangers.

When the stick-up occurs the artist fears the standard community forming common cause and coming after him. Because that community will, by then, have an immovable installed base they will find it easy to agree that a coordinated response is in their best interest. For this reason the stick-up artist will tend to approach and negotiate only with a selected subset of the entire community. Often swearing them to secrecy.

Microsoft consistently reserves the right to be that stick-up artist. This is not entirely insane. Market power may let them get away with it; e.g. setting standards without them is hard and historically they have been able to set standards without reference to any other market participants. Some ethical frameworks hold that they have a fiduciary responsibility to play the game this way. Strategically important for them is that they are near certain candidate for targeting by a some other stick-up artist – there is a kind of mutually assured destruction quality to their approach in these matters. In most firms this section of the strategic policy manual is write once; for quite a few reasons.

Microsoft hasn’t admitted that their standard setting powers are tremendously reduced from a decade ago. The policy they currently use reflects that. It makes them an exceptionally poor partner in standards setting. It does serious damage to the short term chance than any of their proposals will gain any real momentum.

Firms don’t adapt fast unless they are really really scared and they have really really strong leadership. Microsoft has often demonstrated the ability to do that. You don’t survive for long one up from Moore’s law and friends unless you can pull that off! In the absence of fear and leadership, as we see here, firms try to adapt in the usual way, e.g. process improvement or turning up the knob on existing behaviors. The use of one of the creative commons licenses on the text of the spec; as they did in the case Philip is curious about is a bit of both. The existing behavior is PR smoke and mirrors. The process improvement is a bit of dabbling in the techniques of the other side; but it’s just a toe in the water.

Given the abundance of options available for technologists today I can’t quite see why anybody would decide to be an early mover on a standard advocated by Microsoft under terms like this.

Folktales, Folksonomies, and Swallowing the Sea.

I’m a sucker for  grand explain everything attempts to categorize it all.  I find them extremely amusing. Here for example is a five node template that attempts to encompass all folktales. (You can also use this for presentations to your angel investor.)

There is a much more complex template that aspires to the same goal, see here.

The awe inspiring attempt to categorize the universe of folktales is the six volume work by Stith Thompson (who’s name is suspiciously similar to Sith); the Motif-index of Folktales.

The motif-index to distills out the ingredients of all folk tales. “Identity test based on [glass] slipper fitting.” Folktale ingredients is a finite set; at least they are if you limit your self to those Stith could read in a lifetime. Reading some sections makes it clear that’s a fools errand. For example, kinds of fools: “Stupid Wife,” “Extravagant Wife,” “Stupid Husband,” “Stupid Village,” … “Foolish Brahmin,” … “Foolish city dwellers,” “cowardly fools,” … “bungling fools.” Clearly this set is innumerable. 

Many entries in the motif-index are sufficient to let you visualize an entire story. “Fool sent to acquire two 15 year old slaves returns with on 30 year old slave.”

Some of the short comings are quite telling. For example, there are only a handful of entries on slander. Recall Karl Rove’s motto: recovering from slander is like recovering water spilt on dry earth. Other holes arise because Thompson didn’t have access to all the folktales out there. So, others have tried to fill in the gaps. For example: “Sparks of burning cannibal woman become mosquitoes“, has a Native American origin.

Like accounting codes (Horses 10101506), or the Dewey Decimal system (Composting 631.875), the entries in this grand classification have identifiers (F952.1 – Blindness cured by tears).

Systems like these always have an owner, the authority. Thompson’s system evolved from a system designed by Aarne. Aarne’s system was an index to stories; where Thompson’s system finer grain – story elements. I’m not sure I have the history straight, but it appears that some effort was made to update Aarne’s system creating the Aarne-Thompson index of story lines (AT 762 – Woman with three hundred and sixty-five children.).

Thompson failed to establish a clear line of succession for who’d own his system when he died. So if you want to update it there’s nobody to talk too. Which is a problem. As we know, the semantic web wishes to encompass all these systems. (D???. magic helper who can swallow sea).

One might imagine going to the IETF and proposing a new protocol – folktale. (folktale:/AT/1384 – Man sets forth to find three others as foolish as his wife.). You might complain that such a thing wouldn’t have an obvious protocol. But then I’d point out that are already approved URI “protocols” that don’t have what most practitioners would think of as protocols (domain names, ports, etc.) tel:, info:, and tag: are three examples.

The puzzle with these kinds of identifiers is how to bridge from the carefully designed household which is their home into less rule bound global space of URIs. The tel protocol (tel:+1-416-395-5400 Dial-a-story at the Toronto Library) is a exemplar of that process; since there is a highly regulated and extremely complex universe of phone numbers. Phone numbers have high stakes politics, commercial players, property rights, etc. etc.

The tag URI is almost all the way over to the other side of things. Though all tags URI’s have an authority, in this case a domain name or an email address, they are primarily designed to be extremely light weight to create. I like that the tag RFC uses the term mint for the act of creating a tag; many of these identifiers have all the properties of property rights or currency. While I guess there isn’t anything to prevent an authority from announcing that he is governing his tags with great pomp and circumstance that’s not the typical behavior.

Info urls come closest to being a generalized scheme for bridging from the URI identifier space into the identifier spaces ruled by others. For example the info URI: info:ofi/enc:UTF-8 denotes something in the OFI world, i.e. the National Information Standards Organization’s OpenURL Framework. Messy, but necessary, these bridges.

While I still think my favorite entry in the motif index entry is “Fool mistakes pumpkin for asses egg.” I now know that the UPSPC code for asses is 10101509 and that pumpkins don’t have a code.

Universal Demand for Quality

I’ve been reading some books about professionalism. The one I was reading yesterday touched on an interesting model that I’m not quite sure what to make of, but it certainly caught my fancy.

Some commodities have universal demand; i.e. everybody wants some. Some examples: food, education, knowledge, safety, health, mobility, conflict resolution. States naturally are drawn into providing a regulatory function for these industries. A state that fails in these areas finds its legitimacy at risk. Their universal demand assures a strong signal from the citizens to the state, particularly in any functional democratic state.

Meanwhile, universal demand tends to attract numerous suppliers; and in the absence of barrier to entry too many suppliers. Which will lead in short order to market failure if the quality of the goods supplied is hard to measure dependably. The market fails because the horde of suppliers furiously underbid each other until they can’t make a reasonable living, which drives all the competent suppliers to seek other work.

The lack of clear quality measures leads the substitution of alternate sources of legitmacy: pomp, pompus attitude, parasiting on other sources of authority, advertising, character defamation. (A point which deserves a blog posting of it’s own, but since that’s unlikely I’ll toss in this marvelous line. When this happens you see a pattern: consumers hold the trade in very low esteem but hold their personal practitioner in the highest regard. Where have I heard that before?)

It is practically impossible for most buyers to evaluate the quality of what they are buying. If you can’t tell from the plate on the table in the resturant if the kitchen is or isn’t a public health nightmare there is no chance you can evaluate the quality of your teachers, lawyers, groceries, or the city’s levee.

(Oh no, another aside: The library I was reading this book in has taken to using the fire alarm to annouce that the library is closing. It’s a nightmare waiting to happen – the fire that breaks out at 20 minutes before closing time.)

So that’s the story. A commodity with strong demand whose quality isn’t transparently obvious can easily engage in a rush for the bottom, a market failure. If the demand is universal the state will find the pressure to respond irresistible. And so the state will step in to regulate.

As early as the 16th century some European states established regulatory mechanisms for medical providers. Now that’s a great example because it looks to me like those states picked, more or less at random, one class of medical hucksters declared them legit, and declared the others ill-legit. They had to do something.

What I find thought provoking is how granting the state license (the franchise, the monopoly) to one group is a new kind of standards making I’d not recognized before. In the presence of 16th century medical science (i.e. a something totally bogus) and complete market failure (i.e. doctors and barbers sharing the same wages) the state has a chicken and egg problem. No quality, and no market. By tagging one group as responsible it solves the market failure. Wait a few centuries with luck the might science emerges.

If the practitioners don’t capturing the regulator and the regulator keeps demanding that the practitioners address the quality problem this can work. There is some hint that is exactly what happened with the medical profession. That time and time again the profession failed to provide reasonably quality; for example Doctors were very slow to adopt ideas about public health, hygiene, etc. If you want to be nice you could say they were very loyal to their professional practices. Forces would come to bear that would force them to change; in the absence of the professional monopoly there wouldn’t have been anything for those forces to bear down upon.

Building Houses on Sand

Philip Jacob worries, and rightly so, if this current enthusiasm for building websites that rollup bits and pieces isn’t, well, insanely risky.

“…would you incorporate a library in your application that was licensed under terms like this?

  • Required you to read and understand several pages of legalese
  • Is free right now although you might be charged for it at some point in the future at a price that you cannot negotiate in advance
  • Can stop working at any time for any reason without notice
  • Can undergo functional changes at any time without notice
  • Can be rate-limited at any time without notice
  • Does not have any service level agreement
  • Places restrictions on the data you generate using the library, some of which are bizarrely techno-legalistic and open to interpretation”

The licenses of all these fun web APIs are mavalously diverse. It is clear that each firm’s lawyer plays the role of genetic engineer gene splicing together a handful of licenses to create yet another mutation. While in long run (after you’re dead) the ecology will put most of these out of their misery. License interop is hard. This is another example of “standards as a substitute for lawyers.” Phil would like to have some standard licenses; but

We know how this plays out. First we get a bloom of diversity. Then we get consolidation. In the end we get a power-law distribution across some number of licenses. In the final end game the elite members of that distribution engage in a long standards battle who’s outcome can not be predicted – they might make peace, they might split into seperate markets, one might win, etc.

In early phases the smart players move quickly to capture market share that will provide key negotiating power in the later stages. Some of the early movers have to be profitable, others don’t. Some players value a strong position in the future more highly than others. Some players discount risk of having to switching later.

There are adjacent disputed territories to consider. Consider four of the larger players. The communication companies (though they haven’t fully accepted their fate yet) lock in users with physical networks and charge for bandwidth by the month. Companies like AOL, Yahoo, Google charge via advertising and lock-in by capturing a large distribution bottleneck. Platform vendors charge for a stream of updates and lock in via APIs and licenses. Players like the FSF or the ASF are sort of inside out with their focus on freeing developers (their principle consituency) from the licensing/API lock-in points.

All this is just part of the passion play around the emerging Internet OS, and to think I used to believe this was all kind of simple.

Barndoor standards

Javascript is a perfect example of a syndrome in standardization that keeps CIO’s up at night. Let’s call it the barndoor syndrome; after closing folksy saying: “Shutting the barndoor after the horse escapes.”

Javascript escaped into the wild before it was standardized; it then underwent very rapid mutation in the installed base. Three forces drove this rapid emergance of new species: the security nightmare, the fun everbody was having, and high stakes competition. The last is particularly corrosive to the collaboration necessary for standards making. This family of species are now all over the installed base, and as we all know installed bases are very hard to move.

The poor web site designer is stuck with a miserable choice. He can antagonize large numbers of users; or he can make himself miserable. It’s a kind of quanity/quality trade off.

The standard(s) for Javascript aren’t a foundation for innovation; they are more like a beacon in the night toward which their authors hope the installed base slowly migrates.

When learning the language the standard is only a point against which you can measure the distance, in units of exceptions possibly, you must travel to reach this or that subpopulation of the installed base.

Driving the horse out of the barn is very tempting, since it builds momenteum and helps you search for the best design informed by actual use rather than ivory tower mumbling (i.e. security architectures). So we could rename this syndrome entrepeurial standards making rather than barndoor standards making.

When small entrepeurial firms do this it’s reasonably ethical; how else are they going to get traction in the market. When large monopolist firms do it the ethics are much more muddy. Which is something to think about when reading people’s critiques of Microsoft’s infocard. It is of course irrelevant to Microsoft if their designs go thru a legitimate standards process; just as long as it wins in the marketplace. Microsoft has cleverly attempted to substitute for a real standard process a conversation in among bloggers. The technorati are one audience you need to convince before a standard will gain great  momentum, but they are not a substitute for real legitimate standards making.  Assuming that you lack  sufficient  market power to just command it’s success.

Another Hub for Verisign

The Verisign acquisition of the original hub for blog pings ought to deeply concern the high volume ping producers and consumers around Feedmesh. Verisign knows hubs. It has exceptionally large market share of two of the key Internet hubs DNS, and SSL key signing. They know how to encourage further consolidation of those hubs. They are willing to agressively go after revenue, such as selling advertising on every misspelling of a domain name under their stewardship. They are very active in the supply chain RFID space, where we can expect to soon see a bloom of blog-ping like traffic.

The most central struggle in Internet systems is between the architectures dependent on a central authority and those known collectively as end-to-end; i.e. those where the ownership is left with those on the periphery. That choice is not a boolean; there are plenty of architectures that leave large midsized players in the system.

The central authority advocates always emphasis the same things: reliablity, policing, simpler. It’s always simpler to just ask the landlord, and sometimes that works. So Verisign writes about reliablity: “Those days have passed at least for the popular ping servers; pings are well on their way to requiring serious infrastructure. That’s where VeriSign comes in.” Policeman: “there are an enormous number of splogs out there, and the number is growing faster than the number of real blogs.”

The key balance between central and periphery designs is around who owns the rights to innovate an who captures the revenue from those. So Verisign writes: “… a host of new opportunities for delivering network services in a user-friendly (and often user-powered) way. In order for that to happen though, there’s a lot of work to be done underneath the application layer.” That’s the Microsoft slogan “Your potention. Our passion.” Which I can’t resist reframing as: “You take the risks, we’ll take the profit.”

It’s a shame that Dave Winer’s failed in his aspiration to solve the scaling problems without an acquision. The complexities of collabrative open institution building aren’t really his strong suit; he’s a different kind of entrepreneur.

The struggle between concentration toward the center and diffusion toward the periphery isn’t just driven by technology. We make the bed we sleep in.

Standards and the Information Gap

Between buyers and sellers there is always an information gap. The buyer can not know all that the seller knows about the good he is buying. For example when you buy a car you can’t know if that car was used to transport drugs and is now covered in a light dust of narcotics.  Meanwhile, the seller can never know how much value you might extract from the goods after you purchase them.  How is the seller to know that you have coveted that car since childhood?  This information gap makes price setting difficult and risky. If you highly value the thing your buying the seller would raise the price. If you knew more about the history of the good you might be less willing to pay. This is important: adding information makes it less likely a deal can be struck.

Standards are often used to help reduce the information gap. In some cases standards are set that force one or the other side to reveal information. Cars, for example, have a record kept of significant accidents and buyers can query that data to reduce their uncertainty. Certification and government regulation are two other examples. The public health inspector’s job is to reduce the information gap for restaurants; he does what the dinner would rather not have to. In many of these cases both sides of the transaction would prefer to have the regulation, certification, etc. The rules make transactions flow where they might not have otherwise. Sometimes middlemen play role. Restaurant reviewers are an example of that; assuming they aren’t sock puppets.

Models like this one explain why a car looses so much value the moment it leaves the lot of the new car dealer. At that moment the buyer is forced to assume the worst about the car. This is compounded by the suspicion that private sellers only dispose of cars for negative reasons.

Firms often attempt to create certifications, standards, regulations in the hope reducing the information gap, increasing the number of transactions, and raising prices. A good example of that is the way that all the new car companies have a “certified used car” program of some kind. The idea being that this certification reduces the buyer’s uncertainty about the car and allows the dealer to sell the car for more.

Jay Levitt bought a certified Audi recently. Turns out, to hear him tell it, that Audi’s Certification program is a joke. Maybe they are using one of these online make your own certificate sites.

Life Lessons

Leo Simons seems to be having exactly the experiance that pushed me out of Ada and into Lisp 22 years ago.

Java in many ways is kind-of a joke (yes yes its great for some stuff). On the surface its this really type-safe, compiled, predictable language everyone is using for everything. When you dig a little deeper and look at what actually is going on in “real life”, you’ll see that there’s usually some hack to get rid of all that type safety and predictability. For example, you generate source code based on XML, or you generate object code based on XML. And of course we don’t stop there. Since its kind-of hard to sensibly specify chunks of code that are bigger than a “class” (hint: other languages have things like “modules”), we use some huge application library, which we of course configure using more XML. Nevermind that we need to load 500 megs of jars into memory to make all that happen.

Programming languages ought to allow the designer to craft his notations, type models, and execution models so they fit the problem. Not demand that the problem be force fed into the mold handed down by somebody who hadn’t a clue what your problem requires.

Problem is that if you decide to make a switch your forced to write off a vast sunk cost, a network of relationships and a fluent skill set. At the same time your stuck deciding where to jump; and you can not know the color of the grass until your living right on top of it.