Ascription is an Anathema to any Enthusiasm

I believe it was Ray Kurzweil, circa 1989, who advised encouraging a private jargon inside your new company.  I remember because that was just about the time I was starting to think open would totally trump closed in our industry.  The advise seemed to my ears a bit old fashion.  But at the same time I suspected he meant that it was a good way to tighten the bonds inside the team.  By then I knew enough about cults to recognise that’s common inside cults; it supports two of the keys to running a good cult - information control, and thought stopping processes.

But don’t take any of that too seriously.  It is good advise none the less.  I’ll admit to being a Whorfarian, the words you highlight effect your thinking.

These days I’d add to all that though.  Language is the ultimate exchange standard.  So when you decide to innovate a new private language your cutting your self off and creating friction or trade barriers with your outside partners.   Importantly the advantage a new group has is that they can pick and choose what to emphasis.  They can take a run at leveraging some particular competitive advantage.  As Dave Winer says: You can’t win by zigging when he zigs. You have to zag to beat him.  Ray’s advise can be viewed as a bit of implementation advise for that.

So it was with some interest that I saw Google revealing their in house standard for serailizing data.  It’s not hard to see that Protocol Buffers are alternative to XML.  And it is amusing to, at least to me, to think that they did this in the hope of reducing the frictions that occur when they must translate from their in house argot into the dialects used by the outside world.  It’s fun to note that if your start up is as successful as Google you get to promulgate your private jargon.  It is one of the spoils of war.  You can push that friction into your compliments, make them pay the switching costs.

Protocol buffers aren’t anything special: messages are lists of key value pairs, keys can repeat, a small set of value types; including unicode strings and recursively other messages. They are very practical, close to the metal.  Choices were made and they are what they are.  They are quite dense, and easy to parse.  Many messages can be serialized in one pass.  In classic run length encoding style nested structures have their size mentioned in their header.  That makes emitting one pass serialization hard.

Given an array of bytes it wouldn’t be child play to guess that your holding a protocol buffer, you could do it huristically but it would still be a guess.   You need a protocol buffer’s declaration to parse it.  For example you absent the declaration you can’t know if a you’ve got a sint32 or an int64, etc.  All that disappointed me.  It disapointed my inner archivist and the inner peeping tom (who has often debugged tough problems by watching the bytes fly by on the wire).

There is a nice detail that allows optional keys which in turn makes it somewhat easier to add new message variants.  With luck the old message handlers can just ignore the additions.  It made me smile to note that mechanism can be used to pad messages; which in turn makes it more likely that you can serialize in a single pass.

There is another nice detail that allows a key to appear more than once in spite of the metadat saying it is single valued.  The last occurance wins.  This lets you implement a kind of inheritance/defaulting.  For example if your implementing CSS style sheets you read the default style message, and then read the variations from the default, and your ready to go.  They call that merging.

Given the declarative information for a given protocol buffer it’s not hard to convert it to XML or what every else you like.  The observers and archivists will just have to be careful to not loose that metadata; and some of them will no doubt build clever hueristic to cobble together substitute metadata.  Interestingly, inspite of efforts to the contrary, you can’t really work with XML without additional metadata to help.  And, that stuff is horribly complex.

As I like to emphasis what really matters with an exchange standard is how many transactions per second move over it.  No doubt this one has critical mass at least inside the Google cloud.  What matters here for how important this standard might be is how much adoption by non-Google actors.    But, I suspect we will be speaking this dialect more often and for quite a while.  Of course, the rudest way to say that is that we will be chasing their tail lights.  But I don’t really feel that way since I’ve never particularly liked XML and I welcome a substitute.

I’m not particularly proud of the neologism “upgrade forcer.”  It encourages a bad behavior.  Product managers can be a desperate lot, particularly when their bonus is riding on how many copies of the upgrade get sold.  When times are good sweet new product features will draw users to upgrade.  But, as products mature the customers grow content and convincing them to upgrade get’s harder.  Having run out of carrots the product managers are tempted to turn to sticks.

Installed bases are hard to move.  Installed bases without a clear owner, or product manager, are even harder move. You can chat up how nice it would be if we all switched to IP6, but nice isn’t must.  It would be nice if my correspondents encrypted their email; but little drives that upgrade.  Effective upgrade drivers engineer a situation where users move quickly to upgrade.   Y2K was an effective upgrade driver, 1999 was a very good year for upgrade revenue.  I’ve a pet theory that the late 90s high tech bubble owns a debt to that.

One of the open many open standards in the Internet menagerie that badly needs an upgrade is DNS.  DNS is an amazing design for it’s time; but one of it’s failings is security.  It has serious design flaws, and numerous vulnerabilities.  For example you ISP, who your probably should not trust, can trivially intercept DNS queries and inject what ever answers he thinks serves his purposes.  The vulnerabilities make that even worse, since at least you can complain, negotiate, even sue if you catch your ISP playing those games.  But if some evil dude poisons one of the DNS servers your happen to use and your email, IM, or bank traffic is intercepted your unlikely to have much recourse.

Security flaws play an interesting role in driving upgrades.  The product manager can use them to threaten the users, while at least nominally not using force himself.

For years people have been attempting to redesign DNS to add better security.  At first blush it seemed straight forward, but it turned out to be way hard.  My sense is that people now think they have the design problem under control.  So the next step is getting the installed base to move.  Getting a possibly immovable installed base to move generally requires an irresistible force.  Some compelling value or something bad.  There is plenty of bad already, though as is usually the case the immovable base finds it easy to avert their gaze from those horrors.

This posting was triggered by a comment in yesterday’s announcement of yet another really bad flaw in DNS.

There is a update to the DNS standard known as secure dns, or DNSSEC that addresses this problem.  But most people see it as nice to have rather than as a must have.

With luck that changed today.  Yesterday the existence of a really really bad flaw in the the DNS protocol was publicly revealed.  The actual flaw’s details were not revealed, but a massive software upgrade to temper the risk is being rolled out.  But, this line in caught my attention.

“DNSSEC is the only definitive solution for this issue.”

So maybe, just maybe, we have found a upgrade forcer for DNS.  This is extremely good news if your a DNS vendor of any kind.  Profit!  For those who are driven by fun, rather than greed, fixing DNS would allow us to use it safely for a much larger range of light weight database functions.

I have sighted a new urban myth: Electric heating is cheaper than oil heat! Here in Boston people heat with both gas and oil, and the cost per unit of heat between the two has diverged rapidly over the last few years. Those who heat with oil are looking for ways out of their plight. Apparently the rumor making the rounds that it is cheaper to use electric. That’s not true.

In related news Martin brings my attention to a company EnerNoc that sells negative energy, i.e. load shedding, to the utilities. They use telecom and widgets to shift power consumption from high demand time periods into low demand time periods. Martian’s example is the fridge. You chill when power is plentiful and let it coast when others are paying higher prices.

I assume that EnerNoc’s role in all this is to aggregate small power users into a large enough pool to be worthy of selling to the utilities. It’s a interesting example of a coordination problem. There are of course other ways to approach the problem; ones that are less dependent on a thicket of contracts and ongoing coordination signals controlled by a middleman and enabled, as Martian, points out by the telecom infrastructure.

The obvious alternative is to just broadcast signal; and let the demand side react to the signal by selling some simple technology that responds to the signal in reasonably simple ways. That alone would enable substantial contributions from the demand side. But you can improve the incentive structure either thru regulation or by using statistical sampling to tell which customers have gotten with program; and then reduce their tariffs.

The amount of signal that needs to flow from the grid operators to the consumers is small, in the sense that you can broadcast it. A signal only needs to flow back the other way sufficient to assure that the incentives play out right. It is stupid to presume that the only incentives that are available are monetary or that they need to be executed with fastidious accounting. Most social systems have very fuzzy accounting and they work just fine, thank you!

The puzzle to be solved here is how to draw more of the peripheral demand into a load balancing system. Reading about EnerNoc’s approach isn’t the first time I’ve seen discussion of this. For example Bruce Schneier mentioned a regulatory attempt at something similar. I liked that one a lot, it provided a way to signal household thermostats. He was concerned that the resulting system would attract hackers. I presume he’d be just as sanguine about the security of the EnerNoc system; probably more so since it’s a closed system.

Such concerns are appropriate, but for heaven sakes I wish smart people like Bruce would stop pretending that these cases are somehow unique. It is the very rare large scale system that doesn’t have vunerable choke points. Hubs who’s failure can bring the entire system to it’s knees. Telling designers not to build large systems because of those risks is lame. Helping them know how to build them so they are safe and robust is hard, yes. But these systems get built because they generate mind boggling amounts of value. So it’s better to do the hard job and forgo the short term pleasure of a bit of hysteria.

Speaking of load shedding: turning your car’s engine off when you stop is more efficient than you thought.

Standards are often a land war; where members of the standards body act to create de facto standards at the same time they participate in the negotiation of de jour standards. The mix varies. In some contexts the land war dominates. But it’s real politics and certainly tastes dirty. Many standards bodies have clauses in their members agreements to prevent other kinds of dirty gaming. For example clauses to temper the member’s ability to playing stick-up with their patent rights. It’s harder to do in these cases; since it would require the member to temper their striving for market share. This plays off interestingly against the device where a standards body retains for it’s members early access to the specification - a device intended to give the members the reward of early mover advantage. This pattern also reminds me of the scheme were a firm creates a “open standard” so other market players chase his tail lights. Card space is a good example of that move.
Meanwhile there is another delightful introductory paragraph in that posting.

I very much liked this introductory paragraph:

Years ago I read an interesting article about the encyclopedia entry for the keyword “Longitude”. According to the article, the entry merely said “See Latitude”. With that short, two-word sentence the encyclopedia author conflated these two concepts as mere orthogonal dimensions, lumped together, each as boring as the other. This ignored the fact that latitude is boring, easy, trivial, known to the ancients and as easy to calculate as measuring the altitude of Polaris. But longitude, there lies an epic adventure, something fiendishly difficult to calculate accurately, something that propelled a great seafaring nation to a search for accurate timepieces that would work at sea, just in order to more accurately calculate longitude. Books have been written about longitude, lives lost, fortunes made. But latitude — latitude is for children.

Complementary pairs appear through out the world of standards. Often one of these is easier to pull together than the other. After the fact or at if one is only looking casually this difference in cost tends to be forgotten. One of the many places where at first blush two things appear the same, but as you get closer they are not. Delightfully you can actually use this cognitive effect for humor.