Ascription is an Anathema to any Enthusiasm

Brett Porter wrote up a nice summary of his first impressions of that new search engine, Cuil.  He had exactly the same experience I had, and my wife had.  You ego surf only to discover they have a very odd model of your internet presence.  Feeling disappointed you then wander off.   We learn from this is that any new search engine better make us all appear even more above average than we already do.

Having recovered the lamp the genie offers you a wish.  Having rescued his daughter the king grants you a wish.  Having suffered the tortures of the damn’d you can slip anything you want into the final draft of the industrial standard.  What do you ask for?  During his short time at Harvard I gather that Bill Gates mused that it would be nice to own the traffic lights.  I call that owning a standard.  We oft see patent trolls pop up after a standard builds it’s installed base claiming to own the right to tax the traffic.  Gates built his monopoly the old fashion way.  That is somewhat more ethical.

Patents are only one way that the king can grant his minion’s wishes.  The crude example: no-bid contracts are good, and why bother to audit them?   Clever wishing runs along the lines of Bill Gate’s thinking.  Own a piece of the transaction flow.  Find a place with a high traffic standard and get the option to take a piece of the action.  And so I’m very impressed by what Eaton industries managed to pull off in the UK.  They got proprietary ownership of a new light bulb socket, it’s written right into the building code.  The socket’s aren’t that expensive, but the bulbs that go into them!  Twenty dollars a bulb in excess profits!  And you have to admire the crust of greenwash (the intent of the standard was to increase the use of high efficency bulbs).

Institutional failures like this should be fixed.  There should be ways to taking the wish back.  While we wait for that, daring individuals can hack their sockets.

I believe it was Ray Kurzweil, circa 1989, who advised encouraging a private jargon inside your new company.  I remember because that was just about the time I was starting to think open would totally trump closed in our industry.  The advise seemed to my ears a bit old fashion.  But at the same time I suspected he meant that it was a good way to tighten the bonds inside the team.  By then I knew enough about cults to recognise that’s common inside cults; it supports two of the keys to running a good cult - information control, and thought stopping processes.

But don’t take any of that too seriously.  It is good advise none the less.  I’ll admit to being a Whorfarian, the words you highlight effect your thinking.

These days I’d add to all that though.  Language is the ultimate exchange standard.  So when you decide to innovate a new private language your cutting your self off and creating friction or trade barriers with your outside partners.   Importantly the advantage a new group has is that they can pick and choose what to emphasis.  They can take a run at leveraging some particular competitive advantage.  As Dave Winer says: You can’t win by zigging when he zigs. You have to zag to beat him.  Ray’s advise can be viewed as a bit of implementation advise for that.

So it was with some interest that I saw Google revealing their in house standard for serailizing data.  It’s not hard to see that Protocol Buffers are alternative to XML.  And it is amusing to, at least to me, to think that they did this in the hope of reducing the frictions that occur when they must translate from their in house argot into the dialects used by the outside world.  It’s fun to note that if your start up is as successful as Google you get to promulgate your private jargon.  It is one of the spoils of war.  You can push that friction into your compliments, make them pay the switching costs.

Protocol buffers aren’t anything special: messages are lists of key value pairs, keys can repeat, a small set of value types; including unicode strings and recursively other messages. They are very practical, close to the metal.  Choices were made and they are what they are.  They are quite dense, and easy to parse.  Many messages can be serialized in one pass.  In classic run length encoding style nested structures have their size mentioned in their header.  That makes emitting one pass serialization hard.

Given an array of bytes it wouldn’t be child play to guess that your holding a protocol buffer, you could do it huristically but it would still be a guess.   You need a protocol buffer’s declaration to parse it.  For example you absent the declaration you can’t know if a you’ve got a sint32 or an int64, etc.  All that disappointed me.  It disapointed my inner archivist and the inner peeping tom (who has often debugged tough problems by watching the bytes fly by on the wire).

There is a nice detail that allows optional keys which in turn makes it somewhat easier to add new message variants.  With luck the old message handlers can just ignore the additions.  It made me smile to note that mechanism can be used to pad messages; which in turn makes it more likely that you can serialize in a single pass.

There is another nice detail that allows a key to appear more than once in spite of the metadat saying it is single valued.  The last occurance wins.  This lets you implement a kind of inheritance/defaulting.  For example if your implementing CSS style sheets you read the default style message, and then read the variations from the default, and your ready to go.  They call that merging.

Given the declarative information for a given protocol buffer it’s not hard to convert it to XML or what every else you like.  The observers and archivists will just have to be careful to not loose that metadata; and some of them will no doubt build clever hueristic to cobble together substitute metadata.  Interestingly, inspite of efforts to the contrary, you can’t really work with XML without additional metadata to help.  And, that stuff is horribly complex.

As I like to emphasis what really matters with an exchange standard is how many transactions per second move over it.  No doubt this one has critical mass at least inside the Google cloud.  What matters here for how important this standard might be is how much adoption by non-Google actors.    But, I suspect we will be speaking this dialect more often and for quite a while.  Of course, the rudest way to say that is that we will be chasing their tail lights.  But I don’t really feel that way since I’ve never particularly liked XML and I welcome a substitute.

I’m not particularly proud of the neologism “upgrade forcer.”  It encourages a bad behavior.  Product managers can be a desperate lot, particularly when their bonus is riding on how many copies of the upgrade get sold.  When times are good sweet new product features will draw users to upgrade.  But, as products mature the customers grow content and convincing them to upgrade get’s harder.  Having run out of carrots the product managers are tempted to turn to sticks.

Installed bases are hard to move.  Installed bases without a clear owner, or product manager, are even harder move. You can chat up how nice it would be if we all switched to IP6, but nice isn’t must.  It would be nice if my correspondents encrypted their email; but little drives that upgrade.  Effective upgrade drivers engineer a situation where users move quickly to upgrade.   Y2K was an effective upgrade driver, 1999 was a very good year for upgrade revenue.  I’ve a pet theory that the late 90s high tech bubble owns a debt to that.

One of the open many open standards in the Internet menagerie that badly needs an upgrade is DNS.  DNS is an amazing design for it’s time; but one of it’s failings is security.  It has serious design flaws, and numerous vulnerabilities.  For example you ISP, who your probably should not trust, can trivially intercept DNS queries and inject what ever answers he thinks serves his purposes.  The vulnerabilities make that even worse, since at least you can complain, negotiate, even sue if you catch your ISP playing those games.  But if some evil dude poisons one of the DNS servers your happen to use and your email, IM, or bank traffic is intercepted your unlikely to have much recourse.

Security flaws play an interesting role in driving upgrades.  The product manager can use them to threaten the users, while at least nominally not using force himself.

For years people have been attempting to redesign DNS to add better security.  At first blush it seemed straight forward, but it turned out to be way hard.  My sense is that people now think they have the design problem under control.  So the next step is getting the installed base to move.  Getting a possibly immovable installed base to move generally requires an irresistible force.  Some compelling value or something bad.  There is plenty of bad already, though as is usually the case the immovable base finds it easy to avert their gaze from those horrors.

This posting was triggered by a comment in yesterday’s announcement of yet another really bad flaw in DNS.

There is a update to the DNS standard known as secure dns, or DNSSEC that addresses this problem.  But most people see it as nice to have rather than as a must have.

With luck that changed today.  Yesterday the existence of a really really bad flaw in the the DNS protocol was publicly revealed.  The actual flaw’s details were not revealed, but a massive software upgrade to temper the risk is being rolled out.  But, this line in caught my attention.

“DNSSEC is the only definitive solution for this issue.”

So maybe, just maybe, we have found a upgrade forcer for DNS.  This is extremely good news if your a DNS vendor of any kind.  Profit!  For those who are driven by fun, rather than greed, fixing DNS would allow us to use it safely for a much larger range of light weight database functions.

I have sighted a new urban myth: Electric heating is cheaper than oil heat! Here in Boston people heat with both gas and oil, and the cost per unit of heat between the two has diverged rapidly over the last few years. Those who heat with oil are looking for ways out of their plight. Apparently the rumor making the rounds that it is cheaper to use electric. That’s not true.

In related news Martin brings my attention to a company EnerNoc that sells negative energy, i.e. load shedding, to the utilities. They use telecom and widgets to shift power consumption from high demand time periods into low demand time periods. Martian’s example is the fridge. You chill when power is plentiful and let it coast when others are paying higher prices.

I assume that EnerNoc’s role in all this is to aggregate small power users into a large enough pool to be worthy of selling to the utilities. It’s a interesting example of a coordination problem. There are of course other ways to approach the problem; ones that are less dependent on a thicket of contracts and ongoing coordination signals controlled by a middleman and enabled, as Martian, points out by the telecom infrastructure.

The obvious alternative is to just broadcast signal; and let the demand side react to the signal by selling some simple technology that responds to the signal in reasonably simple ways. That alone would enable substantial contributions from the demand side. But you can improve the incentive structure either thru regulation or by using statistical sampling to tell which customers have gotten with program; and then reduce their tariffs.

The amount of signal that needs to flow from the grid operators to the consumers is small, in the sense that you can broadcast it. A signal only needs to flow back the other way sufficient to assure that the incentives play out right. It is stupid to presume that the only incentives that are available are monetary or that they need to be executed with fastidious accounting. Most social systems have very fuzzy accounting and they work just fine, thank you!

The puzzle to be solved here is how to draw more of the peripheral demand into a load balancing system. Reading about EnerNoc’s approach isn’t the first time I’ve seen discussion of this. For example Bruce Schneier mentioned a regulatory attempt at something similar. I liked that one a lot, it provided a way to signal household thermostats. He was concerned that the resulting system would attract hackers. I presume he’d be just as sanguine about the security of the EnerNoc system; probably more so since it’s a closed system.

Such concerns are appropriate, but for heaven sakes I wish smart people like Bruce would stop pretending that these cases are somehow unique. It is the very rare large scale system that doesn’t have vunerable choke points. Hubs who’s failure can bring the entire system to it’s knees. Telling designers not to build large systems because of those risks is lame. Helping them know how to build them so they are safe and robust is hard, yes. But these systems get built because they generate mind boggling amounts of value. So it’s better to do the hard job and forgo the short term pleasure of a bit of hysteria.

Speaking of load shedding: turning your car’s engine off when you stop is more efficient than you thought.