Ascription is an Anathema to any Enthusiasm

If your Microsoft the good news, of 1992, was that you had captured an extremely valuable hub: the OS switching yard, the bridge from desktop hardware and desktop applications. That bridge’s toll generates a steady substantial stream of revenue. The bad news is that those damn innovative dudes, armed with all the weapons that Mr. Moore and his friends can dream up, keep inventing ways around your bridge and off the desktop. It gets worse! Your install base doesn’t like innovation. They want stability. They have real lifes.

The troubles of the rich! The monopolist’s problem. What to do? Seek wasy to temper the forces that seeking a way around the monopoly. The monopolist needs to slow things down. Buy time to captures or marginalize those damn innovations. Give is risk adverse users time to swallow this new stuff.

I call this the forced migration of your ecology. In some industries, like the auto industry, the the ecology moves slowly. But in high tech it can move very fast. In fact high tech moves so fast it keeps pulling slower industries into it’s wake. If you don’t drive your installed base to move forward you’ll get left behind. Mr. Moore and his friends are very hard to negotiate with.

In the accelerating world of high tech keeping the installed base on the move gets harder and harder. You can do it with carrots: “creating a bridge to the future.” You flood the installed base with a vision of how wonderful life is on the other side. You can do it with sticks making the old world less and less habitable. Sometimes you just threaten to pull the plug. Sometimes the only way to get the ecology to migrate is to set the forest on fire.

Apple, to take one example, did an amazing job evolving the original Mac OS over first decade. When they finally started to hit a wall it took quite a few tries to get over the hump. These almost killed the company. But Mac OS X shows them survived.

This is the problem, getting to the future, Microsoft is trying to solve with Longhorn. It’s not the first time Microsoft has had to do this. Recalling that it took them 15 years to swallow graphic user interface illustrates two things. They are persistent. Their installed base is nearly very locked-in. They may not be very good at innovating, but they do know how to herd the cattle.

This essay, which I highly recommend, is Joel Spolsky developer screaming bloody murder that they are setting his forest on fire. Actually he is further along than that. Having notice that the threat of fire he’s trying to work thru understanding what the hell is going on.

Good developers are not cattle, they don’t herd, they hunt. Developers have lots of options about where to go next. High risk choices.

I find it fascinating that while displacement is such a fact of life in the modern world how much effort goes into pretending it is exceptional and then blaming individual actors. This stuff is like plate tectonics. The ground is moving. Vendors and societies strive to temper the rate of change for users. Sooner or later they fail. The result seems like an earthquake.

LOAF is a clever attempt to design a system that would assign a reputation score to a email address. You use these to help reduce spam. The reputations are assembled out of contributions made by your correspondents. They contribute their address books- well not quite. That would be far too revealing. Instead they obfuscate their address books before revealing them.

How’s this work? When correspondents volunteer to reveal their address book via LOAF they actually reveal just a useful bit vector. You collect these bit vectors. Later when you get email from sender X who you’ve never heard of and your wondering is X a spammer you can use these bit vectors and LOAF to make a rough approximate check to see if X is already in any of your friend’s address books. The LOAF calculation might reply there is an 80% chance X is in Bob’s address book. Obviously this is useful input to automation trying to decide if X is a spammer. Obviously there are limits on it’s value. For all you know some spammer has been forged X’s address.

The trouble in that scenario is that you also just invaded Bob’s privacy a little. You since you know have an 80% certainty that Bob and X are correspondents. At least enough so that Bob put X into his address book. But wait! X is your mistress! What’s Bob doing corresponding with her?

Consider how you could map over your address book to get an approximation of who’s in your address book that’s also in Bob’s. What! Bob doesn’t have the boss in his address book?!? Yeah it looks like Bob’s got the same bookie I do! Of course you can use other sets of email addresses to attempt to distill out a better model of Bob’s address book; for example the addresses in mailing lists you think he might lurk in.

So, the question this raises. How well can such a design work? How little of one’s identity can you keep private while contributing to such an enterprise. If you could get this design pattern to work well it might be a big help. It is clear though that it’s easy to do this badly.

The key to thinking thru how to attack such designs is recognizing that the attacker will have a very large set of obfuscated revealing to work with. That’s why the hashed email addresses in FOAF files are a lousy solution. It’s just too easy to start with that data and with a little sophistication and a bit of brute force the attacker can usually reverse the obfuscation.

This paper about how to figure out if a given laboratory is engaged in secret bio-weapons research is another example of how hard it is to avoid incidental revealing.

I’d be very interested to read some material that critiques the LOAF design! I’d be more interested in reading something about how to design systems that do obfuscated public revealing to enable various kinds of questions to be asked without enabling aggregation to overwhelm the partial revealing implicit in the obfuscated revealing. It’s not clear to me it’s actually possible to design such systems.

Update: All the attacks on Bob’s privacy work because we know that we have Bob’s bit vector. For the scheme to be useful what we need is bit vectors of presumably trust worthy people. We don’t need to know which bit vector goes with which person. So if we can launder the bit vectors to remove the identity of who contributed them while keeping some degree of confidence that the person was trust worth then the system works again. For example if your a member of a club you let everybody in the club drop their bit vector anonymously into a box. Then club members could use those to filter their incoming mail. That might tell you that 3 people in your club might be in correspondence with your mistress, but not who of the 200 members that might be. If the club is large enough then you need only trust the members on average for the scheme to work.

About dem WIKIs…

WIKIs are another example of a process framework for solving a class of organizational problems where you have a huge pool of hands and eyes and
you want to leverage that resource to make something good. I sometimes call those “talent scraping.” We have stumbled upon a few things over the last few years about such systems. Some people treat these “discoveries” with near religious reverence - which is fine because we don’t have too many of them so far.

Examples of what know:

• Create a large binding surface, i.e. lots of options, call it modularity, pages, plug-in whatever; but ship these options in preference to shipping finished products.
• Let the many hands self select what options to exercise; this lower’s coordination costs, and moves you closer to the customer/audience.
• Undo is good, it let’s people experiment at much lower risk.
• Stream changes past many eyes to capture free QA
• Very very lightly sort your community and give more power to people closer to the core
• Strive to lower barriers to entry on all community boundries
• Strive for open: a little ownership of turf is good, a little more isn’t, a lot is toxic.
• Bias for action where ever possible
• Ship early - real users embedded in real situations trump designers almost everytime.
• Ship early - highly open systems with lots of options have power-law tendencies.
• Labor to reduce the distinction between audience and creators
• Look for the network effects, nurture them.
• Tone is important - it will drive what you define as “good”
• Know that working and coordinating with infinite tiny options and infinite tiny resources is very different than working on systems with scarce expensive resources.
• …etc. etc.

You can see these in play in numerious systems, Wiki included. People who haven’t see the incredible value these systems can aggregate find them all a little topsy turvy - and indeed they are. They are upside down since they thrive on an abundance of labor all of which is only very marginally coordinated or committed. This upside down quality tends to make it tempting to label their processes as lame. Most traditional process is around managing scarcity and
That doesn’t mean that right way to get them on-board is to label their processes as lame. That’s a bad move. There is plenty of process design here.

(This is the second half of an old posting; but now I want it to stand alone so I’m chopping the old one in half.)

In my last posting on power-laws (this posting) I illustrated a simple simulation. It grows a network by having each node connect to the network by randomly mimicking the connections made by so far in the network. That model’s important because it gives rise to power-law distributions in the connection wealth of individual nodes. Important because it under cuts the argument that the success of a node is based on it’s quality. It’s not a totally unreasonable model for what happens in markets where the supply chain relationships are very sticky (i.e. we don’t simulate yet any breaking of connections) and the new entrants select their supply chain associations based by modeling existing associations. Such markets aren’t that exceptional in the wake of Moore’s law and his friends.

In this posting I want to look at the advantage of being an early mover into such a market. Older nodes in such a networks tend to be richer, but how much more? The advantage is exponential because the advantages of early entry tend to multiple over time.

Older nodes grow wealth for two reasons. First since the game (the simulation if you prefer) lacks any means for a node to lose wealth the longer you play the greater you chance of getting randomly selected for new connections as the game proceeds. Each connections a node garners becomes an additional generator of luck in future rounds of the game. So connections caught early in the game keep returning value thru-out the game. Secondly the effect is multiplies. Secondly early players are situated in a less competitive game. They have a higher change in each round of winning.

Both of these effects compound so that we can expect that the advantages of getting into the network early to be exponential. An in this graph showing which round a node is born in and how many links it manages to accumulate illustrates that. The horizontal axis is linear. The vertical axis is log base two; thus nodes with only two connections get a score of one.

The roughly double expodential distribution of this advantage is clear; but since the fates (i.e. my random number generator) are indeed fickle getting on the bandwagon early doesn’t assure success. One round of the exponential is absorbed in the log vertical scale while the other can be seen if you visualize a fitted line to that noisy data. As you can see, some of the early players gained no long term benefit. I often go to work for those companies.

This graph is only for one run of the simulation but other runs are similar. That graph ought to be a scatter plot, but my graphing tool only does line plots and I’m too lazy to bother to fix it at this point.

All this reminds me of a meeting many years ago where 30 or so very senior technologies sat in a room grilling the senior vice president of a famous desk top software company. The usual problem was a hand. The schedule was slipping, promises had been made, PR as embarrassed, and we had gathered to decide how draconian the cutting would be to get the damn product out.

The question: “Just how important is, time to market?”

The senior vice president allowed as in his experienced it had always appeared to very important, if not more important than all other aspects of the product. Some of us in the room found that to be a less than compelling answer.

He was right. I think we now know why. I’m a believer now!

It’s tended to make less enthusiastic for traditional enthusiasm for process, elegance, care and more enthusiastic for a bias-for-action and a generosity with options that might reveal new networks of oportunity.

The first parts of this paper (pdf) are a readable gloss of the “marketing techniques” used by cults. The second part is much more rough, and it’s certainly not sufficiently tongue-in-check about applying these vile methods to product marketing.

p.s. We love you!