Monthly Archives: June 2004

Don’t Kill Lead Developers

I read my RSS feeds[1] using NetNewsWire, which is great. I see that Apple is use their distribution channel power to destroy that product. They are folding RSS feed reading into Safari

These moments are a real test for a company’s developer network program. They can reward the developers who were early to the market or they can quietly kill them.

Let’s be clear, don’t murder your leading developers! If you kill the ones that begin to succeed; using your hub-power what happens? It will and should frighten off other developers. These days developers have a lot of choices about where to play. In this world a developer network manager to silently kills leading developers on his platform should be fired.

Microsoft wasn’t able to learn this lesson and now they have had to replace a vast army of volunteer developer talent with a huge R&D lab. That won’t work.

Don’t under estimate the value of trust.

[1] Yes, yes, and atom feeds too.

Pointer Rotation

It’s sad how much good material isn’t available on the web. Today I wanted to find a copy of Norihisa Suzuki’s paper Analysis of Pointer “Rotation”. I have access to a good library’s electronic subscriptions; but this paper was printed in 1982 which is too long ago. It appears that if I was an ACM member I could get it. Sigh.

It’s a nice little paper that shows that if you limit your pointer manipulation to rotate and shift you can make the chance of memory leaks less and the programs become more transparent; well they do at least once you get used to this as a programming convention.

For example I think it’s almost impossible for this routine to have a memory leak because the pointers have no way to escape. Rotatef swaps pointers but it can’t lose one.


(defun shuffle-vector (v)
  "Return copy of vector with elements shuffled like a deck of cards."
  (loop
    with result = (copy-seq v)
    finally (return result)
    for i from (length v) downto 1
    as j = (random i)
    do (rotatef (svref result j) (svref result (1- i)))))

Norihisa also wrote a fun paper on extracting 3d objects from 2d line drawings, or at least I think he wrote that paper.

When it Breaks

With few exceptions the QA organization and Engineering organization in software houses are given separate management chains. There are assorted rationales for this. Some firms like to set up the two in a kind of competition in preference to having them working in common cause to create good software. Some firms think of QA as a kind of auditing function who’s role is to temper a presumed tendency of engineering to fraud or self delusion. Some firms do this because they have put the engineering managers into a incentive structure that makes them likely to cheat to ship on time and they need to compensate for that.

This organizational choice results in tension, or worse. The two groups are torn between their common-cause, i.e a great product, and the day-to-day competitive games that are a result of the organization structure.  As an illustrative example of how short term reward structures can generate polarization consider what happens when a bug is found.

When an engineer encounters a bug his reaction is to fall deeply into a trance state.  This is the hunter’s mental state. He slows time down. He rises slowly and closes his door. To the hunter that moment is very valuable. He is in the presence of his prey. Even if it gets away all the information that is at that moment at hand is useful for stalking it tomorrow.

When the QA engineer encounter a bug his reaction is to shout hosanna. He bursts from his office to tell the boss. The reward he seeks is a bug sighted. To the software engineer this reaction is totally inappropriate. Yes, sighting the bug is a very key exciting moment but this excited reaction lays waste to that very valuable moment.

The software engineer takes that exciting moment’s energy and channels it inward seeking a fugue state. The qa engineer takes that moment’s energy and channels in outward seeking a celebration.

The organization structure just amplifies this. The QA engineer goes to his boss; who gives him a warm smile. The QA boss goes to the engineer’s boss and gives him a sly malicious smile. The Engineering boss simulates a gratitude for the valuable sighting. He then simulates the engineer. He asks if they made any effort to stalk the bug. “Can you reproduce it?” “Do you know exactly how it happened?” The QA boss thinks: “Ah you always want us to do your job.” They smile at each other in the shared knowledge of the absurdity of their situation. Later the engineering boss mentions in passing to the engineer that there is another bug in the bug database.

The only way to breakdown this is to allow the QA engineer and the software engineer into more intimate contact. That allows the QA engineer to observe and them model the behavior of the engineer. They they can share the reward to killing the bug. Which is; of course the common cause. Of course, you’ll have to set aside whichever adversarial model you bought into when you established the separation of powers.

Organization design is tough.

The Five Roles of Identity

At it’s heart the problem of network identity is how to manage the model of the user available to web sites. User’s dream of a design that’s explicit, practical, and respects their privacy. Web sites covet different aspects of the user-model model. The fashion web site may desire to know the user’s hair color. The travel web site may desire to know when your employeer is planning a summer shutdown. The bank site may desire to know a statement of account of your current mortgage.

The demand for better models of visitors is what drives the market for solutions in the identity market. For example it’s what keeps DoubleClick in business. DoubleClick aggregates a statistical model of users from their browsing habits and then sell that to web sites. Web sites then use that to target their marketing. For all I know if you tell one of their clients your hair color then DoubleClick may well add that to their model.

Such implicit, statistical models of users don’t scale up to handle the revealing of more serious information (i.e. medical records, mortgage statements, video rental records, etc.), because of regulatory protections. Sadly some cases these regulatory protections are no more solid than the community expectations. I would certainly make a fuss if L.L. Bean sold information about my pants size to Amazon; but I wouldn’t actually be surprised. Few of us are surprised that if you reveal your a wish list or rate a product at Amazon it effects how they customize the web site for you.

The design challenge here is how to make the management of this revealing more explict. Something that users can understand, manage, manipulate, control. Something that regulators can then write practical rules about. Something that can be governed well. Something that tempers how much power concentrates into a few hands. If such hubs are absolutely necessary we presumably want to assure they are well governed. Tough problems.

Any solution will have to respect and balance the concerns of all the market particpants. Broadly there are five roles in this passion play. In the long run none of these is weak. Users, though, tend to be slow in exercising their power.

IDFiveRoles.png

Intermediaries get a lot of the attention here. DoubleClick, Passport, or Gator are comercial examples. These players dream of solutions that tend to concentrate the power in the market into their hands. The regulatory foundation also gets a lot of attention. That includes: standards bodies like the Liberty Alliance; pseudo-standards certification organizations like eTrust; governments (e.g. EU’s privacy regulations). The regulators tend to dream of getting a single standard to “rule them all.” They also tend to work to limit how much market concentration emerges in the roles above them.

The solution vendors, i.e. the folks that don’t actually run services but instead provide tools to those that do, may dream of owning the entire market but they are also very interested in assuring that a large number of customers for their tools survive. There a huge number of examples of in this role just to pick to random examples: the authentication tools found thru-out the open source middleware community; and Novell’s Oblex solution that is widely used inside firms. There is a notable subgroup in the solution provider space, the patent holders. Note also that standards bodies often provide a means to aggregate a patent portfolio.

Many real world examples are hybrids of these five classes. For example Yahoo, which is primarily a site, also does authentication ala Passport for some partners. These hybrids seem to have internal tensions between their roles.

Market concentration in all these catagories is, presumably, power-law distributed. For example DoubleClick and Passport are both in the top hundred traffic sites.

Interesting market, interesting design problem.

Talent Scraping and Microsoft

If there was any doubt that Microsoft’s buisness model is at it’s heart a Talent Scrapping business. Consider their slogan.

“Your Potential. Our Passion.”

People often complain that Microsoft doesn’t innovate. Of course not, that’s not the business they are in. It out to be “Your innovations. Our Passion” or “Your innovations. Our Potential.” or “You innovate ’em. We bank ’em.”

It’s amusing to note that the culture of Microsoft is fixated on how smart they are. They are of course smart in a particular way. Smart at talent scrapping. It is a pretty smart thing to have decided to specialize in. But, it must always lead to a bit of self doubt.

The Structure and Function of Complex Networks

This is a wonderful paper. Your one stop shopping for all things network-science. Seventy five pages of overview! Dozens of pages of bibliography! Enjoy!

1 Introduction
1.1 Types of Networks
1.2 Other Resources
1.3 Outline of the Review
2 Networks in the Real World
2.1 Social Networks
2.2 Information Networks
2.3 Technological Networks
2.4 Biological Networks
3 Properties of Networks
3.1 The Small-World Effect
3.2 Transitivity or Clustering
3.3 Degree Distributions
3.3.1 Scale-Free Networks
3.3.2 Maximum Degree
3.4 Network Resilience
3.5 Mixing Patterns
3.6 Degree Correlations
3.7 Community Structure
3.8 Network Navigation
3.9 Other Network Properties
4 Random Graphs
4.1 Poisson Random Graphs
4.2 Generalized Random Graphs
4.2.1 The Configuration Model
4.2.2 Example: Power-Law Degree Distribution
4.2.3 Directed Graphs
4.2.4 Bipartite Graphs
4.2.5 Degree Correlations
5 Exponential Random Graphs and Markov Graphs
6 The Small-World Model
6.1 Clustering Coefficient
6.2 Degree Distribution
6.3 Average Path Length
7 Models of Network Growth
7.1 Price

Disintermediation

By way of Making Light this marvelous essay on printing. In high school we took a field trip to the New York Times. The Linotype machines were in the next room. Hot, dangerous. In that room you were right at the narrow point in the bottleneck. The throat down which all that journalism flowed thru on it’s way to my parent’s breakfast table. The danger, the hot lead, the diversity of specialized crafts seemed entirely appropriate. Disintermediating these bottlenecks certainly triggered a mess of displacement.

I see that Comdex is dead. Another intermediary who’s time is passed. There was a time when Comdex was the hub of the computer industry; the place were buyer and seller would meet. The folks that ran Comdex used to be able to charge everybody. They would charge the sellers, the buyers, the hotels, the venues. Lots of power at the bottleneck.

When electronic type setting came along the unions negotiated a contract that assured jobs for their children. I wonder what the Comdex guys got; if anything.

Braudel’s marvelous History of Capitalism reports that in the middle ages seasonal fairs would emerge and grow huge and specialized. Villages in France would be entirely turned over to trading some good, horses for example, for a week or two a year. The entire of Europe’s horse industry would descend on the town. And then mysteriously one year or over a period of just a few years it would stop. Some other way for the buyers and sellers of horses to find each other would displace the market fair. I’ve watched that happen to MacWorld, Comdex, Sears. I wonder if it will ever happen to eBay?

Courtier, How To

Impressive! Yes indeed! My readers are so insightful. Rarely does a group gather so blessed by insight. Each of you nurtures a thirst for candor. Your self awareness keeps you immune from the wiles of sycophants. I’m proud to know you.

This book is a hoot!

Equally amusing is the evolution of it’s cover over the years.

flattery1.png

flattery2.png

flattery3.png

The second golden age is at hand. The court of the Sun King is making a come back. Best to brush up on the flattering skills.

oh. Did I mention? Your the best!

Bricolage

Yeah. Notice how the neat vs. scruffy debate in computer science is related to this:

Bricolage: “(French, ‘doing odd jobs’). A characteristic (according to C. Levi-Strauss) of the early human mind, in contrast to modern scientific thinking. But bricolage is entirely rational (i.e. not pre-rational) in its own way. He introduced the term in The Savage Mind. A bricoleur is one who improvises and and uses any means or materials which happen to be lying around in order to tackle a task: ‘The bricoleur is adept at executing a great number of diverse tasks; but unlike the engineer, he does not subordinate each of them to the availability of raw materials and tools, conceptualized and procured specifically for this project; his instrumental universe is closed, and the rule of his game is to make do with the means at hand.’ In the making of myth, bricolage is the use of whatever happens to be ‘lying around,’ so that myth is both rational and improvisatory.” — www.bloomington.in.us/~okolicko/definitions.html

The book I’m reading about Self Efficacy attempts to define efficacy but Garrison Keiler put it well: “The ability to get up and do what needs to be done.” The book labors with some success to block out subspecies of efficacy. For example that ability to get stuff done within the limits of what is at hand v.s. the ability to get stuff done while bending the rules and reworking materials at hand.

I am effective if I bend a coat hanger to keep the hotel window held open at night. To me a coat hander is just a source of soft iron wire. To me that is an act of the first kind; a simple shaping of materials at hand. To the hotel it may appear instead a bit of rule bending bordering on vandalism. Category making in this kind of discussion ain’t easy, which is why the book is labored and long – I guess.

Another suggested partitioning: efficacy via collective vs. individual action. The author reveals his personal and culture preference for the individual. b He sorts the individual before the collective. So in his view collective action is an exception handler for failed individual efficacy. Should the means above prove insufficient then you might shift over to collective action. Using politics to rewrite the rules presumably. That’s too narrow a model of what collective action is good for; there is of course a bricolage of collective action.

All these means of achieving goals form camps: inside/bend/outside the rules; inside/outside the individual/collective boundary. The camps then lay claim to the generic labels primitive or modern, scruffy or neat. The scruffies in computer science have always worn the label with pride; like the some people wear the label hacker. But then a group’s self label always means to entirely different things depending on the speakers membership in the group. When other non-programers call me a “programmer” they know not of what they speak.

Dragged a word into service from a foreign language, like bricolage, brings the possibility of freeing up the discussion. Maybe? We should replace the term hacker with bricoleur. I kind of like that Levi-Strauss was using it to suggest savage.


Yeah! Let’s having an illustration.

EfficacySpace.png