In Praise of Tweaking: A Wiki-like Programming Contest by Ned Gulley, The MathWorks, Inc. is very very cool. The folks at MathWorks make a programming platform for scientists and engineers. So they have a developer network that they manage consciously. Like most developer networks they have contests. Contests can be great for generating some excitement around your platform. If done right they they can cause the community to discover and reveal innovations that might have otherwise remained hidden.

This paper is neat because is shows a really cool hybrid of open source collaborative development, wikis, massively multiplayer games, and objective based management techniques. (I think I deserve a very high score in for that sentence of buzz word bingo!)

What they have set up is a way to run the contests where the entries are like pages in a Wiki. Moves in the resulting game consist of revisions to an existing entry; or possibly the creation of an entirely new entry. Because they can set up a simple score (say number of lines and CPU time) each entry can be scored instantly. This creates a kind of stadium were players and spectators can rendezvous.

Let’s look at some of the ways this solves canonical problems in the open source design space. For example one model of open source says that the work proceeds in two phases; the developer refines the existing code to achieve a benefit that is local and then in a second phase he reveals the work publicly. Open source projects have trouble creating a climate where the second step happens. In this example you can’t play the game unless you reveal your work; so an incentive is created to reveal. But better yet there is an incentive to reveal as soon as possible because somebody else might reveal faster. It creates both a incentive to reveal and a bias for action.

Unlike an eBay auction where bidding last is best this is a game where bidding early and often is your better strategy. Interestingly sooner you enter the game the more chance your entry will become the basis for later entries. Since entries show their pedigree there is an interesting additional bais for action that early movers may become the basis for a large portfolio of entrants. That large portfolio means you have an increased chance of being a contributor to the winning entry.

Another standard problem in Open Source is how to avoid having the project’s core group establish a membrane around the project that rejects contributions from the outside. This is the other side of the how to encourage developers to reveal their contributions. The core group’s problem is how to balance the maintenance of various qualities (design, safety, limited forking, etc.) against the need to make the cost of revealing very very low for outside developers.

This example is a delightfully extreme case. The single naive metric used to compute the score becomes the only definition of quality; so that doesn’t need to be computed using social processes. The question of forking is solved by turning the knob to eleven – forking is the default. Every entry is a fork. At any moment in the game the entry that becomes best might evolve from any of the older entries. Crazy! It’s yet another example of how source control and the nature of information goods make these systems so different than real world ones.

One of the questions around open source is how much of the value in an open source code base flows from the contributions of a few v.s. from the many. People who think that status games are a key salient element of open source tend presume that a few status seeking contributors make the majority of the contributions. This is a question about the statistics of the contribution flows. I assume and there is some data to support that the contribution flows are power-law distributed. I.e. they are highly skewed with a few contributors doing a lot of work and a huge population of contributors doing small amounts of work. I also assume that the severity of the skew is due, in part, to how successfully the projects solve the problem of lowering the barrier to revealing your work back to the core.

One sweet aspect of this paper is that given the simple metric of quality they use to score the game they can cough up some kind of answer to this question. The paper’s title “In Praise of Tweaking” gives it away. Most of the improvement in the final score emerged out of tiny tweaks. But that’s really too simple. I love this bit: “Long stretches of tweaking battles can be suddenly punctuated by dramatic shifts in the code. When one of these big shifts occurs, it also opens up fresh opportunities for tweaking, and swarms of curious competitors descend upon and begin tightening up the new leader.” That pattern happens, of course, in real open source projects just not a quickly.

Well, read the paper! There must be hundreds of places around the edges of open source projects were these techniques could be tried.