Category Archives: power-laws and networks

Network Effect Contests

The “klaws” at Venture blog writes:

…John Maynard Keynes called a “beauty contest”.

He named it after beauty contests that ran in newspapers of his day. A selection of women appeared in the paper, and the idea was to pick the prettiest one (nobody ever said Keynes was politically correct). The paper would award a prize to the people who picked the winner — the one with the most votes. In other words, if you are trying to win the prize, you don’t pick the one you think is prettiest — you pick the one other people will pick …

The venture business is driven by the same logic when setting valuations.

Nice. I like to find examples of simple systems with network effect. So that newspaper contest is a perfect example.

Another example of a game with network effect is the that bit of playground exquipment the turn-table. You can tell it has network effect because mothers stand on the periphery and say “somebodies’ going to get hurt.”

The newspaper contest example seems particularly relevant today when I’m surrounded by people who are being punished by their membership (loyality) to the community that goes by the name Red Soxs fans.

Meanwhile, I guess, I should toss in this quote by Adam Smith:

Are you in adversity? Do not mourn in the darkness of solitude, do not regulate your sorrow according to the indulgent sympathy of your intimate friends; return, as soon as possible, to the day-light of the world and of society. Live with strangers, with those who know nothing, or care nothing about your misfortune; do not even shun the company of enemies; but give yourself the pleasure of mortifying their malignant joy, by making them feel how little you are affected by your calamity, and how much you are above it.

Difficult advice to take when the entire region is in a funk. I gleaned that from Daniel Drezner

Feeding the Link Parasites is a Sin

forest.jpg
Your invisible if your art doll site doesn’t have anybody linking to it!

I know! Let’s make some links! Hmm… blog comments?

We have a problem here. Blog comments are a platform for link parasites just as Microsoft Outlook as a platform for spammers.

That is a problem with the current architecture of the blogging universe.

By virtue of how search engines work web sites accumulate quality ratings from the incoming links they attract. Meanwhile they accumulate reputation by their content, and the out-bound links they create. That reputation gives weight to the links.

Designers should accept some responsibility for creating systems that nurture this. Their designs should help to create good links. At a minimum they should not encourage the creation of links by bad actors.

Good people making lots of good links is a public good. Delightfully it also benefits both the source and the destination of the link. It creates a tiny bit of reputation for the source and a tiny bit of quality rating for the destination.

What’s excellent about the blogging ecology is how it has helped to generate a huge increase in the number of the best kind of links. Links generated by good actors; links that raise the reputation of the source site and raise the quality ranking of the links destination.

Link parasites create links that aim to aid only one side of the link, and manufactured blog comment links tend to drag down the reputation of their hosting blog.

Hacking the search engines with manufactured links is nothing new. Political parities, activist groups, marketing firms, and artists all do it all the time both in the real and the virtual world.

Should one of these link hackers chooses to manufacture a thousand links from art doll blogs to my site, hence slandering my site as being a high quality art doll site, then there isn’t much I can do about that.

But I can complain to the art doll blog
owners, and in turn I can complain about the blog authoring tools that enabled it.

The blog comment mechanisms are a dish of agar for bad actors to manufacture bad links the same way that Microsoft’s mail programs. Just as Microsoft Outlook is a platform for mail virus the blog comment system is a platform for link parasites. That’s a sin!

The good news is that link parasites damage the reputation of the hosting site. Good news? Yes, because it creates an incentive to get the problem resolved. Bad links mislead the search engines. They make the comment pages almost impossible to assign a usable reputation to, and that bleds over to the rest of the site.

The site owner desires a means to protect his reputation and the search engine wants a hint how to treat the links it finds.

A simple solution is to mark the link using attributes in the link. “Please consider this link to be the responsibility of an unknown third party. Your’s sincerely: site owner”

A site authoring tool that fails to do this is doing a disservice both to the public good of the web and to the reputation of the site author. Bad tool!

It has long been a fantasy by hypertext geeks that links would have bundles of meta-data on them. Today you can annotate a link to indicate that following it will take you to the “next page” and most browsers have a keystroke equivalent that will follow that link. This is rarely used for the usual reasons a standard fails to get adopted, i.e. the chicken and egg problem. Chicken: why learn the next-page keystroke if nobody annotates their pages. Egg: why bother to annotate if nobody knows the keystroke.

It seems hopeful that both the search engine and the blog authoring tool have their incentives line up. If they both adopt the standard then the link parasites will have to find someplace else to play their games.

Additionally it appears that we have some hope that mechanism to help is already there, say by adding something like author=”unknown-3rd-party” to the links in comments.

I think it’s neat that while historically putting meta-data onto links hasn’t created much return on the investment in this situation the benefit flows right back to the site author. Now he can defend his reputation and the authoring tools gets to avoid being bad.

If this was a discussion at an international industrial standards body then we would call this annotation a ‘pedigree’ and we would want to use something like SAML to create the assertion. That starts to drag us into the whole identity rat hole, among others.

So while we wait for those guys to get back from their meetings maybe we could just start putting author=”unknown” into blog comment links. If a handful of the big blogging tools and one or two of the search engine leaders indicated that they would get with the program the problem would be solved.

This problem goes by many names in the real world: astroturf, whisper campaign, etc. and you can hire firms to dis-intermediate the bad acting for you. Of course for others it’s just called mobilizing your base.

So! Anybody who’s got this far I want to encourage you to link to this art doll site. It will accrue to your reputation, I’m sure!

Network Maintainance

I’ve been thinking a bit recently about how accounting principles just don’t seem to work when applied to social network, and possibly not even to networks in general.

For example over at unfogged.com Ogged writes a nice little post where in he declines to take too seriously a bit-o-publish/perish research showing that women are generally less effective negotiators than men.

Meanwhile over at Crooked Timber we find Chris talking about the thousands of deaths during the recent heat-wave in France. He quotes Eric Klinenberg’s fascinating work on an analogous situation that happened in Chicago a few years back. Apparently in that disaster men tended to die in much greater numbers than women, in-spite of the data’s suggestion that pool of women was both larger and more vulnerable. Why? Because women have stronger social networks.

Clearly these two are related. You can optimize for the easy to measure short term benefit of optimal negotiated outcome, or you can optimize for a strong social network – which will deliver value in the long run. Of course that’s not black and white.

Finally I’ve been nurturing a model of why the electric grid in the Northeastern US fell apart the other day. A model that works from the assumption that the guys running elements of the grid all knew that there was good chance that it would collapse and were running the business on that basis – creating a feedback loop which accelerates the chance of a collapse. Each owner of an element in the network keeps lowering his trigger points for when he will exit the system because he’s thinking that when the collapse happens he better protect his capital investment. That makes the system increasingly susceptible to cascading failures.

There is another side to the story which is nicely summed up in this article by John Kay. The scenario he outlines involves the complex trade-off a firm makes between short-term income and long-term capital investments. Regulation and standard (ethics, professional standards, inspection regimes, etc.) all work to temper an a cyclic instability in that game.

Consider two identical power generators A and B are selling to the same customers in a competitive market. To first order their costs are identical. Mr. A comes in one morning and discovers that Mr. B is selling power for less than he is. Puzzled he drives by Mr. B’s plant and discovers he’s stopped trimming the trees under his power lines and repairing the roof. Incompetent he thinks. When he get’s back to the plant the next day the accountant comes into his office and explains that due to lost revenue they need to borrow money to meet payroll. The next day Mr. A fires the crew that trims the trees under the power lines and lowers his prices.

Both Mr. A and Mr. B are now caught in a rush to the bottom. If one of them can survive and the other one doesn’t the winner will be sitting pretty. The one with the most stored up fat in their business will survive.

While, there is plenty that’s different between a collapsing social network; and the collapsing electrical grid. In fact all the networks have important subtle differences. For example the triggers in the electrical grid are obviously very sharp, while social networks and the Internets packet networks are breakdown more gracefully, so the nature of the cascade failures are very different. Some networks are more brittle than others.

Another difference is the nature of what happens when the network does fail. Some network communities have what you might call social will to rebuild while others don’t. I suspect that the social network in authoritarian nations have a lot more trouble rebuilding than those in more liberal nations.

The players in the industry around the electric grid probably assume that after the collapse political will emerge to assure a stronger regulator hand and that will allow them to turn again toward investing in maintenance, capital equipment, talented staff, etc.

This faith that the social contract will save us is a beautiful thing. That people believe these things will heal themselves testifies to the faith people have in the social contract. That collectively people will work to create standards of behavior that serve the public good.

Of course the players might be wrong. For example in this article here we see President Bush deciding that upgrading standards for the distribution grid is really not that urgent a priority. Presumably that provides more time for the guys to get together and negotiate an optimal outcome. Meanwhile the old ladies always have their social networks.

Some people don’t believe in the public good, the social contract, need to regulate the networks, the markets, etc… Men!

Simple Measures Kill Diversity

Power-law distributions emerge when new nodes attach to the existing network with some preference for the current winners. You can increase the severity of the distribution by increasing the preferential attachment, and you can temper it by decreasing it.

For example to increase the severity the trick is to encourage the new comers in the belief that current leaders are a good proxy for their quality measures – it’s a simple message: ‘Go with the winner!’ To decrease the severity you need to provide to new commers with a more complex message: “Pick what’s best for you, here’s a bucket of information to help you do just that.”

Improving the information available to newly arriving members of the network, providing them with more complex subjective information, enables them to make their decision more rationally. That will make for a more diverse network and a less severe power-law distribution.

A system like Epinions.com manages this by creating a bit of a feed back loop that allows existing members to report to new members what they now know. Word of mouth can also help, since a new comer

Information & Power-laws

The simplest model of how a power-law distribution emerges presumes both urgency and ignorance. New players enter the network with an urgent need to connect and no time to evaluate their choices. To solve this problem they thrash around for something to substitute for knowledge, a proxy metric. If the proxy they pick leads to their preferentially attaching to already popular nodes the power-law emerges.

This is why advertising can be so powerful in an emerging market.

Bearing that in mind this posting from Clay is amusing. Nightmares are interupting the sleep of marketing executives at major motion picture studios! They wake up in a sweat imagining
that the audience at the first showing of BogusTeenMutantSpacemen IV are
fidgiting in their seats sending instant messages on their phones “mv scks! run awy!”

Always nice to see the vested interests disrupted by the increased ablity of the unwashed masses to coordinate their activities. But…

My nightmare is that this will mean they have to carefully craft movies so they
will get high ratings from the kind of people who watch the first screening at
4:30pm on a Friday afternoon and think that it’s ok to use your phone to IM during the movie. Teenagers!

Mosh Pit

I’m very much enjoying reading Chuq Von Roshpach chew his cud on the issues around running online groups. He is saying a lot of things that
people who do this know. But which I don’t think are written down much. This is the kind of social knowledge that Phil Agre complains that smart
people are often sadly entirely unaware of.

For example I like this peice about a technique he’s tried for drawing forth the folks on the long tail of the power-curve.

It reminded me of a story told to me by a friend about the community
theater group that one lucky day managed have a famous actor offer to
join the company. But, when it came to perform the play the famous actor’s
presense on the stage was so powerful that everybody else seemed to
disappear into the back ground.

That’s not good for you community.

I don’t think I can recall any group I’ve ever been in were this kind of
imbalance didn’t begin to be a problem at some point. Where the slope on
the power-law curve grew to severe. Sometimes that lead to the long tail
just drying up and the group evaporates. Sometimes a way around the
problem is found. I’m surprised how often it resolves by the sad device
that the ‘famous actor’ get’s increasingly peeved that people are not respecting
his role as leader dude or he complains that they are falling further and further
into the background of the stage set. Finally he resolves the problem by
walking off in a huff.

The famous actor problem is yet more subtle when you have a cabal of famous actors at the head of your group.

Power-law graph layout

Nice paper sketching an straight forward algorithum to layout a graph informed by the assumption that the graph’s connectivity exhibits a power-law distribution. The general idea is to sort the graph into rank order and then layout the upper-classes first and pack the lower-classes in latter.

I love the way that a even a graph layout problem becomes a statement on strategy, social engineering, governance, etc. when your working on things like this.


“The highest layer possesses a small number of highly influential nodes. Consequently, it requires the highest temperature so that the nodes are forced to spread out from one another, and explore the global search space. This is essential as these highly connected nodes define the final structure of the graph.”

income power-law visualizations

A simple chart showing the income distribution and an another attempt to give a feel for it via by spreading it across the US (and on into space). These are great. It’s hard to visualize power-law distributions it’s even harder to think straight about how the evolve. Income distribution is always power-law distributed but it’s more severe in some places and eras than others. Much society has come to admit that we must try to manage the interest rate we need to admit that we must manage this.

Silent Reading

reading.jpg

Clay Shirky chimes in on the backchannel discussion (i.e. just how rude is it that in modern meetings people are passing notes in quantity all the time with others both inside and outside the meeting?). He does his bit to move the thread along and rounds up one of the usual suspect, i.e. “Yo vested-interests! Get over it! Too late now. Cat’s out of the bag. Deal.” He throws in the usual story about a previous debate of this kind. The one about calculators in school.

I recall the calculator debate. I was in college. Some students could afford a potent calculator, most couldn’t, I couldn’t Some people had access to symbolic math software and others didn’t. I did. Matlab over the internet could do some of my homework almost with nearly zero effort. My perception at the time was about fairness in the face of widely disparate computational power.

My son recently took a math SAT II test with an approved calculator that could do an awful lot of symbolic math. I wondered how many of the students knew that. How many had been prep’d on the calculator’s capablities? How advantagous to the calculator vendor if he could get an entired generation skilled in his calculator’s UI.

The oldest story of this kind – i.e. stories about out cries over the disruption to existing structures triggered by new technology – I know is about silent reading. Apparently there was a time when men only read outloud, or at least most of them did. About the time the printing press appeared the idea that you could read silently, not even moving your lips, spread across Europe. This created all all kinds of concern amoung those who felt that they were responsible for looking after the thoughts of the population.

That’s of course all entangled in the story of the Protestant revolution. Boy is that a classic story of innovation leading to the displacement of vested interests (aka the hub, aka the intermediary) in that case the role of the chuch as the intermediary between man and God.

Weeds and Tubers

mapleseed.gif

There is a maple tree in my backyard. Each year it drops a carpet of winged seeds on the ground. Tens of thousands of seeds, every year, for 30 to 60 years. One, just one, of those seeds needs to take for this tree to fulfill it’s Darwinian mandate and create the next generation. One survives out of million? The ecologists call this a r-Selected strategy.

I, on the other hand, have only a few children, and unlike the maple tree I am expending vast resources on each one of them. The ecologists call this a K-Selected strategy.

Species with an r-Selected architecture (weeds, annuals, insect pests,bacteria) tend to be opportunistic. They spread fast. The “r” stands for resources. If storm clears a section of forest the nearby maple is ready to seed that clearing. These species quickly cover new territory and quickly compose the dead. They tend to fluctuate quickly with the weather.

Species with K-Selected strategy (coconuts, apples, birds, most mammals) expend more on each offspring. Their populations tend to be more constant and self regulating

I assume that some business models are more r-Selected, with others are more K-Selected. If the weather is good and new markets are emerging fast then one should invest in the r-Selected businesses. For the last hundred years technology has created new markets in quantity – a good time for r-Selected businesses. Ones where it’s best to encourage a large population of lightly funded experiments. Now if you think that’s easy I’d recommend a careful dissection of a maple seed. Conversely I’d assume that if your operating in a very mature market with volatile weather then larger enterprises more careful planning and husbanding of your resources if a better tactic.

All this give rise to my new cartoon of open source. That those tens of thousands of projects at Source Forge are like maple seeds. That Open Source is a species that has adopted a r-Selected strategy. It assumes that that the problem at hand is not husbanding ideas, but covering the fresh earth of possibilities. That the problem isn’t finding options worth executing upon, but finding which of a ten thousand possible options will actually take root.

Now one thing that’s curious about this is that if you look at large Open Source projects you might not see that. Those projects look more like whales. They tend to have a certain overhead where in their resources more carefully managed so the baby doesn’t die.

Of course all this is in total contrast to closed source projects. Every one of those I’ve worked on has expended tremendous resources attempting to manage scarce resources. Those projects seem much more like K-Selected species, carefully hording their fatty reserves so they can survive the next round of lousy weather that the market or management blows down the hallway.

Finally I think this has something critical to say about what goes on as you move along the power-law curve. That the population that resides on the long tail tends to be more r-Selected (limited by resources and driven by opportunism) while the upper classes that reside in the towering heights tend to be more K-selected (durable in the face of bad weather, fat, enthusiastic about property rights that guard their inheritance.).