Here Comes Everybody

Clay Shirky has written a book. Oh joy! Here Comes Everybody is a wise and insightful look into what’s happening as the costs of group forming evaporates. The book is in your bookstore starting today, so run out and get it!

The big question Clay takes on here runs as follows: Given that we are social creatures. Blessed with a having a latent desire to form groups. What happens when it suddenly becomes radically cheaper to form these groups? The straw-man answer: a lot of groups get formed. Groups start popping up, all over, all the time, for all things.

In much the same way that a mole, living underground, is unlike the butterfly living in the summer air these groups are sufficiently different that our intuitions about groups are called into question. While the cost structure does that so does the scale. The traditional organization must cut off it’s own long tail to curtail it’s classic coordination costs from running out of control. These new lower costs groups can push that cut off much further out. That creates entirely new organizational patterns.

This book is roughly targeted at the same audience as Tipping Point or The Long Tail; i.e. the intelligent reader interested in trends. I don’t want to undervalue how critical such books are, but unlike those books this work brings much that is new to the table. Clay frames existing ideas in new ways that gives them additional energy. For example when he says “beneath the Coasean floor” he is illuminating a very radical idea. E.g. We can now solve large swaths of coordination problems in ways that are contrary to architecture of institutions as we have come to know them.

But at the same time Clay introduces many ideas I’d not seen before and which I suspect are fresh. For example it’s well known that modern societies are rich in groups whose domain is very limited. The neighborhood group that forms only to worry about traffic, or the diet club, the knitting group, the professional society each with it’s very specific topic. These go by the name communities of limited liability, and I’ve often encountered people who would prefer that these not be called communities because they are so carefully limited in their scope.

Clay points out a wonderful reason why we choose to create and maintain these limits. First he reminds us of the birthday problem, i.e. that the chance is high that in a group of 30 people two will have the same birthday. That happens because the number of pair wise connections in a group is N!, i.e. grows very fast in the number of people. Now having a common birthday, that’s a delightful coincidence. But the same pattern has a dark side. As the group grows the chances of pairs with some deep-seated antipathy explodes as well. By keeping the scope of the groups limited so some very narrow topic the chance that these will be uncovered and go onto polarize the group is kept low.

There is much more that I’m loving about this book. So I’ll no doubt be talking about it more over the next weeks. But I think if you’re the least bit interested in groups you gotta read this. Like I say it should be in your bookstore today – get a move on!

Power Spikes

072502triglight2.jpgEnergy Storage is a means to smooth out intermittent power supplies or demands. It lets you shift the power from one time period to another. It lets you create a distribution system which sized for average load rather than peak load. The more isolated your power system the more you need all these. The electric power industry calls isolated systems island grids.

The book I’m reading describes an island with abundant power, via hydro, but lousy power due to local industry’s intermittent and high power requirements. So lousy that they were importing fuel to run generators just to raise that quality. In that case fly wheels were the solution. There are a few subway systems that take the power spikes from breaking trains and store them into flywheels to put back into the system when the next train starts up. There’s a huge flywheel at a mine in Alaska that buffers 5 megawatts of power every few minutes as a drag line they have jerks about.

Thinking about power spikes got be thinking about extreme cases. That’s natural for me since I presume that most sources of supply and demand have a very skewed distribution. For example the book includes an example of capturing power spikes from high wind gusts (which are quite skewed) by using fly wheels on an island off the coast of Japan. I wonder how large a spike that system can grab before it let’s it pass. Presumably it isn’t designed to absorb abundant energy from the few huge storms the island probably experiences every year. So there are power spikes the system is designed to survive rather than harvest. Which given the nature of highly skewed distributions means that a lot of the power goes untapped. Your typical wind power system harvests only the long tail of the wind, but not the elite wind.

So I got to wondering about lightning strikes. There is some great nutter material on the web about that. For example at the always amusing site “half bakery” there is a suggestion to running the lightning into an underground cavern and using it to melt tungsten. I love all these crazy ideas; particularly one that mix together other enthusiasms: kites, space elevators, or super conductors.

Poking around looking at these amusing ideas led me to this outrageous gimmick: a solution to an age old problem, the tendency for lightning to never strike twice in the same place.  Lightning rockets! You shoot a rocket into the storm trailing behind it either a wire or a conductive smoke trail. That’s fun to think about, but it’s much more fun to watch the short video!.

Iowa Stored Energy Park

For some reason I’ve become interested in energy storage.  I seem to be thinking that storage is the nub of solving the problem of peak oil (et. al).
So, currently I’m reading Energy Storage: A Nontechnical Guide (isbn, worldcat) which was published in 2006.  It’s a fun book for a geek.  Fun, like reading popular science but a bit more serious.  It focus is on the electric utility industry and it’s working premise is that something is about to happen in that industry which parallels a switch which occurred in the natural gas industry a few years back.  The natural gas industry was deregulated so that a number of elements of the distribution system, storage included, were broken out in ways that created a bloom of different markets.  The author Richard Baxter states that both before and after this change the natural gas industry managed to limit the cost and extent of new distribution pipelines because they used storage as a way to smooth load.
Un-bundling creates distinct markets for lots of different qualities.  For example an electric distribution system needs to keep everything running at 60 Hertz, and some of these storage systems are useful for archiving that.  Dependable steady power is another quality and these systems enable variable demand (such as daily or seasonal variation), variable supply (such as wind or tidal power), or distribution bottlenecks to be smoothed over.  And then there is the usual buy low sell high scenarios where the storage systems enable market timing.  The point is that these systems have lots of value generating scenarios that one might not notice at first blush.

There are only two schemes for storing huge amounts of energy; hydro and compressed air.  The compressed air systems push the air into an underground storage cavity, usually a salt dome.  Which is nice since they are easier to get permits for in developed regions, as contrasted with hydro-systems that sit on top of scenic mountain tops.  I was surprised that the compressed air systems are implemented as complements to natural gas powered generating stations.    Apparently a natural gas power generating station needs a lot of air, and it wants it under pressure; so absent a source of compressed air it spends some of it’s output to compress the air – 66% of it’s output!
Which brings us to this marvelous hack – the Iowa Stored Energy Park.  This place deals in four kinds of energy.  It has a large wind farm.  It has a large compressed energy store.  It has a large natural gas store.  It makes electricity.  The result is a marvel of arbitrage opportunities.  Buying and selling across numerous markets.  For example profiting on seasonal volitility in the natural gas markets.  While at first blush you might think this thing is about smoothing out the wind energy produced by their wind farm I suspect that only time will tell what aspects of the operation are the most valuable.  Baxter points out that there are profitable scenarios where compressed-air/natural gas electricity producers run both the compressors and the generators at the same time to selling frequency stabilization services.    I suspect that their own wind farm is a demonstration, than that they can sell a smoothing service to all the other wind farms in the region.

Of course from here on in I’m going to think of my laptop battery as an energy park and yearn for a compressed air version.

de jure standards versus de facto standards

Standards are often a land war; where members of the standards body act to create de facto standards at the same time they participate in the negotiation of de jour standards. The mix varies. In some contexts the land war dominates. But it’s real politics and certainly tastes dirty. Many standards bodies have clauses in their members agreements to prevent other kinds of dirty gaming. For example clauses to temper the member’s ability to playing stick-up with their patent rights. It’s harder to do in these cases; since it would require the member to temper their striving for market share. This plays off interestingly against the device where a standards body retains for it’s members early access to the specification – a device intended to give the members the reward of early mover advantage. This pattern also reminds me of the scheme were a firm creates a “open standard” so other market players chase his tail lights. Card space is a good example of that move.
Meanwhile there is another delightful introductory paragraph in that posting.

Latitude v.s. Longitude

I very much liked this introductory paragraph:

Years ago I read an interesting article about the encyclopedia entry for the keyword “Longitude”. According to the article, the entry merely said “See Latitude”. With that short, two-word sentence the encyclopedia author conflated these two concepts as mere orthogonal dimensions, lumped together, each as boring as the other. This ignored the fact that latitude is boring, easy, trivial, known to the ancients and as easy to calculate as measuring the altitude of Polaris. But longitude, there lies an epic adventure, something fiendishly difficult to calculate accurately, something that propelled a great seafaring nation to a search for accurate timepieces that would work at sea, just in order to more accurately calculate longitude. Books have been written about longitude, lives lost, fortunes made. But latitude — latitude is for children.

Complementary pairs appear through out the world of standards. Often one of these is easier to pull together than the other. After the fact or at if one is only looking casually this difference in cost tends to be forgotten. One of the many places where at first blush two things appear the same, but as you get closer they are not. Delightfully you can actually use this cognitive effect for humor.

The skewed distribution of Torrents

Oh, yummy. Some data on the distribution of torrent traffic. Not what I expected. Curious!

That’s at Guillaume Besset’s blog, he’s also a collector of highly skewed distributions: More…

He’s also working on a p2p jabber based wiki, Bouillon.  I found this provocative: “… Everything you see in Bouillon is recommended by your friends … ”  Of course given the way marketing people use the term relationship, Google, the Library, Wikipedia, and the Newspaper are my friends, aren’t they?

GDP/Capita v.s. Inequality

Interesting chart (By way of some nice ranting by Dr. Traven.):
gdpcapitavsgini.png

I didn’t click around enough to see the data sources.  I presume that’s Japan up there on the left.  I’d not seen this before, so I’m suspicious.  Also, given how hard it is to pull a meaningful average out of a highly skewed distribution, like the wealth distribution, I’m always a bit uncertain what to make of a concept like GDP per capita.  For example, there are huge number of people in the US who make less than 10K/year.

If this was all you knew, and you want’d to raise GDP you’d start by addressing inequality.

Attack poodles

Nasty polarizing rhetoric dressed up as entertainment is par for the course these days.  Mixing in the entertainment makes them particularly virulent.  James Wolcott is particularly skilled as parsing out the various patterns used in this toxic political theater.    Here for example he sketches the outlines of a nasty attack on Hillary Clinton.  In this example the libertarian attack poodle uses a bad word and then runs around in circles snapping angrily at it’s own tail (nominally punishing it’s self for such naughty behavior).  Meanwhile the other poodles stand around an yap.

Just do it

As requests go “How long will it take?” is one of the tough ones. Taleb’s book Black Swan makes an thought provoking point about it. Like most of that book the underlying question is what statistical distribution the data is drawn. Naturally, that should color our expectations.

Some project durations are reasonable, Gaussian. How long you will live, for example. As the project unfolds the expected end time draws progressively closer. Delay in these projects push out the end date; yes but, you do get always closer to the finish line. Such projects can be standardized.

But many projects are unreasonable, their duration is drawn from a highly skewed distribution. Chock-a-block with extreme cases and little black swans. In this case as the project unfolds it’s end date moves further out. Each delay increases the expected time to complete the project. Taleb’s example is the refugee who imagines that each passing day brings closer the day he will return home, but as that is likely the this second kind of project, these passing days infact push out that day.

You manage these two kinds of project in very different ways. Most projects are a hybrid. The nature of hope is very different for one v.s. the other.

See also: Time to Market

Search v.s. Community

It seems to me that these two are complements.  Search is outward facing and community is inward.  Search is many many short transactional relationships, like shopping, while community is deeper longer transactions – i.e. real relations.  That Google is a search company and Yahoo is a community company.  Even Facebook, for all it’s flaws, is a community company.

Advertising is more at home in a searching context, and less so in a community context.

Huh?  I don’t seem to have told the whispering hammer story.

As the marginal cost of goods decline there is some threshold where advertising can pay.  Information goods, who’s marginal cost approaches zero, are the most common commodity paid for with advertising.  But my barber gives away cheap combs, address books, key chains, credit card magnifiers, and a half dozen other things.  His phone number is printed on them.  I picked up the bag I carry my computer around in at an Apache Software Foundation conference some years ago.  It was paid for by advertisers.  I will note in passing that I carry both my barber’s comb and ASF bag because of my enthusiasm.

So the question arises why advertising doesn’t appear in some venues where the marginal costs are extremely low.  For example why did Eudora’s Ad supported email client fail? Why doesn’t Microsoft sell a discounted version of Word that features embedded advertisements.  I think the reason is because people find it extremely irritating to have their concentration interrupted by their tools.  Advertising is intolerable when in the flow of real work.

My summary of this insight was a joke.  Presumably the marginal cost of a hammer is really quite low.  Not much higher than a plastic comb.  So someday soon we should see the whispering hammer.  As you lift it to strike the nail it would whisper advertisements for band aids and stainless steel nails into your ear.

Well, so it maybe that search v.s. community is one of those complementary balances that shakes out with one side being commoditized while the other garnering all the profits.  I don’t know.  But commerce does have a significant corrosive effect on community and this appears to be more of the same.