UPDATE (December 19, 2009): Peter Staats, in the comments, suggested that belief in anthropogenic global warming is entrenched among scientists and will disappear as the older generation dies (citing Planck, whose point is also made in Thomas Kuhn’s Structure of Scientific Revolutions). I responded that I thought he has it backwards–that AGW has become more and more supported, and the holdouts tend to be older, as some of the data about the anti-AGW organizations above already suggested. So I tested our respective hypotheses against Jim Prall’s data, for IPCC WG1 scientists vs. the signatories of the AGW-skeptical documents. I looked at the average year of the last academic degree awarded, first for those with citation counts for their fourth-most-cited paper >= 200, then, since that was such a small sample for the climate skeptics, for citation counts >= 100, and then for all the 623 IPCC WG1 scientists vs. the 469 signatories of AGW-skeptical documents. Here are the results:

Citation counts of 4th-most-cited >= 200:
IPCC WG1: N=83, 12 w/o year, N=71, average year of last degree = 1981
Skeptics: N=13, 4 w/o year, N=9, average year of last degree = 1965

Citations counts of 4th-most-cited >=100:
IPCC WG1: N=201, 51 w/o year, N=150, average year of last degree = 1983
Skeptics: N=38, 15 w/o year, N=23, average year of last degree = 1968

All IPCC WG1 vs. AGW-skeptical document signers:
IPCC WG1: N=623, 208 w/o year, N=415, average year of last degree = 1989
Skeptics: N=469, 346 w/o year, N=123, average year of last degree = 1973

In short, the skeptics are 15 years older than their opponents.

My mind is settled on this whole climate issue.  But, it would be fun to have a list of other conflicts with a strong generational component.  For example, I think I’ve written about how I think Microsoft is caught on the old side of the swing back to data center computing and it’s associate control of the customer and distribution channels.  I suspect there is one in the Business schools around platform based business architectures.

In a somewhat related bit, I’m subscribed to various pages at wikipedia.  Apparently there is some kind of lower life form that draws satisfaction from swooping into pages and adding “citation needed” after the first period, and so naturally there are mechanisms that chase after this to remove them.   This silly activity happened on one page I was watching the first sentence read “Given two similar rewards humans show a preference for one that arrives sooner rather than later.”

And so, I was delighted to stumble upon a possible citation for that.  Via Crooked Timber, quoting Richard Tol and his co-authors:

Estimates of utility discount rates for individuals are almost always positive – an estimate of 1.5% is considered plausible for the UK for instance (HMTreasury, 2003) – for the simple reason that humans prefer good things to come earlier rather than later. Given the inevitability of death for individuals, a preference for benefits to accrue earlier rather than later is entirely sensible.

John Quiggin adds:

We can sharpen this up a bit by observing that the average annual mortality probability for adults is around 1.5 per cent, suggesting that this factor alone is sufficient to explain positive time preference.

That whole post is thought provoking but what has been stewing in my head since reading it is how immortals might manifest a very different discount rate than we mortals do.  The planet, society, nations, and many corporations act as if they are immortal.    This leads to a pervasive mismatch in the discount rate between an group and it’s members.  That’s the tension of the prisoner’s dilemma, the gang would prefer that the game continue.

In the section I was reading last night he gets to musing that there are two flavors of clubs: inclusive v.s. exclusive.  Both create club goods.  The default tendency of an inclusive club is to welcome new members, while members of exclusive clubs tend to prefer that the membership shrink.   For example a lobbying group seeking lower taxes on industry will be inclusive, the more the merrier; meanwhile a members of an industry will tend to prefer that the number of competitors shrink.   Thus, for example, the members of an industry standards body with associated patent pool may prefer to trend toward a smaller and smaller membership.   In another example a professional society who’s function includes granting professional status to it’s members will might tend toward exclusive; e.g. a profession with a 1o0 members can charge more for it’s services than one with a 10 thousand members.

I sense foreshadowing in Olson’s plot development.   Since any hint of  selfish motivation always excites economists I’ve little doubt he will soon explain how an exclusive club is more sustainable.

The name of this blog is about this topic.  A club of enthusiasts will not casually ascribe membership in their club to others, in fact doing so would offend them.  The enthusiasm acts as a binding force for the group, and dragging in those who lack it is likely to weaken that.   Members of the local model rocket club may engage in outreach, striving to juice up other people’s enthusiasm for the hobby, but it would be quite odd for them to randomly pick people and anoint them as members.

Clubs labor to create club and public goods.   That is their primary goal, but other goals get piled on sometimes intentionally sometimes by happenstance.  An example of that is how members of some clubs often garner status.   To start the goal is to create the good, in support of that the club fiddles with the process that maintains the pool of members.  As an unintended side effect membership becomes selective, and hence elite, and hence confers status.   This status creation wasn’t the goal of the club, and in fact it’s a distraction from the goal.  Good members are engaged with the primary goal, creating the goods; further they are peeved at the way the status becomes a distraction from that.  Meanwhile, outsiders who are much less concerned about creating the good tend to see only the status.  How perverse is that!

Most clubs are actually a hybrid, of course, you want members who contribute to the goal but at the same time you want to extend a broad and enthusiastic welcome to any and all; since how else can you hope to search out the enthusiastic members.

No doubt there are scenarios where the agenda of a club is entirely captured by the status generation.  As an outsider it would be hard to know for sure; for example I know practically nothing about Mensa, which was the first club that came to mind as existing entirely to create status for it’s members, but for all I know if I was inside that club I’d know that it exists to provide a delightful pool of fraternal activities for it’s members.   The insider/outsider problem is fascinating.  For example I certainly feel that people who own Rolex watches are just buying status, but then you discover that lots of those are sold to people who own multiples and have special watch humidors for their collections.  It’s confusing.

Inclusive/Exclusive is not the only dialectics that can be used to sort clubs.  It’s probably worth starting a list.

• Primary goal: create club goods v.s. create public goods.
• Focus is inward facing (i.e. on some project (think barn raising)) v.s. focus is outward facing (for example on some threat)
• Membership – is naturally inclusive v.s. exclusive; e.g. the existing members would naturally prefer more or less members going forward.
• The club tends has high/low capacity to exclude others from access to it’s club goods.
• Clubs that create a broad range of goods v.s. clubs that fix upon a single good
• Clubs who’s principal good’s quality depends upon the quality of the members contributions such that the best contribution sets the quality v.s. those where the worst contribution sets the quality.

These are not entirely orthogonal, are they?

• Social

Demographic Pressure

Displacement

Group Grievance/Paranoia

Flight

• Economic

Changing Inequality

Decline

• Politics

Club losing it’s legitimacy

Deterioration of club services

Arbitrary rules and abuse of member rights

Failed oversight/auditing of those who enforce the rules

Polarization of Elite members

Loss of the club autonomy, intervention by outsiders.

All these pressures ebb and flow in communities.  You get demographic pressure when every you hire or fire employees, or when the students come and go.   Bureaucracies often fall back on arbitrary or abusive enforcement.  Oversight is always spotty.  The elites are rarely always on the same page.  These are always a matter of degree.

When the going get’s rough these all get tangled, so a list helps to tease them apart.   But they certainly reinforce each other.

For example.  Harvard, about whom I have zero personal knowledge, has suffered a substantial decline in it’s economics.  I gather there has been a decline in the services (no hot breakfasts) the university provides to it’s students.  Is talent in flight?  I’ve no idea if any of the other metrics have taken a hit.

Actors seeking to increase the level of discontent, i.e. violence entrepreneurs, can talk up all these to create an impression of pending failure.  For example talk up fear of foreign influences, immigrants, police abuse, battling elites, reduction of public services.  Sounds like the insurance industries and it’s right-wing agent’s battle against healthcare, eh?

If I scroll down through the list of things in my password wallet it’s quite nostoglic.  There dozens of mailing lists and web forums which I have warm memories.  For example I used to lurk on the tornado chasing list server back in the late 1980s – good times!  I particularly remember a long thread about some product they would all buy and spread over their wind sheilds; apparently once applied you didn’t need to turn on your wipers.  They all swore by it.  Rain-X I think it was called.

The impressive thing about this spam I got this morning was it worked.  They successfully reminded me of why I enjoyed hanging out at that site and drew me back for a visit.  This was the key: what do you call that thing where an egg is cooked in a hole in a slice of bread?

Lots of organizations have huge peripheral networks of related parties.  Alumni networks for example.  Community networks.  Product owners. etc. etc.   When I think about the kind of communications I get from various groups targeted to me as a member of this or that peripheral network it’s just sad.  They are so heavy handed, over done, and often kind of needy.   As the example above illustrates that doesn’t need to be the case.

So, something to think about.  Why do community organizers do such a lousy job of this kind of sustaining of the long term low key relationships.  Why do we all do such a lousy job of it.  Shouldn’t this internet thing should make it a lot easier?

Good monitoring will draw our attention to surprising behaviors.  Surprising behaviors trigger debugging projects.  The universe of tools for gleaning out surprising behavior from systems is very large.   Years ago, when I worked at BBN, the acoustics’ guys were working on a system that listened to the machine room noise on a ship hoping to sense that something anomalous was happening.

I attended a talk “Using Influence to Understand Complex Systems” this morning by Adam Oliner (the same talk performed by his coauthor Alex Aiken is on youtube) where I was again reminded of how you can often do surprisingly effective things with surprisingly simple schemes.

Adam and Alex are tackling an increasingly common problem.  You have a huge system with numerous modules.  It is acting in surprising ways.  You’ve got a vast piles of logging data from some of those modules.  Now what do you do?

Their scheme works as follows.  For each of the data streams convert the stream into a metric that roughly measures how surprising the behavior was at each interval in time.  Do time series correlation between the modules.  That lets you draw a graph: module A influence B (i.e. surprising behavior in A tends to precede surprising behavior in B).  You can also have arcs that say A and B tend to behave surprisingly at the same time.  These arcs are the influence mentioned in their title.

If you add a pseudo module to include the anomalous behavior your investigating, then the graph can give you some hints for were to investigate further.

At first blush you’d think that you need domain expertise to convert each log into a metric of how surprising the log appears at that point in time.  But statistics is fun.  So they adopted a very naive scheme for converting logs into time series of surprise.

They discard everything in the log except the intervals between the messages.  Then they keep a long-term and a short-term histogram.  The surprise is a measure of how different these appear.  The only domain knowledge is setting up what short and long-term means.

The talk includes a delightful story about applying this to a complex robot’s naughty behaviors, drawing attention first to the portion of the system at fault and further revealing the existence of a hidden component where the problem actually was hiding out.  Good fun!

I gather that they don’t currently have a code base you can download and apply in-house, but the system seems simple enough that cloning it looks straight forward.

They would love to have more data to work on, so if you have a vast pile of logs for a system with lots and lots of modules, and your willing to reveal the inter-message timestamps, module names, and some information about when mysterious things were happening.  I suspect they would be enthusiastic about sending you back some pretty influence graphs to help illuminate your mysterious behaviors.

It would be fun to apply this to some social interaction data (email/im/commit-logs).  I suspect the histograms would need to be tinkered with a bit to match the distributions seen in such natural systems better.  Just trying various signals as to what denotes a surprising behavior on the part of the participants in the social network would be fun.   But it would be cool to reveal that when Alice acts in a surprising way shortly there after Bob does; and a bit later the entire group descends into a flame war.

Back in high school I Jane Jacob’s books on the economics of urban regions schooled me in a cynical attitude about these stories about site optimization.  I recall learning that the most powerful predictor of where a large firm would sight it’s new office park was the distance from the CEO’s wife’s horses.  So I wasn’t terrible surprised that one of Microsoft’s big data centers is in the country side of east of Redmond.

FYI – the drawing above is terribly misleading.  For wholesale power West Texas is a steal right now, wind power.  For a residential power consumer per month cost to connect to the grid tends to be a large additional cost.   I wrote about that under the heading of “micro-utilitity coops” using the gas company as an example.  Since then I’ve learned there is a nice term of art in the utility industry “islanding.”   That’s worth reading about if your want yet another way to look at the issues around localism.

Islanding is one of the themes that runs thru the discussions of cloud computing.  But it goes under various guises (security, control, specialization, cost or ops, capital equipment, bandwidth, latency).   That I continue to presume that anybody who can make a credible case for building their own island will be able to negotiate a pricing deal with their cloud vendor means I’m starting to think that people who run their own data centers feel like fellow travels with other the off-grid enthusiasts.  You gotta love ‘em.

Presuming we have solved both the problem of aggregating the group and extracting the value then points on that surface more valuable per size-of-group * value-of-member.  This is the calculation that any site with an audience makes, or any shop with regular customers, or any club.  The definition of value varies a lot.  A knitting group wants something different from it’s members than does a standards body.   The word authentic get’s tossed around to label the miss-match between what affiliate marketing platforms (like Amazon, or Google ad-sense) value in site visitors v.s. what makes a site attract an authentic membership with some particular enthusiasm.   A lack of appreciation for how diverse of value is goes a long way toward explaining how dismissive people are of sites with narrow enthusiasms.   People dismissed open source for years because they were blind to the values that attracted it’s participants, people are no less blind today even if they are less dismissive.   It really pull my cord to watch observers rapidly dismiss sites of other enthusiasms just because they can’t be bothered to puzzle out what might be the value those members (or the site operator) has managed to find in there.

It seems useful to be clear that value-of-member has at least four aspects.  There is the member’s value the members see in each other (a p2p network scoped by the group).  The value the members contribute to the common cause of the group (a sarnoff kind of value to the groups barn raising).  The value the site owner (or steward) values in his members (i.e. a site for lawyers wants the lawyers who are highly respected and well networked to participate).   The value that feeds clearly into value extraction (i.e. the lawyer site values those who click thru on the ads or regularly subscribe to premium services).   Value is messy.

Presumably the universe of groups, the population, is distributed on that chart such that most groups are down near the origin.  Again it frustrates me how people are dismissive of those.   [Apparently I'm easily peeved ]  So they complain about the how github’s fuzz of forked projects is confusing, or how google code and source forge are cluttered with tiny projects; or how the numbers yahoo groups or ning has are inflated by groups with little or no traffic.

Which brings us to the question of aggregating groups.   We can visualize this as regions on that chart.  Consider the local Dayton Business Journal, it’s got an audience that is valuable in a particular way and when rolled up into a company like American City Business Journals (see the select city pulldown on one of their sites).  Or consider this set of  local newspapers around Boston.  In both cases the set of groups aggregated is some range of distances from the origin the definition value-of-member axis has been narrowed down.   That narrowing is in part tied to the cost of rolling up the aggregate, which presumably involved negotiation and money.   Sourceforge is a different story.  They rolled up their groups organically which goes to explain why they have a lot of groups close to the origin.  Sourceforge’s value-of-member definition isn’t very broad spectrum.  But there are platforms where you see extremely broad spectrum value-of-member definitions.  Plenty of examples: Yahoo groups, Meetup, Ning, vBulletin, WikiSpaces or Acquia are all examples.   I’ve often thought that Yahoo’s strategy was to roll up these kinds of companies; and it’s a puzzle why that aggregate hasn’t turned out to be more valuable.

Well, it’s all food for thought.