Category Archives: power-laws and networks

Paleotempestology

Leave a reply

Sitting in the small Vietnamese restaurant in Western Massachusetts an ominous dark cloud slowly delivered one of those marvelous downpours that sometimes end hot summer days. For the woman at the next table the sky was bright one moment; the next the windows were sheeted with water. A young man walked across the square, hood up, jacket unbuttoned, tee shirt glued to his chest.

The woman spoke of global warming to her companions: a friend and a husband. She mentioned that she thought such storms were becoming more common. She mentioned hurricanes. His counter point was that we don’t know much about hurricanes, maybe a few decades. No doubt this was only idle conversation; but I quietly leaned across my table and whispered to my wife. “This is important. Somebody is wrong on the Internet!”

I’ve been haunted by this conversation. He’s wrong. She’s right.

Finally the ghost of the stairwell was sufficiently frustrated to manifest himself, at least in virtual form. Fresh off the presses “Weather and Climate Extremes in a Changing Climate” – From the Brochure: “More frequent and intense heavy downpours and higher proportion of total rainfall in heavy precipitation events.” – “Very Likely.” From volume 3, chapter 2 of the final report: “Paleotempestology is an emerging field of science that attempts to reconstruct past tropical cyclone activity using geological proxy evidence and historical documents.”

Ah, if only. I could have leaned over and said “Paleotempestology!” Well, as we like to say around my house “Oh, tell it to the blog!”

Negative Energy

4 Replies

I have sighted a new urban myth: Electric heating is cheaper than oil heat! Here in Boston people heat with both gas and oil, and the cost per unit of heat between the two has diverged rapidly over the last few years. Those who heat with oil are looking for ways out of their plight. Apparently the rumor making the rounds that it is cheaper to use electric. That’s not true.

In related news Martin  brings my attention to a company EnerNoc that sells negative energy, i.e. load shedding, to the utilities. They use telecom and widgets to shift power consumption from high demand time periods into low demand time periods. Martian’s example is the fridge. You chill when power is plentiful and let it coast when others are paying higher prices.

I assume that EnerNoc’s role in all this is to aggregate small power users into a large enough pool to be worthy of selling to the utilities. It’s a interesting example of a coordination problem. There are of course other ways to approach the problem; ones that are less dependent on a thicket of contracts and ongoing coordination signals controlled by a middleman and enabled, as Martian, points out by the telecom infrastructure.

The obvious alternative is to just broadcast signal; and let the demand side react to the signal by selling some simple technology that responds to the signal in reasonably simple ways. That alone would enable substantial contributions from the demand side. But you can improve the incentive structure either thru regulation or by using statistical sampling to tell which customers have gotten with program; and then reduce their tariffs.

The amount of signal that needs to flow from the grid operators to the consumers is small, in the sense that you can broadcast it. A signal only needs to flow back the other way sufficient to assure that the incentives play out right. It is stupid to presume that the only incentives that are available are monetary or that they need to be executed with fastidious accounting. Most social systems have very fuzzy accounting and they work just fine, thank you!

The puzzle to be solved here is how to draw more of the peripheral demand into a load balancing system. Reading about EnerNoc’s approach isn’t the first time I’ve seen discussion of this. For example Bruce Schneier mentioned a regulatory attempt at something similar. I liked that one a lot, it provided a way to signal household thermostats. He was concerned that the resulting system would attract hackers. I presume he’d be just as sanguine about the security of the EnerNoc system; probably more so since it’s a closed system.

Such concerns are appropriate, but for heaven sakes I wish smart people like Bruce would stop pretending that these cases are somehow unique. It is the very rare large scale system that doesn’t have vunerable choke points. Hubs who’s failure can bring the entire system to it’s knees. Telling designers not to build large systems because of those risks is lame. Helping them know how to build them so they are safe and robust is hard, yes. But these systems get built because they generate mind boggling amounts of value. So it’s better to do the hard job and forgo the short term pleasure of a bit of hysteria.

Speaking of load shedding: turning your car’s engine off when you stop is more efficient than you thought.

Man’s work – depolarization

Leave a reply

barfight.jpgThis is a long, interesting, carefully written article by Alex Kotlowitz about a program to change community behaviors in service of decreasing violence. It is nice to see a discussion of how to reduce polarization. There is so much entertainment value in polarization that the media revel in it. It is further to the article’s credit that it doesn’t play the card of Chicago’s most recent violent troubles into the text.
If I was to be critical of the article it would only be how it leaves the impression that the means being used here were entirely gin’d up by the folks involved. I certainly hope that many of these were imported from other violence reduction efforts, and most of the methods outlined I’ve read about in other venues; particularly the negotiation handbooks.

Given my interest in group dynamics I was interested by the story about a kid who wanted to resign from a gang. Much negotiation through various possible outcomes, “no”, “well, maybe, after a beating,” until they settled with the kid paying a fee. Not unlike getting out of your cell phone contract, eh? Joining a group often involves a hazing, and I suspect leaving often involves a beating – or at least a standing debt.  You can’t make the books balance in such situations. They never balance between members inside a club, that’s the point, so you can’t make them balance when you exit.

That groups are a sources of violence is clear. You get discipline inside, and polarization between, but I hadn’t noticed before that movements of members across the group member boundaries can also generate violence.

One other thing that caught my interest, because it’s analogous to atomization effect we see in other systems these days. “Many of the interrupters seem bewildered by what they see as a wilder group of youngsters now running the streets and by a gang structure that is no longer top-down but is instead made up of many small groups – which they refer to as cliques – whose membejoebenadam_ahd.jpgrs are answerable to a handful of peers.” When the middle scale systems die off you left with nothing but long-tail, and elites in their walled-gardens.

The skills these “interrupters” have are skills that we all should have. Skills the media should model. A culture rich in these skills can tolerate a lot more diversity. That there is honor in turning the other check. It is man’s work to intervene in a fight. The article is here.

Bamboo Mast Year

Leave a reply

Facinating article about a mast year in bamboo. Rat’s thrive and the following year people starve.

… Mizoram and Manipur … rats … army … teach … eradicate … flowering … bamboo … increase fertility rates … population explosion … crops … severe famines … the mautam in 1958-59 … triggered an insurgency … redrawing of boundaries…. Bamboo Flowering and Famine Control Schemes … rat tail … bounty … 200,000 rat tails… registering more tails… hunger currently prevail … grain production falling by 87% …collapse of the administrative machinery … refugee flows … worst is still to come….

Tip of the hat to Koranteng Ofosu-Amaah.

Levy Walks

3 Replies

Search in a space requires poking around. In fact random search is the canonical strong method. Presuming you have time, can cover the search space, and recognize a solution it can solve any problem. Of course, drunken random walks are well known to result in repeatedly bumping into the same parking meter. So it is with some pleasure that I stumble upon a different kind of random walk. Better yet these walk’s principle feature is that their moves are power law distributed.

Levy walks are observed in nature. Searching animals have only a limited awareness of where food might be found. Outside that awareness they are reduced to random search. But they don’t move randomly in the sense of Browning motion, nor do they randomly select some destination and then move to it. Rather they appear to draw a random number from a highly skewed distribution and then make that move. If you browse the literature on Levy walks you can see that monkeys, humans, and many marine animals all use Levy walks to organize their search. Here’s a picture showing an example of those three kinds of walks.
randomwalks.png

I find this fascinating. As an engineer I’m surprised that I don’t think I’ve ever saw seen a search algorithm using this method. Looking down from above, this means you could observe the trajectory of any object and this simple statistic would tell you if it was alive. Looking out from within, since this is presumably a largely scale free phenomenon it’s quite suggestive about the trajectory of one’s life.

The skewed distribution of Torrents

Leave a reply

Oh, yummy. Some data on the distribution of torrent traffic. Not what I expected. Curious!

That’s at Guillaume Besset’s blog, he’s also a collector of highly skewed distributions: More…

He’s also working on a p2p jabber based wiki, Bouillon.  I found this provocative: “… Everything you see in Bouillon is recommended by your friends … ”  Of course given the way marketing people use the term relationship, Google, the Library, Wikipedia, and the Newspaper are my friends, aren’t they?

Just do it

Leave a reply

As requests go “How long will it take?” is one of the tough ones. Taleb’s book Black Swan makes an thought provoking point about it. Like most of that book the underlying question is what statistical distribution the data is drawn. Naturally, that should color our expectations.

Some project durations are reasonable, Gaussian. How long you will live, for example. As the project unfolds the expected end time draws progressively closer. Delay in these projects push out the end date; yes but, you do get always closer to the finish line. Such projects can be standardized.

But many projects are unreasonable, their duration is drawn from a highly skewed distribution. Chock-a-block with extreme cases and little black swans. In this case as the project unfolds it’s end date moves further out. Each delay increases the expected time to complete the project. Taleb’s example is the refugee who imagines that each passing day brings closer the day he will return home, but as that is likely the this second kind of project, these passing days infact push out that day.

You manage these two kinds of project in very different ways. Most projects are a hybrid. The nature of hope is very different for one v.s. the other.

See also: Time to Market

Visa risk

Leave a reply

Income inequality effects firms and individuals. Rising inequality moves hand and hand with increasing skew in the distribution of firm size. For firms this goes by various cliched names, consolidation for example. When it becomes extreme then we talk of anti-trust. In this framing talk of what’s good for small business is analogous to talk about what’s good for the poor and in those terms the ideas of consumer protection are mimic’d by regulations, like anti-trust, that temper the power of the large firms over the small ones.

The payment’s industry is a good exemplar for all these dynamics. Follow the money, and all that. There is a lot of high stakes antitrust litigation (2.1 Billion example) in the payments industry. Personally I think these cases should get more press coverage.  Visa has filed to go public (10 Billion).  These exchange networks are big money; e.g. UPS, or AT&T wireless (10.6 Billion) As an aside I find this is interesting from an industrial standards point of view. It makes me wonder if the folks that run the Universal Product Code hub could go public too?

The risk section in Visa’s S1 makes for interesting reading. It’s a guided tour of the business model. I love S1 since they are a rare attempt to speak clearly about the business. A good example: “Our management team is new and does not have a history of working together.” Where else would you get to see that? Or that they have a complex partnership with Visa Europe full of technical stress; and options for significant financial events they don’t control.
Regarding the introduction above it says that both banking and retailing are becoming more consolidated. It’s nice to have diffuse and weak neighbors in the supply chain; it’s a pain to have powerful neighbors since they tend to be difficult to negotiate with.

Consumer protection, i.e. government regulation targeted to protect consumers, hardly shows up in their list of concerns. Government regulation does. Privacy regulation is the closest the come to a consumer protection worry; and that appears to be less of a worry than dealing with the thicket of regulations thrown up after 9/11.

Its telling that they frame their concerns about credit card fraud entirely in terms of it’s effect upon the brand.
I love to anthropomorphize and these dozen pages are a great window into what keeps a payment network up at night.

Do you feel lucky, Punk?

Leave a reply

Brad DeLong picks up some postings arguing that Google destroy’d the garden wall revenue model that the New York Times was using for Times Select and that others (e.g. the Financial Times and the Wall St. Journal) aren’t long for this world.

My first nit to pick here is that it wasn’t Google, but the tsunami of disintermediated content that blew up that business model. If you looking for the institution to blame it was the internet and it’s end-to-end design principles. Google had nothing to do with it. Well maybe it had a tiny bit to do with it; but it pains me how people are unable to distinguish the value of Google from the value of the content it is now the intermediary for.  This is like confusing the card catalog for the library, or Sony for the Shastakovich.
Brad then goes on to say something much more interesting to me.

I suspect we are headed for a winner-take-all situation here as well: journalists who acquire reputations as experts will do very well as they become draws for advertisers. Institutions–not so much. Anybody who trusted the New Republic under Michael Kinsley and then encountered the New Republic under Andrew Sullivan, Michael Kelly, and Peter Beinart learned a very painful lesson about focusing on institutions rather than people.

That is indeed the pattern. Winner take all maybe a bit exaggerated, but the pattern is clear; and this is the yet again the question of what the highly skew’d distribution of economic entities will look like going forward. As we skew more severely we end up with fewer intermediaries and everybody else cast into the role of a solitary player. Institutions fail, certainly, but it is ironic that in a single paragraph Brad can suggest that we should expect a single institution, Google, to win it all and then advise us not to focus on institutions; just on the solitary players.