Category Archives: power-laws and networks

Match Making Machine

1 Reply

Match making is at the heart of most middleman functions.  The buyer and seller work thru the middleman to improve the chance of closing a transactions.  Both sides reveal information to the match maker who then puzzles out if a deal can be closed.

The negotiation literature is full of examples where the negotiations fail, in spite of the existence of a deal to be had, because the two parties reveal information along a path that leads to a lousy outcome.  The simplest example is negotiating over price.  The rule of thumb is that once each side has bid a price the only possible outcome, short of a great deal of negotiation, is to split the difference.  The rub here is that each party has a space of acceptable deals and the challenge of the negotiation is to both discover if they overlap and then give that find a good point to close on.  But the moment one side reveals anything about his space of acceptable outcomes the other side will adjust what he reveals.

The following illustration is taken from a paper that suggests a way, using cryptography, that two parties might check if they have found common ground without actually revealing anything about the ground that is acceptable to them.  The paper frames the match making as a romantic problem (Romantic Cryptography (pdf) in the Journal of Craptology).  The two sides want to know if they love each other without revealing their love.

The balance beam below tries to solve this problem.  The players place tokens on the right side.  These tokens are identical to the eye, but are either light or heavy.  In the drawing the heavy tokens have a dot shown in their center.  A player who loves the other player places a heavy token on the scale.  Once both players have placed their tokens a pin is removed and  if the scale falls they both love each other.

You could use this in price negotiations.  You using a spinner to suggest a price and then both sides play a round; repeating until a mutually acceptable price is discovered.  It could also be used to make all kinds of self revealing games; say with a deck of cards that ask questions.

Of course in this story the device, the scale, is acting as the middleman.  The paper sketches out how to do this kind of thing with cryptography.

I wonder if there is a simple way to cobble together a scale and tokens like this out of stuff likely to be lying about.  The need does arise.  Say for example when a team wants to know if they all think it’s time to cancel the project, delay the release, switch database engines, fire the project manager, hire this guy, …

The paper goes on to describe variation on the technique using transparencies.

Each player is provided with two transparencies, a yes and no vote, that look similar the ones above.  If the two yes votes are overlaid you get a heart.  Otherwise you get nothing.  The first approach requires some carefully made tokens and scale that works just os.  This technique requires preparing the transparencies for each round; but they can be manufactured before hand into a deck for use during the negotiation.  It’s fun to note that a similar technique can be used so the players can distinguish if which of their cards are yes or no.

They point out that there is a problem with lying.  If the game reveals both sides love each other it’s still a problem that they might say “Oh, just kidding!”    Placing a bond or playing the game in a group can help to put a price on that behavior.  There is somewhat different problem that raising a question reveals a lot, as in the example “Do you think we should fire Bob?”  Some of that can be addressed by creating a swarm of random but oft raised questions.

Is advertising spend as skewed as income?

Leave a reply

We know how skewed the distribution of wealth and income are. But I suspect that if you had data about the flux of advertising dollars you would discover that those dollars are targeted disproportionately at tail of the distribution.

I gather that it is well known that if you look at the ads in a given newspaper of magazine the ads are targeted toward the high end of their readership. Hence the ads for thousand dollar watches in the New York Times.

Possibly selling into the high end of the distribution is so lucrative that you don’t really use advertising as most of us think of it. The selling channels are different; you give talks at Ted and get booth space at the superbowl.

Which makes me wonder how different the ads are if my IP address is in 10028 (Upper East Side of Manhattan, avg income $227K/year) v.s. 10454 (South Bronx, avg. income $18K/year).  There must be advertisers who have figured that out.

Density, Hierarchy, and Power

Leave a reply

This is a post about why gerrymandering might not be as unethical as it appears; or maybe it’s about the hidden agenda of those who argue against it.

Each time you encounter a highly skewed (power-law) distribution the population spread out on the long tail can be assumed to suffer from a severe coordination problem.  Being numerous, banding together to advocate for their common interests is harder.  Meanwhile the population at the elite end of the distribution have a much an easier time coordinating their actions.  By an example consider overdraft fees – it’s a lot easier for banks to get it together for their prefered regulation than it is for bank customers.

The natural disadvantage of the diffuse and numerous doesn’t prevent the elite from worrying about  the little guy ganging up on them.  This is old news.  In Dicken’s Hard Times we get this dialog.

” All is shut up, Bitzer ?” said Mrs. Sparsit.
“All is shut up, ma’am.”
” And what,” said Mrs. Sparsit, pouring out her tea, ” is the news of the day ? Anything ?”
” Well, ma’am, I can’t say that I have heard anything particular. Our people are a bad lot, ma’am; but that is no news, unfortunately.”
“What are the restless wretches doing now?” asked Mrs. Sparsit.
“Merely going on in the old way, ma’am. Uniting, and leaguing, and engaging to stand by one another.”
“It is much to be regretted,” said Mrs. Sparsit, making her nose more Roman and her eyebrows more Coriolanian in the strength of her severity, ” that the united masters allow of any such classcombinations.”
“Yes, ma’am,” said Bitzer.
” Being united themselves, they ought one and all to set their faces against employing any man who is united with any other man,” said Mrs. Sparsit.
” They have done that, ma’am,” returned Bitzer; “but it rather fell through, ma’am.”

” All is shut up, Bitzer ?” said Mrs. Sparsit.

“All is shut up, ma’am.”

” And what,” said Mrs. Sparsit, pouring out her tea, ” is the news of the day ? Anything ?”

” Well, ma’am, I can’t say that I have heard anything particular. Our people are a bad lot, ma’am; but that is no news, unfortunately.”

“What are the restless wretches doing now?” asked Mrs. Sparsit.

“Merely going on in the old way, ma’am. Uniting, and leaguing, and engaging to stand by one another.”

“It is much to be regretted,” said Mrs. Sparsit, making her nose more Roman and her eyebrows more Coriolanian in the strength of her severity, ” that the united masters allow of any such class combinations.”

“Yes, ma’am,” said Bitzer.

” Being united themselves, they ought one and all to set their faces against employing any man who is united with any other man,” said Mrs. Sparsit.

” They have done that, ma’am,” returned Bitzer; “but it rather fell through, ma’am.”

In spite of that we encourage this uniting in numerous cases.  The California Raisin Marketing Board for example brings together small producers.  We often write regulations to encourage or force the pooling of common interests like that.  From trade association to the interlibrary loan system there are plenty of examples.

Often these schemes involve a certain amount of coercion.  Clearly an urban voter could be a little peeved to discover a Montana vote is twice as potent as his and no doubt there are some California raisin producers who would rather go it alone.

So the menu of schemes for enabling, encouraging, or forcing solidarity on the members of the long tail interest me.  These are the tools for community forming, and one of the most venerable is representative government.  You can’t run any large organization with direct democracy; sooner or latter it’s a good idea to encourage a bit of intermediation.  At some point the town meeting doesn’t work.  So you start electing representatives instead.

Ok, so here’s the point of this post.

Andrew Gelman draws our attention to an political science article (pdf) that throws new light on ethical puzzle of how to draw congressional districts.  The whinging crowd loves to make fun of how bizarrely shaped congressional districts are.  Oh the horror, they cry, look politics has tainted this process.  I’m not sure what they expected, but yeah.

Let’s say your state has two kinds of people, good people and bad people, and they are split evenly 50/50.  Your a good person and lucky you you’ve been given the power to draw your three congressional districts.  So, obviously you draw them so that 80% of the bad people are in one district; thus come election time you’ll get two good congressmen and you can limit the damage the bad voters can do to one congressman.  Such cartographic manipulation is obviously fraught with ethical implications.  Not the least of which is how it leads to increasing polarization, reduced competition and absence of discourse.

You might be able to achieve the above goal indirectly.  For example if all the good voters live in the country and all the bad voters live in the cities then you could lobby for rules that requires congressional districts to have small perimeters, i.e. to be compact.  That would tend to trigger districts bundle up those bad urban voters in compact little districts.  As a fall back position you can’t get of rules adopted that arguing for compact districts you need only argue for such districts on cosmetic grounds.

The article asks: Does a political party who’s voters live in dense regions have some significant structural advantage or disadvantage?  Their answer: yes, a big one.  If representatives are drawn from geographic regions and there is a preference for compactly shaped regions then a party that draws principally from rural areas will have a substantial advantage.  They focus on Florida and use detailed data for the two parties to show that a preference for compact districts gives the Republicans a significant structural advantage given the concentration of Democratic voters in urban districts.

One last thing, communities of common interest aren’t always aligned with geography.  While the raisin growers of California probably have some geographic coherence other groups, e.g. doctors or school teachers, don’t.  For the last century or two communities have been becoming increasingly a-geographic.  It is a fun sort of sci-fi like fantasy to imagine representative district lines that are dawn in virtual rather than physical space.  Then the senator from the Auto Industry wouldn’t have to live in Michigan.

Any x Any

1 Reply

Having spent far too much time thinking, writing, and attempting to bootstrap some two sided network businesses I’m impressed by this slogan from Google’s efforts to become the hub of bookville: “buy anywhere, read anywhere”, very clever.

Searching for Alternate Routes

1 Reply

RNA viruses may well be the ultimate r-selected species.  The life cycle of an RNA virus includes a few steps.  Infecting the cell, coopting the machinery of the cell, making copies of its self, assemble those copies into viral particles.  Then the offspring need to escaping the cell, avoid the immune system, and find a new cell to infect.  If it’s that simple then it’s seven steps.

I very much doubt it’s that simple.  In fact the illustration above shows just the bit where the virus enters the cell and off it’s coat.  There is an antiviral drug that works by frustrating it’s attempt shed it’s coat.  Obviously it get’s even more complex yet again if we add in how the virus moves between host animals.

But the copy step is notable. In quantity and quality.

a typical RNA viral genome of 10,000 bases, a mutation frequency of 1 in 10,000 corresponds to an average of 1 mutation in every replicated genome. If a single cell infected with poliovirus produces 10,000 new virus particles, this error rate means that in theory, about 10,000 new viral mutants have been produced.

The quantity is high, but the quality is low.  Amazingly there is method in this madness.  The combination of high errors and high numbers creates something useful, a search scheme.

If you want to frustrate a virus then you need to shutdown, or a least narrow, the pathway through which one of the steps in the reproductive cycle.  For example, improved hygiene and increasing social distancing works by making the movement between host animals harder.  Anti-viral drugs target individual steps in the cycle.  The immune system learns to recognize the virus and pick it off as it moves between cells.  In all these case the challenge for the virus is to route around the resulting bottleneck.

Since most of it’s offspring are mutants, most of it’s offspring are sacrificed to searching for these alternate pathways.  Like most r-selected reproductive strategies the vast majority of the offspring fail in the process.  When the spider has a thousand babies it works out because takes that many to searches opportune door into the next cycle of reproduction.  When the maple tree throws off billions of seeds during it’s life that works because it needs to run that many searches to find one that let’s it pass it’s genes into the next generation.

It must be vary frustrating for the inventor of an anti-viral.  The stupid viruses can mindlessly find a route around his clever invention.  Adamantine was approved for use in 1966, by the 2005-2006 US flu widespread flu strains had routed around the hole it plugged.  I think we can assume that the route around was found quickly and what took most of the time was propogating it around the larger community of flu viruses.

There are interesting analogies to be drawn between this and the way we use r-selected designs in open source, platform, social-network, strategies.  I need to stew on that.

As with most things these days, I draw analogies twix this and my job search.  I keep trying to have the options be numerous and to try to treat the failed attempts casually.  But my species is not naturally given to r-selected tactics.

Blow Up Rich

1 Reply

I’ve been trying to think about the financial structures around processes that exhibit highly skewed distributions.  The insurance industry is a great place to find the examples.  We buy insurance to hedge against the small but awful.  Most of our houses don’t burn down, but it does happen.  The chance of a fire is scale free, the insurance company protects it’s clients at the scale they care about, but who protects the insurance company against the rare event the burns down the entire town.  There are three ways the insurance industry handles that scenario: they don’t cover it (excluding acts of god for example), they reinsure into a yet larger pool, or they avoid it by not insursing in certain venues. Over here at Bronte Capital is a posting arguing that Warren Buffet, who moved into the insurance industry in a big way over the last few years, has been working this third angle.

When process with a highly skewed distribution delivers it’s rare but powerful shock into the system, it’s black swans, everything designed to work with the median shocks is blows up.  I’d be interested to know how the insurance industry handled the New England hurricane of 1938.  I’d be interested to know how the insurance industry in Thailand handled the AID’s epidemic.

Another place I’ve been musing about exceptional, but inevitable, events is where you situate your career planning.  I’ve a friend who likes to say that almost all the people he knows who made a fortune in their life “fell off a log into a pile of money” thru no special merit of their own except in some cases they consciously picked a good log to sit on.  On the other hand a lot of people just fall off a log sooner or latter.  It would be nice if, as you plan your career, you had a better sense of what the chances are in the trade you pick, in the economy at large.  The fetish people have for presuming that career path probablities are entirely a matter of personal merit seem wreckless.  I was quite impressed when an acquantance of mine with a degree in biology explained he was moving into lawyering because, well he didn’t put it this way, the climate was more predictable.

Recently I’ve been trying to explain how wily US cell phone pricing is.  They sell monthly plans with N minutes and then when the exceptional crisis comes down the pike, you fall in love example, they charge you huge over charges.  The typical plan delivers minutes at about five cents each and forty cents a minute.  Better, at least for them, is that as little crissis come and go your start changing your plan to buy more minutes, which in the absense of a crissis you don’t use.  That in turn raises the real cost of even you noncrisis minutes.  It’s a very impressive pricing scam isn’t it!  I recomend prepaid (t-mobile for gsm, pageplus for cdma on verizon).

If we ignore prepaid cell phone service, the cell phone contracts with a bundle of minutes every month are a bit like lousy insurance policies. You buy the option to use five hundred minutes, not because you need them, but because your insuring against the risk that you’ll run over and get stuck with the over charges.  That’s great, and I mean that sarcasticly, they are selling you insurance against a risk they created.

It amuses me to wonder what would happen if everybody in the country could be coordinated into using all those free minutes one month.  I very much doubt the phone companies can fufill that promise.

The options contracts implicit in those monthly cell phone contracts are analogous to the insurance pools.  If we could coordinate the month of the phone it would be the analogous to a hurricane or a plague, at least from the point of view of the phone company.

That scenario has been playing out with the internet service providers, at least for the incompetent ones.  For example Comcast sells me a package with certain assurances about what bandwidth I get into the Internet.  Unsurprisingly the consumption patterns of their customers is highly skewed, and I’m one of the higher users since this site runs over that connection.  Inspite of 20 plus years of history showing that Internet consumption grows extremely fast and quickly grows to fill the pipe provided Comcast was suprised when more users actually exercised the option they had bought.  It is not relevant what these users are doing with the bandwidth (P2P, video, voice over IP, spam) because if it hadn’t been one of those it would have been something else.

This last example, the ISP’s problems, is not actually an example of pricing design in the face of a highly skewed distribution.  It just looks like one at first blush.  The real problem the ISPs face is the rapidly rising tide of usage.  They thought they had a slower growing usage situation, something more like what is seen with the cell phones, but they were wrong.  When they discovered some of the users were consuming all the bandwidth they thought they had purchased the ISPs presumed those users were little trouble makers rather than early movers.    But that’s a mistake, soon everybody will consume all the bandwidth they can get.

Gridlock Economy

2 Replies

Michael Heller’s new book looks interesting.  Heller was, for the last decade, been working to introduce a bit of balance into the discussion down stream from the idea that goes by the name “Tragedy of the Commons.”  He originally called his idea “The Tragedy of the Anticommons.”  Those who public goods coming to tragic ends often prescribe a dose of property rights.  Heller is interested in situations where too many property rights create grid lock.

His Authors@Google talk is a good introduction.  About a half hour long it touchs on various coordination failures with substantial social costs that arise from an abundance of property rights: Drugs that don’t get developed, families displaced from their legacies, urban development frustrated, air traffic congestion, foul ups in the post soviet privization programs.  Good stuff, and he is reasonably straight forward about how societies should be more aware about the balance they strike when they architect their property rights schemes.

That last point is of particular interest to me, since it goes to the question of how you shape the power law curves.  Is the single property owner who frustrates the urban developer the hero of the long tail; or is he just the worse case of ground cover strangling urban vitality?  Guess I’ll need to read the book.

I’m bemused, or confused, by the realization that both these tragedies arise because some coordination problem blows up when too many parties have simultaniously have rights.  I guess you might say the anticommons goes down the tubes when one player says no (or more often just lies silent) while the commons blow up when too many people say yes.  After a bit I can’t see these as really different, it’s back to the group forming coordination problem.

The No Carrot, No Stick Zone

Leave a reply

This talk by Clay Shirky is a basicly the first bit of his book performed live.

He cut from the book the suggestion that the phase transition we are going thru is going to lead to chaos.

I don’t recall hearing before the delightful idea that Institution rely of carrots and sticks, but that if you want to tap into the the long tail of one off contributors you can’t do that, making the long tail a no carrots, no stick zone. That is very line nice. While it’s probably not true, since systems that work by filtering value out of that thin soup of long tail contributors can to a lot to manage their incentive structures, it is a very good rough approximation of the right mindset.