Monthly Archives: May 2006

Lower Latency Secondary Storage

These three items come together in my head.

Saw this today:

Samsung Electronics said Tuesday that it will launch two mobile computers in early June that will do away with hard drives altogether, replacing them with 32 gigabytes of NAND flash memory. … Unfortunately for U.S. consumers, both will be sold in Korea only.

While previously I saw this go by:

Owners of Apple Computer’s new MacBook consumer notebooks will find that upgrading or replacing the computer’s hard disk is as simple as adding more memory.

Together those reminded me of this…

…prompted South Korea’s Samsung to offer Apple a deep discount and be willing to dedicate 40% of its flash-memory manufacturing capacity to seal the deal.

Even though I recall the details of that last item being very murky, I think it’s pretty clear we can expect the first US release of a hard driveless portable to be from Apple. August MacWorld?

Flash based secondary storage creates some interesting options in physicality, power, programing, database, and OS design; it will be interesting to see how that shakes out. Can anybody think of some new applications it enables?

Nice people scratching the itch I gave them.

If you visit my blog you might notice that at the bottom of each posting there is a link for reporting typos. Since I installed that I get about one report for each posting I make, and about one a week from some random visit to an old posting. From time to time somebody reports two or three typos in a batch. I also get torrent of spam thru that UI, but that’s no big deal.

Recently some kind soul has been working submitting a lot of typos. Here’s the list of posting he (or she) has happened to report typos on so far.

Who ever you are, thank you very much!

kill -9 $$

Back in the dark ages, before web sites had feeds, I’d visit the website of a Bay area consulting firm once or twice a month. It was a fun place were the nutty ideas of the sixty’s counter culture would get framed up for sale to moguls of silicon valley. Never mind that Apple sues Apple; it was all very chummy; these folks all went to school together during the 60s.

One of the meta-meme’s I captured from that venue was the trick of importing ideas from the natural world – e.g. ecology, biology, agriculture, sociology, evolution – into my own problem spaces – i.e. software architecture, project management, business architecture, etc. Though fraught with opportunities for error, this is an extremely fecund technique.

Here’s a nice example, pointed out by Sam of the technique.

Which brings our attention to the term apotheosis.

A mechanism by which one cell dies if it becomes severely mutated as a means of protecting the entire organism.

Traditional Apache HTTPD has a very very primitive example of apotheosis. There you have a swarm of child processes that handle the incoming HTTP requests. You can configure these children to commit suicide after N requests.

I have built systems where processes commit suicide for assorted reasons. Lack of customer is one. Memory bloat is a common one. It is often a lot easier to have them die than fix all the legacy problems that cause them to leak this or that resource. And then I have built systems where suicide and murder are used when various handshaking patterns fail. These are useful when you can’t control some of the components.

But to tell the truth I find the apotheosis idea much more interesting at the social network level. Since it connects to the whole suite of puzzles that are discussed in Group Thing, Exit, Voice, and Loyality, and go by various cliches: “Seppuku,” “Fall on Your Sword,” “Spend more time with my family.” And in turn reminds me of this marvelous quote.

To the untraveled, territory other than their own familiar heath is invariably fascinating. Next to love, it is the one thing which solaces and delights. Things new are too important to be neglected, and mind, which is a mere reflection of sensory impressions, succumbs to the flood of objects. Thus lovers are forgotten, sorrows laid aside, death hidden from view. There is a world of accumulated feeling back of the trite dramatic expression–“I am going away.” – Theodore Dreiser in “Sister Carrie”

This is the kind of blog posting that requires a disclaimer. No this is not about any current situation in any portion of my many lives. If it were I would not have posted it!

Engineering Information Asymmetries

Consistently following the advice to “Buy cheap, sell dear” requires that you know something the market doesn’t. Consider a very simple example. Two towns one has abundant amounts of oysters, so abundant that the local population is sick of them; meanwhile in the second town oysters are a delicacy. A trader can make a good living moving oysters between the two; but only so long as he can keep his sources secret. The lack of transparency enables the profit; or more generally the information asymmetries between the two towns.

I was very amused by a story that appears in the early pages of Frank Partnoy’s book Infectious Greed. In this story a trader discovered way to trade between two markets; rather than moving oysters he moved information. The markets were currency markets; a private market and a public market. So in one market his actions were visible, i.e. they generated information, while in the second market they were largely invisible.

While all commerce, markets, etc. are, to a degree, about risk it’s particularly helpful at this point to introduce a point about betting. If you wish to place a $100 bet on your home team there are an infinite number of ways you might do that. For example you might place a $300 dollar bet on the home team and a compensating $200 bet on the opposing team. That may seem like an odd choice but notice that it allows you, with nearly total honesty, to go around telling everybody you bet $200 dollars on the opposing team. It lets you signal the opposite of your true intentions. That creates an information asymmetry, one that you control.

The public/private markets enable the same pattern. If you can trade in both a public and a private market for the same good; but only one of these trades will generate an signal about your intentions. If your trades are large enough you can move the market with that signal.

In the story the trader played this game with the international currency markets. He actually had two pairs of markets he could play this game in. First he had the traditional currency market and the currency options markets which were at that time not well connected. Secondly he had public exchanges and private, so called over the counter, deals he could make. There is a short paragraph in the midst of the story about how his boss got a call from New Zealand’s central banker demanding that he stop toying with their currency.

Because benefits can flow to market actors from information asymmetries most commercial dialog is permeated by a subtext of information hoarding. In some scenarios, like the one above, the appearance of an abundance of information might be a signal that information is scarce in an adjacent market. I think this is a rarely realized element of why open source appears so suspicious to some commercial observers.

The Struggle to Govern the Commons

Appearing in Tilly’s book “Why”:

Effective commons goverance is easier to achive when (i) the resources and the use of the resources by humans can be monitored, and the information can be verified and understood at relatively low cost (e.g. trees are easier to monitor fish, and lakes are easier to monitor than rivers); (ii) rates of change in resources, resource-user populations, technology, and economic and social conditions are moderate; (iii) communities maintain frequent face-to-face communication and dense social networks – sometimes called social capital – that increase the potential fo trust, allow people to express and see emotional reactions to distrust, and lower the cost of monitoring behavior and inducing rule compliance; (iv) outsiders can be excluded at relatively low cost from using the resources (new entrants add to the havested pressure and typically lack understanding of the rules); and (v) users support effective monitoring and rule enforcement.

That sentences is from “The Struggle to Govern the Commons” (pdf). Very interesting how poor the match is between that list and the situation with Open Source.

iRates

Kieran Healy mentions at the end of this post about identity theft that the rumor has reached him that call center workers have a minted a new noun: irate; as in the contraction of irate customer.

I’ve spent much of my career dealing with post start up code bases. Along the way I’ve picked up a strategies for approaching the beasts. For example always pick apart the garbage collection, i.e. those portions of the system where they built the code to clean up afterward. They never get that right. For example how do they close accounts; particularly accounts where multiple parties are involved.

An even more productive strategy is to look at the exceptions; they are always a very generative area to dig into. Irates are a class of exceptions; the customers who fail to conform to the call center’s scripts. These folks reside in the the long tail of the cases the call center is expected to handle. The common cases all have well worn and efficient scripts. When these customers discover that the center’s scripts don’t resolve their problems they either give up and move on, or they keep pushing. Their persistence gets them labeled irate. Calling them exceptions might be a better choice; but in point of fact the call center wouldn’t mine “firing those customers.”

Long time ago, when bank machines first came on the scene, I once asked the teller if the bank machine meant that the only people who still came to her desk were the incompetent customers. I suspect that call centers have a number of neologisms for labeling the stream of customers they deal with; since to a degree all these customers are exceptions. Interesting place for a case study on tagging; since I bet their software includes the tools to label a customer: malingerer, idiot, irate, etc. E.g. all the words used to label a person as failing to conform to some norm.

I’ve been thinking a lot recently about a dialectic I first encountered explicitly named in the semantic web community: open v.s. closed world. It is, apparently, a term stolen from the world of logic and automated proof systems; but that doesn’t actually interest me about it. Inside the closed world you can achieve the illusion of being in control. All you data will be will formed and fully populated. The exceptions won’t happen. The data will respect your designs. When these things turn out not to be true you diagnositc gaze can move backward in time to seeking the place where things went astray.

Most of my rules for how to think about legacy software involve a generalization of an old rule I learned about hardware: “It’s always the connectors.” The slight generalization of that is that it’s all about the border of the closed world.

In contrast to the closed world are the worlds where you have progressively less and less ablity to predict the shape of the data. The cases to be handled become more and more diverse. You get a lot of that in real time control systems. No two pumps in the plant are the same, they were all bought at different times, they all have different quirks, and differing maintainance histories. All the components are irate, passive agressive, moody, and certainly they are all idiots. I’ve got a problem these days with my VOIP phone number; it’s taken 10 emails to slowly convince their customer support organization that I am none of those and that my problem demands handing off to more expensive labor.

The real world is open. The code and data in real world system should be expected to be mostly about the boundaries, the exceptions, the startup, the shutdowns. When things go wrong in these systems the you tend to look forward to how to bake in the handlers for this new case. This suggests that this is a fundimentally different kind of programming – an insight that makes me recall how some of the early languages back in the 1960 were structured so that each statement consisted of three parts: what to do, what to do next if that worked out, what do do next if it didn’t.

When one of the parties in a multi-player relationship announces the identity of a shared client has been stolen what does your software do? You, of course, are welcome to become irate when that happens.

Skype Confrence Calling

Here’s a dumb trick you can do with skype. Put a few 800 numbers into your contacts list; it’s best if the ones you put in have automated voice recognition. Then start a conference call and invite them all to talk to each other. Very silly.

Inalienable

An interesting triple.

… arguments in favour of having secret ballots …; most obviously the argument that secret ballots obscure the information needed to perfect a market in votes; so that the vote remains effectively inalienable …

These three: privacy, markets, and inalienable are deeply linked.

Emigrants Who Refuse to be Assimilated

One of the delights of growing up in New York and going to school in Pittsburg was access to a number of authentic ethnic enclaves. Little communities that hadn’t melted into the gruel of american culture, and which, better yet, had prospered. Boston, where I now live, has some ethnic enclaves too: Chinatown, the North End, etc. Of course, our most largest segregated community is black and poor. Most people though think of Boston as the home of those smaller segregated communities, those where we house displaced populations of high school students. We keep the first segregated with the shape of the public transportation system. The second problem is kept in control by carefully walling them off in the universities.

Emegrants into a new land always try to reproduce their native homeland. I’ve read that the Spanish methodically leveled the tropical ecology in Mexico until they had reproduced the arid plains of Spain. At regular intervals some new emigrant to my town opens a small grocery store selling an assortement of foods identical that of an analagous store back in his homeland. A trunk from New York or Montreal visits once a weak so he can restock. I love these stores, but sadly they are apparently incompatible with the local social and economic climate and six to eighteen months later they close down. If we are lucky they evolve quickly into something else.

But really! I am rolling on the floor laughing to see what the lawyers at the Berkman center did when they got off the boat on the shores of the new virtual world!