Skip to content

ePOST

Sunday, August 7, 2005

This is a very cool example of how peer to peer systems might displace centralized hubs.

Most email today is stored on centralized servers; i.e. hotmail, gmail, the firm’s email servers, etc. Mail, i.e. the post office, is one of the oldest centralized services. The network effects are strong and they tend toward creating a monopoly.

ePOST is a serverless peer to peer email system. Download it, backup your copy, fire it up, setup your email client, exchange mail. Later blow up your computer (this step is optional). Pull the backup, fire it up, and magic! All your mail reappears.

Your copy is part of a peer to peer swarm that is collaborating to exchange encrypted mail and store all the mail redundently amoung the members.

The hub is unnecessary. Hotmail? gmail? AOL? Uneccessary!

Now to be honest this system is clearly a first generation; a proof that these hub killers are possible.

Christenson’s last book on innovation put a model into my mind about how innovation proceeds. It’s kind of systolic. On one hand innovation proceeds by cobbling together out of parts at hand solutions to problems at hand. These solutions are, well, Rube Goldberg like. But they are cool because the solve problems that weren’t solved before; which makes them valuable. Time passes and these solutions are refined. And then, enough knowledge is accumulated that the modular boundries in the solutions become apparent. These modules are then broken out and fall out as peices. The peices can then engender another round of problem solving.

A system like ePOST feels like one of the highly integrated systems of the first phase. At the same time it’s designers are part of a storm of activity going on in the peer to peer community to find the modular boundries so the component parts can be distilled out.

If you pop the lid on ePost you find first, second, third drafts of a lot of these modules.

  • Peers self assign a place around a ring of integers; that ring is the swarm.
  • FreePastry – allows one peer to send messages the peer nearest any point on the ring.
  • PAST – allows distributed, reliable, key/value hash table lookups in the swarm.
  • POST – allows encrypted objects storage in the swarm along with encrypted user to user messaging.
  • ePOST – build mail on POST
  • Glacier – provides durable storage so that huge percentages of the swarm die you can still recover all your data.

There is the most marvalous amount of research going on around all these modules these days. ePOST is a beautiful example of how what is becoming possible. This work on peer to peer DNS lookups is another.

Filed in General | Comments (0)

End to End is Back?

Sunday, August 7, 2005

I got an Onion of the day calendar for Christmas and this gem from 2002 came up recently.

U.S. Middlemen Demand Protection From Being Cut Out

WASHINGTON, DC—Some 20,000 members of the Association of American Middlemen marched on the National Mall Monday, demanding protection from such out-cutting shopping options as online purchasing, factory-direct catalogs, and outlet malls. “Each year in this country, thousands of hard-working middlemen are cut out,” said Pete Hume, a Euclid, OH, waterbed retailer. “No one seems to care that our livelihood is being taken away from us.” Hume said the AAM is eager to work with legislators to find alternate means of passing the savings on to you.

The classic paper by Saltzerz, Reed, and Clark End-to-End Arguments in System Design which gave rise to the stupid network is principally about how to expend your design resources. It argues that your communication subsystem needn’t address a range of seemly necessary functions: bit recovery, encryption, message duplication, system crash recovery, delivery confirmation, etc. etc. This is a relief for the system designer, he can ship earlier.

The end-to-end principle drove a lot of design thinking for the Internet. For example DNS, the mapping of names to IP addresses, is layered above UDP, which is above IP. The end-to-end principle drove DNS up the stack like that.

The designer in the thrall of the end-to-end principle strives to leave problems unsolved. That makes it a kind of lazy evaluation technique. Leaving problems for later increases the chances the will get solved by the end users rather than by the system designer. It pushes the locus of problem solving toward the periphery. It creates option spaces for third party search, innovation, etc.

It is possible to look on the design principle as shifting risk. By leaving the problem resolution to later the design is relieved of the risk that he will screw it up. While users might prefer to have their problems solved by some central authority they do get a bundle of benefits if the problem is handed off to them. These benefits are otherside of the coin of agency risks.

The end-to-end principle is always about managing the risk associated with agency. The Internet’s designers were well aware that they were attempting to create a communication subsystem that would remain open, robust, and hard to capture. Those goals were complementary with designing a system that could survive in battle.

When ever you clear the fog around one of these communication or distribution networks you find a power-law distribution. I.e. you find hubs. I.e. you find middlemen. I.e. you discover the risks of agency.

I don’t think that the original designers of the Internet expected to see the concentration of power we see in Internet traffic, domain name service, email, instant messaging, etc. etc. Nor do I suspect they expected to see the concentration of power that the internet has triggered in the industries that are moving on top of it; i.e. a single auction hub, a handful of payment hubs, a single world wide VOIP hub, an handful of book distributors, a handful of music distributors, one browser, one server, etc. etc.

Nothing in the end to end principle actually frustrates that outcome. It argues that there are a collection of reasons why a middleman, i.e. designer of a distribution/communication cloud, might find it advantagous to limit what functions he preforms in his role as intermediary. It doesn’t argue that intermediaries shouldn’t exist. The middlemen in the Onion piece are not being displaced by other middlemen. Middlemen rarely disappear completely.

This is why it is an ongoing effort to keep the network open. While we have a bag of tricks for shifting problems toward the edges and out of the center we seem largely at sea about how to control the degree to which hubs condense on the layers above us.

Filed in General | Comments (0)

Mozilla Corp.

Wednesday, August 3, 2005

Cool, the Mozilla Foundation has budded off a commercial taxible subsidiary. I agree with Karim. This is a very exciting development.

While we have seen numerous attempts by commercial firms to capture some of that Open Source magic. Most of these have come from people who’s motives are principally commercial. Now there is nothing wrong with those motives, but it tends to color their attempts. The motivations that serve the establishment and stewardship of a rich open commons tend to move progressively (sic) to the back burner.

It is difficult to create a hybrid in the space between these two very distinct ethical frameworks. It is not entirely clear if one even exists. What is clear though is that a lot of people from the commerical side are searching really hard to find one. I’m always happy to see search parties heading out from the nonprofit side of the space.

This is a particularly important one though.

My bemused characterization of the driving force for most open source start ups goes as follows: On the one hand we have free stuff! On the other hand we have rich CTO/CIOs! We will just stand in the middle and make money! It’s a plausible premise.

If you stick a firm into that gap there are a lot of other aspects to bridging between those two, it’s not just money. For example on the Open side you have a high value placed on the creation of a huge pool of options; while on the commerical side you have a high value placed on minimizing risk and maximizing predictablity. On the open side you have a enthusiasm for rapid release and adaptation. On the commercial side your required to synch up in tight lock step with the buying organization’s schedules. On the open side the evolution of the project is a continous negotiation among the projects particpants; a deep relationships. Participants are often locked-in. On the commercial side the relationships are kept at arms length with contracts, specifications. Buyers strive to commoditize markets with multiple vendors, avoiding lock-in. I could go on.

There is arguement to be made that the CTO/CIO side of these businesses should adapt. I have no doubt that over time they will. For example I suspec that CTOs will adapt before CIOs. But it is always hard to shift an installed base. It’s obviously hard when you dig into all the APIs of complex peice of software, like Microsoft Windows. But it even harder when you dig into the complex tissue of social webs. Changing the rules for how firms manage software isn’t easy. That’s why the CIO organizations will shift more slowly than the CTO organization; one has a much more complex social web to adapt. At minimum a much larger one.

But back to the reason why the Mozilla move strikes me as important. It’s not just that I’m glad to see experimentation comming out of the open side of things.

Firefox is key. Installed base on the client side is key. To reach large swaths of market share the Mozilla community needs to solve a consumer marketing problem. That includes finding the ways and means to move the product down the existing distribution channels. Thos channels are directly analagous to gaps between the open source community and the needs of the CTO/CIO software users.

It’s my hope that the Mozilla Corp. can enable them to leverage those channels.

Just to mix the two examples together. Consider how hard it is for a CIO to justify installing Firefox rather than IE given how extensible it is. While for a open source guy that extensiblity looks like oportunity for the CIO it looks like increased risk and hightened support costs. An open source guy thinks Grease Monkey is cool. It makes the guys in the IT department quake in their boots. A varient of Firefox that addresses their concerns is a no brainer. It gives the CIO access to the vibrant innovation around Firefox, but it allows him to limit the risks.

Exciting.

Filed in business modeling, open source | Comments (0)

Phisher’s Trick

Wednesday, August 3, 2005

Most web login schemes work the same way. When the site becomes curious and wants to know more about the user it delegates that task to identity provider. The identity provider then authenticates the user and sends him back to the original site. Information about the user can then be sent via back channel of one kind or another. Lots of schemes exist for implementing these back channels: shared cookies, extra CGI parameters, full fledged TCP back channels.

Building a good identity service is delicate work. People screw it up because it’s hard. Criminals spend a lot of time working out schemes to trick the identity server into misbehaving.

Here’s a way to get it wrong: let untrusted parties bounce off your login server. In this example eBay’s the one that messed up. The problem? Phisher’s entice users to go to their eBay account an log in. Once log on the user is bounced over to the phisher’s web site. The user at that point thinks he’s talking to eBay; but he’s not. He’s talking to a bad guy.

The user thinks, when he logs into eBay, that he’s entering the world of eBay. This is similar to what in Liberty is call the circle of trust. As a rule of thumb identity servers shouldn’t provide services to unknown sites. The users presume that sites reached directly via the login server have some degree, no matter how slight, of trust among them.

It’s possible that an extremely lite weight identity system, like OpenID, could break this rule. The cost of conforming to the rule is two fold. It makes it harder to run an ID service, since you have to provision and maintain an account relationship with every site you service. More critical from a business model point of view – you raise the barrier to entry for additional sites adopting your service.

Personally I think ID services should always have an account relationship with the sites they serve. They can then hang a lot of useful junk off that account data; for example a list of URL patterns they are willing to bounce back to.

Every time you see data stuffed into a URL you obviously need to worry about what that might be revealing about the user which ought to be kept private. What’s less obvious is how every time you see an URL getting bucket brigaded along you need to puzzle out how the user’s trust model is changing as he travels. That URL can tap that trust. The first worry is about stealing bits of the user’s privacy. The second is about stealing bits of trust off the intermediaries.

Filed in identity | Comments (0)

Intelligence

Wednesday, August 3, 2005

The President and Bill Gates are throwing some love to the anti-science anti-secular crowd. The president signals that he thinks teaching anti-evolution poppycock in the schools is just fine. Meanwhile Bill gives 9+ Million dollars to the creationist PR mill’s front office, i.e. the Discovery Institute

Filed in politics | Comments (0)

Cell Phone Tracking

Wednesday, August 3, 2005

Here’s an interesting example of what a huge wake we leave behind as we move thru modern life. The CIA team that kidnapped a guy in Italy and sent him off to Egypt to be “interrogated” left behind a nearly complete record of their movements (down to a few feet) and contacts. They left their cell phones on and since modern cell systems track and record everybody’s movements the Italians only had to get a court order and a database query. There is an unbelievable amount of casual revealing about you that your totally unconscious of.

A few years back the Irish cellphone company discovered that they had neglected to discard ten years of this data. Traces of every cell phone user in Ireland for a decade! Kinda puts to shame the data set MIT collected recently. They got a 100 volunteers and recorded 40 years of data (i.e. a couple months each). But you can download large swaths of their data and they have been doing some really fun things; like discovering who your real friends are and where your likely to be next tuesday at 6:45pm.

No worries.

Filed in identity | Comments (0)

The Five Great Philosophies of Life

Saturday, July 30, 2005

The book I was looking for was not on the shelf, but this was a few books down. The original copyright is 1904. Prudence Hyde renewed the copyright in 1938 by Prudence Hyde. The five great philosophies?

  • Epicurean Pursuit of Pleasure
  • Stoic Self-Control by Law
  • Platonic Subordination of Lower to Higher
  • Aristotelian Sense of Proportion
  • The Christian Spirt of Love

But then I’m sure you knew that. The author was William De Witt Hyde, a president of Bowdoin College. (Amazon)

Filed in frameworks | Comments (0)

Loyality is very skew’d

Friday, July 29, 2005

Chris Anderson points out an interesting power point presentation from a company called DVD stations ptt/pdf Their business is DVD rental kiosks. The kiosks print DVDs on demand and have really fast connection back to the mother ship. Like all attacks on a distribution channel the upside comes from unlocking value that couldn’t get thru the channel beforehand. In this case the long tail of content. There are some nice charts showing how much value there is out there. One curiousity, they have a little budge in their revenue for movies that are around 9-10 years old, what’s up with that?

I was interested in two charts. The first one shows a portion of their sales pipeline, it shows that a few customers account for a large chunk of their revenue. You gotta love the labels marketing people put on the folks in various stages of their pipeline.

The second chart shows which channels distribute thru and how much revenue comes out of each. Since I don’t watch cable TV I was surprised how much is premium cable and video-on-demand. This is pie is of course a slice of yet larger pies, i.e. the entertainment pie. In the future the big slice is going to be neighborhood puppet theaters – you heard it here first!

Of course I suspect that both these pie charts are just power-law curves, but ironically they don’t display that. They want to be an elite. For example, in the customer catagory space that leads to inevitably into pricing discussions and then into the new dark ages of DRM content.

Filed in power-laws and networks | Comments (0)

Silver Spoon 0.0.2

Thursday, July 28, 2005

Fresh version at http://www.cozy.org/silverspoon/.

If you compile up the test program big then you can use it to glean out a list of items that occur (aprox) a certain number of times. For example if “yesterday -ip” dumps the log of yesterday’s web server traffic with just the IP addresses, then: “yesterday -ip | big -c 50 > bl50” will show you the IP addresses that dropped by 50 or more times. Or you can then do “yesterday -ip | big -c 40 -b bl50” to see the ones that dropped by from 40 to 50 times. The “-b” switch stands for black list and all items in that set are discarded.

Filed in programming | Comments (0)

Silver Spoon

Wednesday, July 27, 2005

htp-0.0.1.tgz is a tar file containing some code I hacked together this afternoon implementing bloom counters. Minimal doc and source views.

It has got lots of bugs. While is also an implementatiof of bloom filters it’s totally untested. There is one test executable named big. It is unix pipe fitting. Given a stream of lines it prints out all the lines appear more N times.

The code depends on APR.

The mnemonic HTP was for Hot Spot, but only late in the day did I realize that’s too much like HTTP – oops. That’s change if I get back to this.

No promises, no warrenty, lousy license.

Filed in programming | Comments (0)