Webmachine

Leave a reply

Recently I’ve been taking a recreational climb up the Erlang learning curve.  I’ve been up this hill before, it’s a lovely walk.  This afternoon I took a restful detour to see one of the sights.  Gosh, it’s so pretty.  Webmachine an elegant implementation of an HTTP server (code and wiki, blog, talk and video) written by Andy Gross and Justin Sheehy from Basho Technologies along with  numerous  contributors.  Bryan Fink added this lovely view, a flow chart for visualizing a HTTP request:

The slightly darker line in that traces out a single very simple request; it starts on the lower left, rises up and flows down in a staircase until right at the end it turns back and into a box which you might notice is outlined in red. That is the end state, the 200 OK response.

Back when I was actually sculpting web servers out of blocks of raw bits my white board had baked on drawing like that, but mine wasn’t as nicely laid out.  There are other ways to get it right, but most serious HTTP server implementors have, or at least had, some version of that which haunted their dreams.  These folks have been doing their homework.

What make’s Bryan drawing totally amazing is that that it is traced out an actual request.  One my instance of Webmachine did a few minutes ago.  It’s drawn onto a canvas element.  Tracing is cool, but wait,  there’s  more!

You’ve doubt heard about how Erlang’s a functional language, which to the naive listener means no side effects.  But here you get to see what that can enable.

Unsurprisingly as that request is handled the http server has a data structure which represents the request, and as the trace proceeds that state matures until finally a response can be sent.  For example at some point it has to pick what character set to respond in, while at another point it needs to pick if and how to compress the result.  So all HTTP server’s I’m familiar with have a request data structure around which this process is organized.

Because Erlang is a functional language each time we augment the request state we actually create a fresh data structure (bla, bla, bla, sharing substructure, whatever).  Which makes it easy for the trace to record how the request state looked at each step along the way.

Bryan’s trace is alive.  If you click on nodes in that graph it will show you what request state function got and produced.  Which I must say takes all the challenge out of writing this stuff.

The win here is that you can quickly make sure your resources are served up in ways that actually use all the sweet features in HTTP.

Get 40% more done, keep it secret

7 Replies

Wow!  This a wonderfully counter intuitive bit of social science!

Imagine that you would like to be sword swallower.  In service of achieving that goal  you set out to accumulate assorted accouterments: a sword, some books on sword swallowing, you study your vocabulary, you watch some videos, you take a course.  Each of these moves you closer to your goal, and slowly you begin to think of your self as actually a sword swallower.

Or imagine you want to be a small businessman.  You print up some business cards.  You pick a name for your business.  You register with the town, open a bank account. You buy a suit and start wearing it.  Again each of these is a step toward constructing your new identity.

Wait!  Printing up business cards doesn’t make you a small businessman.  Owning a sword you intend to swallow does not a sword swallower make.  Intention is a far cry from being.  But, people get confused and questions arise.  How often to we declare success based only on our good intentions?  Do we confuse the means with the end?  Of course we do.  And, while the worse case delusions are obvious – a good costume doesn’t make yo a superhero – in most cases the lines are quite fuzzy.

What kicked off that musing was this wonderful article reporting four studies reported in Psychology Science: “When Intentions go Public” (pdf or pdf).  I was interested in this paper because I’m interested in privacy.  What happens if I show other people that sword I’ve bought?  What happens when students sign one of those honor codes?  What happens when a hacker publicly reveals some source code he’s working on.  Does it help?  Are we more or less likely to get a sword-swallower, an honorable student, or a open source contributor?

Obviously, we think that public revealing strengthens identity.  We certainly wouldn’t be asking students to sign honor codes if we thought otherwise.  But what if!  What if it undermines their aspirations to be honorable people!  I was already suspicious because of story we tell for why these things must work.  I.e. that this public revealing raises the stakes, that it puts our reputation at risk.  If everybody knows we have started a small business or taken up sword swallowing then we risk embarrassment when down the road we have to admit it all was a sham.  But I have my doubts about that, since I doubt that stakes is an effective means to controlling behavior.

So here is the surprise.  These studies suggest that public revealing of our intentions undermines our follow through.  These studies are all very simple.  Let’s look at one.  They get a group of students who really want to become psychologists and having them filled out a survey.  On the last page they ask them to write down what the intend to study next week in service of their goal.  The subjects are randomly split into two groups public, and private.  The kids in the private group are told that intend-to-study question was included by mistake, the experimenter tears it off and throws it away.  For the public group the experiment reads over the survey with the kid.  Nominally this is to check for errors, but in reality it just assures the kid knows that the experimenter knows of his study plans.

A week later they have the kids fill out a form to on which days they studied.  Those who’s intentions remained private studied 40% more!

What’s going on?  The theory suggests that we are keeping score on our progress toward achieving out goal.  Naturally we take credit for various steps along the way.  We give our self some credit for opening that business bank account, or buying that sword, or signing that honor code.  The bizarre bit is that these students apparently gave themselves some points toward achieving their goal as a consequence of having it be known to another person that they intended to work on toward the goal, and feeling that they were closer they didn’t work as hard.

This is worse than confusing the means with the end.  Worse than confusing having business cards with being a businessman.  In this case the just telling somebody your going to buy some business cards creates a sufficiently powerful fantasy that you are a businessman that you stop working on the project.

Now if this is all true it suggests that asking students into signing honor codes make them less likely to follow through on becoming honorable.  It suggests that asking your staff to outline their plan for the next week actually makes it less likely they will follow through on that plan.  It suggests that signing a contract undermines rather than increases the chance the parties will do the work agreed.  How weird is that!

It’s interesting that nobody I’ve told this story to finds this particularly surprising.  They all agree, revealing their intentions seems to undermine their follow through.  It’s all just too delightfully weird!

Google the Gorilla hearts RDF

1 Reply

I spent a few years working a few houses down the street from the Semantic Web and I came to have serious doubts about the it.  My concerns ranged from irritations with the specifications through frustrations with  interoperability  and tools, and on into angry critiques about  innocence  in the execution of the standard’s effort.  But yet, I remain a  thoroughly  enthusiastic supporter of the  need for widely adopted standards of its kind  and I suspect it’s our best hope in the long run.

There are some bright spots in the specs.  Turtle (v.s. n3) and RDFa are both pretty nice.    They increase the chances that tools will rendezvous and thus decrease the frustrations around  interoperability.  I think it is nice that Google’s giving RDFa some love.

The tough part of making any successful standard is getting a large volume of transactions taking place over it.  After years of effort, to first order, no data moves over RDF.  In other words the Semantic Web has been a non-starter.  Maybe it has a long runways but few Internet success stories do.

So one of the things that angered me: people at the core of the Semantic Web standardization  movement  seemed largely disinterested in working on getting a large volume of data to flow.  And, wanting to work on that I rarely found people to join in common cause with.

That is a classic problem in making standards.  The story is often the same.  The shape of the exchange network that emerges almost always the same.  The distribution of  producers, by volume, shakes out into a highly skewed distribution, a power-law curve.  It is the same story with the consumers.  You wee that pattern everywhere you look from seen in phone call traffic to web traffic.  For example on the consuming side Google and a dozen other entities try to consumer everything.

The lesson in this, if your advocating a standard, is that you need to be consciously puzzling out how to bringing both consumers and producers onto the standard; and you need to be conscious about bringing both the big players and the vast long tail of small players on was well.  You always have these four audiences to bring on board.  And of course there are others, i.e. experts, tool makers, etc. etc.  Everybody has to understand the value proposition, and that story is always the same.  You get on board because it substantially improves the transactions with the other guys.  The producers desire the consumers and visa versa.

This is the classic chicken and egg problem.  Why would anyone waste scarce capital to produce RDF is there is nobody interesting in consuming it, and visa versa?  Sometimes you can solve this problem bottom up; you seed some exchange between small producers and consumers and then other see what fun they are having and join in.  Given the vast number of little producers and consumers it is typically easy to get this started, but getting it to spread  requires a lot of tinder and it’s best if there aren’t alternate ways to do these exchanges.  The trouble for the Semantic Web has always been XML.  XML provides a well supported means for the little guys to do their exchanges.

A while back somebody at Google asked me: “What can we do to help the Semantic Web along?”  I recall thinking, but not saying, that they should loudly signal that they are going to consume RDF, that search results would depend on that.  That would be enough, thought I, to trigger the entire search engine optimization industry into frenzy.  RDF would start popping out all over in the hope it would improve one’s Google juice.  If that actually happened then they could decide if the resulting data was useful.  But, I’m  embarrassed  to say I kept my mouth shut.  It didn’t seem to me like my  interlocutor  was likely to take kindly to the idea.  I doubted he’d cotton to the idea of Google actually playing hardball with their market power to shape standards.  Thinking back I discover I’m  embarrassed.  I froze like a deer in the headlights. That makes me smile.

I’m happy to see that Google appears to have done exactly that.

So, what I find most encouraging about the Google announcement is that it might actually do something to draw out more semantic goodness.  With luck we might start to observe some RDFa in the wild now.  No doubt, at this moment, hordes of SEO consultants are reading the RDFa spec.  I wonder how long before all the big vending sites (Amazon, eBay, Walmart, Etsy, etc. etc.)  are too.

Off to the Races?

Leave a reply

The term “platform” misleads people.  The metaphor is flawed.  It suggests land, and it can be made to work, if you insist.  Accepting the metaphor then applications are built on the platform, like houses on the landscape.  I read recently a brief summary of why even if you set aside the housing bubble the cost of housing has risen in the US.  Two reasons: zoning and tax caps.  Zoning has made it ni-impossible to increase the density of the existing  metropolitan  areas.  Tax caps doubled down on the primary problem of public goods, under provisioning.  In the absence of public goods (schools, roads, security, environment, public health, …) individuals are forced to provide substitutes; and by definition these are higher cost and lower quality.  For the platform metaphor to work it’s critical to think not just about the  applications  it supports.  You need to dig into the governance; i.e. the costs, rules, and services provided.  That is an improvement and it does illuminate the question but it is not my preference.

There are at least three aspects of that metaphor that I find lacking.  You need a metaphor that gives equal weight to both sides of the equation.  The services a platform provides are just as important as the applications it enables.  You need a metaphor that gives more weight to life-cycle of platforms; that in each round we  experience  a race to see who will own the platform.  The platform as land metaphor is far to lead us to ‘pay no attention to the man behind the curtain.’  You need a metaphor that embraces how important the network effects are.  All these can be see thru the lens of each other; particularly in the early days of as a platform emerges.

Consider the current state of play.  Developers seek out fresh real estate to build on and these days they appear to be gathering in two regions: smart phones and cloud computers.  So there are two species of platforms, two competitive games in play, two industrial standards battles.  In the life cycle of these platforms both horse races are well out of the gate.  Apple and Amazon respectively have grabbed substantial early leads.

Picking the right metaphor helps to assure you stay focused the right things and that you have the right expectations.  For example some applications, payments for example, are probably destined to become key platform features.  That in turn informs the question of who’s in the game.  For example is eBay/Paypal a cloud computing OS waiting to happen?  I think so.  It also helps to explain why Google or Amazon have a payments offering.

We know to expect an operating system to provide a file-system and a GUI.  We know to expect that a local government will provide public schools and some regulation of the sewage.  So, presumably we should be forming expectations about what features a cloud computer offers, or a smart phone.  Here’s a nice long list for cloud computing.  Here’s a shorter list for smart phones.  When the  column  fodder charts are that messy you can be sure of a lot of condensation and turbulence ahead.

The early days in the life-cycle of a platform are interesting in part because where the lines are drawn is under discussion.  Things settle down during the midlife.  I can recall the heady early days of the Mac when every release of the OS brought with it new extremely exciting APIs.  But also how each of these APIs was actually prototyped by somebody else, often an application builder.  There is always a tension between what will be owned by the platform and what will be owned by those around him.  This is a bit like how some wags like to complain that the town’s public produce markets or schools competes unfairly with private enterprise.  Right now, for example, there is a firm that is dominate in the geo-location via wifi market and there are three clear ways that might go.  They might be rolled up into one of the platform players.  They might be displaced by an open substitute (based on say open street maps), or of course they might survive as a vendor.

There is one place where I seem to most often run into confusion caused by the platform as land metaphor and that is with websites that are playing the open API card.  The metaphor causes them to focus  primarily  on getting developers to adopt their API.  That’s an unalloyed good thing, not just because it is actionable.  But it tends to make them blind to the dynamics of the battle unfolding all around them.  For example; for various reasons a service offered inside of EC2 is  preferable  to a service offered outside, at minimum it will be more performance and the bandwidth costs will be zero.  So I suspect we will see a trend toward all firms offering a open API moving some or all their offering inside of EC2.  More generally, and presuming that real competitors to EC2 emerge, they will have to build the same kind of branch offices inside each EC2 competitor.  That in turn is exactly like the “Render unto Caesar the things which are Caesar’s” dynamic seen around older operating system platforms.  Where, for example, a hardware maker or application maker has to carefully assure his offerings are supported by the OS vendor.

Why Do We Pay Attention?

2 Replies

Why do we read those blogs, email, chats, twitter, voice mails, newspapers, magazines, etc. etc.  Presumably there is some logic to that.  Some motivational schema.  There’s money in the answer to this question.  Will my students pay attention?  Will my novel be a hit?  Will my newspaper survive?  So, surely this question has been extensively studied?  I can think of a few examples.  There are handbooks on teaching, writing, advertising that all look into the question.

Here is an another attempt, coming at this from the currently popular puzzle of what might stop the free-fall of newsprint and it’s codependents (i.e. investigative reporting, PR, local advertising, etc).  He blocks out four reasons why we expend resources to accumulate new information:

  • Entertainment – is everybody animated now?
  • Deciding – in a yellow wood?
  • Staying Expert – sort of a service contract model i guess
  • Paid To – diagnosing, trained,  flattery?

These are not independent.  For example, the author of a highly technical paper targeted at a community of experts will often include a significant amount of entertaining content since he knows that makes the material more memorable or more viral.  But one reason it’s clear these are disjoint categories is how when your goal is drawn from one category it can be  irritating  to have content from one of the others popping up.

I found it disconcerting and then amusing that what I’ve labeled “paid to” he named flattery.

I’m not particularly comfortable with this framework.  Why do fans pay attention?  But, it is fun to compare it various other schemes: story templates, selling scripts, etc.  For example in the  typical  fairy tale our hero is cast out of one’s home, goes on a quest, and then returns home.  That has all four elements.  For example when we are influenced by the use of social proof in a situation that has elements of deciding; but helps to highlight how there is a social aspect to all four.  When we tell a story by opening with a mystery to hook our readers, a standard bit of teaching advise which I used in this posting, then we are pulling on a few cords from all four.  And where does the phrase “breaking news” fit into that framework?

And what’s up with cliff hangers?  People do pay to have those resolved.  Did Ben find a job yet?  Tune in tomorrow!

based on The 4 reasons anybody ever consumes information…

Cognitive Shortcuts

Leave a reply

I’m beginning to have the same reaction to studies involving brain scans as I do to those damn evolutionary just so stories.  That is not entirely fair.  The former at least have some data.  The later don’t.  What pulls my cord about the brain scan stories is how often the researchers appear unaware of the existing work on what ever effect they are measuring.

Here’s a fun example.  The summary “Experts make us dumb.”  These guys stuck subject noodles in the scanner and had them make financial decisions.  The scanner let them see how hot the decision making bits got.  Sometimes they gave them expert advise, sometimes not. Unsurprisingly given advice the analytical bits took a breather.

The wonderful book Influence covers this result, and quite a few others.  People have a lot of cognitive shortcuts.  It’s  practically  a  tautology  that shortcuts often output the wrong answer.  But then that was my primary critique of that fun book.

Now that I’ve gotten that off my chest.  It does seem worth recognizing that in sentences like the following (via)

John Haywood, a prominent restaurant concept designer, agrees.  Processing, he says, creates a sort of “adult baby food.”  By  processing, he means removing the elements in whole food – like fiber and gristle – that are harder to chew and swallow.

The first sentence is serving two functions toward winning the  argument.  The paragraph is adding evidence to the  argument.  Labeling that evidence with it source and tagging it so we can decide if the source is credible is all well and good.  The more  wily  purpose of that lead in sentence is to trigger our decision making  apparatus  into shutting down.  Skimming that the brain is likely to note only the word prominent.  Aggressive  debaters will want to take note how you can apply a marginally credible expert like this.  Restaurant  concept designer?  The Wired article plays a variation on this game by dressing up their article with a picture of a brain scan.

I should have a category for rants.

Groups and Value

Leave a reply

Thinking here about group forming and group forming networks; here’s one of those typical B-school 2D drawings:

Presuming we have solved both the problem of aggregating the group and extracting the value then points on that surface more valuable per size-of-group * value-of-member.  This is the calculation that any site with an audience makes, or any shop with regular customers, or any club.  The definition of value varies a lot.  A knitting group wants something different from it’s members than does a standards body.  The word authentic get’s tossed around to label the miss-match between what affiliate  marketing platforms (like Amazon, or Google ad-sense) value in site visitors v.s. what makes a site attract an authentic membership with some particular enthusiasm.  A lack of appreciation for how diverse of value is goes a long way toward explaining how dismissive people are of sites with narrow  enthusiasms.  People dismissed open source for years because they were blind to the values that attracted it’s participants, people are no less blind today even if they are less dismissive.  It really pull my cord to watch observers rapidly dismiss sites of other enthusiasms just because they can’t be bothered to puzzle out what might be the value those members (or the site operator) has managed to find in there.

It seems useful to be clear that value-of-member has at least four aspects.  There is the member’s value the members see in each other (a p2p network scoped by the group).  The value the members contribute to the common cause of the group (a sarnoff kind of value to the groups barn raising).  The value the site owner (or steward) values in his members (i.e. a site for lawyers wants the lawyers who are highly respected and well networked to participate).  The value that feeds clearly into value extraction (i.e. the lawyer site values those who click thru on the ads or regularly subscribe to premium services).  Value is messy.

Presumably the universe of groups, the population, is distributed on that chart such that most groups are down near the origin.  Again it frustrates me how people are dismissive of those.  [Apparently I’m easily peeved :)]  So they complain about the how github’s fuzz of forked projects is confusing, or how google code and source forge are cluttered with tiny projects; or how the numbers yahoo groups or ning has are inflated by groups with little or no traffic.

Which brings us to the question of aggregating groups.  We can visualize this as regions on that chart.  Consider the local Dayton Business Journal, it’s got an audience that is valuable in a particular way and when rolled up into a company like American City Business Journals (see the select city pulldown on one of their sites).  Or consider this set of  local newspapers  around Boston.  In both cases the set of groups aggregated is some range of distances from the origin the definition value-of-member axis has been narrowed down.  That narrowing is in part tied to the cost of rolling up the aggregate, which presumably involved negotiation and money.  Sourceforge is a different story.  They rolled up their groups organically which goes to explain why they have a lot of groups close to the origin.  Sourceforge’s value-of-member  definition  isn’t very broad spectrum.  But there are platforms where you see extremely broad spectrum value-of-member definitions.  Plenty of examples: Yahoo groups, Meetup, Ning,  vBulletin, WikiSpaces or Acquia are all  examples.  I’ve often thought that Yahoo’s strategy was to roll up these kinds of companies; and it’s a puzzle why that aggregate hasn’t turned out to be more valuable.

Well, it’s all food for thought.

Searching for Alternate Routes

1 Reply

RNA viruses may well be the ultimate r-selected species.  The life cycle of an RNA virus includes a few steps.  Infecting the cell, coopting the machinery of the cell, making copies of its self, assemble those copies into viral particles.  Then the offspring need to escaping the cell, avoid the immune system, and find a new cell to infect.  If it’s that simple then it’s seven steps.

I very much doubt it’s that simple.  In fact the illustration above shows just the bit where the virus enters the cell and off it’s coat.  There is an antiviral drug that works by frustrating it’s attempt shed it’s coat.  Obviously it get’s even more complex yet again if we add in how the virus moves between host animals.

But the copy step is notable. In quantity and quality.

a typical RNA viral genome of 10,000 bases, a mutation frequency of 1 in 10,000 corresponds to an average of 1 mutation in every replicated genome. If a single cell infected with poliovirus produces 10,000 new virus particles, this error rate means that in theory, about 10,000 new viral mutants have been produced.

The quantity is high, but the quality is low.  Amazingly there is method in this madness.  The combination of high errors and high numbers creates something useful, a search scheme.

If you want to frustrate a virus then you need to shutdown, or a least narrow, the pathway through which one of the steps in the reproductive cycle.  For example, improved hygiene and increasing social distancing works by making the movement between host animals harder.  Anti-viral drugs target individual steps in the cycle.  The immune system learns to recognize the virus and pick it off as it moves between cells.  In all these case the challenge for the virus is to route around the resulting bottleneck.

Since most of it’s offspring are mutants, most of it’s offspring are sacrificed to searching for these alternate pathways.  Like most r-selected reproductive strategies the vast majority of the offspring fail in the process.  When the spider has a thousand babies it works out because takes that many to searches opportune door into the next cycle of reproduction.  When the maple tree throws off billions of seeds during it’s life that works because it needs to run that many searches to find one that let’s it pass it’s genes into the next generation.

It must be vary frustrating for the inventor of an anti-viral.  The stupid viruses can mindlessly find a route around his clever invention.  Adamantine was approved for use in 1966, by the 2005-2006 US flu widespread flu strains had routed around the hole it plugged.  I think we can assume that the route around was found quickly and what took most of the time was propogating it around the larger community of flu viruses.

There are interesting analogies to be drawn between this and the way we use r-selected designs in open source, platform, social-network, strategies.  I need to stew on that.

As with most things these days, I draw analogies twix this and my job search.  I keep trying to have the options be numerous and to try to treat the failed attempts casually.  But my species is not naturally given to r-selected tactics.

Journalist disappears, he complains

3 Replies

Here’s something worth chewing on.

Justin Fox is steamed:

“powers that be saw fit to delete from existence … all web-only content that had previously resided on fortune.com, including the ‘London Calling’ columns I wrote every week in 2000 and 2001.”

and in turn he notes that International Herald Tribune reporter Thomas Crampton is too:

So, what did the NY Times do to merge these sites?

They killed the IHT and erased the archives.

  1. Every one of the links ever made to IHT stories now points back to the generic NY Times global front page.
  2. Even when I go to the NY Times global page, I cannot find my articles. In other words, my entire journalistic career at the IHT – from war zones to SARS wards – has been erased.

This is really a fact of modern life and work done for hire.  Thinking about my own my career: there are three pieces of code I was particularly delighted with.  My  employers owned  rights to all three.  They have all since been acquired, sometimes multiple times.  I suspect some of that code is still running, but I can’t see it nor can I show it off.

These stories are situated in three distinct frames: individual v.s. institutions, the mindless stupidity  of institutions, and society’s knowledge archives.  Justin and Thomas are principally pissed about the first.  And we should all be paying attention, since more of our identity and work is parked in 3rd party sites like Wikipedia, Flickr, Blogger and Facebook.  Companies are forever shooting themselves in the foot.  And, as Thomas highlights, these media firms are being extremely foolish.  They acquire another firm and just discard all the link juice, that’s just lame.  And finally the question an  archivist  would ask, i.e. of who looks after backing up society’s collective knowledge in these scenarios.  Remember the Times is if not on it’s death bed at least in intensive care.  I think we can confidently predict that the  accessibility  of all Newspaper archives are going to have a very rough ride over the next few years and the end state isn’t particularly clear.

But, most work disappears like this.  If your work product is less concrete (keeping a team on course say) or more concrete (building a boat say) then the disappearing work problem is possibly even more common.  Tempering this problem requires work in all three frames.

QA: Impulse Control

1 Reply

People lose control and act impulsively all time.  It is important to forgive ’em.  If you never act  impulsively  then you seem humorless, uptight, officious,  bureaucratic.  When professionals act impulsively we wonder: should let this guy steer the ship? One scheme to temper impulses is to smooth things a bit using a group.  When your managerial team act out impulsively it’s a signal to go short.

Here is an example that is kind of meta.

“I have a friend who works in a small company who have just put the three testers on notice of redundancy. The developers have been told that they will have to do all the testing. The testers have been told they have five days to write a letter justifying why they should be kept.

I need as many reasons possible why getting rid of the testers is a bad idea in a scrum environment please.”

The managers in question probably should have run that idea thru a bit of quality assurance.

Boy is that bad personal management, totally unprofessional.  It’s going to be impossible to get much enthusiasm for the job going forward.  I’d file that under “examples of trying to control behavior by raising the stakes.”

Whenever you do layoffs the remaining staff is soured.  Dissipating the emotional cloud that falls over the team critical.  It is really hard.  The people laid off are the most  susceptible.  That is one reason firms try to get them out of the building as quickly as possible.  That bum’s rush is another example of what the public health guys call social distancing, and the tactic outlined above just about assures maximal infection.

It also signals that management lacks much clue about the role of QA, or that things are much worse than they are saying.

I can’t help thinking that these poor testers have been invited to write their own obituary.  It could be like one of those scenes in a comedy of bad behavior where the lawyer reads the will that  going on for pages as the  deceased  enumerates each and every flaw of his  descendants.  But yeah, here’s a impulsive suggestion:  Once they are gone, who will fill the role of canary for their code miners?