Caesar’s Render

Leave a reply

ceaserJohn Siracusa has posted a quite thought provoking piece about the power games unfolding between Google, Apple, Facebook and others.  Let’s say you wanted to own a cirtical open source project, i.e. own a key standard.  How might you go about that.  He makes a case, that has some merit(sic), that contribution rates can be used by a firm to buy control of an open source project.  If so, then it’s only a matter of money and time.

It’s a good essay and worth reading.  In my limited experience firms are rarely this strategic; but in retrospect they can stumble into outcomes like these – moving fast, hill climbing, and a bit of impatient greed can goes a long way.  But, yeah – who knows … he might be right.

Weird day

Leave a reply

It is a warm humid day here in Boston, early summer rather than spring.  Nature is bursting out all over and the authorities have told close to a million people (Watertown, Newton, Waltham, Belmont,Cambridge and Alston Brighton) to shelter in their homes so they can look for a young man who, which his older brother, is suspected of bombing the Boston Marathon on Monday.

My town is not on the list, but just the same everybody is indoors.  It’s been incredibly quiet here.  No traffic on the main streets, no pedestrians on the sidewalks.  The occasional dog walker.  It’s quieter than when we have a major snow storm and the air is full of sound deadening snow flakes and the authorities have begged the populous to stay off the roads.  It’s weird.

At this point they have caught the 2nd guy, the 19 year old.  Late in the day I went to Trader Joe’s. Things were quiet on the way over, but it was crowed.  People seems sort of giddy, glad to be out of the house.  It was kind of sweet, for some reason.

I don’t have any complaints.  I feel the authorities did what they needed to do, and for logical good reasons.  I’m happy they got the 2nd kid alive; i hope he survives.  It’s been a tough ride for people in the media, they just can’t quite get that really they shouldn’t speculate.  And gosh, they ought to to keep their hands off the emotional levers.

Let’s hope, it’s over.

another dialectic

2 Replies

Andrew Gelman reminds us of an an amusing thought.  Which I will paraphrase… Social Scientist explain surprising behavior in two ways.

  1. Yes, that is surprising, but here let me explain why that happens.
  2. Yes, that is surprising, but here let me explain how to tinker things so that doesn’t happen.

In Gelman’s version is a more focused.  The social science in question is economics.  For economists surprising is called “irrational” and is largely synonymous with costly, inefficient, etc; i.e.  it has monetary metrics.  He quotes Steve Levitt: “… economists … by training … (don’t) think.. about whether something is right or wrong, they think about it in terms of whether it’s efficient…”

What’s delightful about this dialectic is how these two are identical except for the underlying agenda of the social scientist.  In the first he’s adopted the role of naturalist observer and in the second he’s tapping into his inner entrepreneur.  While it is pretty trivial to switch back and forth; the trick is to be sure you do that efficiently and rationally.

In the end this is the dialectic between understanding and effecting.

My Common Lisp Buildpack for Heroku

Leave a reply

Meanwhile, I reworked an existing buildpack for my needs.  It’s very easy to try it.   Assuming you’ve signed up for heroku following their quickstart instructions then you just:

curl https://gist.github.com/bhyde/5383182/raw/gistfile1.txt | bash

That will create a directory on your machine with the example application’s sources, over on Heroku it will build and launch that application, and finally it will open your web browser visiting the home page of the application.

herokuCCL64

This is all free.

This application is written in Common Lisp.  There are lots of nice open source Common Lisp compilers, in this case it’s using Clozure Common Lisp.  The sources of the app amount to 16 lines of code.  Another few lines implement the hook used by the buildapp to compile the application.   Tiny applications like this are made possible thru the excellent build and library support in the modern lisp community; so most of the meat is defining how to use them (i.e. the system definition).

It’s interesting that none of the above installs common lisp on your own machine, nor does it check out the buildpack I built.  In total it adds ~200K to your local machine, most of which is the git repository.  The actual sources are about 14K, of which the image is 13.

To undo the above you need only delete the directory it creates and destroy the application on heroku: heroku apps:destroy <name>.

Heroku Buildpacks

1 Reply

buildpacks

Heroku is a cloud computing platform, i.e. a place where you can run your applications.  When you author an application on Heroku it is split into two parts.  One part, the buildpack, is responsible for building your application; while the other part is the actual application’s sources.

I found this interesting.  Modularity always serves to separate concerns.  What concerns, and who’s concerns, are natural next questions.  Buildpacks make it easier to get started with Heroku; so they address one of Heroku’s concerns – i.e. “How can we get people to use our product.” – by lowering the barrier to entry.

There are many buildpacks.  Heroku provides a half dozen, and other people have build a few hundred more.  Each build packs makes it easier to get an app based on some programming framework started.  For example there are ones for: ruby, node.js, erlang, wordpress, common lisp, go, etc. etc. etc.

Of course how exactly any given application get’s built tends to quickly become unique.  I suspect that most serious app developers customize their buildpacks.  Heroku makes extensive use of git to move things around, so naturally a lot of the buildpacks are on github.  I got thinking it would be interesting to look at some statistics about them.

The chart above has one dot for each of N buildpacks (in some cases multiple buildpacks have landed on the same dot).  Notice that the chart is log-log.   The vertical axis indicates how many +1 like stars the buildapp has received – so that’s a proxy for how popular it is.  The horizontal axis shows how often the buildapp has been forked.

In one’s fantasy the perfect build app for a given programming platform would fufill all the needs of it’s app developers.  In that case it would never need to be forked.  But three things work against that fantasy.  First off the buildapps aren’t perfect, in fact they tend to simplistic because their short-term goal is to make it trival to get started.  Secondly – I think – the building of applications tends to sport all kinds of unique special cases.  And finally the usual third reason – it’s hard work to push patches around, so even if you improve a given buildpack the chances your enhancments flow back to the master buildpack are – well – iffy.

Anyhow, I wanted to share the picture. (oh, and zero was mapped to .1)

Compass and Straight Edge

6 Replies

EuclideProgramming languages often have a juicy core of one kind or another.  Back in the 60s and 70s we had a lovely assortment of languages each of which took some particular idea to heart and then ran as far was they could with that idea.  SETL – using sets – is a good example.  It was a lot of fun to write within it’s framework.   I still my  delight when, at one point, they managed to get the compiler’s optimizer to the point where it spontaneously discovered assorted famous graph algorithms.

Other examples include:

  • SIMULA – using what we’d now call lightweight threads.  
  • SNOBOL – centered around pattern matching which informed a whole tangle of other languages like SL5, and Prolog, and such.
  • LISP – with its symbols, lists, etc. etc.
  • APL – with its arrays
  • etc. etc.

There others that stand atop a big data structure; SQL, Emacs, and AutoCAD.

Some stand on an unusual computational model.   Rule based (truth maintenance?) systems like Unix make or Prolog.   The constraint based systems.  Lazy evaluation.

A very few are almost only about some syntactic or semantic gimmick; like forth, postscript, or Python.

Is that era is largely over?   The search space has been mined out?  I guess some work on genetic programming or machine learning are the modern decedents of this style of language design.

All these languages have a kind of inward looking quality to them.  They don’t really care much about their users.  If there are applications, well that’s nice.  Their enthusiasm is rooted in their the juicy (often somewhat eccentric) center, not the tedium of actually putting them to use.  To a greater or lesser degree you can make that critique about all programming languages.

DSC02333Which brings me to a last night.  Harry Mairson gave a nice little talk to the Boston Lisp meeting about a spin off of his hobby – which is making string instruments.

Apparently we don’t actually have a good handle on how our ancestors designed and built their instruments.  Insta-theories might include that they traced existing instruments, or maybe they had templates they handed down, etc. etc.

One recent theory is that they had recipes that guided the making of patterns using only compass and straight edge.  There is a book that makes this case.  “Functional Geometry and the Traité de Lutherie.”

When Harry found and read this book he got to wondering if the descriptions in the book might be converted into something more algorithmic.  He spun off a little language where the juicy core was a compass and straight edge.  … time passes … and now he can write programs that almost sketch out the designs for cellos and such.   It was an awesome, eccentric, fun talk.

It’s notable that he did this backward.  He started from the application and ended up with a cute new language based on a curious juicy center.

I found myself wondering.  To what extent the design languages used by craftsmen in the Middle Ages rested on the compass and straightedge.  Architecture?  Furniture?  Music?  Here’s graphic I found showing a bit of (presumably modern) font design.

NayeraAbusteit-ArabicTypeface-2013

His work is not yet published, so all of you who are suddenly tempted to write a web server using only a compass and straight edge are best advised to wait until it is.

Update: Cool, there is now a paper you can read.  I look forward to seeing your web servers.

When Flags are Necessary

Leave a reply

flag_makingBack in January Jeff Hodges wrote a wonderful essay: “Notes on Distributed Systems for Young Bloods.”  There is a lot of good stuff in there.  I was remembering this recently because this section highlights something I’d not thought so clearly about; i.e. that feature flags run contrary to the usual software engineering best practice – as explained in the next to the last paragraph of this snippet:

“Feature flags are how infrastructure is rolled out. “Feature flags” are a common way product engineers roll out new features in a system. Feature flags are typically associated with frontend A/B testing where they are used to show a new design or feature to only some of the userbase. But they are a powerful way of replacing infrastructure as well.

Suppose you’re going from a single database to a service that hides the details of a new storage solution. Have the service wrap around the legacy storage, and ramp up writes to it slowly. With backfilling, comparison checks on read (another feature flag), and then slow ramp up of reads (yet another flag), you will have much more confidence and fewer disasters. Too many projects have failed because they went for the “big cutover” or a series of “big cutovers” that were then forced into rollbacks by bugs found too late.

Feature flags sound like a terrible mess of conditionals to a classically trained object-oriented developer or a new engineer with well-intentioned training. And the use of feature flags means accepting that having multiple versions of infrastructure and data is a norm, not an rarity. This is a deep lesson. What works well for single-machine systems sometimes falters in the face of distributed problems.

Feature flags are best understood as a trade-off, trading local complexity (in the code, in one system) for global simplicity and resilience.”

I think that helps to explain why the things that get built out over time so often seem to be in such desperate need of a total rewrite.  The litter left behind by the search (a/b testing) and the vistigal organs of the switch overs all frustrate some people’s sense of good design.  The design patterns at one scale or dimension are at odds here those of another.

Using Freight Trains to Store Energy

1 Reply

Grid scale energy storage systems are all such cool hacks.  Here’s a new one which will be popular with all you model train geeks.  In this scheme they build a specialized railroad, based on those used in mining, and haul freight cars full of somehting heavy (gravel?) back and forth between two rail yards.  The video (which is only tollerable if you turn the sound off, also skip the 1st 2 minutes) illustrated the idea.

668MW Facility from Advanced-RES on Vimeo.

Sadly one of my favorite schemes, Iowa Stored Energy Park, didn’t work out (lessons learned (pdf)).