Docker, part 2

The San Francisco Hook

I played with Docker some more.  It’s still in beta so, unsurprisingly, I ran into some problems.   It’s cool, none the less.

I made a repository for running OpenMCL, aka ccl, inside a container.   I set this up so the Lisp process expects to be managed using slime/swank.  So it exports port where swank listens for clients to connect.  When you run it you export that port, i.e. “-p 1234:4005” in the example below.

Docker shines at making it easy to try things like this.  Fire it up: “docker run –name=my_ccl -i -d -p 1234:4005 bhyde/crate-of-ccl”.   Docker will spontaneously fetch the everything you need.   Then you M-x slime-connect to :1234 and you are all set.  Well, almost, the hard part is  .

I have run this in two ways, on my Mac, and on DigitalOcean.  On the Mac you need to have a virtual machine running linux that will hold your containers – the usual way to do that is the boot2docker package.  On Digital Ocean you can either run a Linux droplet and then installed Docker, or you can use the application which bundles that for you.

I ran into lots of challenges getting access to the exported port.  In the end I settled on using good old ssh LocalForward statements in my ~/.ssh/config to bring the exported port back to my workstation.  Something like “LocalForward 91234” where that IP address that of an interface (docker0 for example) on the machine where the container is running.  Lots of other things look like they will work, but didn’t.

Docker consists of a client and a server (i.e. daemon).  Both are implemented in the same executable.  The client chats with the server using HTTP (approximately).  This usually happens over a Unix socket.  But you can ask the daemon to listen on a TCP port, and if you LocalForward that back to your workstation you can manage everything from there.  This is nice since you can avoid cluttering you container hosting machine with source files.  I have bash functions like this one “dfc () { docker -H tcp://localhost:2376 $@ ; }” which provides a for chatting with the docker daemon on my Digital Ocean machine.

OpenMCL/ccl doesn’t really like to be run as a server.   People work around by running it under something like screen (or tmux, detachtty, etc.).  Docker bundles this functionality, that’s what the -i switch (for interactive) requests in that docker run command.  Having done that you can then uses “docker log my_ccl” or “docker attach my_ccl” to dump the output or open a connection to Lisp process’ REPL.   You exit a docker attach session using control-C.  That can be difficult if you are inside of an Emacs comint session, in which case M-x comint-kill-subjob is sometimes helpful.

For reasons beyond my keen doing “echo ‘(print :hi)’ | docker attach my_ccl” get’s slightly different results depending on Digital Ocean v.s. boot2docker.  Still you can use that to do assorted simple things.   UIOP is included in the image along with Quicklisp, so you can do uiop:runprogram calls … for example to apt-get etc.

Of course if you really want to do apt-get, install a bundle of Lisp code, etc. you ought to create a new container built on this one.  That kind of layering is another place where Docker shines.

So far I haven’t puzzled out how to run one liners.  Something like: “docker run –rm bhyde/crate-of-ccl ccl -e ‘(print :hi)'” doesn’t work out as I’d expect.  It appears that argument pass thru, arg. quoting, and that the plumbing of standard IO et. al. is full of personality which I haven’t comprehended.  Or maybe there are bugs.

That’s frustrating – I undermines my desire to do sterile testing.


5 thoughts on “Docker, part 2

  1. Patrick Stein

    Lol… “full of personality which I haven’t comprehended.” That perfectly explains many such things…. I promise to always attribute you when I use that phrase.

  2. Faré

    Is it using to connect to the container on the other side? Have you tried giving it an extra level of shell quoting?

    In zsh, it’s available with “${(qqq)A}” where A is an array, e.g. A=(“a b” “c d” “e”)

    In CL, it’s availalbe with uiop:escape-command and/or uiop:escape-sh-command depending on how you interact with local Windows.

    Otherwise, I recommend you first create a trivial executable that echos its arguments, e.g. using cl-launch, and then figure out what kind of expansion is taking place.

  3. bhyde Post author

    Faré, thanks for the suggestions. I was trying to avoid getting to down in the weeds in that portion. But, I suspect the issue is mostly bugs in the docker code; given that it varies depending on which hosting scheme is in use.

    Meanwhile, I considered putting cl-launch and N implementations in a container … maybe another day.

  4. Faré

    Worse comes to worst, if you find you can’t reliably pass spaces, even quoted, you can use the magic quoting strategy in the minimakefile branch of asdf in tools/invoke-lisp.lisp — or your favorite variant thereof.

  5. bhyde Post author

    Here’s a dependable work around for the personality issues I was encountering.

    Write a script, put it in a directory, mount that when you launch container, run the script. Something like:

    db2d run --rm --volume=`pwd`/work-to-do/:/w2d bhyde/crate-of-ccl /w2d/do-the-work

    Your work to do script might look like this:

    cd $( dirname $0 )
    ccl > log 2>&1 <

Leave a Reply

Your email address will not be published. Required fields are marked *