Category Archives: common-lisp

Graphical Programming and yEd

Leave a reply

Graphical programming languages are like red sports cars.  They have lots of curb appeal, but they are rarely safe and reliable.

I long worked for a company whose product featured a very rich graphic programming. It allowed an extremely effective sales process.  The salesman would visit the customer who would sketch a picture of his problem on the whiteboard, and the salesman would enquire about how bad things would get if the problem didn’t get solved.

Meanwhile in the corner the sales engineer would copy the drawing into his notebook.  That night he would create an app in our product who’s front page looked as much like that drawing as possible.  It didn’t really matter if it did anything, but it usually did a little simulation and some icons would animate and some charts’ would scroll.  The customers would be very excited by these little demos.

I consider those last two paragraphs a delightful bit of sardonic humor.  But such products do sell well.   Customers like how pretty they look.  Sales likes them.  Engineering gets to have mixed feelings.  The maintenance contracts can be lucrative.  Thathelps with buisness model volatility.  So yeah, there is plenty of value in graphical programming.

So one of the lightning talks at ILC 2014 caught my attention.  The speaker, Paul Tarvydas, mentioned in passing that he had a little hack based on a free drawing application called yEd.  That evening I wrote a similar little hack.

Using yEd you can make an illustrations, like this one showing the software release process for most startups.

My few lines of code will extract the topology from the drawing, at which point you can build whatever strikes your fancy: code, ontologies, data structures.  (Have I mentioned how much fun it is to use Optima to digest into a glob of XML?  Why yes I have.)

I was also provoked by Fare Rideaus‘ talk.  Fare is evangelizing the idea that we ought to start using Lisp for scripting.   He has a package, cl-launch, intended to support this.  Here’s an example script.   Let’s dump the edges in that drawing:

bash-3.2$ ./topology.sh abc.graphml
Alpha -> Beta
Beta -> Cancel
Beta -> Beta
Beta -> Beta
bash-3.2$

I’ve noticed, dear Reader, that you are very observant.  It’s one of the things I admire about you.  So you wondering: “Yeah Ben, you found too many edges!”   Well, I warned you that these sports cars are rarely safe.  Didn’t I?

Docker, part 2

5 Replies

The San Francisco Hook

I played with Docker some more.  It’s still in beta so, unsurprisingly, I ran into some problems.   It’s cool, none the less.

I made a repository for running OpenMCL, aka ccl, inside a container.   I set this up so the Lisp process expects to be managed using slime/swank.  So it exports port where swank listens for clients to connect.  When you run it you export that port, i.e. “-p 1234:4005” in the example below.

Docker shines at making it easy to try things like this.  Fire it up: “docker run –name=my_ccl -i -d -p 1234:4005 bhyde/crate-of-ccl”.   Docker will spontaneously fetch the everything you need.   Then you M-x slime-connect to :1234 and you are all set.  Well, almost, the hard part is  .

I have run this in two ways, on my Mac, and on DigitalOcean.  On the Mac you need to have a virtual machine running linux that will hold your containers – the usual way to do that is the boot2docker package.  On Digital Ocean you can either run a Linux droplet and then installed Docker, or you can use the application which bundles that for you.

I ran into lots of challenges getting access to the exported port.  In the end I settled on using good old ssh LocalForward statements in my ~/.ssh/config to bring the exported port back to my workstation.  Something like “LocalForward 91234 172.17.42.1:1234” where that IP address that of an interface (docker0 for example) on the machine where the container is running.  Lots of other things look like they will work, but didn’t.

Docker consists of a client and a server (i.e. daemon).  Both are implemented in the same executable.  The client chats with the server using HTTP (approximately).  This usually happens over a Unix socket.  But you can ask the daemon to listen on a TCP port, and if you LocalForward that back to your workstation you can manage everything from there.  This is nice since you can avoid cluttering you container hosting machine with source files.  I have bash functions like this one “dfc () { docker -H tcp://localhost:2376 $@ ; }” which provides a for chatting with the docker daemon on my Digital Ocean machine.

OpenMCL/ccl doesn’t really like to be run as a server.   People work around by running it under something like screen (or tmux, detachtty, etc.).  Docker bundles this functionality, that’s what the -i switch (for interactive) requests in that docker run command.  Having done that you can then uses “docker log my_ccl” or “docker attach my_ccl” to dump the output or open a connection to Lisp process’ REPL.   You exit a docker attach session using control-C.  That can be difficult if you are inside of an Emacs comint session, in which case M-x comint-kill-subjob is sometimes helpful.

For reasons beyond my keen doing “echo ‘(print :hi)’ | docker attach my_ccl” get’s slightly different results depending on Digital Ocean v.s. boot2docker.  Still you can use that to do assorted simple things.   UIOP is included in the image along with Quicklisp, so you can do uiop:runprogram calls … for example to apt-get etc.

Of course if you really want to do apt-get, install a bundle of Lisp code, etc. you ought to create a new container built on this one.  That kind of layering is another place where Docker shines.

So far I haven’t puzzled out how to run one liners.  Something like: “docker run –rm bhyde/crate-of-ccl ccl -e ‘(print :hi)'” doesn’t work out as I’d expect.  It appears that argument pass thru, arg. quoting, and that the plumbing of standard IO et. al. is full of personality which I haven’t comprehended.  Or maybe there are bugs.

That’s frustrating – I undermines my desire to do sterile testing.

 

MOCL demo

Leave a reply

When I first heard about MOCL a few years ago I was pretty sure it wouldn’t survive, but it looks like I was wrong.  See this nice video that the folks at wukix.com have recently posted.  It’s an impressive 15 minute video demo of using MOCL to author in Common Lisp an application targeted to IOS.

Love that remote REPL for debugging your application!

MOCL has a fair amount of extended syntax so it can play nice with Objective C.

I’m surprised they don’t have a free demo version.  But then, I’m a cheapskate!

So, go watch the video 🙂.

cl-xmpp

Leave a reply

Cl-xmpp has, sadly, fallen into disrepair. It can’t connect to anything I tried to connect oo. It’s hard to find it’s development list archives.   Common-lisp.net is working thru some troubles. FYI – you can subscribe to common-lisp.net mailing lists by adding +subscribe to the list’s name, as in cl-xmpp-devel+subscribe I built my own index to the archives as so:


cl-user> (loop for y from 2008 to 2014 do (loop for m in '("January" "February" "March" "April" "May" "June" "July" "August" "September" "October" "November" "December") as u = (format nil "http://lists.common-lisp.net/pipermail/cl-xmpp-devel/~A-~A/thread.html" y m) do (multiple-value-bind (a b) (drakma:http-request u :method :head) (declare (ignore a)) (when (= 200 b) (print u)))))

"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2008-January/thread.html" 
"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2008-June/thread.html" 
"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2008-July/thread.html" 
"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2008-August/thread.html" 
"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2009-January/thread.html" 
"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2009-September/thread.html" 
"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2010-March/thread.html" 
"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2010-August/thread.html" 
"http://lists.common-lisp.net/pipermail/cl-xmpp-devel/2011-March/thread.html" 
nil

Some random patches mentioned there … I’ll have to try those.

And all I wanted to do was send an IM when ever the bouy outside of Boston harbor is forecast to have big waves.

tidy up the output of lisp macros

2 Replies

For some reason it makes my teeth hurt to have my macros generate code that I wouldn’t have written by hand.  For example it’s not hard to get code like this out of a macro expansion.

(let ()
  (progn
    (if (fp x)
      (progn 
         (f1 x)
         (f2 x)))))

v.s. what I might like:

(when (fp x)
   (f1 x)
   (f2 x))

I probably ought to just relax and ignore it, but instead I often revise macros so the code they generate is nicer to look at.   So that:

`(let ,vars ,@body)

becomes

(if vars
    `(let ,vars ,@body)
    `(progn ,@body))

This is silly!   Now I have ugly macros instead of ugly output.   I’m just moving the ugly bits around.

So I’ve started doing this:

(tidy-expression `(let ,vars ,@body))

where tidy-expression is something like this:

(defun tidy-expression (x)
  (match x
    (`(or ,a (or ,@b)) `(or ,a ,@b))
    (`(progn ,a (progn ,@b)) `(progn ,a ,@b))
    (`(progn ,x) x)
    (`(and ,a (and ,@b)) `(and ,a ,@b))
    (`(if ,a (progn ,@b)) `(when ,a ,@b))
    (`(if ,a (progn ,@b) (progn ,@c)) `(cond (,a ,@b) (t ,@c)))
    (`(if ,a (progn ,@b) ,c) `(cond (,a ,@b) (t ,c)))
    (`(if ,a ,b (progn ,@c)) `(cond (,a ,b) (t ,@c)))
    (`(let ,vs (progn ,@body)) (tidy-expression `(let ,vs ,@body)))
    (`(let nil ,@body) (tidy-expression `(progn ,@body)))
    (_ x)))

It’s another chapter in my crush on optima.

I write these tidy up functions as necessary.

That example only chews on the top of the form.   If you wanted something to clean up the first example you’d need to write tidy-expression-all.

(tidy-expression-all
 '(progn
   (if (fp x)
       (progn 
         (f1 x)
         (f2 x)))))
-->
(when (fp x) (f1 x) (f2 x))

This all reminds me of Warren Teitelman’s programmer’s assistant in Interlisp.  It reminds me of some of the things that flycheck in Emacs does for other programming languages.   It reminds me that I’ve been wondering what would a lint for Common Lisp would look like.

I bet somebody already wrote a generalized tidy expression and I just don’t know were to look.

,@ v.s. ,.

2 Replies

I’m surprised that I didn’t know about the ,. construct in Common Lisp’s backquote syntax.  It is equivalent to ,@ except that it licenses the implementation to destructively modified the tail of the list being inlined.

cl-user> (defparameter *x* '(1 2 3))
*x*
cl-user> *x*
(1 2 3)
cl-user> `(a b ,@*x* c d)
(a b 1 2 3 c d)
cl-user> *x*
(1 2 3)
cl-user> `(a b ,.*x* c d)
(a b 1 2 3 c d)
cl-user> *x*
(1 2 3 c d)
cl-user>

Tasty Languages

Leave a reply

haggisMy mother, or so I am told, had a device for dealing with that frustrating syndrome where you wake up in the middle of the night and your damn brain decided to tour all the things that are making you anxious.   The trick was to try and enumerate something, for example vegetables in alphabetical order.

So I’ve be trying this out but I needed something to enumerate.  At first I was enumerating foods, and then I started adding constraints.  … gizzards, haggis, intestines, …   Then I started enumerating programing languages: APL, Basic, C, Datalog, …   But it was became fun to try and include a language only if it has some aspect that makes me smile.  For example the function calling weirdness in SL5.   Elang is a layer cake of odd smilies, though then I can’t use E.  R is a must have for how evaluates function arguments.

You might pick Caja[1,2] for C.   And that lead me to recall Dart.  I’ve been meaning to go back and see what’s up with Dart.   This talk is a fascinating entry point into Dart, at least for me.  Google managed to find a good guy, Gilad Bracha, for this design challenge!

For the me the tasty thing in this talk is that they are trying to stand in the middle ground between a dynamically typed language and something, well, something else.

This decades after what Common Lisp did about this problem.  In Common Lisp you can declare that something is, for example, a small integer; but the programming environment can ignore that; it’s just advice.  In fact even in a single language environment how that statement effects things can vary depending on other stuff like optimization settings.   If you implementation ignores the declarations then they practically comments.  I think it was an Algol manual that documented comments by saying “the compiler makes no effort to check the correctness of the comments.”  This wiggle room makes some people’s skin crawl.  And, it’s certainly enticed a lot of engineering effort on the part of Common Lisp implementors.

In fact a Common Lisp programmer can do wonderfully weird things with the type system and declarations.  For example we can define a type that asserts our graphs are acyclic and provide a predicate that we only use in desperate situations.

(defun acyclic-graph-p-for-type (g)  (if *desperate-debug* (acyclic-graph-p g) t))

(deftype acyclic-graph () (and graph (satisfies #'acyclic-graph-p-for-type))

The reason I found that Dart talk so interesting is the how Gilad deals with the skin crawling issue. He adopts a war weary Eeyore-ish manner.  I can totally relate.  He makes no real effort to argue why this is a useful such a powerful and useful approach.  In fact I’d say he baits his audience into an absence of sympathy.

I’ve done some very fun things with type systems that are analogous to what he is calling optional typing, for example this diagnostic typing I described a while back.

I’ll have to dig some more to see if any of the Caja ideas survived into Dart.  But they are similar, in the sense that there turn out to be many type system like blankets one might want to throw over your program to make you feel more cozy.  And if you insist that on exactly when in the program life cycle, if ever, they are checked (or worse proven) it’s just not as much fun.

The talks also made me sad.  It’s clear there is a lot of language design argot that I haven’t kept up with.

… junket, keratin, …

Eliza in BBN-Lisp

Leave a reply

Over in comp.lang.lisp Jeff writes.

With the permission and assistance of the author himself, Bernie Cosell, I have added the original Lisp Eliza to the Eliza Generations collection. Cosell wrote this Lisp version of Eliza at BBN in the mid-late 1960s. (Weizenbaum’s original was written in about 1966 in SLIP, a long-dead Fortran-based symbol processing package.)

See: http://elizagen.org/index.html

Thanks to Peter Seibel for connecting me with Bernie Cosell.

!!YOU CAN HELP!!

One way that you can help is by writing to Deborah Cotton (cot…@hq.acm.org) at the ACM permissions office and encourage them to open source Weizenbaum’s paper, which is still inaccessible under copyright protections.

Second, I’m hoping to create a “perfect” OCR of this code and then macrify it to run on CL with as little modification as possible. You can help create the codebase for this by choosing a single page at random from among the 48 TIFFs, manually entering the code as precisely as possible (including indentation) into a text file, and then emailing it to me: jshr…@stanford.edu. If you decided to do this, here are a few details worth attending to: So that we get good coverage, please really choose a TIFF file at random, e.g., via the moral equivalent of (1+ (random 48)). Please don’t just OCR the TIFF file; I’ve already tried this with very high end OCR tools, and they make terrible encodings of this sort of thing! It would help me do the reconstruction is you put the name of the TIFF file in a leading comment. Finally, if you would like to be explicitly acknowledged for your efforts, please include a comment line for yourself as well. The codebase will be released on github or some such public location. Then you’ll be able to help actually hack it!

Finally, if you know of open source Elizas, in any language, roaming around the net, please send me links to them so that I can update the “more recent” section of the page.

Thanks!

Cheers,
‘Jeff

Which is a delightful boondoggle. And, as I’ve currently got a lousy cold transcription is amount the most strenuous activity I’m up for. So, I’ve done two pages.  This gave me Interlisp flashbacks, which was fun.

They are very short, so you should do a few too.  Grab a random page here.

You can use this bash oneliner to pick a random page

curl -s http://shrager.org/eliza/20131112-Eliza600dpiRawScansRenamed/index.html | grep -o '>Eliz.*TIF' | sed -n $(( 1 + $RANDOM % 48 ))p

and then grab that page from here: http://shrager.org/eliza/20131112-Eliza600dpiRawScansRenamed/index.html

Quicklisp new packages, October 2013

2 Replies

Xach has announced the the October 2013 quicklisp release.  Below are short summaries of the new packages.

asdf-package-system 2K -- No license specified?
  No description provided.
  git: git://common-lisp.net/projects/asdf/asdf-package-system.git

ayah-captcha 4K -- MIT
  A simple interface to the API of the play-thru captcha of areYouAHuman.com
  author: Andy Peterson
  git: https://github.com/aarvid/ayah-captcha.git
  more: https://github.com/aarvid/ayah-captcha#readme

cl-binaural 4K -- GPL
  Utilities to generate binaural sound from mono
  author: Alexander Popolitov
  git: https://github.com/mabragor/cl-binaural.git
  more: https://github.com/mabragor/cl-binaural#readme

fgraph 6K -- EUPL V1.1
  No description provided.
  author: Thomas Bartscher
  mercurial: https://bitbucket.org/thomas_bartscher/cl-fgraph
  more: https://bitbucket.org/thomas_bartscher/cl-fgraph

cl-larval 20K -- GPL
  Lisp syntax for assembler for AVR microcontrollers
  author: Alexander Popolitov
  git: https://github.com/mabragor/cl-larval.git
  more: https://github.com/mabragor/cl-larval#readme

cl-ply 10K -- LLGPL
  A library to handle PLY file format which is known as the Polygon File Format or the Stanford Triangle Format in Common Lisp.
  author: Masayuki Takagi
  git: https://github.com/takagi/cl-ply.git
  more: https://github.com/takagi/cl-ply#readme

cl-server-manager 5K -- MIT
  Manage port-based servers (e.g., Swank and Hunchentoot) through a unified interface.
  author: Wei Peng
  git: https://github.com/pw4ever/cl-server-manager.git
  more: https://github.com/pw4ever/cl-server-manager#readme

cl-spark 10K -- MIT License
  Generates sparkline string for a list of the numbers.
  author: Takaya OCHIAI
  git: https://github.com/tkych/cl-spark.git
  more: https://github.com/tkych/cl-spark#readme

curry-compose-reader-macros 2K -- GPL V3
  reader macros for concise function partial application and composition
  author: Eric Schulte
  git: https://github.com/eschulte/curry-compose-reader-macros.git
  more: https://github.com/eschulte/curry-compose-reader-macros#readme

drakma-async 26K -- MIT
  An asynchronous port of the Drakma HTTP client.
  author: Andrew Danger Lyon
  git: https://github.com/orthecreedence/drakma-async.git
  more: https://github.com/orthecreedence/drakma-async#readme

draw-cons-tree 2K -- Public Domain
  Makes and ascii picture of a cons tree
  author: Ported by:CBaggers - Original Author:Nils M Holm
  git: https://github.com/cbaggers/draw-cons-tree.git
  more: https://github.com/cbaggers/draw-cons-tree#readme

filtered-functions 5K 
  An extension of generic function invocation that add a preprocessing step before the usual dispatch.
  No description provided.
  author: Pascal Costanza
  darcs: http://common-lisp.net/project/closer/repos/filtered-functions/
  more: http://common-lisp.net/project/closer/filtered.html

graph 37K -- GPL V3
  Simple library for building and manipulating graphs.
  author: Eric Schulte, Thomas Dye
  git: https://github.com/eschulte/graph.git
  more: https://github.com/eschulte/graph#readme

lisphys 19K -- Specify license here
  Describe lisphys here
  author: Guillaume
  git: https://github.com/kayhman/lisphys.git
  more: https://github.com/kayhman/lisphys#readme

mailbox 2K -- MIT
  Simple multithreading mailboxes.
  author: Lucien Pullen
  git: https://github.com/drurowin/mailbox.git
  more: https://github.com/drurowin/mailbox#readme

map-set 2K -- BSD 3-clause (See LICENSE)
  Set-like data structure.
  author: Robert Smith
  mercurial: https://bitbucket.org/tarballs_are_good/map-set
  more: https://bitbucket.org/tarballs_are_good/map-set

cl-oneliner 3K -- wtfpl
  Given a piece of text, summarize it with a one-liner
  author: mck-
  git: https://github.com/mck-/oneliner.git
  more: https://github.com/mck-/oneliner#readme

pooler 4K -- MIT
  A generic pooler library.
  author: Abhijit Rao
  git: https://github.com/quasi/pooler.git
  more: https://github.com/quasi/pooler#readme

readable 443K -- MIT
  'Readable' extensions to Lisp s-expresions: curly-infix, neoteric, and sweet expressions
  author: David A. Wheeler
  git: git://git.code.sf.net/p/readable/code
  more:  http://readable.sourceforge.net/

simple-currency 13K -- BSD, 2 clause.
  Currency conversions using data published daily by the European Central Bank.
  author: Peter Wood
  git: https://github.com/a0-prw/simple-currency.git
  more: https://github.com/a0-prw/simple-currency#readme

smackjack 17K -- MIT
  A small Ajax framework for hunchentoot using parenscript
  author: Andy Peterson
  git: https://github.com/aarvid/SmackJack
  more: https://github.com/aarvid/SmackJack#readme

snappy 7K -- New BSD license.  See the copyright messages in individual files.
  An implementation of Google's Snappy compression algorithm.
  author: Robert Brown
  git: https://github.com/brown/snappy.git
  more: https://github.com/brown/snappy#readme

spellcheck 2,376K -- MIT
  Peter Norvig's spell corrector.
  author: Mikael Jansson
  git: https://github.com/RobBlackwell/spellcheck.git
  more: https://github.com/RobBlackwell/spellcheck#readme

tagger 1,107K -- No license specified?
  No description provided.
  git: https://github.com/g000001/tagger.git
  more: https://github.com/g000001/tagger#readme

As usual many older packages were revised.