One project I’ve been doing while unemployeed is to review how the Common Lisp landscape has changed since I was last paying attention. And the big change is the emergance of a rapidly growing pool of libraries (900+) all distributed via Quicklisp.
My current favorite is a library called Optima. Optima adds constructs for doing rich pattern matching. It can match lists, arrays, objects, structs, strings; and if you load fare-quasiquote you can write patterns using backquote.
Let’s build a JavaScript to Parenscript translator. First we can use a Javascript parsing library.
> (ql:quickload '("optima" "fare-quasiquote" "parse-js" "parenscript"))
...
> (defpackage #:js2ps (:use #:cl #:optima #:parenscript #:parse-js))
> (in-package #:js2ps)
> (pprint (parse-js "a[3] += f(x,y)")))
(:toplevel
((:stat
(:assign :+
(:sub (:name "a") (:num 3))
(:call (:name "f")
((:name "x") (:name "y")))))))
Then we can write a recursive function using Optima to translate that into parenscript like so:
(defun js2ps (txt)
(labels
((r (x)
(match x
(`(:toplevel ,stms) `(progn ,@(mapcar #'r stms)))
(`(:stat ,s) `(progn ,(r s)))
(`(:assign :+ ,left ,right) `(incf ,(r left) ,(r right)))
(`(:call ,func ,args) `(,(r func) ,@(mapcar #'r args)))
(`(:name ,name) (js2ps-name name))
(`(:num ,n) n)
(`(:sub ,array ,index) `(aref ,(r array) ,(r index)))
(eck (error "TBD ~S" eck)))))
(r (parse-js txt))))
Let’s try it, and then using ps*
we can have parenscript translate it back into Javascript.
js2ps> (js2ps "a[3] += f(x,y)")
(progn (progn (incf (aref a 3) (f x y))))
js2ps> (ps* *)
"a[3] += f(x, y);"
You have already noticed that the patterns look just like the code for building the result. Pretty eh?
A complete translator is easy. The fiddly bits arise around handling variable bindings and around what in Parenscript are called @ and chain. There code here that does much of that.
I’ve also used Optima is to scrap web pages. It’s trivial to load and parse an page into an s-expression, just quickload “drakma” and “closure-html”, and then do (parse (http-request url) (closure-html:make-lhtml-builder)). Then let’s say you wanted to know the prices that macbook airs have sold for on ebay.
(defun snarf-prices (page-sxpr)
;; for example: (snarf-prices (fetch-price-page "macbook air 13" 3))
(labels ((recure (x)
(match x
(`(:div ((:class "g-b bidsold") (:itemprop "price"))
,(ppcre "\\$([\\d.]+)" price))
(push (parse-number price) result))
(`(,(satisfies keywordp) _ ,@children)
(map nil #'recure children)))))
(recure page-sxpr)
(nreverse result))))
The second pattern match `(,(satisfies keywordp) _ ,@children)
uses two new tricks. The statisfies
shows how we can put arbitrary predicates into your patterns, and the _
is a wildcard.
The first pattern shows how to match strings to regular expressions. The pattern (ppcre “\\w*\\$([\\d.]+)” price) would match the string
” $647.10″ and binds price to the string “647.10”.
The code for an earlier version of this page scraping example can be seen here.
One problem with Optima is that it’s a bit addictive. Once you start using it you stop using lots of other techniques. For example I hardly ever use cl-ppcre directly anymore. For example if I want to pluck the host out of some URLs I’ll write: (match url ((ppcre "//([-_.\\w]+)/" host) host))
. It also lets me write code in the style of awk or perl, i.e. with a set of pattern matches.
And I should have mentioned, it expands into pretty good raw code. I say pretty good rather than great because it doesn’t currently factor out common subexpressions.
So try it out, you’ll like it.