An ugly security hole has been uncovered in Grease Monkey, the Firefox extension, if your a user act now.
Grease Monkey Troubles
Leave a reply
Category Archives: programming
Private Mail & RSS
I think this is very clever: use Grease Monkey to decrypt content that’s stored on 3rd party servers. In the example the body of the RSS entries are encrypted. Then the user subscribes to the feed via Bloglines. Finally when viewing the entry a Grease Monkey script notices the encrypted text and decrypts it so the user can view it. The same idea would work for email sent to a hotmail or gmail.
I find it interesting that this use case wasn’t a standard part of browser design.
domain specific languages
Back in the 1970s when I first learned Unix the key idea was that you built lots of clever tools which you’d then hook up in pipelines and shell scripts to solve larger problems. Tools like grep, sed, awk where exemplars of this approach. Tools like lex and yacc (now flex and bison) made it trivial to invent new micro-languages for the problem at hand. It was common to work on systems with a dozen lexer’s and parsers. The unix macro language m4 provides another scheme for getting a micro language.
One reason I got into Lisp was that after building dozens of systems using those design patterns I began to yearn for something that would unify it all. Perl, which came later, is one example of that urge comming to bearing fruit. Two things really got me interested in Lisp; symbols and macros.
Rainer Joswig put up a video (Quicktime via BitTorrent) showing a simple example of how you can use macros in Lisp to build a micro-language. It looks like he stumbled across an article illustrating the micro-language approach and it made him sad to see how horribly complex this approach is in other languages.
The example sets up a micro-language for parsing for call records dumped from a phone switch.
You get to watch a not a-typical Lisp coding session. He grabs some example data to play with. Then he grabs the text from the article that outlines the record formats. Thru the demo that text will get reformated into the statements in the new micro-language. The largest change that happens to that text happens early because he changes all the tokens so they conform to lisp coding conventions.
He then writes a bit of code to parse one line. After testing that code he will reformat it into a Lisp macro so he can stamp out parsers for each flavor of record. This is cool part. For a nonLisp programmer the going gets a bit rough at this point. The tricks of the trade for writting macros aren’t explained and so it’s probably hard to see how the macros work. For example he uses something called “backquote” to create templates that guide the expansions. The backquote templates are denoted with a single character almost the smallest character that could be used, i.e. the single backquote. When the templates are expanded they are copied and as they are copied bits are filled into them were ever comma (and some other syntax sugar) appear.
Lots of other things are glossed over, for example that when you compile a thing like this the macros are all expanded at compile time and don’t necessarily even survive into the runtime environment.
But the final finished micro language is very small and concise so it makes a nice introduction into the kind of programming that I miss when programming in other languages.
Thanks to Zach Beane for the putting together the BitTorrent, it only took 7 minutes to download from 20+ peers in the swarm. Rainer Joswig original post is here.
Meanwhile. Can it really be true that there isn’t a version of lex/flex that handles utf-8 or 16 bit characters? That is too bizzare. It’s as if the corner stone of an entire culture of programming hasn’t made the transition to the world of international data handling.
Trust in Exchange Swarms (and Streaming)
Exchange networks are full of trust and reliability issues. How do you know that the intermediaries aren’t modifying the content? How do you know that swarm members in a p2p distribution scheme aren’t freeloading?
You can tackle the freeloading problem with a reputation scheme. The participants in the swarm can report on how helpful their peers are. The system can create statistics on those reports. The statistic for a given peer become his reputation. In networks where peers have durable identities and participate over long periods this is probably sufficient to reduce freeloading to reasonable levels. Though you would still need generous participants to enable new participants to enter the swarms.
That participants can collaborate to keep an eye on each other. These models are statistical models of the behavior reports. Successful collaborative efforts demand a lot forgiveness and generosity, so for many measures it’s not a problem that the models are only estimates. What you sum up into the participants reputations will define what the participants aspire to. For example if you model how long a participant stays in the swarm and you can create incentives to stay longer.
The standard solution to middleman problems is to insist on validation end-to-end. That’s one reason the .torrent files in BitTorrent have checksums in them. An intermediary can’t modify the distributed file without the end user noticing.
I’m interested in how to use swarming approaches on streaming data; for example video or audio broadcast, blog pings, weather or other sensor networks. Harder to get an end to end solution for these streams. The broadcaster could sign every packet, I guess, but that would put significantly add to the costs.
Another possible approach is to assure that the checksums and the packets they validate travel by different and hard to predict routes. Participants can report on their peers adherence to a rule that checksum’s should never travel with their associated packets.
BitTorrent’s module dependency
I’ve been reading the BitTorrent code. Which lead to discovering this nice tool for drawing python module dependency graphs.
The drawing shows the modules of a swarm member in the current beta version (4.1.2) of BitTorrent. In the middle, the brown tangle, is the new DHT (or distributed hash table). Using that a swarm implement their own tracker. The two purple bits on the lower right implement the http server for the tracker. The purple bits on the left include the BitTorrent encoder, or bencode, as well as configuration and argument parsing. The stuff at the top implements the swarm’s collaborative file exchange; e.g. is orchestrating the exchange of pieces over connections with peers at reasonably rates. Here is a pdf version of that drawing.
If you want to use the module drawing tool you really need to subclass it’s main class and fine tune what’s shown. That’s not hard but you probably have to climb the foot hills of both the graphviz and python learning curves. If you enjoy playing with graphviz, as I do, then you can edit’s it’s output before handing it off to the graph layout program. The Macintosh application that wraps up graphviz in a pretty Mac GUI is sweet. (Or, LOL, you could paypal me $998.35, email me your code and I’ll make one for you! Fourth of July sale, mention this ad before the 10th of July and get 10% off!)
Running Lispworks from the shell.
Shell script for Mac OS X that launchs Lispworks Personal edition and then uses applescript to initialize it. The initialization then presumably loads swank, so slime can talk to it. This works with the recently release 4.4.5 LispWorks.
I spend an inordinate amount of time working thru an AppleScript error where the line activate application "LispWorks Personal" would fail on one machine but not on another. Finally deleting LispWorks, empting the trash, rebooting the machine, reinstalling LispWorks, and firing it up via the finder before running this script fixed it. I think some sort of directory of known applications was wedged.
#!/bin/sh
# Because we fork, and then wait for the fork to end you can't do all
# this in AppleScript.
# 1. Fork Lispworks.
# 2. Run an applescript to step it thru start-up.
# 3. Finally wait till it exits.
cd ~/w/lispworks
"/Applications/LispWorks Personal 4.4.5/LispWorks Personal.app/Contents/MacOS/lispworks-personal-4-4-5-darwin" &
osascript <<EOS
-- Wait for it.
tell application "System Events"
repeat until process "lispworks-personal-4-4-5-darwin" exists
delay 1
end repeat
activate application "LispWorks Personal"
-- Clear splash screen, force feed the init, hide the application.
tell process "lispworks-personal-4-4-5-darwin"
repeat until window "LispWorks Personal Edition" exists
delay 1
end repeat
click button "Close" of window "LispWorks Personal Edition"
tell window "Listener 1"
keystroke "(load \"~/w/lispworks/init\")"
keystroke return
end tell
-- set visible to false
end tell
end tell
EOS
wait
Space: the API
My car’s check engine light came on. Being both modern and a cheapskate I found the online community where Passat owners hang out. My old car there had a secret pattern of key turns and button pushes that would cause the car reveal engine’s fault. I was hoping for something similar.
But car makers love to keep things secret. German car maker are more secretive than Japanese makers. So it is verboten for Passat owners to know what lies behind their check engine lights. My fellow owners showed me a way. Autozone will read out your check engine codes for free. Autozone has solidarity with those who to fix their own cars.
The man at Autozone climbed under my steering wheel and plugged in. He and I copied the codes down onto a peice of paper. He then placed his finger just under a large yellow button on the read out device. He held it up, away from his body, toward me. He turned and gazing away into the distance. My eyes followed, a large warehouse sat on the far side of the parking lot. He said speaking into the empty car. “If it’s still under warrenty we can’t press the big yellow button. That clears the codes.” He paused, gazing more closely at the warehouse. “I” “Can’t push the yellow button.”
The inside of Autozone is full of gadgets to hack your car. An entire asle of things that plug into the cigarette lighter. A wall of steering wheel covers. Two racks of things to hang from the rear view mirror. Devices that clip to the air vents on the dash board.
I once built a store front. A place where developers could sell their wares. The developer products all plugged into an existing product. Our product. We called it the developer’s market place, it was a way for the developers to reach our customers, our product’s users.
As mentioned car makers like to horde their options. They like to sell you the radio rather than relinquish that option to some random 3rd party. So generally auto makers are not very big on open APIs. I suspect they grumble about that cigarette lighter.
But standing looking at the Hawain shirt steering wheel covers at Autozone I had an epiphany. How little it takes to create an API. How small an affordance. The convex shape of the steering wheel, the latent hook on the rear view mirror, the trickle of electricity in the cigarette lighter.
A few days later I was listening to a talk by one of the dudes who have been hacking on the Prius. He throws up a slide showing the floor under the hatchback. There’s a panel. The panel is removed in his next slide and there is a small shallow space. Just a few inches deep. Useless really.
But, immediately I knew. That empty space! Full of latent energy. Waiting for their hack. Drawing them in. “Fill me!”
What happened? Why did Toyota leave that space? Did they actually know what they were doing. Did they know that empty space is an open API for developers? It’s not as hard to create open APIs as you might think.
Brand Religiosity
There a lot of fun “statically improbable phrases” in this paper (sadly hidden behind a garden wall) about religiosity in brand communities. It’s about the Newton community, which like the Lisp community, can be described as “operating in a threatened state.”
Like all communities these brand communities have rituals, including stories. And when the community is threatened then you can look for these kinds of stories: “(1) tails of persecution, (2) tales of faith rewarded, (3) survival tales, (4) tales of miraculous recovery, and (5) tales of resurrection.”
I particularly liked the idea of “highly visible stigma symbol.” The damn back lite apple on my powerbook for example. Until recently I could use the word closure as a stigma symbol but it apparently it’s making a comeback. The stigma symbol attracts persecution. Communities have what salesmen call objection handling techniques for responding to those. In this paper we get the wonderful phrase “taming the facts.”
Of course Brand communities have product at their center, often technological products. Technology is magical. That leads to the wonderful phrase “technopagan magic.” The best heroic fantasy tales deal with persecution with a burst of tecnopagan magic.
But the real reason I needed to write this posting was this marvalous signature line used by somebody in the Newton community:
“Would the last person to leave the platform, please turn off the backlight.”
Minuteman Library Grease Monkey
One sign of a good open source project is that it has lots of rough edges for people to rub down.
For example there is a little monkey that will tinker with Amazon pages to add a link to your local library, when they have the book your considering. This script needs to be reworked to get to your library. Here’s the modification so it checks the bigger of the two Boston area interlibrary systems.
var libraryUrlPattern = 'http://library.minlib.net/search/i?SEARCH=' var libraryName = 'Minuteman Libraries';
Get the modified script here.
Interface Sistering
I drove by this yesterday and had to go back and take it’s picture. They replaced the telephone poll but avoided having to rework all the wires on the old poll. They chop out the section of the old pool holding the wires and suspend that from the new poll.
I assume moving the wires requires getting each of the N utility companies to come and move their wires. On this street all the polls were these ugly things. I particularly loved this one because you can’t actually move any of the wires. It’s on a curve and they aren’t long enough reach the new poll.
In software rework you rarely get to provide a clear visualization of what a mess things are, and how what sounds simple actually has cascade effects far beyond your wildest imagination. I used to work with a guy who would quietly say, at times like this: “You may not use the word ‘just’.”
I took this picture a short distance from where I took this one. I’m starting to think this neighborhood vast metaphor for software reengineering.