“My mistake is perhaps instructive, because it exemplifies some cognitive biases that bedevil all of us: wishful thinking, overvaluing the causal impact of character traits (and individual agency, generally) relative to structural tendencies, and an unfortunate tendency to take sides” — Books Do Furnish a Room
Why does the culture so love to overvalue individual agency relative to structural tendencies?
This is incompetent public relations. It is childish blame shifting. Google slandered every single web site on the planet, and where do they place the responsibility?
… What happened? Very simply, human error. Google flags search results with the message “This site may harm your computer” if the site is known to install malicious software in the background or otherwise surreptitiously. We do this to protect our users against visiting sites that could harm their computers. We maintain a list of such sites through both manual and automated methods. We work with a non-profit called StopBadware.org to come up with criteria for maintaining this list, and to provide simple processes for webmasters to remove their site from the list. … we push these updates in a staggered and rolling fashion
On a non-profit supplier, a human, and a nameless process. That is bull. Google is a large complex mind bogglingly rich and powerful institution. That institution needs to take and act responsibility. Distracting the public by attempts to shift the blame is dishonorable and unprofessional. Now institutions don’t have honor. I hope the PR department and the Vice President who signed that posting can do better.
I have a rule of thumb about what to do in a crisis. Fight the fire, but do not every do the postmortem during the fire fighting. After it’s over it is best if you can get somebody uninvolved to do the diagnosis and block out the lessons learned. For example there is no way that the Vice President who signed the blog posting is likely to fully block out what’s so wrong with it. It is unlikely that the people on either side of the vendor relationship were the bit-o-data was handed off that triggered the fire are likely to avoid finger pointing; which is frankly a stupid waste of breadth. This single sentence: “Since we push these updates in a staggered and rolling fashion, the errors began appearing between 6:27 a.m. and 6:40 a.m. and began disappearing between 7:10 and 7:25 a.m., so the duration of the problem for any particular user was approximately 40 minutes.” suggests a huge amount of work is going to be required to preempt this kind of foul up in the future.”
None of that will happen if the firm embraces the dodge of blaming human error, low quality supplier, and blind process. Good PR is not excuse making. Google needs to wake up the fact that people are now afraid of them. Cute dog ate my homework dodges don’t cut it. People need to see both transparency and professionalism.
Wife and I are off to NYC. Taking the bus down tommorrow (Ok, I’m an idiot since there is a snow storm predicted), and then returning late in the day on Saturday. It would be fun to met up with people. If anybody would like to get together; please get in touch! There’s a Korean place on 36th near 5th that I love and hopefully we will have dinner on Friday night. We are also hoping to get to see Dan Hurlin’s Disfarmer, tommorrow night if the God’s cooperate. We have a foolish plan involving ferryboats!
I seem to be building logging infrastructure today. I keep recalling one or another of the rules for playing this game. Might as well try to put them down.
This is great, and my first reaction is yes, yes.
For many cultural and artistic movements there is a “counter-cultural moment” when a previously obscure and out-of-the-way tendency suddenly gains prominence. It may be the surrealists in the ’30s, Bauhaus in the ’20s, the hippy movement in the late ’60s, punk in the late ’70s – each had a few years in which it was both important and yet still opposed to the mainstream. But that counter-cultural moment is short, often just a handful of years, and once it is over, the movement either becomes the mainstream or fades away. Only nine years after the summer of ’67 the hippies were a thing of the past and punk set itself up in direct opposition to the pompous and irrelevant dinosaurs that “progressive rock” bands had become.
“Geek culture” has had its counter-cultural moment. It’s over.
It is now eight years today since Wikipedia started, four years since blogging, ten years of Google. These formerly “hungry upstarts” are now the establishment. Netflix partners with Wal-Mart, Google partners with CBS, Amazon is bigger than any of those “establishment” bookstores it challenged and pushes around publishers with impunity. Geek culture is now mainstream. Google, Amazon, Netflix have passed their counterculture sell-by date.
But yet again. There are some distinctions to be teased out. For example, some of what is “geek culture” is indistinquishable from the techno-scientific-industrial revolution; and that firestorm just keeps going and going. Some of what is “geek culture” is the rise of groups (scale free) condensing in the network; and that just keeps happening. And while the cultural bit is all true there it’s bigger than that – touching all the social sciences.
On balance, and I’ve felt for quite a while now, we went thru a real phase change when the Internet went mainstream. At the moment it transititioned it was a real oh-dear/oh-my-goodness moment. We are on the otherside of that. There is plenty of remaining turbulance yet to be played out. Plenty of solid bits waiting to be be liquidated. Some, like the newspaper industry or walmart, are plenty big. But the big-bang appears to be behind us.
I’ve been meaning to write this up for the last few months but I wanted to get the code into a shape I liked first. The way EC2 works you spin up fresh virtual machines by doing: ec2-run-instance AMI-123 … and so the natural thing to do is if you need machines configured for different roles you take the time to stamp out a different AMI (aka Amazon Machine Image) for each role. That’s tedious. Alternately you can gin up an image which learns what role it should fill as it starts up. Some folks already build their AMI to let you do that. For example the kind folks at Alestic provide assorted AMI that include this feature: “On first boot, runs instance user-data script if it starts with #!”.
So here’s what I do. First I arrange to have the custom personality packed up in what I call a start-up-bundle. I put that someplace. In the example below it’s at: <http://example.s3.amazonaws.com/starters/start-up-example>. I prefer to have the file publicly available, so that fetching it trivial. But since it’s content is private I encrypt the file. In the example below the password for decrypting the file is “stirringdullroots”.
To start an machine up and have it take on this personality I then do: “ec2-run-instance ami-115db978 -f start-up-example-userdata …”
This is what you’d find in the file start-up-example-userdata, i.e. the script which grabs the starter bundle, decrypts it, and then recurs into the bundle to let the personality take shape.
#!/bin/bash
trap 'echo ERROR Bootstrapping failed; exit 1' ERR
echo "Bootstrapping from <http://example.s3.amazonaws.com/starters/start-up-example>"
set -o pipefail
cd /tmp
curl -s "http://example.s3.amazonaws.com/starters/start-up-example" | openssl enc -d -aes-256-cbc -k stirringdullroots | tar x
echo "Unpacked: `find start-up-bundle | wc -l` files"
cd start-up-bundle
echo "Run start-up scripts for various modules"
for i in */start-up.sh ; do
MODULE=`dirname $i`
echo "Starting module $MODULE: via $i"
(cd $MODULE; bash start-up.sh)
echo "Finished starting module $MODULE"
done
So each startup bundle has two parts the userdata script, and the encrypted lump-o-personality.
Of course there are scripts for building start-up-bundles and other orchustration of the scheme, but that’s the heart of the story.
As you can see each machine’s personality consists of a set of modules. I have a collection of standard modules for things like java, perl, erlang, tinydns, rabbitmq, sbcl, etc. etc. One nice feature of this design is that it should be possible to make and share these with other folks.
Economists will appreciate that anthropologists have noted that scarcity is one the reasons why old ladies sometimes transition from Nana, to crone, to shunned, to witch, and are then murdered. Food gets scarce and somebody’s got to go. I see that Google let go “about a 100” recruiters.
bAnd, I see that Google is setting some products aside. I’ll miss one of these: catalog search. It was fun, take a peek before it’s gone.
Those that live on the bottom lands, out on the long tail, suffer the most when the storms come or times get tough. Easier on the conscious if you label them as unworthy. Of course declaring them witches is a bit over the top. There are other moves. For example you can set them free. Sort of like how polygamist societies shun their young men due to a shortage of wives you can always donate your weaker products to the open source community (don’t forget the patents)
Some of those kids will thrive. Sometimes they get adopted by a stronger family. I wonder if the mashup editor team is getting folded into the Google App Engine team. But then I also wonder about the health of Google App Engine. I recently paged thru their example application catalog and it’s a lot thinner than I would have expected. There was almost nothing in there that I thought: “Yeah! I should blog this.” That is not good.
Kremlin watching.
Idiots. Comcast upgraded their email system over night and now I have thousands of duplicate messages in dozens of mail folders. And, so does everybody in my household. Guess who get’s to clean up this mess?
The email announcing this wonder (Smartzone(tm) Communications Center) includes the phrase: “You Asked for It!” It would have been more polite of course if they had given us all fair warning, before throwing this punch. You know as in: “Ok buddy, you asked for it.” This is has an abusive relationship. Their product managers must be really really happy about their switching costs.
Don’t want to lose this:
You can disable ldb … at runtime …:
(funcall (sb-alien:define-alien-routine
(“disable_lossage_handler” cl-user::disable-sbcl-ldb)
sb-alien:void))
About once a year I dabble with trying to setup a video streaming server so I can watch movies on my Mac which doesn’t have a DVD reader on it. Usually this all falls apart because the debug loop is the scale of a DVD, the free tools are all full of personality, and the world of video encoding is quite confusing. This time I think I got it. So here some tricks I picked up along the way.
Darwin Streaming Server isn’t hard to set up. I have it serving up files who’s extension is m4v, which I assume stands for mp4 video.
I use HandBrake on the Mac to rip the DVDs. In fact I use HandBrakeCLI to rip them. A command like: HandBrakeCLI --preset Universal --input /dev/disk3 --output foo.m4v rip and convert DVD into a single file that is 1 to 2.5 Gig in size.
While that foo.m4v file can be served by Darwin Streaming Server your better off if you add so called hints to the file. These are apparently nonstandard mark up that Apple invented to make it possible to seek to arbitrary points in your movie. While there are signs that “QT Sync” can do hinting I ended up using mp4creator to do it. In addition to hinting you can optimize, so while I have no idea what that does, I do that as well. Hinting is a pain because you need to do one pass to hint the video track and another pass to hint the audio track. So you end up doing something like this: mp4creator --hint=1 "foo.m4v" ; mp4creator --hint=2 "foo.m4v" ; mp4creator -optimize "foo.m4v"
In that example the numbers 1 and 2 are denote the track number to hint. Those are reasonably dependable; but to be save you can use mp4creator -list foo.mp4v to be sure. I wrote a bit of perl to glean out the right numbers and gin up the commands to do the hinting.
Usually when you stick a DVD into a mac the operating system launches the DVD player, but you can change that behavior in the CDs and DVDs Preference Panel. Using that I launch an apple script. Since I’m more fluent in shell scripting I have that turn around an launch a shell script. That script devolved from other examples I found around the net. The essoteric bit-o-clever in it are useful; so it appears below. It has an unfortunate flaw that the heuristic for guessing which disks are DVDs sometimes gets the wrong answer; but it’s good enough for now.
My shell script then: 1) runs HandBrakeCLI, 2) ejects the DVD, 3) hints, optimizes, and 4) moves the result into appropriate directory for the Darwin Streaming Server. A little cgi script then enumerates what’s available to machines on my household’s local network.
global dvdPath
try
tell application “System Events” to set DVDs to name of disks whose capacity is less than 8.589934592E+9
set dvdPath to quoted form of (POSIX path of item 1 of DVDs)
do shell script “/Users/bhyde/bin/snarf_dvd.sh “ & dvdPath
end try