Campbell’s law: “The more any quantitative social indicator is used for social decision-making,” he wrote, “the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
So true, and it also tends to drive out other things. … (ht: karim)
I don’t recall why but I installed brew’s verison of openssh using the instructions here on my OSX 10.8.5. 10.8.5 is Mountain Lion.
Ssh stopped working. ssh -v revealed it would hang at “debug1: SSH2_MSG_SERVICE_ACCEPT received” which after I while I traced down to a hint that ssh-agent might be wedged.
Sure enough launchd’s org.openbsd.ssh-agent was failing. Console was reporting: ” (org.openbsd.ssh-agent) Throttling respawn: Will start in 10 seconds”, a lot.
This failure arises because the org.openbsd.ssh-agent plist passes a -l switch to ssh-agent, and the one brew provided doesn’t have this undocumented switch. You can see the switch in apple’s variant, just look for l_switch.
There are assorted pages complaining about this problem, but no solution. My solution was to change the org.openbsd.ssh-agent plist to use the apple version of ssh-agent that resides in /usr/bin/ssh-agent.
I think I’m happy now.
Log, from Blamo: Civil War Reenactor
I am enjoying this extremely long blog post about how logs can form the hub for a distributed system, by Jay Kreps from Linked-in. It’s TLMR “too long, must read?” It reminds me of my post about listening to the system, but more so.
He has a wonderful example of batch v.s. continuous processing. A dialectic worthy of its own post at some point.
The US census provides a good example of batch data collection. The census periodically kicks off and does a brute force discovery and enumeration of US citizens by having people walking around door-to-door. This made a lot of sense in 1790 when the census was first begun. Data collection at the time was inherently batch oriented, it involved riding around on horseback and writing down records on paper, then transporting this batch of records to a central location where humans added up all the counts. These days, when you describe the census process one immediately wonders why we don’t keep a journal of births and deaths and produce population counts either continuously or with whatever granularity is needed.
Cute. My goto example has always been the difference between the annual cycle(s) that arises from agriculture and tax law revisions v.s. the newspaper’s daily cycle in service of the demand for fish wrapping.
But of course that’s not really continuous, it’s just batch with different cycle times. And yet I once encountered a continuous system that involved a pipeline across a desert. Each time the sun would emerge from behind the clouds the pipe would warm up and a vast slug of material would be ejected out the far end into a hastily build holding pit at the refinery. Maybe slug processing would be a good fall back term for the inevitable emergence of batches in continuous systems. Blame the clouds.
Clay Shriky tells a story in his book “Here comes everybody:…”
“Fans of the TV show Jericho were so upset when CBS canceled the show that they started mailing peanuts to CBS in protest using the NutsOnline delivery service. This effort cost the fans real money, so there was no mistaking their commitment, especially not when twenty tons of peanuts eventually arrived at CBS.”
As an illustration to one of the “solutions” to what an apparently insoluble problem in distributed systems known as a Sybil Attack. Sybil attacks are all about sock puppets. For example I knew somebody, let’s call them Alice, once who maintained a handful of email addresses (let’s call those Bob, Bill, and Betty) all subscribed to the same mailing list. When Alice ever got into a debate on the mailing list with another party one of her sock puppets would speak up to support her.
Socket puppet are the bane of lots of systems. Ebay reputation, Google linking algorithm, voting of all kinds – but particularly on-line voting. And the only solution that works dependably is to shift the problem to some other identity system – i.e. a central authority that hands out unique identities. And that solution isn’t hard to critique either.
Assorted schemes have been invented to temper the problem. The peanuts in the example above are one instance of schemes where you use cost (sometimes called proof of work) to help. Which of course only means that the rich get more votes.
If you interested here’s a survey of tricks people have come up with.
Years ago i needed a way to generate unique identifiers for components in a distributed system. This was years before people invented GUIDs. I was pleased with my little trick. In the manual I prescribed a little algorithm for generating the component’s ID: take a one dollar bill and use its serial number, then deface the bill slightly with a smile-face so nobody else will use that bill for this purpose. Cheaper than a peanut. As an aside I continue to be surprised how currency serial numbers don’t have check digits, etc.
I was reminded about that trick recently when I bought a candy bar. This candy bar was part of a contest Google was running. Each wrapper had number printed inside for use in entering a contest. I got to thinking that could be used for a unique identifier. What is nice about the dollar bill and these candy wrapper numbers it they are pretty anonymous.
But if you were going to try to mix those into your trick to temper a Sybil attack you could encode some more information into the candy wrapper numbers, such as information about where they were distributed. That might be used to invalidate batches of them when an attacker, who would obviously buy in bulk, was identified.
Apparently in the 2nd world war the US stamped all the currency in Hawaii so that if it fell to the Japanese they could declare currency with that stamp upon it to be invalid. I inherited from my father the habit of occasionally asking a retail clerk if they accept Hawaii money?
In the 1930s a traffic engineer in England noticed a curious pattern in the data about highway deaths. Here is the chart from the article he published.
The vertical axis shows deaths/car and the horizontal shows cars/person with one dot for each country. That’s for 1938. In 1938 few people in Spain(19) owned a car, but those that did were causing a lot of deaths. Switzerland(2) wasn’t fitting the model very well. You can make up your own insta-theory for why countries with few cars/person kill more people with each car.
Here’s a chart from 1980. More countries, more years, more confirmation of the model. The data are shown twice, the second time is a log-log graph.
Note that there are lots of things you might think would affect the numbers for a given country. For example: seat belts, population density, driver median age, safety regulations, insurance, policing, road quality, dash-board cams… But those aren’t part of this simple curve and so can only effect the residuals.
I stole these charts from J.G.U. Adams short article “Smeed’s Law: some further thoughts” in Traffic Engineering and Control, 1987
I find this all weird. You would think the traffic engineers would have a polished consensus by now of what this is saying. Adams’ article has some interesting things to say. For example societies learn to manage the cars as their numbers increase. But I don’t sense there is a consensus in the profession. Even now, 80+ years after the pattern was first noticed.
My mother, or so I am told, had a device for dealing with that frustrating syndrome where you wake up in the middle of the night and your damn brain decided to tour all the things that are making you anxious. The trick was to try and enumerate something, for example vegetables in alphabetical order.
So I’ve be trying this out but I needed something to enumerate. At first I was enumerating foods, and then I started adding constraints. … gizzards, haggis, intestines, … Then I started enumerating programing languages: APL, Basic, C, Datalog, … But it was became fun to try and include a language only if it has some aspect that makes me smile. For example the function calling weirdness in SL5. Elang is a layer cake of odd smilies, though then I can’t use E. R is a must have for how evaluates function arguments.
You might pick Caja[1,2] for C. And that lead me to recall Dart. I’ve been meaning to go back and see what’s up with Dart. This talk is a fascinating entry point into Dart, at least for me. Google managed to find a good guy, Gilad Bracha, for this design challenge!
For the me the tasty thing in this talk is that they are trying to stand in the middle ground between a dynamically typed language and something, well, something else.
This decades after what Common Lisp did about this problem. In Common Lisp you can declare that something is, for example, a small integer; but the programming environment can ignore that; it’s just advice. In fact even in a single language environment how that statement effects things can vary depending on other stuff like optimization settings. If you implementation ignores the declarations then they practically comments. I think it was an Algol manual that documented comments by saying “the compiler makes no effort to check the correctness of the comments.” This wiggle room makes some people’s skin crawl. And, it’s certainly enticed a lot of engineering effort on the part of Common Lisp implementors.
In fact a Common Lisp programmer can do wonderfully weird things with the type system and declarations. For example we can define a type that asserts our graphs are acyclic and provide a predicate that we only use in desperate situations.
(defun acyclic-graph-p-for-type (g) (if *desperate-debug* (acyclic-graph-p g) t)) (deftype acyclic-graph () (and graph (satisfies #'acyclic-graph-p-for-type))
The reason I found that Dart talk so interesting is the how Gilad deals with the skin crawling issue. He adopts a war weary Eeyore-ish manner. I can totally relate. He makes no real effort to argue why this is a useful such a powerful and useful approach. In fact I’d say he baits his audience into an absence of sympathy.
I’ve done some very fun things with type systems that are analogous to what he is calling optional typing, for example this diagnostic typing I described a while back.
I’ll have to dig some more to see if any of the Caja ideas survived into Dart. But they are similar, in the sense that there turn out to be many type system like blankets one might want to throw over your program to make you feel more cozy. And if you insist that on exactly when in the program life cycle, if ever, they are checked (or worse proven) it’s just not as much fun.
The talks also made me sad. It’s clear there is a lot of language design argot that I haven’t kept up with.
… junket, keratin, …
I let myself be baited into one of those soul sucking debates that occur on mailing lists.
So, here’s an interesting paper on distracted driving. The researchers ran a very carefully designed experiment using a driving simulator and then measure maybe a dozen different things as the drivers engaged in various tasks. They then summed up those measures into a metric they call workload. What I found surprising is how all forms of conversation are basically equivalently distracting – talking to a passenger, talking on a classic cellphone, and talking on via a handsfree device. Their scale runs from 1 to 5 where the task that anchors #1 is driving their course, and #5 is an obnoxious task where the driver has to listen to and solve word and match puzzles.
In the US the legal limit for drunk driving is 0.08, and apparently talking on a classic cell phone is equivalent. I’ve not found any research on how distracting thinking about internet trolls is.
The social sciences can be dangerous stuff. Get you public health policies wrong people die. Get your diplomacy wrong and all hell breaks loose. Follow the wrong economic policies and folks starve or worse.
One of the charts that most effected me over the years is this one that shows when various nations abandoned the gold standard during the depression and the lead up to the 2nd world war. I think you make a pretty straight forward argument that the 2nd world war might have been avoided if the sequence had been different. Hard money kills.
All currencies have an agenda. Sometimes their designers are too foolish to know what it is, but still. Gift cards, frequent flier miles, credit cards, check clearing networks all have an agenda. You can manage your nation’s currency to make workers insecure and increase the level of unemployment.
This essay by Charlie Stross about why Bitcoin is evil says many of the things I have been thinking. He calls it a weapon by design.
I’ve wondered if nation states engage in cold warfare by viciously engaging in PR campaigns designed to advance the bad economic policies in their rivals. I don’t see why not, all the other players in the democratic policy strive to guide social policies to their benefit.
This chart comes from the Financial Times by way of Roger Pielke. The horizontal axis runs from poor to rich. The vertical axis show how much their income changed in the last 20 years. For example the poorest 10-15% saw their incomes rise by 50% over the last 20 years. That’s about 2% a year.
So how much did the world economy grow (per person or household) during this 20 years. I’m not clever enough to answer that question quickly. This other chart shows world GDP (it has a very cool horizontal axis). And that suggests that growth was 4% during the first decade and effectively zero during the second. But I’d take that with a grain of salt.
Of course, given how skewed the income distribution is, it takes a lot more money to raise the top 10 percent by 2% v.s. the bottom 50%. This chart totally obscures that. So while it’s good that the 5%..70% saw their incomes rise I’m not confident that they got a reasonable share of the over all growth.
None the less, a thought-provoking chart.
I don’t think I ever mentioned here, in the blog, a little hack I enjoyed doing a while back. I bought an old android phone ($10 on eBay, boy was it junk!) and repurposed it to fill the role of an Airport Express, i.e. so I could route music from iTunes to the phone and then to a set of speakers.
Naturally that got me thinking of all the curious things you might do with an Android Phone. For example put it in the car as a nav system, use it as a webcam, or tracking/security device. If you poke around you can file lists of suggestions (for example).
Some of my more self amused fantasies involve using the camera to look at a sensor, say a thermometer for an input signal and then using the flash, speaker, or bluetooth as the output signal in some over engineered control system. Say to boil water, or heat a fish tank.
I see that you can now buy a new android phone for $30 at Radio Shack, or online from Best Buy.
