I’m a bit of crank about the right answer to the question: “What has improved the quality of life the most for humans?”  Since pretty clearly the answer is Public Health.

Here’s a nice example of one of the many stories of that kind:  A Striking Change in Lightning Deaths.

I wonder if there is an estimate of how many lives the slogan: “”when thunder roars, go indoors” saved?

Listeners on Private Ports – race condition edition

Ha!  I have here a service X that is listens for it’s clients on private port N.  It failed to start because another activity Z was using N.  Looking at the state of things it’s clear that Z got N randomly assigned when it established a connection to a service on another machine.

This is the kind of bug you discover only if you get a short debug loop around what is typically a rare activity – rebooting the server in this case.  Gosh the chances of this happening are small.

I have been parking my private listeners in the private port range (49152 to 65535) for almost 40 years.  Maybe, I need to stop doing that.  Or at least assure that all listeners get started before any other activities start using the net – yeah right.

But now I’m confused.  I see that there is a concept of “ephemeral ports,” but the Wikipedia article leaves one with the impression that in practice you don’t know what range they are being drawn from.

So now I’m a bit confused what best practice might be.  Advocates of resource discovery score a few more points?

Frequency-hopping a server’s port

Here’s one of those ideas you have when you are not sleeping: why don’t we use frequency hopping to make it hard for attackers to find listeners to attack?

In scenarios where you want to keep the port number a secret,  you could randomly vary it’s location.  You could use TOPT, so both sides can rendezvous.  Seems this wouldn’t be that hard to add to ssh.  The sshd_config file might look something like this:

# Enable dynamic port listening, and the TOPT secret
Port dynamic 6000 16000
PortSecret 12345678901234567890

And the user’s ~/.ssh/config file would then have something like this in it

   Port dynamic
   PortSecret 12345678901234567890

You could let the PortSecret default to something derived from host key.

A Good Day

A few misc. items…

Happiness & Economics … What a wonderfully weird chart this is.  It’s weird in two ways.  First off what the heck is going on in the US?  Secondly this is basically the inverse of the chart of happiness v.s. income.

Programming languages – There is a very nice dialect of Lisp build on top of the Python ecology.   Sort of analogous to the way clozure is built on top of the Java ecology.  It’s called hy. Very smooth interoperability with Python, across many Python implementations.  For example you can casually load libraries written in hy into python code and visa versa.  Macros, backquote, real lambdas, everything is value returning, etc.  Surprisingly it even works pretty well with the Python debugger, such as it is.

Pricing games – As an collector of amusing pricing games this article that attempts to puzzle out the details of MTA ticket pricing is fun.

Programming – I wish I could find a standard tool that would let me make a compressed archive and then insert a descriptive header of unpredictable size at the front of it.  Something suitable for when you are building an archive by streaming and after the fact you want to prepend the cataloging metadata.  I guess I’m just a bit surprised that this use case isn’t so common that we don’t have a widely used tool that supports it.

Current events – NYPD?  What a bunch of babies!

Tourist Info:  The Brooklyn Art Museum is amazing.

Market Signals

Getting the annual before Thanksgiving hair cut I amused myself by trying to convince the Barber that he should introduce surge pricing.  I mean!  How are Uber drivers to know they should lay down their keys and take up their clippers.  Who is he to deny the market it’s signal? Does not the market have rights?

Optical character recognition for PDF files.

Pypdfocr is very nice.   The input is a PDF file, for example the output of scanner.  The output is another PDF, which looks like the original but now has the words recognized in it.  That lets you can search it, and if you index all your documents then that’s very useful.  Spotlight on my Mac sees into these.

You can extract the raw text using pdftotxt, which is nice for reading on the train.

I was delighted the it understands columns pretty well.  It is not so good at paragraph breaks though.

I gather that a some of people use this to scan all the paper, receipts, et. al. that comes into their home.  It has some clever switches to help with that usecase.

It is a bit of a pain to install, lots of homebrew packages and pip packages are required; and then – at least in my case – it works but it complains that I didn’t get it right.  There are pages that talk about these things; but I’m happy enough now.

Claw Back

Recently I’ve added Ponzi Tracker to my RSS reader and it’s fun in that way that we all enjoy things that feed our confirmation bias.  And who doesn’t love a story about a criminal.  Today’s post included this bit:

“… the court-appointed Receiver, Kenneth D. Bell, begins his quest to recover “false profits” from thousands of victims that were fortunate enough to profit from their investment.  The receiver’s efforts to recover these “false profits” will become markedly easier in the event that Burks pleads guilty to the fraud, since the guilty plea or conviction of a Ponzi schemer allow the use of the “Ponzi presumption” that significantly simplifies the burden of proof required in the so-called “clawback” actions.”

I didn’t know that.  It seems like a big gaping hole in the investor protections that encourage corporate risk taking.  The reason we have limits on investor liability is that it lets the investors delegate risk taking to the corporation while avoiding the worst case scenarios that they will be held responsible for the evil that firm does.  Their risk is limited to the amount of their investment.  Back in the day only the king had the power to get away with murder, but then it devolved to his friends.

So I’d love to know why Ponzi schemes are unique in this regard.  And I’d love to know that if we convicted a few large financial firms of just the right crimes we could then claw back the money from the “lucky” ones who cashed out early.

Any amateur social scientist knows the next question: What about incentives?  If you threaten investor class it creates an incentive.  Presumably the kings friends let this loophole appear because the victims of Ponzi schemes are somehow unique when compare to the other victims of corporate malfeasance.  Maybe it’s about affinity.  Which is ironic, as affinity is a common feature of Ponzi schemes, but in this case I think it might be that the Ponzi victims are called “investors.”

If only the victims of the mortgage crisis had called themselves investors.  If only we could learn use that phrase “false profits” more.

Bash Quoting

For years I’ve been frustrated by my inablity to puzzle out how to write this in bash:

H=$(dirname $0)

so it’s safe if $0 has spaces in it.


H="$(dirname $0)"

is better, but still the $0 isn’t usefully quoted.

I finally complained to the shellcheck author that this should be in the FAQ, though there isn’t a FAQ. And he assured me if you do it right it isn’t a problem.  He also happened to mention the answer:

H="$(dirname "$0")"

I complained that just because it wasn’t a question if you’d ask if you knew the answer didn’t mean that it wasn’t a commonly asked question.  So he added (or updated) a wiki page to say that the $(…) creates a new “context”.

So there you go.