Today’s peeve: people dealing poorly with randomness. First, an article:

• Oh No, Not Steely Dan Again (linked from The Geomblog)

Here’s the money shot:

We perceive trends when there are none. Poker players invariably believe they can lock into streaks. Backgammon champions swear that dice can go hot or cold. Likewise, people think they can cosmically predict what song will come next on their shuffle. The blogger Kapgar, who claimed this power, remembers vividly the times when he predicted a song and the iPod amazingly delivered it. But there may have been a thousand times when his iPod played songs he didn’t guess – non-memorable circumstances that, not surprisingly, didn’t make an impression. Likewise, is it really so astounding that Kapgar’s iPod played a run of what he considers great songs? Dude – he’s loaded the iPod with his favourite tunes. You know what would be really freaky? If his iPod came up with a run of tunes that he despised.

The idea that a random source can “remember” what it’s just produced and somehow correct itself to “balance things out” is called the gambler’s fallacy, and like all fallacies it’s wrong. A “random” source that picks new output based on past output is, er, no longer random.

(Amusingly, this is how pseudorandom number generators — including the one in Steven Levy’s iPod — work. Extracting a “random” number from a PRNG tweaks the PRNG’s internal state a little bit, influencing the next number in an entirely deterministic way. A PRNG’s output depends entirely upon its structure and upon its initial state — its random seed. Picking a sufficiently random seed is critical to getting sufficiently random numbers out of your PRNG. But that kind of determinism is far more subtle than the “patterns” people see in their iPods. The whole process is explained in detail in Bruce Schneier’s excellent Applied Cryptography.)

People expect “random” sequences to have a certain amount of variation. We’ve evolved to look for patterns where none may exist (it’s less dangerous to think you see a tiger where there isn’t one than to think you don’t see a tiger where there is), and the same intuition applies to sequences. For instance, if I flip a coin eight times, and get the sequence

• H H H H H H H H

you’re going to be pretty suspicious of my coin. If, however, I get

• H T H H T T T H

you’re not likely to have a problem. (The sequence of three tails might make you a bit nervous, but that’s all.) The fun part is that I’m just as likely to get the first sequence as the second. If I conduct Sagans of eight-flip tests, I should get as many eight-heads sequences as HTHHTTTH sequences. (Well, assuming a fair coin that never lands on its edge, but that’s why this is a thought experiment.)  See, the probability of any given fixed eight-flip sequence coming up is the same, regardless of whether it “looks” random or not.
Generalizing horribly, we perceive patterns where we see clusters. For instance, there’s a “spike” in the eight-heads sequence above, and another much smaller “spike” in the three-tails section of the second sequence. If you place points completely at random on a plane, it won’t “look” random — you’ll have tight clusters of dots in some places, and wide gaps in others, as if the random point-producer favours some regions and shuns others.

A more “random-looking” point placement routine works much the same way — only, when it places a point, it makes sure there are no other points within a small radius. This is a Poisson disc distribution:

(image from Dunbar and Humphreys’ paper). It has a pleasing look to it — uniform, but not too uniform. No big gaps, no big clusters — this is how we like to think of randomness. (It’s also very closely related to the distribution of photoreceptors in the human eye. Coincidence? I doubt it.)

This is basically the approach Apple uses in their “less-random more-random-feeling shuffle” approach, as far as I can tell from the article on the Guardian.