What about the unknown knowns?

Reading Nate Silver’s On the Edge and The Art of Uncertainty by David Spiegelhalter for the Spectator

The Italian actuary Bruno de Finetti, writing in 1931, was explicit: “Probability,” he wrote, “does not exist.”

Probability, it’s true, is simply the measure of an observer’s uncertainty, and in The Art of Uncertainty British statistician David Spiegelhalter explains how his extraordinary and much-derided science has evolved to the point where it is even able to say useful things about why things have turned out the way they have, based purely on present evidence. Spiegelhalter was a member of the Statistical Expert Group of the 2018 UK Infected Blood Inquiry, and you know his book’s a winner the moment he tells you that between 650 and 3,320 people nationwide died from tainted transfusions. By this late point, along with the pity and the horror, you have a pretty good sense of the labour and ingenuity that went into those peculiarly specific, peculiarly wide-spread numbers.

At the heart of Spiegelhalter’s maze, of course, squats Donald Rumsfeld, once pilloried for his convoluted syntax at a 2002 Department of Defense news briefing, and now immortalised for what came out of it: the best ever description of what it’s like to act under conditions of uncertainty. Rumsfeld’s “unknown unknowns” weren’t the last word, however; Slovenian philosopher Slavoj Žižek (it had to be Žižek) pointed out that there are also “unknown knowns” — “all the unconscious beliefs and prejudices that determine how we perceive reality.”

In statistics, something called Cromwell’s Rule cautions us never to bed absolute certainties (probabilities of 0 or 1) into our models. Still, “unknown knowns” fly easily under the radar, usually in the form of natural language. Spiegelhalter tells how, in 1961, John F Kennedy authorised the invasion of the Bay of Pigs, quite unaware of the minatory statistics underpinning the phrase “fair chance” in an intelligence briefing.

From this, you could draw a questionable moral: that the more we quantify the world, the better our decisions will be. Nate Silver — poker player, political pundit and author of 2012’s The Signal and the Noise — finds much to value in this idea. On the Edge, though, is more about the unforeseen consequences that follow.

There is a sprawling social ecosystem out there that Silver dubs “the River”, which includes “everyone from low-stakes poker pros just trying to grind out a living to crypto kings and adventure-capital billionaires.” On the Edge is, among many other things, a cracking piece of popular anthropology.

Riverians accept that it is very hard to be certain about anything; they abandon certainty for games of chance; and they end up treating everything as a market to be played.

Remember those chippy, cheeky chancers immortalised in films like 21 (2008: MIT’s Blackjack Team takes on Las Vegas) and Moneyball (2011: a young economist up-ends baseball)?

More than a decade has passed, and they’re not buccaneers any more. Today, says Silver, “the Riverian mindset is coming from inside the house.”

You don’t need to be a David Spiegelhalter to be a Riverian. All you need is the willingness to take bets on very long odds.

Professional gamblers learn when and how how to do this, and this is why that subset of gamblers called Silicon Valley venture capitalists are willing to back wilful contrarians like Elon Musk (on a good day) and (on a bad day) Ponzi-scheming crypto-crooks like Sam Bankman-Fried.

Success as a Riverian isn’t guaranteed. As Silver points out, “a lot of the people who play poker for a living would be better off — at least financially — doing something else.” Then again, those who make it in the VC game expect to double their money every four years. And those who find they’ve backed a Google or a SpaceX can find themselves living in a very odd world indeed.

Recently the billionaire set has been taking an interest and investing in “effective altruism”, a hyper-utilitarian dish cooked up by Oxford philosopher Will MacAskill. “EA” promises to multiply the effectiveness of acts of charity by studying their long-term effectiveness — a approach that naturally appeals to minds focused on quantification. Silver describes the state of the current movement, “stuck in the uncanny valley between being abstractly principled and ruthlessly pragmatic, with the sense that you can kind of make it up as you go along”. Who here didn’t see that one coming? Most of the original EA set now spend their time agonising over the apocalyptic potential of artificial intelligence.

The trick to Riverian thinking is to decouple things, in order to measure their value. Rather than say, “The Chick-fil-A CEO’s views on gay marriage have put me off my lunch,” you say, “the CEO’s views are off-putting, but this is a damn fine sandwich — I’ll invest.”

That such pragmatism might occasionally ding your reputation, we’ll take as read. But what happens when you do the opposite, glomming context after context onto every phenomenon in pursuit of some higher truth? Soon everything becomes morally equivalent to everything else and thinking becomes impossible.

Silver mentions a December 2023 congressional hearing in which the tone-deaf presidents of Harvard, Penn and MIT, in their sophomoric efforts to be right about all things all at once all the time, managed to argue their way into anti-Semitism. (It’s on YouTube if you haven’t seen it already. The only thing I can compare it to is how The Fast Show’s unlucky Alf used to totter invariably toward the street’s only open manhole.) No wonder that the left-leaning, non-Riverian establishment in politics and education is becoming, in Silver’s words, “a small island threatened by a rising tide of disapproval.”

We’d be foolish in the extreme to throw in our lot with the Riverians, though: people whose economic model reduces to: Bet long odds on the hobby-horses of contrarian asshats and never mind what gets broken in the process.

If we want a fairer, more equally apportioned world, these books should convince us that we should be spending less time worrying about what people are thinking, and concern ourselves more with how people are thinking.

We cannot afford to be ridden by unknown knowns.

 

“One cannot fail to overcome the brutality of blind chance”

Reading David Flusfeder’s Luck for New Scientist, 6 April 2022 

The Russian novelist Fyodor Doystoyevsky was a gambling addict. He believed, if he could only maintain his composure, the various strategies and “systems” he dreamt up to beat the roulette wheel would one day pay off. But no strategy can game pure chance.

David Flusfeder’s book is not about randomness, or statistics, or the behaviour of numbers. It is, quite specifically, about luck, defined as “the operations of chance taken personally.” Flusfeder is a semi-professional poker player, and knows whereof he speaks.

His eccentric, insightful meditations focus on Fortune’s favourites and its gulls — from the Marquis de Dangeau, 18th-century Versailles’ wiliest card shark, to Thomas Bastard, Oxford University’s most comprehensively shunned epigrammatist. Weaving backwards and forwards through western history, Flusfeder encounters more familiar figures too, from Ludwig Wittgenstein to Niels Bohr to four-times lottery winner Joan Ginther, “the luckiest woman on Earth”.

In an effort to stop his project sprawling, Flusfeder has given its structure over to chance. The 14 essays here are largely self-contained, and they have to be, since they’re presented in an order determined by an on-line randomiser.

Doystoyevsky’s experience predominates. Even after he managed to cure himself of his addiction, the novelist retained the conviction “that in games of chance, if one has perfect control of one’s will, so that the subtlety of one’s intelligence and one’s power of calculation are preserved, one cannot fail to overcome the brutality of blind chance and to win.”

Flusfeder reckons poor Fyodor was born in the wrong place; he should have been playing poker with French settlers in New Orleans, for the card game they invented in 1829 really does reward composure, and nerve, as well as luck.

Not that poker is an altogether rational pursuit. If it were, then Flusfeder would not be wearing green underpants to every important game. Superstition abounds on the poker circuit, as it does wherever people wield little or no control over their lives. Professional tennis, where “there is so much time to think, and doubt, and lose the learned rhythms of technique, and to be afraid”, is awash with fetishes, tics, and absurd pre-match “routines”.

Superstition, according to Flusfeder, is not some primitive psychic excrescence that can be discarded. It’s merely the florid expression of heuristic thinking, without which we wouldn’t be able to function at all. We don’t constantly re-evaluate the world. We make reasonable assumptions about how it works, and we rely on those assumptions. We develop habits. We conjure up a deterministic world in which what happened yesterday reliably guides our actions tomorrow.

All Nature is but Art, unknown to thee;
All Chance, Direction, which thou canst not see;
All Discord, Harmony not understood.

This attitude, caught so superbly by the poet Alexander Pope, works well enough for day-to-day life, but trouble attends our efforts to institutionalise such thinking. Swathes of economic practice, and not a little theory, are predicated on this nonsense — the idea that past results are a guide to future performance.

Statistician David Spiegelhalter, who studies the public perception of risk, is blunt about this: probability does not exist outside the mind, he says: “It is not an objective aspect of the world. It’s a way to operationalise a belief.” At best it’s the map for a territory that, being unbounded, is immeasurable and unknowable.

Vulnerable to the vagaries of chance, how should we conduct ourselves?

Rather than cower timidly, avoiding all randomness, we might develop prudence, seizing opportunities while sidestepping unnecessary risks. Still, against the vicissitudes of fortune, prudence is a thin shield indeed.

Virtue may be our better armour. A life lived with honesty and integrity will at least feel consistent, whether we are suffering adversity or enjoying good fortune.

Or as the early renaissance poet Petrarch put it, “Many times whom fortune has made bond, virtue has made free.”