What about the unknown knowns?

Reading Nate Silver’s On the Edge and The Art of Uncertainty by David Spiegelhalter for the Spectator

The Italian actuary Bruno de Finetti, writing in 1931, was explicit: “Probability,” he wrote, “does not exist.”

Probability, it’s true, is simply the measure of an observer’s uncertainty, and in The Art of Uncertainty British statistician David Spiegelhalter explains how his extraordinary and much-derided science has evolved to the point where it is even able to say useful things about why things have turned out the way they have, based purely on present evidence. Spiegelhalter was a member of the Statistical Expert Group of the 2018 UK Infected Blood Inquiry, and you know his book’s a winner the moment he tells you that between 650 and 3,320 people nationwide died from tainted transfusions. By this late point, along with the pity and the horror, you have a pretty good sense of the labour and ingenuity that went into those peculiarly specific, peculiarly wide-spread numbers.

At the heart of Spiegelhalter’s maze, of course, squats Donald Rumsfeld, once pilloried for his convoluted syntax at a 2002 Department of Defense news briefing, and now immortalised for what came out of it: the best ever description of what it’s like to act under conditions of uncertainty. Rumsfeld’s “unknown unknowns” weren’t the last word, however; Slovenian philosopher Slavoj Žižek (it had to be Žižek) pointed out that there are also “unknown knowns” — “all the unconscious beliefs and prejudices that determine how we perceive reality.”

In statistics, something called Cromwell’s Rule cautions us never to bed absolute certainties (probabilities of 0 or 1) into our models. Still, “unknown knowns” fly easily under the radar, usually in the form of natural language. Spiegelhalter tells how, in 1961, John F Kennedy authorised the invasion of the Bay of Pigs, quite unaware of the minatory statistics underpinning the phrase “fair chance” in an intelligence briefing.

From this, you could draw a questionable moral: that the more we quantify the world, the better our decisions will be. Nate Silver — poker player, political pundit and author of 2012’s The Signal and the Noise — finds much to value in this idea. On the Edge, though, is more about the unforeseen consequences that follow.

There is a sprawling social ecosystem out there that Silver dubs “the River”, which includes “everyone from low-stakes poker pros just trying to grind out a living to crypto kings and adventure-capital billionaires.” On the Edge is, among many other things, a cracking piece of popular anthropology.

Riverians accept that it is very hard to be certain about anything; they abandon certainty for games of chance; and they end up treating everything as a market to be played.

Remember those chippy, cheeky chancers immortalised in films like 21 (2008: MIT’s Blackjack Team takes on Las Vegas) and Moneyball (2011: a young economist up-ends baseball)?

More than a decade has passed, and they’re not buccaneers any more. Today, says Silver, “the Riverian mindset is coming from inside the house.”

You don’t need to be a David Spiegelhalter to be a Riverian. All you need is the willingness to take bets on very long odds.

Professional gamblers learn when and how how to do this, and this is why that subset of gamblers called Silicon Valley venture capitalists are willing to back wilful contrarians like Elon Musk (on a good day) and (on a bad day) Ponzi-scheming crypto-crooks like Sam Bankman-Fried.

Success as a Riverian isn’t guaranteed. As Silver points out, “a lot of the people who play poker for a living would be better off — at least financially — doing something else.” Then again, those who make it in the VC game expect to double their money every four years. And those who find they’ve backed a Google or a SpaceX can find themselves living in a very odd world indeed.

Recently the billionaire set has been taking an interest and investing in “effective altruism”, a hyper-utilitarian dish cooked up by Oxford philosopher Will MacAskill. “EA” promises to multiply the effectiveness of acts of charity by studying their long-term effectiveness — a approach that naturally appeals to minds focused on quantification. Silver describes the state of the current movement, “stuck in the uncanny valley between being abstractly principled and ruthlessly pragmatic, with the sense that you can kind of make it up as you go along”. Who here didn’t see that one coming? Most of the original EA set now spend their time agonising over the apocalyptic potential of artificial intelligence.

The trick to Riverian thinking is to decouple things, in order to measure their value. Rather than say, “The Chick-fil-A CEO’s views on gay marriage have put me off my lunch,” you say, “the CEO’s views are off-putting, but this is a damn fine sandwich — I’ll invest.”

That such pragmatism might occasionally ding your reputation, we’ll take as read. But what happens when you do the opposite, glomming context after context onto every phenomenon in pursuit of some higher truth? Soon everything becomes morally equivalent to everything else and thinking becomes impossible.

Silver mentions a December 2023 congressional hearing in which the tone-deaf presidents of Harvard, Penn and MIT, in their sophomoric efforts to be right about all things all at once all the time, managed to argue their way into anti-Semitism. (It’s on YouTube if you haven’t seen it already. The only thing I can compare it to is how The Fast Show’s unlucky Alf used to totter invariably toward the street’s only open manhole.) No wonder that the left-leaning, non-Riverian establishment in politics and education is becoming, in Silver’s words, “a small island threatened by a rising tide of disapproval.”

We’d be foolish in the extreme to throw in our lot with the Riverians, though: people whose economic model reduces to: Bet long odds on the hobby-horses of contrarian asshats and never mind what gets broken in the process.

If we want a fairer, more equally apportioned world, these books should convince us that we should be spending less time worrying about what people are thinking, and concern ourselves more with how people are thinking.

We cannot afford to be ridden by unknown knowns.

 

How we went from mere betting to gaming the world

Reviewing The Perfect Bet: How science and maths are taking the luck out of gambling by Adam Kucharski, for The Spectator, 7 May 2016.

If I prang your car, we can swap insurance details. In the past, it would have been necessary for you to kill me. That’s the great thing about money: it makes liabilities payable, and blood feud unnecessary.

Spare a thought, then, for the economist Robin Hanson whose idea it was, in the years following the World Trade Center attacks, to create a market where traders could speculate on political atrocities. You could invest in the likelihood of a biochemical attack, for example, or a coup d’etat, or the assassination of an Arab leader. The more knowledgeable you were, the more profit you would earn — but you would also be showing your hand to the Pentagon.

The US Senate responded with horror to this putative “market in death and destruction”, though if the recent BBC drama The Night Manager has taught us anything at all (beyond the passing fashionability of tomato-red chinos), it is that there is already a global market in death and destruction, and it is not at all well-abstracted. Its currency is lives and livelihoods. Its currency is blood. A little more abstraction, in this grim sphere, would be welcome.

Most books about money stop here, arrested — whether they admit it or not, in the park’n’ride zone of Francis Fukuyama’s 1989 essay “The End of History?” Adam Kucharski — a mathematician who lectures at the London School of Hygiene and Tropical Medicine — keeps his foot on the gas. The point of his book is that abstraction makes speculation, not just possible, but essential. Gambling isn’t any kind of “underside” to the legitimate economy. It is the economy’s entire basis, and “the line between luck and skill — and between gambling and investing — is rarely as clear as we think.” (204)

When we don’t know everything, we have to speculate to progress. Speculation is by definition an insecure business, so we put a great deal of effort into knowing everything. The hope is that, the more cards we count, and the more attention we pay to the spin of the wheel, the more accurate our bets will become. This is the meat of Kucharski’s book, and occasions tremendous, spirited accounts of observational, mathematical, and computational derring-do among the blackjack and roulette tables of Las Vegas and Monte Carlo. On one level, The Perfect Bet is a serviceable book about professional gambling.

When we come to the chapter on sports betting, however, the thin line between gambling and investment vanishes entirely, and Kucharski carries us into some strange territory indeed.

Lay a bet on a tennis match: “if one bookmaker is offering odds of 2.1 on Nadal and another is offering 2.1 on Djokovic, betting $100 on each player will net you $210 — and cost you $100 — whatever the result. Whoever wins, you walk away with a profit of $10.” (108) You don’t need to know anything about tennis. You don’t even need to know the result of the match.

Ten dollars is not a great deal of money, so these kinds of bets have to be made in bulk and at great speed to produce a healthy return. Which is where the robots come in: trading algorithms that — contrary to popular myth — are made simple (rarely running to more than ten lines of code) to keep them speedy. This is no small problem when you’re trying to automate the business of gaming the entire world. In 2013 — around the time the US Senate stumbled across Robin Hanson’s “policy market” idea, the S&P 500 stock index took a brief $136 billion dive when trading algorithms responded instantly to a malicious tweet claiming bombs had gone off in the White House.

The subtitle of Kucharski’s book states that “science and maths are taking the luck out of gambling”, and there’s little here to undercut the gloomy forecast. But Kucharski is also prosecuting a cleverer, more entertaining, and ultimately more disturbing line of argument. He is placing gambling at the heart of the body politic.

Risk reduction is every serious gambler’s avocation. The gambler is not there to take part. The gambler isn’t there to win. The gambler is there to find an edge, spot the tell, game the table, solve the market. The more parts, and the more interactions, the harder this is to do, but while it is true that the world is not simply deterministic, at a human scale, frankly, it might as well be.

In this smartphone-enabled and metadata-enriched world, complete knowledge of human affairs is becoming more or less possible. And imagine it: if we ever do crack our own markets, then the scope for individual action shrinks to a green zero. And we are done.