Sod provenance

Is the digital revolution that Pixar began with Toy Story stifling art – or saving it? An article for the Telegraph, 24 July 2021

In 2011 the Westfield shopping mall in Stratford, East London, acquired a new public artwork: a digital waterfall by the Shoreditch-based Jason Bruges Studio. The liquid-crystal facets of the 12 metre high sculpture form a subtle semi-random flickering display, as though water were pouring down its sides. Depending on the shopper’s mood, this either slakes their visual appetite, or leaves them gasping for a glimpse of real rocks, real water, real life.

Over its ten-year life, Bruges’s piece has gone from being a comment about natural processes (so soothing, so various, so predictable!) to being a comment about digital images, a nagging reminder that underneath the apparent smoothness of our media lurks the jagged line and the stair-stepped edge, the grid, the square: the pixel, in other words.

We suspect that the digital world is grainier than the real, coarser, more constricted, and stubbornly rectilinear. But this is a prejudice, and one that’s neatly punctured by a new book by electrical engineer and Pixar co-founder Alvy Ray Smith, “A Biography of the Pixel”. This eccentric work traces the intellectual genealogy of Toy Story (Pixar’s first feature-length computer animation in 1995) over bump-maps and around occlusions, along traced rays and through endless samples, computations and transformations, back to the mathematics of the eighteenth century.

Smith’s whig history is a little hard to take — as though, say, Joseph Fourier’s efforts in 1822 to visualise how heat passed through solids were merely a way-station on the way to Buzz Lightyear’s calamitous launch from the banister rail — but it’s a superb short-hand in which to explain the science.

We can use Fourier’s mathematics to record an image as a series of waves. (Visual patterns, patterns of light and shade and movement, “can be represented by the voltage patterns in a machine,” Smith explains.) And we can recreate these waves, and the image they represent, with perfect fidelity, so long as we have a record of the points at the crests and troughs of each wave.

The locations of these high- and low-points, recorded as numerical coordinates, are pixels. (The little dots you see if you stare far too closely at your computer screen are not pixels; strictly speaking, they’re “display elements”.)

Digital media do not cut up the world into little squares. (Only crappy screens do that). They don’t paint by numbers. On the contrary, they faithfully mimic patterns in the real world.

This leads Smith to his wonderfully upside-down-sounding catch-line: “Reality,” he says, ”is just a convenient measure of complexity.”

Once pixels are converted to images on a screen, they can be used to create any world, rooted in any geometry, and obeying any physics. And yet these possibilities remain largely unexplored. Almost every computer animation is shot through a fictitious “camera lens”, faithfully recording a Euclidean landscape. Why are digital animations so conservative?

I think this is the wrong question: its assumptions are faulty. I think the ability to ape reality at such high fidelity creates compelling and radical possibilities of its own.

I discussed some of these possibilities with Paul Franklin, co-founder of the SFX company DNEG, and who won Oscars for his work on Christopher Nolan’s sci-fi blockbusters Interstellar (2014) and Inception (2010). Franklin says the digital technologies appearing on film sets in the past decade — from lighter cameras and cooler lights to 3-D printed props and LED front-projection screens — are positively disrupting the way films are made. They are making film sets creative spaces once again, and giving the director and camera crew more opportunities for on-the-fly creative decision making. “We used a front-projection screen on the film Interstellar, so the actors could see what visual effects they were supposed to be responding to,” he remembers. “The actors loved being able to see the super-massive black hole they were supposed to be hurtling towards. Then we realised that we could capture an image of the rotating black hole’s disc reflecting in Matthew McConaughey’s helmet: now that’s not the sort of shot you plan.”

Now those projection screens are interactive. Franklin explains: “Say I’m looking down a big corridor. As I move the camera across the screen, instead of it flattening off and giving away the fact that it’s actually just a scenic backing, the corridor moves with the correct perspective, creating the illusion of a huge volume of space beyond the screen itself.“

Effects can be added to a shot in real-time, and in full view of cast and crew. More to the point, what the director sees through their viewfinder is what the audience gets. This encourages the sort of disciplined and creative filmmaking Melies and Chaplin would recognise, and spells an end to the deplorable industry habit of kicking important creative decisions into the long grass of post-production.

What’s taking shape here isn’t a “good enough for TV” reality. This is a “good enough to reveal truths” reality. (Gargantua, the spinning black hole at Interstellar’s climax, was calculated and rendered so meticulously, it ended up in a paper for the journal Classical and Quantum Gravity.) In some settings, digital facsimile is becoming, literally, a replacement reality.

In 2012 the EU High Representative Baroness Ashton gave a physical facsimile of the burial chamber of Tutankhamun to the people of Egypt. The digital studio responsible for its creation, Factum Foundation, has been working in the Valley of the Kings since 2001, creating ever-more faithful copies of places that were never meant to be visited. They also print paintings (by Velasquez, by Murillo, by Raphael…) that are indistinguishable from the originals.

From the perspective of this burgeoning replacement reality, much that is currently considered radical in the art world appears no more than a frantic shoring-up of old ideas and exhausted values. A couple of days ago Damien Hirst launched The Currency, a physical set of dot paintings the digitally tokenised images of which can be purchased, traded, and exchanged for the real paintings.

Eventually the purchaser has to choose whether to retain the token, or trade it in for the physical picture. They can’t own both. This, says Hirst, is supposed to challenge the concept of value through money and art. Every participant is confronted with their perception of value, and how it influences their decision.

But hang on: doesn’t money already do this? Isn’t this what money actually is?

It can be no accident that non-fungible tokens (NFTs), which make bits of the internet ownable, have emerged even as the same digital technologies are actually erasing the value of provenance in the real world. There is nothing sillier, or more dated looking, than the Neues Museum’s scan of its iconic bust of Nefertiti, released free to the public after a complex three-year legal battle. It comes complete with a copyright license in the bottom of the bust itself — a copyright claim to the scan of a 3,000-year-old sculpture created 3,000 miles away.

Digital technologies will not destroy art, but they will erode and ultimately extinguish the value of an artwork’s physical provenance. Once facsimiles become indistinguishable from originals, then originals will be considered mere “first editions”.

Of course literature has thrived for many centuries in such an environment; why should the same environment damage art? That would happen only if art had somehow already been reduced to a mere vehicle for financial speculation. As if!

 

Run for your life

Watching Gints Zilbalodis’s Away for New Scientist, 18 November 2020

A barren landscape at sun-up. From the cords of his deflated parachute, dangling from the twisted branch of a dead tree, a boy slowly wakes to his surroundings, just as a figure appears out of the dawn’s dreamy desert glare. Humanoid but not human, faceless yet somehow inexpressibly sad, the giant figure shambles towards the boy and bends and, though mouthless, tries somehow to swallow him.

The boy unclips himself from his harness, falls to the sandy ground, and begins to run. The strange, slow, gripping pursuit that follows will, in the space of an hour and ten minutes, tell the story of how the boy comes to understand the value of life and friendship.

That the monster is Death is clear from the start: not a ravenous ogre, but unstoppable and steady. It swallows, without fuss or pain, the lives of any creature it touches. Perhaps the figure pursuing the boy is not a physical threat at all, but more the dawning of a terrible idea — that none of us lives forever. (In one extraordinary dream sequence, we see the boy’s fellow air passengers plummet from the sky, each one rendered as a little melancholy incarnation of the same creature.)

Away is the sole creation of 26-year-old Latvian film-maker Gints Zilbalodis, and it’s his first feature-length animation. Zabalodis is Away’s director, writer, animator, editor, and even composed its deceptively simple synth score — a constant back-and-forth between dread and wonder.

There’s no shading in Zabalodis’s CGI-powered animation, no outlining, and next to no texture, and the physics is rudimentary. When bodies enter water, there’s no splash: instead, deep ripples shimmer across the screen. A geyser erupts, and water rises and falls against itself in a churn of massy, architectonic white blocks. What drives this strange retro, gamelike animation style?

Away feels nostalgic at first, perhaps harking back to the early days of videogames, when processing speeds were tiny, and a limited palette and simplified physics helped players explore game worlds in real time. Indeed the whole film is structured like a game, with distinct chapters and a plot arranged around simple physical and logical puzzles. The boy finds a haversack, a map, a water canteen, a key and a motorbike. He finds a companion — a young bird. His companion learns to fly, and departs, and returns. The boy runs out of water, and finds it. He meets turtles, birds, and cats. He wins a major victory over his terrifying pursuer, only to discover that the victory is temporary. By the end of the film, it’s the realistic movies that seem odd, the big budget animations, the meticulously composited Nolanesque behemoths. Even dialogue feels clumsy and lumpen, after 75 minutes of Away’s impeccable, wordless storytelling.

Away reminds us that when everything in the frame and on the soundtrack serves the story, then the elements themselves don’t have to be remarkable. They can be simple and straightforward: fields of a single colour, a single apposite sound-effect, the tilt of a simply drawn head.

As CGI technology penetrates the prosumer market, and super-tool packages like Maya become affordable, or at any rate accessible through institutions, then more artists and filmmakers are likely to take up the challenge laid down by Away, creating, all by themselves, their own feature-length productions.

Experiments of this sort — ones that change the logistics and economies of film production — are often ugly. The first films were virtually unfollowable. The first sound films were dull and stagey. CGI effects were so hammy at first, they kicked viewers out of the movie-going experience entirely. It took years for Pixar’s animations to acquire their trademark charm.

Away is different. In an industry that makes films whose animation credits feature casts of thousands, Zabalodis’s exquisite movie sets a very high bar indeed for a new kind of artisanal filmmaking.