Tag Archives: animation

The strange, the off-kilter and the not-quite-right

Posted on by

The release of Mufasa, Disney’s photorealistic prequel to The Lion King, occasioned this essay for the Telegraph on the biota of Uncanny Valley

In 1994 Disney brought Shakespeare’s Hamlet, or something like it, to the big screen, In turning the gloomy Dane into an adorable line cub, and his usurping uncle into Scar (arguably their most terrifying villain ever) the company created the highest-grossing movie of the year. Animators sat up and marveled at the way the film combined hand-drawn characters with a digitally rendered environment and thousands of CGI animals. This new technology could aid free expression, after all!

Well, be careful what you wish for.

When in 2019, Disney remade its beloved The Lion King (1994), it swapped the original’s lush hand-drawn animation for naturalistic computer-generated imagery. The 2019 reboot had a budget of $260 million (£200 million) and took more than $1.5 billion (£1.1 billion) at the box office, making it one of the most expensive, and highest-grossing, films of all time – and the focus of a small but significant artistic backlash. Some critics voiced discomfort with the fact that it looked more like an episode of Planet Earth than a high-key musical fantasy. Its prequel Mufasa: The Lion King (directed by Moonlight’s Barry Jenkins), released this month, deepens the trend. For Disney, it’s a show of power, I suppose: “Look at our animation, so powerful, you’ll mistake it for the world itself!” In time, though, the paying public may well regret Disney’s loss of faith in traditional animation.

What animator would want to merely reflect the world through an imaginary camera? The point of the artform, surely, is to give emotion a visual form. But while a character drawn in two dimensions can express pretty much anything (Felix the Cat, Wile E Coyote and Popeye the Sailor are not so much bodies as containers for gestures) drawing expressively in 3D is genuinely hard to do. Any artist with Pixar on their resume will tell you that. All that volumetric precision gets in the way. Adding photorealism to the mix makes the job plain impossible.

Disney’s live-action remake of The Jungle Book (2016) at least used elements of motion capture to match the animals’ faces to the spoken dialogue. In 2024, even that’s not considered “realistic” enough. Mufasa, Simba, Rafiki the mandrill and the rest simply chew on air while dialogue arrives from out of space, in the manner of Italian neorealist cinema (which suggests, incidentally, that, along with the circle of life, there’s also a circle of cinema).
Once you get to this point, animation is a distant memory; you’ve become a puppeteer. And you confront a problem that plagues not only Hollywood films, but the latest advances in robotic engineering and AI: “the uncanny valley”.

The uncanny valley describes how the closer things come to resembling real life, the more on guard we are against being fooled or taken in by them. The more difficult they are to spot as artificial, the stronger our self-preserving hostility towards them. It is the point in the development of humanoid robots when their almost-credible faces might send us screaming and running out of the workshop. Or, on a more relatable level, it describes the uneasiness some of us feel when interacting with virtual assistants such as Apple’s Siri and Amazon’s Alexa.

The term was invented by the Japanese roboticist Masahiro Mori in 1970 – when real anthropomorphic robots didn’t even exist – who warned designers that the more their inventions came to resemble real life-forms, the creepier they would look.

Neurologists seized on Mori’s idea because it suggested an easy and engaging way of studying how our brains see faces and recognise people. Positron emission tomography arrived in clinics in the 1970s, and magnetic resonance imaging about twenty years later. Researchers now had a way of studying the living human brain as it saw, heard, smelled and thought. The uncanny valley concept got caught up in a flurry of very earnest, very technical work about human perception, to the point where it was held up as a profound, scientifically-arrived-at insight into the human condition.

Mori was more guarded about all the fuss. Asked to comment on some studies using slightly “off” faces and PET scans, he remarked: “I think that the brain waves act that way because we feel eerie. It still doesn’t explain why we feel eerie to begin with.” And these days the scientific community is divided on how far to push the uncanny valley concept – or even whether such a “valley” (which implies a happy land beyond it, one in which we would feel perfectly at ease with lifelike technology) exists at all.

Nevertheless, the uncanny valley does suggest a problem with the idea that in order to make something lifelike, you just need to ensure that it looks like a particular kind of living thing – a flaw that is often cited in critical reviews of Disney’s latest photorealist animations. Don’t they realise that the mind and the eye are much more attuned to behaviour than they are to physical form? Appearances are the least realistic parts of us. It’s by our behaviour that you will recognise us. So long as you animate their behaviour, whatever you draw will come alive. In 1944 psychologists Fritz Heider and Marianne Simmel made a charming 90-second animation, full of romance, and adventure, using two triangles, a circle and a rectangle with a door in it.

There are other ways to give objects the gift of life. A few years ago, I met the Tokyo designer Yamanaka Shunji, who creates one-piece walking machines from 3D vinyl-powder printers. One, called Apostroph (a collaboration with Manfred Hild in Paris), is a hinged body made up of several curving frames. Leave it alone, and it will respond to gravity, and try to stand. Sometimes it expands into a broad, bridge-like arch; at other times it slides one part of itself through another, curls up and rolls away.

Engineers, by associating life with surface appearances, are forever developing robots that are horrible. “They’re making zombies!” Shunji complained. Artists on the other hand know how to sketch. They know how to reduce, and abstract. “From ancient times, art has been about the right line, the right gesture. Abstraction gets at reality, not by mimicking it, but by purifying it. By spotting and exploring what’s essential.”

This, I think, gets to the heart of the uncanny valley phenomenon: we tend to associate life with particular outward forms, and when we reproduce those things, we’re invariably disappointed and unnerved, wondering what sucked the life out of them. We’re looking for life in all the wrong places. Yamanaka Shunji’s Apostroph is alive in a way Mufasa will never be.

***

We’re constantly trying to differentiate between living and the non-living. And as AI and other technologies blur the lines between living things and artefacts, we will grapple with the challenge of working out what our moral obligations are towards entities — chatbots, robots, and the like — that lack a clear social status. In that context, the “uncanny valley” can be a genuinely useful metaphor.

The thing to keep in mind is that the uncanny is not a new problem. It’s an evolutionary problem.

Decades ago I came across a letter to New Scientist magazine in which a reader recalled taking a party of blind schoolchildren to London Zoo. He wanted the children to feel and cuddle the baby chimps, learning about their hair, hands, toes and so on, by touch. The experiment, however, proved to be a disaster. “As soon as the tiny chimps saw the blind children they stared at their eyes… and immediately went into typical chimpanzee attack postures, their hair standing upright all over their bodies, their huge mobile lips pouting and grimacing, while they jumped up and down on all fours uttering screams and barks.”
Even a small shift in behaviour — having your eyes closed, say, or not responding to another’s gaze, was enough to trigger the chimpanzee’s fight-or-flight response. Primates, it seems, have their own idea of the uncanny.

Working out what things are is not a straightforward business. When I was a boy I found a hedgehog trying to mate with a scrubbing brush. Dolphins regularly copulate with dead sharks (though that might just be dolphins being dolphins). Mimicry compounds the problem: beware the orchid mantis that pretends to be a flower, or the mimic octopus that’ll shape-shift into just about anything you put in front of it.

In social species like our own, it’s especially important to recognise the people you know.
In a damaged brain, this ability can be lost, and then our nearest and our dearest, our fathers, mothers, sons, daughters, spouses, best friends and pets become no more in our sight than malevolent simulacra. For instance, Capgras syndrome is a psychiatric disorder that occurs when the internal portion of our representation of someone we know becomes damaged or inaccessible. This produces the impression of someone who looks right on the outside, but seems different on the inside – you believe that your loved one has been taken over by an imposter.

Will Mufasa trigger Capgras-like responses from movie-goers? Will they scream and bark at the screen, unnerved and ready to attack?

Hopefully not. With each manifestation of the digital uncanny comes the learning necessary for us not to be freaked out by it. That man is not really on fire. That alien hasn’t really vanished down the actor’s throat. After all, the rise of deepfakes and chatbots shows no sign of slowing. But is this a good thing?

I’m not sure.

When push comes to shove, the problem with photorealist animation is really just a special case of the problem with blockbuster films in general: the closer it comes to the real, the more it advertises its own imposture.

Cinema is, and always has been, a game of sunk costs. The effort grows exponentially, to satisfy the appetites of viewers who have become exponentially more jaded.

And this raises a more troubling thought – that beyond the uncanny valley’s lairs of the strange, the off-kilter and the not-quite-right is a barren land marked, simply, “Indifference”.

The uncanny valley seemed deep enough, in the 1970s, to inspire scientific study, but we’ve had half a century to acclimitise to not-quite-human agents. And not just acclimitise to them: Hanson Robotics’ wobbly-faced Sophia generated more scorn than terror when the Saudi government unveiled her in 2017. The wonderfully named Abyss Creations of Las Vegas turned out their first sexbot in 1996. RealDoll now has global competition, especially from east Asia.

Perhaps we’ve simply grown in sophistication. I hope so. The alternative is not pretty: that we’re steadily lowering the bar on what we think is a person.

 

Look! The Astrodome! Glen Campbell! Hippies!

Posted on by

Watching Richard Linklater’s Apollo 10-1/2 for New Scientist, 13 April 2022

“What we really seek in space is not knowledge, but wonder, beauty, romance, novelty – and above all, adventure.” So said science fiction writer Arthur Clarke, speaking at the American Aeronautical Society in 1967, and with the gloss already beginning to flake off the Apollo project.

By the time Apollo 11 launched on 16 July 1969, NASA’s bid to land astronauts the moon — the costliest non-military undertaking in history — could not help but be overshadowed by the even more enormous cost of the Vietnam War.

Only a very little of this realpolitik trickles into the consciousness of ten-year-old Stanley (newcomer Milo Coy) as he propels himself on his Schwinn bike around Houston — north America’s own Space City. His father is one of NASA’s smaller cogs — one of the 400,000 people who contributed to the programme — but this is enough to inspire a whole other reality in Stanley’s head: one in which he’s hired for a secret test flight of Apollo equipment before the grown-ups, Armstrong, Aldrin and Collins, blast off to glory.

Jack Black (whose mother, incidentally, was a NASA engineer; she worked on Apollo 13’s life-saving abort-guidance system) plays Stanley in the present: a narrator whose perspectives have widened to take in the politics of the time, but not in a way that undercuts the story. Apollo 10½ is, in the best sense, an innocent film: a film about wonder, and beauty, and adventure. Though full of Boomer catnip (Look! The Astrodome! Glen Campbell! Hippies!) — it is not so much a nostalgic movie as a movie about childhood, about its possibilities and its fantasies.

To that end the film, an animation, harnesses the “interpolated rotoscoping” technique first developed by art director Bob Sabiston for Linklater’s 2001 film Waking Life. Sabiston’s “Rotoshop“ software essentially allowed an artist to draw over the top of QuickTime files, much as inventor Max Fleischer drew over movie stills to create the first Rotoscoped animations in the 1910s.

The software worked a treat for the surreal philosophical meanderings of Linklater’s 2006 Waking Life (a documentary of sorts about consciousness) but keeled over somewhat when a frantic studio expected it to actually speed up the production of A Scanner Darkly.

Unsurprisingly, it didn’t.

An adaptation of Philip Dick’s paranoid classic (in which an undercover policeman is assigned to follow himself), this unfairly rushed film wobbles uncertainly between visionary triumph (type “scramble suit” into Youtube) and the sort of rather flat, literal animation that looks as if a computer could have done it unaided (though it couldn’t, and it didn’t).

Sixteen years on, Apollo 10½ realises Sabiston’s original 2½-D conception with perfect consistency. But that’s only partly down to improved technology. In fact traditional rotoscoping techniques were used in preference to the computer-aided “interpolated” rotoscoping of Scanner and Waking Life. The two-year industry hiatus triggered by COVID-19 gave Linklater and his animators the time they needed to hand-craft their film.

Time is rarely on the side of the filmmaker, but Linklater has chiselled out a unique relationship with the stuff. Boyhood (2014), about one boy’s childhood and adolescence, was filmed in episodes from 2002 to 2013 with the same cast. Merrily We Roll Along, based on Stephen Sondheim’s musical spanning 20 years, will take 20 years to complete. Apollo 10½, which the director had been noodling around for 18 years, has taken longer than the whole space race.

These are approaches to production that any traditional film studio would struggle to accommodate. So it’s no surprise to find an odd duck like Apollo 10½ streaming as a Netflix original. The streaming company’s 222 million subscribers are already sat at their screens, waiting to be entertained. Relieved of the need to recoup single investments in single cinema-going weekends, Netflix can afford to work in a more constructive fashion with its artists. That, anyway, was Linklater’s view when interviewed by IndieWire in March 2022, and he’s by no means the first auteur to sing the company’s praises.

Streaming will kill the feature film? On the evidence of Apollo 10½ alone — a charming, moving, and intelligent movie — I think we should bury that particular worry.

Sod provenance

Posted on by

Is the digital revolution that Pixar began with Toy Story stifling art – or saving it? An article for the Telegraph, 24 July 2021

In 2011 the Westfield shopping mall in Stratford, East London, acquired a new public artwork: a digital waterfall by the Shoreditch-based Jason Bruges Studio. The liquid-crystal facets of the 12 metre high sculpture form a subtle semi-random flickering display, as though water were pouring down its sides. Depending on the shopper’s mood, this either slakes their visual appetite, or leaves them gasping for a glimpse of real rocks, real water, real life.

Over its ten-year life, Bruges’s piece has gone from being a comment about natural processes (so soothing, so various, so predictable!) to being a comment about digital images, a nagging reminder that underneath the apparent smoothness of our media lurks the jagged line and the stair-stepped edge, the grid, the square: the pixel, in other words.

We suspect that the digital world is grainier than the real, coarser, more constricted, and stubbornly rectilinear. But this is a prejudice, and one that’s neatly punctured by a new book by electrical engineer and Pixar co-founder Alvy Ray Smith, “A Biography of the Pixel”. This eccentric work traces the intellectual genealogy of Toy Story (Pixar’s first feature-length computer animation in 1995) over bump-maps and around occlusions, along traced rays and through endless samples, computations and transformations, back to the mathematics of the eighteenth century.

Smith’s whig history is a little hard to take — as though, say, Joseph Fourier’s efforts in 1822 to visualise how heat passed through solids were merely a way-station on the way to Buzz Lightyear’s calamitous launch from the banister rail — but it’s a superb short-hand in which to explain the science.

We can use Fourier’s mathematics to record an image as a series of waves. (Visual patterns, patterns of light and shade and movement, “can be represented by the voltage patterns in a machine,” Smith explains.) And we can recreate these waves, and the image they represent, with perfect fidelity, so long as we have a record of the points at the crests and troughs of each wave.

The locations of these high- and low-points, recorded as numerical coordinates, are pixels. (The little dots you see if you stare far too closely at your computer screen are not pixels; strictly speaking, they’re “display elements”.)

Digital media do not cut up the world into little squares. (Only crappy screens do that). They don’t paint by numbers. On the contrary, they faithfully mimic patterns in the real world.

This leads Smith to his wonderfully upside-down-sounding catch-line: “Reality,” he says, ”is just a convenient measure of complexity.”

Once pixels are converted to images on a screen, they can be used to create any world, rooted in any geometry, and obeying any physics. And yet these possibilities remain largely unexplored. Almost every computer animation is shot through a fictitious “camera lens”, faithfully recording a Euclidean landscape. Why are digital animations so conservative?

I think this is the wrong question: its assumptions are faulty. I think the ability to ape reality at such high fidelity creates compelling and radical possibilities of its own.

I discussed some of these possibilities with Paul Franklin, co-founder of the SFX company DNEG, and who won Oscars for his work on Christopher Nolan’s sci-fi blockbusters Interstellar (2014) and Inception (2010). Franklin says the digital technologies appearing on film sets in the past decade — from lighter cameras and cooler lights to 3-D printed props and LED front-projection screens — are positively disrupting the way films are made. They are making film sets creative spaces once again, and giving the director and camera crew more opportunities for on-the-fly creative decision making. “We used a front-projection screen on the film Interstellar, so the actors could see what visual effects they were supposed to be responding to,” he remembers. “The actors loved being able to see the super-massive black hole they were supposed to be hurtling towards. Then we realised that we could capture an image of the rotating black hole’s disc reflecting in Matthew McConaughey’s helmet: now that’s not the sort of shot you plan.”

Now those projection screens are interactive. Franklin explains: “Say I’m looking down a big corridor. As I move the camera across the screen, instead of it flattening off and giving away the fact that it’s actually just a scenic backing, the corridor moves with the correct perspective, creating the illusion of a huge volume of space beyond the screen itself.“

Effects can be added to a shot in real-time, and in full view of cast and crew. More to the point, what the director sees through their viewfinder is what the audience gets. This encourages the sort of disciplined and creative filmmaking Melies and Chaplin would recognise, and spells an end to the deplorable industry habit of kicking important creative decisions into the long grass of post-production.

What’s taking shape here isn’t a “good enough for TV” reality. This is a “good enough to reveal truths” reality. (Gargantua, the spinning black hole at Interstellar’s climax, was calculated and rendered so meticulously, it ended up in a paper for the journal Classical and Quantum Gravity.) In some settings, digital facsimile is becoming, literally, a replacement reality.

In 2012 the EU High Representative Baroness Ashton gave a physical facsimile of the burial chamber of Tutankhamun to the people of Egypt. The digital studio responsible for its creation, Factum Foundation, has been working in the Valley of the Kings since 2001, creating ever-more faithful copies of places that were never meant to be visited. They also print paintings (by Velasquez, by Murillo, by Raphael…) that are indistinguishable from the originals.

From the perspective of this burgeoning replacement reality, much that is currently considered radical in the art world appears no more than a frantic shoring-up of old ideas and exhausted values. A couple of days ago Damien Hirst launched The Currency, a physical set of dot paintings the digitally tokenised images of which can be purchased, traded, and exchanged for the real paintings.

Eventually the purchaser has to choose whether to retain the token, or trade it in for the physical picture. They can’t own both. This, says Hirst, is supposed to challenge the concept of value through money and art. Every participant is confronted with their perception of value, and how it influences their decision.

But hang on: doesn’t money already do this? Isn’t this what money actually is?

It can be no accident that non-fungible tokens (NFTs), which make bits of the internet ownable, have emerged even as the same digital technologies are actually erasing the value of provenance in the real world. There is nothing sillier, or more dated looking, than the Neues Museum’s scan of its iconic bust of Nefertiti, released free to the public after a complex three-year legal battle. It comes complete with a copyright license in the bottom of the bust itself — a copyright claim to the scan of a 3,000-year-old sculpture created 3,000 miles away.

Digital technologies will not destroy art, but they will erode and ultimately extinguish the value of an artwork’s physical provenance. Once facsimiles become indistinguishable from originals, then originals will be considered mere “first editions”.

Of course literature has thrived for many centuries in such an environment; why should the same environment damage art? That would happen only if art had somehow already been reduced to a mere vehicle for financial speculation. As if!

 

Run for your life

Posted on by

Watching Gints Zilbalodis’s Away for New Scientist, 18 November 2020

A barren landscape at sun-up. From the cords of his deflated parachute, dangling from the twisted branch of a dead tree, a boy slowly wakes to his surroundings, just as a figure appears out of the dawn’s dreamy desert glare. Humanoid but not human, faceless yet somehow inexpressibly sad, the giant figure shambles towards the boy and bends and, though mouthless, tries somehow to swallow him.

The boy unclips himself from his harness, falls to the sandy ground, and begins to run. The strange, slow, gripping pursuit that follows will, in the space of an hour and ten minutes, tell the story of how the boy comes to understand the value of life and friendship.

That the monster is Death is clear from the start: not a ravenous ogre, but unstoppable and steady. It swallows, without fuss or pain, the lives of any creature it touches. Perhaps the figure pursuing the boy is not a physical threat at all, but more the dawning of a terrible idea — that none of us lives forever. (In one extraordinary dream sequence, we see the boy’s fellow air passengers plummet from the sky, each one rendered as a little melancholy incarnation of the same creature.)

Away is the sole creation of 26-year-old Latvian film-maker Gints Zilbalodis, and it’s his first feature-length animation. Zabalodis is Away’s director, writer, animator, editor, and even composed its deceptively simple synth score — a constant back-and-forth between dread and wonder.

There’s no shading in Zabalodis’s CGI-powered animation, no outlining, and next to no texture, and the physics is rudimentary. When bodies enter water, there’s no splash: instead, deep ripples shimmer across the screen. A geyser erupts, and water rises and falls against itself in a churn of massy, architectonic white blocks. What drives this strange retro, gamelike animation style?

Away feels nostalgic at first, perhaps harking back to the early days of videogames, when processing speeds were tiny, and a limited palette and simplified physics helped players explore game worlds in real time. Indeed the whole film is structured like a game, with distinct chapters and a plot arranged around simple physical and logical puzzles. The boy finds a haversack, a map, a water canteen, a key and a motorbike. He finds a companion — a young bird. His companion learns to fly, and departs, and returns. The boy runs out of water, and finds it. He meets turtles, birds, and cats. He wins a major victory over his terrifying pursuer, only to discover that the victory is temporary. By the end of the film, it’s the realistic movies that seem odd, the big budget animations, the meticulously composited Nolanesque behemoths. Even dialogue feels clumsy and lumpen, after 75 minutes of Away’s impeccable, wordless storytelling.

Away reminds us that when everything in the frame and on the soundtrack serves the story, then the elements themselves don’t have to be remarkable. They can be simple and straightforward: fields of a single colour, a single apposite sound-effect, the tilt of a simply drawn head.

As CGI technology penetrates the prosumer market, and super-tool packages like Maya become affordable, or at any rate accessible through institutions, then more artists and filmmakers are likely to take up the challenge laid down by Away, creating, all by themselves, their own feature-length productions.

Experiments of this sort — ones that change the logistics and economies of film production — are often ugly. The first films were virtually unfollowable. The first sound films were dull and stagey. CGI effects were so hammy at first, they kicked viewers out of the movie-going experience entirely. It took years for Pixar’s animations to acquire their trademark charm.

Away is different. In an industry that makes films whose animation credits feature casts of thousands, Zabalodis’s exquisite movie sets a very high bar indeed for a new kind of artisanal filmmaking.