Tag Archives: technology

Making abstract life

Posted on by

Talking to the design engineer Yamanaka Shunji for New Scientist, 23 January 2019

Five years ago, desktop 3D printers were poised to change the world. A couple of things got in the way. The print resolution wasn’t very good. Who wants to drink from a tessellated cup?

More important, it turned out that none of us could design our way out of a wet paper bag.

Japanese designer Yamanaka Shunji calls forth one-piece walking machines from vinyl-powder printers the way the virtuoso Phyllis Chen conjures concert programmes from toy pianos. There’s so much evident genius at work, you marvel that either has time for such silliness.

There’s method here, of course: Yamanaka’s X-Design programme at Keio University turns out objects bigger than the drums in which they’re sintered, by printing them in folded form. It’s a technique lifted from space-station design, though starry-eyed Western journalists, obsessed with Japanese design, tend to reach for origami metaphors.

Yamanaka’s international touring show, which is stopping off at Japan House in London until mid-March, knows which cultural buttons to press. The tables on which his machine prototypes are displayed are steel sheets, rolled to a curve and strung under tension between floor and ceiling, so visitors find themselves walking among what appear to be unfolded paper scrolls. If anything can seduce you into buying a £100 sake cup when you exit the gift shop, it’s this elegant, transfixing show.

“We often make robots for their own sake,” says Yamanaka, blithely, “but usefulness is also important for me. I’m always switching between these two ways of thinking as I work on a design.”
The beauty of his work is evident from the first. Its purpose, and its significance, take a little unpacking.

Rami, for example: it’s a below-the-knee running prosthesis developed for the athlete Takakura Saki, who represented Japan during the 2012 Paralympics. Working from right to left, one observes how a rather clunky running blade mutated into a generative, organic dream of a limb, before being reined back into a new and practical form. The engineering is rigorous, but the inspiration was aesthetic: “We hoped the harmony between human and object could be improved by re-designing the thing to be more physically attractive.”

Think about that a second. It’s an odd thing to say. It suggests that an artistic judgement can spur on and inform an engineering advance. And so, it does, in Yamanaka’s practice, again, and again.

Yamanaka, is an engineer who spent much of his time at university drawing manga, and cut his teeth on car design at Nissan. He wants to make something clear, though: “Engineering and art don’t flow into each other. The methodologies of art and science are very different, as different as objectivity and subjectivity. They are fundamental attitudes. The trick, in design, is to change your attitude, from moment to moment.” Under Yamanaka’s tutelage, you learn to switch gears, not grind them.

Eventually Yamanaka lost interest in giving structure and design to existing technology. “I felt if one could directly nurture technological seeds, more imaginative products could be created.” It was the first step on a path toward designing for robot-human interaction.

2nd_Prototyping_in_Tokyo_exhibition_at_Japan_House_London_16Jan-17Mar2019_showing_work_by_Professor_Yamanaka_Shunji_Image©Jeremie_Souteyrat-(2)

Yamanaka – so punctilious, so polite – begins to relax, as he contemplates the work of his peers: Engineers are always developing robots that are realistic, in a linear way that associates life with things, he says, adding that they are obsessed with being more and more “real”. Consequently, he adds, a lot of their work is “horrible. They’re making zombies!”

Artists have already established a much better approach, he explains: quite simply, artists know how to sketch. They know how to reduce, and abstract. “From ancient times, art has been about the right line, the right gesture. Abstraction gets at reality, not by mimicking it, but by purifying it. By spotting and exploring what’s essential.”

Yamanaka’s robots don’t copy particular animals or people, but emerge from close observation of how living things move and behave. He is fascinated by how even unliving objects sometimes seem to transmit the presence of life or intelligence. “We have a sensitivity for what’s living and what’s not,” he observes. “We’re always searching for an element of living behaviour. If it moves, and especially if it responds to touch, we immediately suspect it has some kind of intellect. As a designer I’m interested in the elements of that assumption.”

So it is, inevitably, that the most unassuming machine turns out to hold the key to the whole exhibition. Apostroph is the fruit of a collaboration with Manfred Hild, at Sony’s Computer Science Laboratories in Paris. It’s a hinged body made up of several curving frames, suggesting a gentle logarithmic spiral.

Each joint contains a motor which is programmed to resist external force. Leave it alone, and it will respond to gravity. It will try to stand. Sometimes it expands into a broad, bridge-like arch; at other times it slides one part of itself through another, curls up and rolls away.

As an engineer, you always follow a line of logic, says Yamanaka. You think in a linear way. It’s a valuable way of proceeding, but unsuited to exploration. Armed with fragile, good-enough 3D-printed prototypes, Yamanaka has found a way to do without blueprints, responding to the models he makes as an artist would.

In this, he’s both playing to his strengths as a frustrated manga illustrator, and preparing his students for a future in which the old industrial procedures no longer apply. “Blueprints are like messages which ensure the designer and manufacturer are on the same page,” he explains. “If, however, the final material could be manipulated in real time, then there would be no need to translate ideas into blueprints.”

Rami---Additively-manufactured-running-specific-prosthetics_Image-©-KATO-Yasushi-

It’s a seductive spiel but I can’t help but ask what all these elegant but mostly impractical forms are all, well, for.

 

Yamanaka’s answer is that they’re to make the future bearable. “I think the perception of subtle lifelike behaviour is key to communication in a future full of intelligent machines,” he says. “Right now we address robots directly, guiding their operations. But in the future, with so many intelligent objects in our life, we’ll not have the time or the patience or even the ability to be so precise. Body language and unconscious communication will be far more important. So designing a lifelike element into our machines is far more important than just tinkering with their shape.”

By now we’ve left the gallery and are standing before Flagella, a mechanical mobile made for Yamanaka’s 2009 exhibition Bones, held in Tokyo Midtown. Flagella is powered by a motor with three units that repeatedly rotate and counter-rotate, its movements supple and smooth like an anemone. It’s hard to believe the entire machine is made from hard materials.

There’s a child standing in front of it. His parents are presumably off somewhere agonising over sake cups, dinky tea pots, bowls that cost a month’s rent. As we watch, the boy begins to dance, riffing off the automaton’s moves, trying to find gestures to match the weavings of the machine.

“This one is of no practical purpose whatsoever,” Yamanaka smiles. But he doesn’t really think that. And now, neither do I.

Praying to the World Machine

Posted on by

Attending Belief in AI in Dubai for New Scientist, 4 January 2018

In late spring this year, the Barbican Centre in London will explore the promise and perils of artificial intelligence in a festival of films, workshops, concerts, talks and exhibitions. Even before the show opens, however, I have a bone to pick: what on earth induced the organisers to call their show AI: More than human?

More than human? What are we being sold here? What are we being asked to assume, about the technology and about ourselves?

Language is at the heart of the problem. In his 2007 book, The Emotion Machine, computer scientist Marvin Minsky deplored (although even he couldn’t altogether avoid) the use of “suitcase words”: his phrase for words conveying specialist technical detail through simple metaphors. Think what we are doing when we say metal alloys “remember” their shape, or that a search engine offers “intelligent” answers to a query.

Without metaphors and the human tendency to personify, we would never be able to converse, let alone explore technical subjects, but the price we pay for communication is a credulity when it comes to modelling how the world actually works. No wonder we are outraged when AI doesn’t behave intelligently. But it isn’t the program playing us false, rather the name we gave it.

Then there is the problem outlined by Benjamin Bratton, director of the Center for Design and Geopolitics at the University of California, San Diego, and author of cyber bible The Stack. Speaking at Dubai’s Belief in AI symposium last year, he said we use suitcase words from religion when we talk about AI, because we simply don’t know what AI is yet.

For how long, he asked, should we go along with the prevailing hype and indulge the idea that artificial intelligence resembles (never mind surpasses) human intelligence? Might this warp or spoil a promising technology?

The Dubai symposium, organised by Kenric McDowell and Ben Vickers, interrogated these questions well. McDowell leads the Artists and Machine Intelligence programme at Google Research, while Vickers has overseen experiments in neural-network art at the Serpentine Gallery in London. Conversations, talks and screenings explored what they called a “monumental shift in how societies construct the everyday”, as we increasingly hand over our decision-making to non-humans.

Some of this territory is familiar. Ramon Amaro, a design engineer at Goldsmith, University of London, drew the obvious moral from the story of researcher Joy Buolamwini, whose facial-recognition art project The Aspire Mirror refused to recognise her because of her black skin.

The point is not simple racism. The truth is even more disturbing: machines are nowhere near clever enough to handle the huge spread of normal distribution on which virtually all human characteristics and behaviours lie. The tendency to exclude is embedded in the mathematics of these machines, and no patching can fix it.

Yuk Hui, a philosopher who studied computer engineering and philosophy at the University of Hong Kong, broadened the lesson. Rational, disinterested thinking machines are simply impossible to build. The problem is not technical but formal, because thinking always has a purpose: without a goal, it is too expensive a process to arise spontaneously.

The more machines emulate real brains, argued Hui, the more they will evolve – from autonomic response to brute urge to emotion. The implication is clear. When we give these recursive neural networks access to the internet, we are setting wild animals loose.

Although the speakers were well-informed, Belief in AI was never intended to be a technical conference, and so ran the risk of all such speculative endeavours – drowning in hyperbole. Artists using neural networks in their practice are painfully aware of this. One artist absent from the conference, but cited by several speakers, was Turkish-born Memo Akten, based at Somerset House in London.

His neural networks make predictions on live webcam input, using previously seen images to make sense of new ones. In one experiment, a scene including a dishcloth is converted into a Turneresque animation by a recursive neural network trained on seascapes. The temptation to say this network is “interpreting” the view, and “creating” art from it, is well nigh irresistible. It drives Akten crazy. Earlier this year in a public forum he threatened to strangle a kitten whenever anyone in the audience personified AI, by talking about “the AI”, for instance.

It was left to novelist Rana Dasgupta to really put the frighteners on us as he coolly unpicked the state of automated late capitalism. Today, capital and rental income are the true indices of political participation, just as they were before the industrial revolution. Wage rises? Improved working conditions? Aspiration? All so last year. Automation has  made their obliteration possible, by reducing to virtually nothing the costs of manufacture.

Dasgupta’s vision of lives spent in subjection to a World Machine – fertile, terrifying, inhuman, unethical, and not in the least interested in us – was also a suitcase of sorts, too, containing a lot of hype, and no small amount of theology. It was also impossible to dismiss.

Cultural institutions dabbling in the AI pond should note the obvious moral. When we design something we decide to call an artificial intelligence, we commit ourselves to a linguistic path we shouldn’t pursue. To put it more simply: we must be careful what we wish for.

Whose head is it anyway?

Posted on by

Reading Hubert Haddad’s novel Desirable Body for the Guardian, 22 December 2018

English speakers have only two or three translations from the French by which to judge the sometimes dreamy, sometimes nightmarish output of Tunisian poet and novelist Hubert Haddad. He began writing long prose in the 1970s and has been turning out a novel a year, more or less, since the turn of the century.

First published as Corps désirable in 2015, this novel sews a real-life maverick neurosurgeon, Sergio Canavero, into a narrative that coincides with the bicentenary of the first ever neurosurgical horror story, Mary Shelley’s Frankenstein.

Cédric Allyn-Weberson, scion of a big pharma plutocrat, has set sail for the coast of Paros with his war correspondent girlfriend Lorna Leer, on a yacht called Evasion. A horrible accident crushes his spine but leaves his head intact. Funded by Cédric’s estranged father Morice, Canavero sets about transplanting Cédric’s head on to a donor body. Assuming the operation succeeds, how will Cédric cope?

Nevertheless, this short, sly novel is not about Canavero’s surgery so much as about the existential questions it raises. Emotions are physiological phenomena, interpreted by the mind. It follows that Cédric’s head, trapped “in a merciless battle … abandoned to this slow, living enterprise, to the invading hysteria of muscles and organs”, can’t possibly know how to read his new body. His life has, sure enough, been reduced to “a sort of crystalline, luminous, almost abstract dream”.

Cédric doesn’t forget who he is; he simply ceases to care, and adopts a creaturely attitude in which self hardly matters, and beings are born and die nameless. In his world, “There was no one, with the exception of a few chance encounters and sometimes some embraces. Did birds or rats worry about their social identity?”

There is something dated about Haddad’s book: an effect as curious as it is, I am sure, deliberate, with piquant hints of Ian Fleming in his use of glamorous European locations. It’s in its glancing, elliptical relationship to technology that Desirable Body takes its most curious backward step. Yet this elusive approach feels like a breath of fresh air after decades spent wading through big infrastructure-saturated fictions such as Don DeLillo’s Underworld and Richard Powers’s The Overstory. Haddad focuses succinctly on formal existential questions: questions for which there are no handy apps, and which can in no way be evaded by the application of ameliorating technology.

The besetting existential problem for the book, and, indeed, for poor Cédric himself, is pleasure. He discovers this with a vengeance when he once again (and at last) goes to bed with his girlfriend: “Getting used to this new body after so much time seems like an appropriation of a sexual kind, a disturbing usurpation, a rape almost.” Lorna’s excitement only adds to his confusion: “The last straw is the jealous impulse that overtakes him when he sees her writhing on top of him.”

French critics have received Desirable Body with due solemnity. Surely this was a mistake: Haddad’s nostalgic gestures are playful, not ponderous, and I don’t think we are required to take them too seriously. Following Cédric’s dismal post-operative sexual experience, the book changes gear from tragedy to farce; indeed, becomes laugh-out-loud funny as he finds himself king-for-a-day in a buffoonish and clockwork world where “no one is really loved because we constantly go to the wrong house or the wrong person with the same extraordinary obstinacy”.

Desirable Body is about more than one decapitated man’s unusual plight; it’s about how surprisingly little our choices have to do with our feelings and passions. A farce, then, and a sharp one: it’s funny to contemplate, but if you fell into its toils for a second, you’d die screaming in horror.

Liquid Crystal Display: Snap judgements

Posted on by

Lead_Waad-AlBawardi,-The-Hidden-Life-of-Crystals,

Visiting Liquid Crystal Display at SITE Gallery, Sheffield, for New Scientist, 31 October 2018

Untitled Gallery was founded in Sheffield in 1979. It specialised in photography. In 1996 it was renamed Site Gallery and steadily expanded its remit to cover the intersection between science and art. Nearly 30 years and a £1.7million refit later, Site Gallery is the new poster child of Sheffield’s Cultural Industries Quarter, with an exhibition, Liquid Crystal Display, that cleverly salutes its photographic past.

Most shows about art value the results over the ingredients. The picture matters more than the paint. The statue matters more than the stone. Exhibitions about photography give rather more space to process because photography’s ingredients are so involved and fascinating.

Liquid Crystal Display follows this photographic logic to its end. This is a show about the beauty, weight and messiness of materials we notice only when they’ve stopped working. It’s about the beauty created by a broken smartphone screen, a corroded battery, a cracked lens.

Site Gallery’s new exhibition – a cabinet of curiosities if ever there was one – collides science and art, the natural and the manufactured, the old and the new. It puts the exquisite sketches of 19th-century Scottish chemist and photographer Mungo Ponton (detailing his observations of how crystals polarise light), next to their nearest contemporary equivalent: microscopic studies (pictured) of liquid crystals caught in the process of self-organisation by Waad AlBawardi, a Saudi molecular biologist who’s currently in Edinburgh, researching the structure of DNA organisation inside cells.

This provocative pairing of the relatively simple and the manifestly complex is repeated several times. Near a selection of crystals from John Ruskin’s mineral collection sit the buckets, burners and batteries of Jonathan Kemp, Martin Howse and Ryan Jordan’s The Crystal World project, a tabletop installation recording their hot, smelly, borderline-hazardous effort to extract the original minerals from bits of scavenged computers. Curated by Laura Sillars, assisted by Site Gallery’s own Angelica Sule, Liquid Crystal Display reveals the material, mineral reality behind our oh-so-weightless holographic world of digital imagery. “Liquid crystals polarise light, produce colour and yet, as a material form, recede into the background of technology,” Sillars wrote in the catalogue to this show.

JulesLister_24Sept2018Large_4024-Large

This awareness is not new, of course. In the 1960s, liquid crystals were being burned on overhead projectors to create psychedelic light shows. J G Ballard’s novel The Crystal World (1966) concocted a paranoid vision of a world and a civilisation returned (literally) to its mineral roots. That story receives a handsome homage here from the scifi-obsessed Norwegian artist Anne Lislegaard, whose stark monochrome animation (above) turns the sharp shadows and silhouettes cast by contemporary domestic furniture into insidious crystalline growths.

Arrayed within Anna Barham’s peculiar hexagonal cabinetwork, a gigantic piece of display furniture that is itself an artwork, the pictures, objects, films and devices in Liquid Crystal Display speak to pressing topical worries – resource depletion, environmental degradation, the creeping uncanny of digital experience – while at the same time evoking a peculiar nostalgia for our photochemical past.

The exhibition lacks one large signature object against which visitors can take selfies. A peculiar omission in a show that’s relaunching a gallery. And a bit of a shame for an exhibition that, in its left-field way, has handsomely captured the philosophical essence of photography.

Tomás Saraceno: Beneath an ocean of air

Posted on by

Visiting Tomás Saraceno’s Berlin studio for New Scientist, 13 October 2018

THE Argentine-born artist Tomás Saraceno maintains a studio in Berlin – if you can call a disused chemicals factory a studio. There is nothing small about this operation. Saraceno, who trained as an architect in Buenos Aires, now employs hundreds of people, with specialisms ranging from art history and architecture to biology and anthropology. If you’re serious about saving the world, you need this kind of cross-disciplinary team, I suppose.

Though Saraceno hasn’t exactly promised to save the world, he has been dropping some big hints. His utopian installations include Cloud Cities at Hamburger Bahnhof, Berlin, in 2011 – a collection of geometric, inflated shapes. Even by the time of his Observatory/Air-Port-City show at London’s Hayward Gallery in 2008, these shapes contained autonomous residential units. A network of habitable cells floated in the air, combining and recombining like clouds.

A year later at the Metropolitan Museum in New York, gallery-goers got to explore these spaces via 16 interconnected modules made up of glass segments held in place by steel cables. And in June 2013, the K21 gallery in Düsseldorf invited visitors to wander more than 25 metres above the gallery’s piazza across a web dotted with inflated PVC spheres.

This is Saraceno’s answer to our global problems: he wants us to take to the air. That’s why he coined the term “Aerocene” for one of his projects. He wants people to think of climate change in terms of possibility, playfulness and, yes, escape. “We live beneath an ocean of air,” he once wrote, as he sketched his utopian vision of a city in the clouds. “But we’ve yet to find a way to inhabit it.”

Near his Berlin studio is a scruffy public park. Part of it is marked out for football. Behind one goal stands a graffitied stretch of the Berlin Wall. Today there’s another attraction: two men are running back and forth, trying to fill a black bag as big as a minivan with air. It is a fine, windless day; the air in the bag heats up quickly, and once it is sealed, the container rises into the sky. A bag no longer, it is clearly recognisable as one of Saraceno’s signature tetrahedral solar balloons.

These black balloons have been plying the skies since 2007. They are mascots of the artist’s multi-stranded effort to combine engineering, architecture and the natural sciences to create a new, democratic kind of environmental art, made of bubbles and aerial platforms and webs. An art that mitigates climate change, he says, and makes the sky habitable, by establishing a modular, transnational settlement in the skies through solar balloons that require no fuel at all. An art that ushers in utopia.

Could it be that this chap is just playing about with balloons? Trying to calculate Saraceno’s level of seriousness is half the fun. Over lunch, for instance, he tells me that he wants to return us “to a sort of Mayan sensitivity towards celestial mechanics”.

But some of his efforts are admirably practical. The balloon I’d just seen being demonstrated was an Aerocene Explorer: it comes in a backpack complete with instructions on how to create and fly lightweight sensors. Any data collected can be uploaded and shared with Aerocene’s online community, via a website where participants from all over the world are sharing their experiments and innovations.

Practicalities aside, much of Saraceno’s work is simply beautiful. For a show opening at the Palais de Tokyo in Paris on 17 October, the team is busy building playful orreries, mechanical models of the solar system that combine planetary orbits with the physics of soap bubbles and webs spun by his pet Cyrtophora citricola spiders.

These unbelievably delicate confections will be on show with some mirrored umbrellas that also double as solar cookers. When arranged in concentric circles, Saraceno imagines that in the manner of a solar thermal power plant, the umbrellas might even concentrate enough heat to inflate a large balloon. He hopes to try out the idea when Audemars Piguet – a Swiss watch manufacturer that has recent form in backing innovative science-inflected art – takes parts of his sprawling Aerocene endeavour to Miami this December for the Art Basel fair.

Meanwhile, there are myriad things to organise for Paris: workshops, concerts, public symposiums uniting scientific institutions, researchers, activists, local communities, musicians and philosophers. As he says: “People aren’t very interested in simple ideas. You have to give things a little bit of complication to get the audience to engage.”

balloons

He found this out the moment he started using solar balloons. The balloons, which work by simply zipping up some air in a heat-absorbing bag, have been around since the 1970s. His own projects have demonstrated their usefulness in meteorology, pollution monitoring, even passenger transport. In 2015, he flew in a tethered solar balloon over the dunes of White Sands in New Mexico, where the US launched its first rockets and where the world’s first tourist spaceport is located. The Massachusetts Institute of Technology got in on the act, and created technology so that you can use the Aerocene.org website to plan a meteorologically feasible journey, by balloon, from Point A to Point B, anywhere on Earth.

“Rats saved at the point of giving up fought for life 240 times longer when returned to danger”

Here’s the paradox. Saraceno’s work has always been playful, and part of the game, he explains, has been “trying to sell this work as some sort of global solution to something”. But while his visions of an airborne utopia remain as remote as ever, his Aerocene project has spawned a foundation that uses lightweight balloons for climate activism and pollution monitoring. And even the absurd spectacle of someone jetting from country to country to fly fuel-less balloons has become part of the art, as Saraceno’s studio begins to record his own carbon footprint.

Saraceno makes an important point about how we address climate change in our lives. The trick, he says, is not to let the perfect get in the way of the good. Escapism is fine. He has no time for the way so many artists and pundits are ringing humanity’s death knell. He has a special contempt for the lazy way the word Anthropocene crops up now in every climate conversation, as if, with the advent of this putative new era, our doom was sealed. “What a great way for a small number of people to disempower and demotivate us,” he says.

Given the seriousness of our environmental bind, isn’t escapism a bit irresponsible? Saraceno points me to a 1957 paper by psychobiologist Curt Richter. His gruesome experiments left rats to drown in water-filled containers from which they could not escape. But if he briefly rescued rats at the point they gave up swimming, and then returned them to the water, those rats continued to fight for life 240 times longer. Richter concluded that they had learned that there was hope. Faced with challenges on a planetary scale, we are scrambling for our lives, and can see no way out. “We need the energy those rats got when they saw some small hope,” says Saraceno.

I hadn’t expected our conversation to take this dark turn, but creating such small glimmers of hope is his business. If he is a joker, then he is one in the best sense of the word.

Should we take Saraceno’s work seriously? I was doubtful, but now I think, why look a gift horse in the mouth? He enthuses people. He gets us thinking. And he is right: a little hope goes a long way.

Pierre Huyghe: Digital canvases and mind-reading machines

Posted on by

Visiting UUmwelt, Pierre Huyghe’s show at London’s Serpentine Gallery, for the Financial Times, 4 October 2018

On paper, Pierre Huyghe’s new exhibition at the Serpentine Gallery in London is a rather Spartan effort. Gone are the fictional characters, the films, the drawings; the collaborative manga flim-flam of No Ghost Just a Shell; the nested, we’re not-in-Kansas-any-more fictions, meta-fictions and crypto-documentaries of Streamside Day Follies. In place of Huyghe’s usual stage blarney come five large LED screens. Each displays a picture that, as we watch, shivers through countless mutations, teetering between snapshot clarity and monumental abstraction. One display is meaty; another, vaguely nautical. A third occupies a discomforting interzone between elephant and milk bottle.

Huyghe has not abandoned all his old habits. There are smells (suggesting animal and machine worlds), sounds (derived from brain-scan data, but which sound oddly domestic: was that not a knife-drawer being tidied?) and a great many flies. Their random movements cause the five monumental screens to pause and stutter, and this is a canny move, because without that  arbitrary grammar, Huyghe’s barrage of visual transformations would overwhelm us, rather than excite us. There is, in short, more going on here than meets the eye. But that, of course, is true of everywhere: the show’s title nods to the notion of “Umwelt” coined by the zoologist Jacob von Uexküll in 1909, when he proposed that the significant world of an animal was the sum of  things to which it responds, the rest going by virtually unnoticed. Huyghe’s speculations about machine intelligence are bringing this story up to date.

That UUmwelt turns out to be a show of great beauty as well; that the gallery-goer emerges from this most abstruse of high-tech shows with a re-invigorated appetite for the arch-traditional business of putting paint on canvas: that the gallery-goer does all the work, yet leaves feeling exhilarated, not exploited — all this is going to require some explanation.

To begin at the beginning, then: Yukiyasu Kamitani , who works at Kyoto University in Japan, made headlines in 2012 when he fed the data from fMRI brain scans of sleeping subjects into neural networks. These computer systems eventually succeeded in capturing shadowy images of his volunteers’ dreams. Since then his lab has been teaching computers to see inside people’s heads. It’s not there yet, but there are interesting blossoms to be plucked along the way.

UUmwelt is one of these blossoms. A recursive neural net has been shown about a million pictures, alongside accompanying fMRI data gathered from a human observer. Next, the neural net has been handed some raw fMRI data, and told to recreate the picture the volunteer was looking at.

Huyghe has turned the ensuing, abstruse struggles of the Kamitani Lab’s unthinking neural net into an exhibition quite as dramatic as anything he has ever made. Only, this time, the theatrics are taking place almost entirely in our own heads. What are we looking at here? A bottle. No, an elephant, no, a Francis Bacon screaming pig, goose, skyscraper, mixer tap, steam train mole dog bat’s wing…

The closer we look, the more engaged we become, the less we are able to describe what we are seeing. (This is literally true, in fact, since visual recognition works just that little bit faster than linguistic processing.) So, as we watch these digital canvases, we are drawn into dreamlike, timeless lucidity: a state of concentration without conscious effort that sports psychologists like to call “flow”. (How the Serpentine will ever clear the gallery at the end of the day I have no idea: I for one was transfixed.)

UUmwelt, far from being a show about how machines will make artists redundant, turns out to be a machine for teaching the rest of us how to read and truly appreciate the things artists make. It exercises and strengthens that bit of us that looks beyond the normative content of images and tries to make sense of them through the study of volume, colour, light, line, and texture. Students of Mondrian, Duffy and Bacon, in particular, will lap up this show.

Remember those science-fictional devices and medicines that provide hits of concentrated education? Quantum physics in one injection! Civics in a pill! I think Huyghe may have come closer than anyone to making this silly dream a solid and compelling reality. His machines are teaching us how to read pictures, and they’re doing a good job of it, too.

Jenna Sutela: Mars in a dish

Posted on by

Contemplating nimiia cétiï for New Scientist, 11 September 2018

The artist Jenna Sutela normally divides her time between London and Helsinki, but she has spent the last four months at London’s Somerset House Studios, thanks to a residency with Google Arts & Culture. Here, she’s been either making a video, learning about computers, teaching artificial intelligences to dream, or mastering Martian. Perhaps all of the above. It depends who you speak to.

The artists Google invites to explore the potentials of its machine learning systems normally wind up in its lab in Paris. This time, however, Google engineer Damien Henry (the co-inventor, incidentally of the Google Cardboard VR headset) has been travelling to London to assist Sutela and her artistic mentor, the Turkish-born data artist Memo Akten, in a project that, the more you learn about it, resembles an alchemical operation more than a work of art.

Here – so far as I understand it – is the recipe.

  1. Take one nineteenth-century French medium, Hélène Smith, who made much of her communications with Martians. (The Surrealists lapped this stuff up: they dubbed her “the muse of automatic writing”.) Make up some phonemes to match her Martian lettering. Speak Martian.
  2. Prepare a dish of Bacillus subtilis, a bacterium that we expect would cope rather well with conditions on Mars. Point a camera at it, and direct the video signal through a machine learning system. (Don’t call this an AI, whatever you do. Atkin has issued a public warning that “every time someone personifies this stuff, every time someone talks about ‘the AI’, a kitten is strangled.”)
  3. Lie to your AI. Tell it that your dish of wiggling bacteria is in fact a musical score. Record the music your AI makes as it tries to read the dish. Hide kittens.
  4. Keep lying. Tell it your dish of wiggling bacteria is a text.
  5. – a language.
  6. – a map.
  7. Write down the text. Speak the language. Read the map. Put the whole enterprise into a single twelve-minute video and hang it up in the foyer of London’s Somerset House Studios.

Titled nimiia cétiï and on view in Somerset House Studios until 15 September, Sutela’s video installation is heavy-going at first, but well worth some close scrutiny. Everything you see and hear came from that petri dish: the landscape, the music, the alien script, even its eerily convincing Martian vocalisation. “There is,” Henry tells me with avuncular pride, “absolutely no scientific goal to this project whatsoever.”

The point being that Sutela is one of the first artists, if not the first, to appropriate the rules of machine learning entirely to her own ends. It’s a milestone of sorts. She’s not illustrating an idea, or demonstrating some technical capability. She’s using machine learning like a brush, to conjure up imaginary worlds.

Which is to take nothing away from nimiia cétiï‘s considerable technical achievement. Sutela is forcing her recurrent neural network to over-interpret its little petri dish-shaped world. We’re a long way from inventing a machine that sees pictures in a fire, but these results are certainly suggestive.

«e tesi leca rizini nirnemea riechee sat ze po mizi» as a Martian might say.

Parade of the Possible

Posted on by

Watching the Elfwegentocht parade spool by for New Scientist, 18 July 2018

Astronaut André Kuipers has enjoyed his share of travel, and has no doubt racked up some air miles. Who better, then, to wave the start flag for a parade of futuristic vehicles?

Spooling along at a sedate 30 miles an hour down the motorway from Drachten to Leeuwarden, this year’s European Capital of Culture, they lacked a certain Mad Max flair. But that’s Friesland for you: a land of 60 languages and 128 nationalities, birthplace of Mata Hari – and when you drive through there’s nothing to see but cows.

The Parade was the festive conclusion of the Elfwegentocht: for two weeks, people have got about the region without using a drop of fossil fuel. “And now we’ve shown it’s possible,” says Bouwe de Boer, the municipality’s energy coordinator at the municipality of Leeuwarden, “we’ve shown that it is possible also for the rest of the Netherlands.”

De Boer is now project leader of Fossylfrij Fryslân, the fossil-free movement in Friesland, bringing disparate environmental campaigns and start-up technologies together to achieve real goals in tiny time frames. Electric vehicles dominate the parade but as de Boer points out, there are other ways to drive fossil-free. “Think of trucks and buses on hydrogen, cars on blue diesel, buses on green gas, Segways, bicycles, mobility scooters, go-karts…”

Go-karts? It’s a gimcrack future, this – but then, what else can the future ever be but an amalgam of new and old, complex and homespun?

The two big innovative technologies on display here aren’t actually “on display” in a physical sense. You’ll have to take my word for it that the “Blauwe Diesel” manufactured by Neste in Rotterdam from restaurant waste and residues is, indeed, satisfyingly blue. It’s a pure HVO (“hydrotreated vegetable oil” to you), low on emissions and so similar to regular diesel in the way it behaves that it requires no modifications to existing diesel engines or distribution systems. At a pump near you – assuming you live in this go-ahead region of the Netherlands – it could be the saving of an industry that some manufacturers and governments have already written off. Meanwhile Neste is trying to make its blue diesel from other sources, including old car tires, waste paper and algae.

Elsewhere in the parade, under the bonnets of a handful of electric cars, sit batteries from MG Energy Systems. These are the batteries you most often find in racing cars and speedboats, and they’re the brainchild of local entrepreneurs Gerard van der Schaar and Mark Scholten, whose first project, back in 2006, was a vessel to compete in the world’s first solar boat race (another de Boer initiative).

They quickly discovered that batteries were the boats’ Achilles’ heel. There was simply no good battery management system available. A little over a decade later their products power the Furia solar boat, which has finished first in just about every international solar boat event; Solarwave 62, the first hybrid yacht with electric propulsion to cross the Atlantic Ocean; the Elektra One Solar, the first electric-and-solar aircraft to fly over the Alps; and the Nuna7 car, winner of the World Solar Challenge in 2013, having achieved an average speed of 90.71 kmh for over 30 hours.

De Boer is proud of his region’s achievements but he has his eye on the bigger picture, too. As of 27 June the Netherlands has set in train one of the world’s most ambitious climate laws, which if it’s finalised in 2019, will set a target to reduce greenhouse gas emissions by 95 per cent by 2050, with the introduction of a carbon-neutral electricity system. (The UK’s mandated 2050 emissions target is an 80 per cent reduction. Sweden and Norway are set to go carbon neutral by 2045 and 2050 respectively)

De Boer talks excitedly about Friesland’s circular energy economy. Cleaning up waste water in this region generates methane which is being harvested to boost biogas production. He talks excitedly about advances in renewable energy. Solar panels power MG’s entire battery factory. He talks excitedly about everything, quite frankly. But it’s an incidental detail which captures my attention: fruit, I am told, of another one of de Boer’s endless stream of friendly, chivvying phone-calls.

The police looking after the parade are riding electric bikes.

Technology vs observation

Posted on by

Losing my rag at the Royal Academy for New Scientist, 13 December 2017

When the schools of London’s Royal Academy of Arts were opened in 1769, life drawing — the business of sketching either live models or the plaster casts of worthy sculptures — was an essential component of an artist’s training.

As I wandered around From Life, an exhibition devoted to the history as well as the future of the practice, I overheard a curator explaining that, now life drawing is no longer obligatory in Royal Academy art courses, a new generation of artists are approaching the practice in a “more expressive” way. The show’s press release claims even more: that life drawing is evolving “as technology opens up new ways of creating and visualising artwork”.

There was little of this in evidence when I visited, however: two-and-a-half of the three virtual-reality experiences on offer had broken down. Things break down when the press turns up – you might even say it’s a rule. Still, given their ubiquity, I’m beginning to wonder whether gallery-based VR malfunctions are not a kind of mischievous artwork in their own right. In place of a virtual sketch, a message in an over-friendly font asks: “Have you checked your internet connection?” At least Swiss artist Jean Tinguely’s wild mobiles of the 1960s had the decency to catch fire.

How can new technologies like Google’s Tilt Brush and HTC’s Vive VR platform bring artists into a more intimate relation to their subject — more intimate than might be achieved by, say, standing a metre away from a naked stranger armed only with a bit of charcoal?

Jonathan Yeo has had a stab at the problem, using Tilt Brush’s 3D painting tech to fashion a sculptural self-portrait. The outsize bronze 3D print of his effort — an assemblage of short, wide, hesitant virtual “brushstrokes” — has a curiously dated feel and wouldn’t look out of place in a group retrospective of 20th-century British sculpture. As an advert for a technology that prides itself on its expressivity (videos of the platform at work usually resemble explosions in a paint factory), it’s a curiously laborious piece.

On a nearby wall hang Gillian Wearing’s photographic self-portraits, manipulated using the sort of age-progression technology employed by forensic artists. In this way, Wearing has captured her appearance 10, 20, 30 years into her future. It’s an undeniably moving display, and undeniably off the point: life drawing is about capturing the present moment, which leaves Wearing’s contribution resembling those terms and conditions that appear at the bottom of TV advertisements – Other Moments Are Available.

Yinka Shonibare (best known for his ship-in-a-bottle sculpture on the fourth plinth of Trafalgar Square) comments on the show, rather than contributes to it, with a 3D VR conceptualisation of a painting by the 18th-century Scots artist and dealer Gavin Hamilton.

Hamilton once sold a Roman sculpture to a collector. Shonibare has scanned a plaster cast of this Townley Venus, then placed it on a plinth in a largely imaginary VR garden (you catch only a glimpse of this space in Hamilton’s painting). He has covered its plaster-white surface with batik designs (referring to common sub-Saharan African fabric, though it was originally a Dutch export) and as a coup de grâce, he has stuck a globe on Venus’s torso in place of her head. The point is that we can never copy something without to some degree appropriating it. Whether you like what he’s done will depend on whether you like art that makes a primarily intellectual point.

In a gallery environment increasingly besotted by (and bested by) technology, such acts of cultural orienteering may be necessary; they’re certainly inevitable. The new work gracing From Life at least attempts to address the theme of the show, and its several failures are honest and interesting.

Still, I keep coming back to the historical half of the exhibition — to the casts, the drawings, the portraits of struggling young artists from 1769 to now. Life drawing is not obligatory for artists? It should be obligatory for everyone. If we never learn to observe honestly, what the devil will we ever have to be expressive about?

It’s coming at you!

Posted on by

Exploring volumetric capture for New Scientist, 13 December 2017

OUTSIDE Dimension Studios in Wimbledon, south London, is one of those tiny wood-framed snack bars that served commercial travellers in the days before motorways. The hut is guarded by old shop dummies dressed in fishnet tights and pirate hats. If the UK made its own dilapidated version of Westworld, the cyborg rebellion would surely begin here.

Steve Jelley orders us breakfast. Years ago he left film production to pursue a career developing new media. He’s of the generation for whom the next big thing is always just around the corner. Most of them perished in the dot-com bust of 2001, but Jelley clung to the dream, and now Microsoft has come calling.

His company, Hammerhead, makes 360-degree videos for commercial clients. Its partner in this current venture, Timeslice Films, is best known for volumetric capture of still images – the business of cinematographically recording forms in three dimensions – a practice that goes back to founder Tim MacMillan’s art-school experiments of the early 1980s.

Steve Sullivan, director of the Holographic Video initiative at Microsoft, is fusing both companies’ technical expertise to create volumetric video: immersive entertainment that’s indistinguishable from reality.

There are only three studios in the world that can do this with any degree of conviction, and Wimbledon is the only one outside the US. Still, I’m sceptical. It has been clear for a while that truly immersive media won’t spring from a single “light-bulb” moment. The technologies involved are, in conceptual terms, surprisingly old. Volumetric capture is a good example.

MacMillan is considered the godfather of this tech, having invented the “bullet time” effect central to The Matrix. But The Matrix is 18 years old, and besides, MacMillan reckons that pioneer photographer Eadweard Muybridge got to the idea years before him – in fact, decades before cinema was invented.

Then there’s motion capture (or mocap): recording the movement of points attached to an actor, and from those points, constructing the performance of a three-dimensional model. The pioneering Soviet physiologist Nikolai Bernstein invented the technique in the early 1920s, while developing training programmes for factory workers.

Truly immersive media will be achieved not through magic bullets, but through thugging – the application of ever more computer power, and the ever-faster processing of more and more data points. Impressive, but where’s the breakthrough?

“Well,” Jelley begins, handing me what may be the largest bacon sandwich in London, “you know this business of the ‘uncanny valley’…?” My heart sinks slightly.

Most New Scientist readers will be familiar with Masahiro Mori’s concept of the uncanny valley. It’s a curiously anglophone obsession. In the 30 years since the Japanese engineer published his paper in 1970, it has been referred to in Japanese academic literature only once. Mori himself says the idea was never meant to be taken scientifically. He was merely warning robot designers at a time when humanoid robots didn’t exist that the closer their works came to resemble people, the creepier we would find them.

In the West, discussions of the uncanny valley have grown to a sizeable cottage industry. There have been expensive studies done with PET scans to prove the existence of the effect. But as Mori commented in an interview in 2012: “I think that the brainwaves act that way because we feel eerie. It still doesn’t explain why we feel eerie to begin with.”

Our discomfort extends beyond encounters with physical robots to include some cinematic experiences. Many are the animated movies that have employed mocap to achieve something like cinematic realism, only to plummet without trace into the valley.

Elsewhere, actor Andy Serkis famously uses mocap to transform himself into characters like Gollum in The Lord of the Rings, or the chimpanzee Caesar in Rise of the Planet of the Apes, and we are carried along well enough by these films. The one creature this technology can’t emulate, however, is Serkis himself. Though mocap now renders human body movement with impressive realism, the human face remains a machine far too complex to be seamlessly emulated even by the best system.

Jelley reckons he and his partners have “solved the problem” of the uncanny valley. He leads me into the studio. There’s a small, circular, curtained-off area – a sort of human-scale birdcage. Rings of lights and cameras are mounted on scaffolds and hang from a moveable and very heavy-looking ceiling rig.

There are 106 cameras: half of them recording in the infrared spectrum to capture depth information, half of them recording visible light. Plus, a number of ultraviolet cameras. “We use ultraviolet paint to mask areas for effects work,” Jelley explains, “so we record the UV spectrum, too. Basically we use every glimmer of light we can get short of asking you to swallow radium.”

The cameras shoot between 30 and 60 times a second. “We have a directional map of the configuration of those cameras, and we overlay that with a depth map that we’ve captured from the IR cameras. Then we can do all the pixel interpolation.”

This is a big step up from mocap. Volumetric video captures real-time depth information from surfaces themselves: there are no fluorescent sticky dots or sliced-through ping-pong balls attached to actors here. As far as the audience is concerned, volumetric video is essentially just that, video, and as close to a true record as anything piped through a basement full of computers is ever going to get.

So what kind of films are made in such studios? Right now, the education company Pearson is creating virtual consultations for trainee nurses. Fashion brands and car companies have shot adverts here. TV companies want to use them for fully immersive and interactive dramas.

On a table nearby, a demo is ready to watch on a Vive VR headset. There are three sets of performances for me to observe, all looping in a grey, gridded, unadorned virtual space: the digital future as a filing cabinet. There are two experiments from Sullivan’s early days at Microsoft. Thomas Jefferson is pure animatronic; the two Maori haka dancers are engaging, if unhuman. The circus gymnast swinging on her hoop is different. I recognise her, or think I do. My body-language must be giving the game away, because Jelley laughs.

“Go up to her,” he says. I can’t place where I’ve seen her before. I try and catch her eye. “Closer.”

I’m invading her space, and I’m not comfortable with this. I can see the individual threads, securing the sequins to her costume. More than that: I can smell her. I can feel the heat coming from her skin.

I know she’s not real, but my body doesn’t. Every bit of me that might have rejected a digitised face as uncanny has fallen hook, line and sinker for this super-real gymnast. And this, presumably, is why the bit of my mind that enables me to communicate freely and easily with my fellow humans is in overdrive, trying to plug the gaps in my experience, as if to say, “Of course her skin is hot. Of course she has a scent.”

Mori’s uncanny valley effect is not quantifiable, and I don’t suppose my experience is any more measurable than the one Mori identified. But I’d bet the farm that, had you scanned me, you would have seen all manner of pretty lights. This hasn’t been an eerie experience. Quite the reverse. It’s terrifyingly ordinary. Almost, I might say, human.

Jelley walks me back to the main road. Neither of us says a word. He knows what he has. He knows what he has done.

Outside the snack shack, three shop dummies in pirate gear wobble in the wind.