For Ken Goldberg, it's a chance to see like a fish. For Christian Möller, it's an opportunity to question the ubiquitous happy face. For Jennifer Steinkamp, it's the prospect of using a historic building as a canvas for interactive, computer-generated art.
And those are only half of the adventures-in-progress for "Neuro," a yearlong art-science collaboration that will culminate in an exhibition opening in April at two Pasadena institutions: Art Center College of Design and Caltech. Simon Penny, for example, is designing an environment that will allow people to experience the perceptions of electro-sensing fish. Jessica Bronson is devising an image or text that can be seen only with peripheral vision.
As for Martin Kersels, don't ask. He might take information gathered for a particular purpose by one device -- a seismograph, microphone, pollution indicator or rain gauge, say -- and "trans-code" it to do something completely different, possibly play music on an electronic keyboard. Or maybe not.
"I'm fumbling in an ill-lit room," Kersels says of his nascent project. "Not a dark room, but a room that needs a little more light."
"Neuro" itself -- which has teamed six tech-savvy, California-based artists with Caltech scientists and students -- is a work in progress. But whatever it turns out to be when lights go on at the exhibition, it has already generated a lot of interdisciplinary investigation. And that's the point, the organizers say.
"Artists see things with different eyes and allow us to take a step back and reflect on what we do," says Pietro Perona, a specialist in human and computer vision who directs Caltech's Center for Neuromorphic Systems Engineering. The center, where scientists attempt to endow machines with senses and sensory-like behavior, is Art Center's partner in the project. "Scientific research is supposed to be about everything, but even at universities we are pushed to be quite narrow," he says. "We lose sight of the big picture, so this is a good thing for us."
Perona admits to approaching the project with a certain trepidation. "The artists I had met before were so non-concrete, so touchy-feely. I appreciate their work, but I didn't think the collaboration would be easy," he says. "But the artists we are working with understand our mind-set very well, maybe more than we understand theirs. We are much closer than I thought."
For Stephen Nowlin, director of Art Center's Alyce de Roulet Williamson Gallery, the crux of the project is "bringing these two cultures together" to "find common ground between art and science that sparks creativity."
At the forefront of art and technology ventures, Nowlin has explored the territory for nearly 30 years and has organized crossover exhibitions at Art Center for the past decade. But "Neuro" is different.
"What is new here is that the artists and the scientists are equal partners in learning about what each other does," says Jill H. Andrews, Caltech's assistant to the provost for educational outreach. " 'Neuro' is like a research project. When you combine the element of expectation or surprise with the creative aspect of trying to figure out how you depict an aspect of science -- and throw in the interactive element -- that's a real challenge."
Still, everyone involved seems to have risen to the challenge. "Neuro" is billed as an investigation of "aesthetic possibilities at the intersection of art, science and engineering."
MEETING THEIR MATCHES
It's a busy intersection. Some of the scientists grapple with machines that can detect human movement and recognize faces; others develop sensors that can control low-altitude surveillance aircraft or sniff out illegal drugs. The artists are equally diverse in their interests and approaches, and each is charting a different course for "Neuro."
The collaboration began last March, when the artists visited Caltech to learn about research projects at the neuromorphic center. In April, a group of scientists and students went to Art Center, where the artists presented their work. Then came a couple of months of matchmaking, in consultation with Perona and Andrews.
The choice of a partner was easy for Ken Goldberg, a conceptual artist who often combines robotics with cultural criticism in his artwork and is a professor of engineering at UC Berkeley. He and Perona already knew each other as robotics researchers, and they grabbed the opportunity to work together.
At their first meeting, over lunch, Goldberg talked about his interest in using new technology to reveal aspects of contemporary human experience. As they tossed ideas around, Perona mentioned UC Berkeley scientist Michael Dickinson's groundbreaking studies of insect aerodynamics. Dickinson filmed tethered flies, then reconstructed their movement and, from that, what they see in flight.
The latter discovery captured Goldberg's interest and led to his project, "Infiltrate." "I was immediately enthusiastic about the possibility of viewing the world through a new perspective," he says. "I had been struggling with the idea of art after 9/11, and I was thinking about how two cultures can view the same events completely differently. So the new technique of tracking motion, then digitally reconstructing the viewpoint, was compelling."
At the end of their lunch, "Pietro agreed to study the tracking problem with his students at Caltech, and I agreed to study the graphical display problem with my students at Berkeley," Goldberg says. "A few weeks later, via e-mail, we decided that the project was feasible and began work."
As for the fish, "they are intrinsically interesting to watch and a very different species from humans, but much more elegant than flies," Goldberg says. "They also move slower, so we conjectured that it would be possible, using advanced hardware and software, to reconstruct a fish's view of its environment in real time, as the fish swims through the space."
In the exhibition, a 110-gallon tank containing six koi -- one orange and five white -- will be installed at eye level in the center of one gallery. Three cameras will track the movement of all the fish; at the same time, a computer running a program adapted from Dickinson's experiments will digitally reconstruct the images of what the orange fish sees. Those moving images -- which appear abstract but are actually magnified views of the other fish and the tank -- will be projected on a wall. The relationship between the fish and the images may not be clear at first, but "if we do the job right, visitors will figure out the correlation between the movement in the tank and the movement on the screen," Goldberg says.
As Perona puts its: "Hopefully, when you come in, you become the fish."
Christian Möller, a German artist who is internationally renowned for making interactive, multimedia installations and who teaches art at UCLA, also matched wits with Perona. But his project, "Cheese," will scrutinize human facial expression.
"One thing that strikes all Europeans who come to America is the friendliness of the locals and the value of smiling," Perona says. "Another thing that interested him is a long-term objective here: to make a machine that estimates your feelings, in part by reading your face and gestures, and relates to you as humans relate to each other."
Möller decided to use a computer to rate the sincerity of a smile, by reading a subject's facial structure in terms of muscle definition, how the eyes crinkle, how much of the teeth show. Perona referred him to Machine Perception Laboratories at UC San Diego, which specializes in motion recognition. The lab's scientists hadn't rated smiles before, but they came up with software to fit Möller's needs. He issued a casting call for women who would be willing to smile for hours while being videotaped --and to be judged by a computer. If it senses that the face has relaxed too much to be genuinely smiling, it sets off an alarm and the subject tries again. Tapes of six women engaged in this ordeal will appear on flat, wall-mounted screens in the Art Center gallery where Möller's work is shown.
"I wanted to do something more relevant than technology itself, something a bit more controversial," Möller says. "This is about the architecture of sincerity. It's a computer-human interaction, but the computer is dominant."
Other projects are still in the planning stages. Simon Penny, a professor of arts and engineering at UC Irvine, is hard at work with Malcolm MacIver, a Caltech postdoctoral scholar whose research is on the sensory system of fish that "see" their world through a self-generated electric field. Penny's project, "Electrosenster," is an interactive sound and light environment that will allow visitors to play the role of electric fish in search of prey.
Jennifer Steinkamp, who alters architectural spaces with projected, moving images of computer-generated graphics, was immediately smitten with the possibility of doing a piece on semicircular wall spaces under the vaulted ceiling of Caltech's stately Athenaeum. She got permission to use the space -- after a series of meetings with authorities -- but she is still grappling with the images to be projected.
Two things are certain, however. The video imagery will be related to Caltech research on optical illusions and the artwork will be interactive. Motion detectors will pick up the movement of people in the space, Steinkamp says. "When someone passes through, the image will shift or change somehow, but it will keep moving."
INSPIRATION IN THE LAB
Jennifer Bronson, who did her undergraduate work in science and worked in a biochemistry lab before becoming a video installation artist, looked for "Neuro" inspiration while visiting several scientists in their labs. She met with Richard Andersen, who is developing a way of implanting electrodes in the brain that can guide prosthetic limbs; Christof Koch, who studies visual perception and consciousness; Shinsuke Shimojo, who focuses on how the brain resolves perceptual ambiguities; and MacIver.
Bronson thought of creating a room that would respond to an occupant. "It would read certain information about the person, like heart rate, breath, body temperature, and assess the person's emotional state," she says. "Then, through some kind of computer interface and lighting or sound phenomenon, the room would create a presence that opposes what the person is feeling. The scientists I met with said, 'That's great,' but I realized that the technology to do that would cost like half a million dollars. Also, I had to step back and remember that I want viewers to be partly involved in constructing meaning instead of being totally overwhelmed and immersed in an environment."
In consultation with Shimojo, Bronson eventually decided to work with a visual phenomenon known as retinal painting, which allows the human eye to fill in missing parts of a fragmented image. Visitors who look directly at her work will merely see a thin metal strip on a wall. But those who pass by will glimpse a fleeting image or text, possibly a palindrome, which Bronson hasn't chosen yet.
"There will be a technological wonder about the piece," she says, "but it's going to be very simple -- something between the technology and understanding that there's some kind of brain chemistry at work. Hopefully, that will cause people to think about peripheral vision that happens just moving through life every day."
Martin Kersels, a conceptual sculptor who incorporates a strong dose of humor in his work, found his scientific soul mate at an introductory "Neuro" meeting. "It was a gut-level thing," says Peter Schroeder, a professor of computer science and mathematics. "I just thought that Martin did terrific stuff. I thought he was having a lot of fun with his art and I really enjoyed that." They hit it off so well that Kersels taught a class at Caltech at the invitation of Schroeder, who attended most sessions. But their work for "Neuro" is only beginning to evolve.
"We have been meeting and talking about art history, art and life, mathematics and life," Kersels says. "It's been very casual, but it's getting more pointed all the time in terms of how we are going to work on a project together and what each of us wants from it. It's just a negotiation about what we want to do and how it fits within the realm of a collaboration between an artist and a scientist."
The deadline may be approaching fast, but the art-science collaboration had a long gestation. It began three or four years ago when the National Science Foundation, the major funder of the Center for Neuromorphic Systems Engineering, asked the center to produce a creative outreach program. The goal was to make the public aware of the center's research, but the method was up for discussion.
"They asked us to come up with ideas that would be novel, not your usual, run-of-the-mill science curriculum for an elementary teacher," Caltech's Andrews says.
In a brainstorming session led by Perona and Andrews, Daniela Meeker, a graduate student, came up with idea of organizing "an exhibition that would depict the science in the center," Andrews says.
THE IDEA CATCHES ON
The initial idea was to do a show at Caltech composed of projects by students who had an interest in art. But as more people heard about the proposal, the idea grew and someone suggested collaborating with Art Center College of Design, which has a reciprocal class enrollment program with Caltech.
When the Art Center gallery's Nowlin got involved, he said he had come full circle to Caltech -- where he got a temporary job nearly 30 ago so that he could participate in an art and technology project. He recruited the artists for "Neuro" and made plans to host the bulk of the show. Steinkamp's project is the only one that will be installed at Caltech.
Expenses, about $120,000, are being split between the National Science Foundation and Art Center, which is producing a catalog that will document the collaborative process and the exhibition.
"Neuro" is one of two ongoing initiatives between Art Center and Caltech. In the other, Caltech scientists teach graduate industrial design classes at Art Center on the evolution of the nervous system and the operation of the brain's visual cortex; the graduate students, in turn, develop thesis projects in consultation with Caltech faculty and Jet Propulsion Laboratory staff.
In addition, Nowlin and Andrews are discussing a potential series of future collaborations. "We both feel this overlap between science and art -- where science becomes an aesthetic and art informs us about the world -- is important to pursue beyond this single project," Nowlin says.
As for "Neuro," "it's fun," Perona says. "But the proof of the pudding is in the eating, so we will find out in April if this was just a big intoxication."
If you want other stories on this topic, search the Archives at latimes.com/archives. For information about reprinting this article, go to www.lats.com/rights.
Copyright 2003 Los Angeles Times