Following the people and events that make up the research community at Duke

Students exploring the Innovation Co-Lab

Tag: art

AI Time Travel: Reimagining Ancient Landscapes

Sticky post

You are looking at a field of fluffy, golden grass dotted with yellow flowers. There are trees in the background and mountains beyond that. Where are you?

Now you’re facing a terracotta sarcophagus. Where are you? When are you?

A new exhibit in the Rubenstein Arts Center uses AI to bring viewers into ancient Roman and Etruscan landscapes spanning 1300 years, from about 1000 BCE to 300 CE. (The field is Roman, the sarcophagus Etruscan.)

An AI-generated image of a summer meadow near Vulci (Viterbo, Italy). Preserved pollen evidence has revealed which plant species dominated these landscapes, and the prompts used to generate images like this one include lists of plant species.

Along one wall, screens show springtime landscapes representing ancient Rome. The written prompts AI used to create each image include detailed information on plant species found in each landscape. One titled “Sedges in shallow water of an ephemeral pond” mentions “sparse trees of alder (Alnus glutinosa), white willow (Salix alba), and white poplar (Populus alba), and few herbaceous plants.” You can view examples of the written prompts on the exhibit’s website, AI Landscapes – Rethinking the Past.

Models of pollen grains from different plant species. Real pollen grains are microscopic, but these magnified representations help show how different their shapes can be.

Historians know what plants were likely to be in these landscapes because of evidence from preserved pollen grains. Different species have distinct pollen shapes, which makes it possible to identify plants even centuries or millennia later.

Part of the exhibit uses AI and a camera to turn interactive prompts into ancient Roman scenes.

An interactive display near the front of the room has a camera pointed at props like building models, pillars, toy horses, and pieces of styrofoam. An AI model reinterprets the camera’s images to create hypothetical scenes from ancient Rome. “See how the columns get reinterpreted as statues?” says Felipe Infante de Castro, who helped program the AI. The AI attempts to add detail and backgrounds to simple props to create realistic scenes. “The only thing that we’re forcing,” he  says, “are essentially shapes—which it may or may not respect.” It may reinterpret a hand as a horse’s head, for instance, or a strangely shaped building.

The model is more precise with plants than buildings, says Augustus Wendell, Assistant Professor of the Practice in Art, Art History and Visual Studies and one of the exhibit designers. Latin names for plants are widely used in modern taxonomy, and the AI is likely to have encountered more plants in its training than ancient Roman architecture styles. The AI is a “generic model” asked to “draw on its presuppositions” about Roman buildings, says Felipe. It “wasn’t trained on specifically Roman landscapes…. It just tries its best to interpret it as such.” The results aren’t always completely authentic. “In the background,” Wendell says, “the city is often quite modern Tuscan, not at all ancient Roman.”

It’s interesting to see how the AI responds when you place unfamiliar objects in front of the camera, like your hand. Here, it tried to turn my hand into some sort of building.

“We can use an AI,” Felipe says, “to give us a representation of the past that is compatible with what we believe the past should look like.”

In another part of the exhibit, you can use an AI chatbot to talk to Pliny the Elder, a Roman scholar. Caitlin Childers, who helped design the exhibit, explains that the chatbot was trained on Pliny the Elder’s 37 books on natural history. When I asked Pliny what the chatbot was designed for, he told me, “I do not have the ability to access external articles or specific information beyond the knowledge I possess as Pliny the Elder up to the year 79 AD.”

He can give you information on plants and their uses in ancient Rome, but when I asked Pliny what his favorite plant was, he couldn’t decide. “I find it challenging to select a favorite plant among the vast array of flora that the Earth provides. Each plant contributes uniquely to the balance and beauty of nature.” According to Professor Maurizio Forte, “This AI chatbot can speak in English, French, Italian and also in Latin! So it is possible to formulate questions in Latin and requiring a response in Latin or ask a question in English and expect a reply in Latin as well.”

A virtual reality headset lets you see a three-dimensional model of an Etruscan sarcophagus. The real sarcophagus is encased in glass in the Villa Giulia Museum in Rome, but the virtual reality experience puts it right in front of you. The experimental VR-AI installation also allows viewers to ask questions to the sarcophagus out loud. The sarcophagus has a statue of a man and woman, but historians don’t know whose ashes are buried inside. “It’s not important how they look,” says Forte. “It’s important how they want to be.”

The sarcophagus would have been a “symbolic, aristocratic way to show power,” Forte explains. The design of the sarcophagus represents an intentional choice about how its owners wanted the world to see them after their death. “This is eternity,” Forte says. “This is forever.”

A display of quotes at the “Rethinking the Past” exhibit.

The exhibit, called “Rethinking the Past,” is on display at the Rubenstein Arts Center until May 24.

When Art and Science Meet as Equals

Artists and scientists in today’s world often exist in their own disciplinary silos. But the Laboratory Art in Practice Bass Connections team hopes to rewrite this narrative, by engaging Duke students from a range of disciplines in a 2-semester series of courses designed to join “the artist studio, the humanities seminar room, and the science lab bench.” Their work culminated in “re:process” – an exhibition of student artwork on Friday, April 28, in the lobby of the French Family Science Center. Rather than science simply engaging artistic practice for the sake of science, or vice versa, the purpose of these projects was to offer an alternate reality where “art and science meet as equals.”

The re:process exhibition

Liuren Yin, a junior double-majoring in Computer Science and Visual and Media Studies, developed an art project to focus on the experience of prosopagnosia, or face blindness. Individuals with this condition are unable to tell two distinct faces apart, including their own, often relying on body language, clothing, and the sound of a person’s voice to determine the identity of a person. Using her experience in computer science, she developed an algorithm that inputs distinct faces and outputs the way that these faces are perceived by someone who has prosopagnosia.

Yin’s project exploring prosopagnosia

Next to the computer and screen flashing between indistinguishable faces, she’s propped up a mirror for passers-by to look at themselves and contemplate the questions that inspired her to create this piece. Yin says that as she learned about prosopagnosia, where every face looks the same, she found herself wondering, “how am I different from a person that looks like me?” Interrogating the link between our physical appearance and our identity is at the root of Yin’s piece. Especially in an era where much of our identity exists online and appearance can be curated any way one wants, Yin considers this artistic piece especially timely. She writes in her program note that “my exposure to technologies such as artificial intelligence, generative algorithms, and augmented reality makes me think about the combination and conflict between human identity and these futuristic concepts.”

Eliza Henne, a junior majoring in Art History with a concentration in Museum Theory and Practice, focused more on the biological world in her project, which used a lavender plant in different forms to ask questions like “what is truthful, and what do we consider real?” By displaying a live plant, an illustration of a plant, and pressings from a plant, she invites viewers to consider how every rendition of a commonly used model organism in scientific experiments omits some information about the reality of the organism.

Junior Eliza Henne

For example, lavender pressings have materiality, but there’s no scent or dimension to the plant. A detailed illustration is able to capture even the way light illuminates the thin veins of the leaf, but is merely an illustration of a live being. The plant itself, which is conventionally real, can only further be seen in this sort of illustrative detail under a microscope or in a diagram.

In walking through the lobby of FFSC, where these projects and more are displayed, you’re surrounded by conventionally scientific materials, like circuit boards, wires, and petri dishes, which, in an unusual turn of events are being used for seemingly unscientific endeavors. These endeavors – illustrating the range of human emotion, showcasing behavioral patterns like overconsumption, or demonstrating the imperfection inherent to life – might at first glance feel more appropriate in an art museum or a performing arts stage.

But the students and faculty involved in this exhibition see that as the point. Maybe it isn’t so unnatural to build a bridge between the arts and the sciences – maybe, they are simply two sides of the same coin.

Post by Meghna Datta, Class of 2023

Anyone Can Be a “Math Person”

Dr. Francis Su, a mathematician and professor at Harvey Mudd College and the author of “Mathematics for Human Flourishing,” wants you to know that math can be beautiful. As these “infinitely quartered” squares show, 1/4 + 1/(42) + 1/(43) + … = 1/3. Image attribution: Tdadamemd, via Wikimedia Commons, under Creative Commons CC0 1.0 Universal Public Domain Dedication

Francis Su, Ph.D., visited Duke to talk about math. He began by talking about art.

Su, a mathematician and professor at Harvey Mudd College, displayed “Hope,” an 1886 painting by George Frederic Watts. He asked the audience to look at it, really look at it, and think about what’s happening in the painting. At first glance, it shows a blindfolded woman holding a wooden object. She seems to be in pain. But the more time we spend looking, the more we notice. We might notice that there’s a single star above her. We might notice that the wooden object is a lyre with only one string left attached. We might notice, too, that the woman is plucking that final string and straining to hear its music. 

If we take the time to explore the history of the painting, we might learn that Martin Luther King, Jr., talked about the same painting in a sermon. Su quoted a line from that sermon: “Who has not had to face the agony of blasted hopes and shattered dreams?” We find beauty in art, and often we find it relatable as well. Art invites us to look closer, to wonder, to feel, to ask questions, to imagine.

“Why,” Su asks then, “don’t we approach mathematics the way that we approach art?”

Professor Francis Su’s book, “Mathematics for Human Flourishing.”

Whether we consider ourselves “math people” or not, we rarely if ever hear mathematics discussed as an affirmation of human virtues and desires—love, beauty, truth, the “expectation of enchantment.” Su wants to change that. In his book “Mathematics for Human Flourishing” and in his talk at Duke, he envisions mathematics as beautiful, inclusive, and accessible to anyone.

Along with the painting “Hope,” Su’s first slide shows a quote by Simone Weil: “Every being cries out silently to be read differently.” Simone Weil, according to Su, was a “French religious mystic” and “widely revered philosopher,” but she also had a deep interest in math. Her older brother, André Weil, was an influential mathematician whose mathematical achievements often overshadowed her own. In a letter to a friend published posthumously in the book “Waiting for God,” Simone Weil wrote: “I did not mind having no visible successes, but what did grieve me was the idea of being excluded from that transcendent kingdom to which only the truly great have access and wherein truth abides.” Su sometimes wonders how Simone’s relationship to mathematics would have been different if André had not been her brother. Again, “Every being cries out silently to be read differently.” According to Su, when Simone Weil speaks of “reading” someone, she means “to interpret or make a judgment about them.”

Su has a friend, Christopher Jackson, who is an inmate in a high-security prison, serving a thirty-two year sentence for involvement in armed robberies as a teenager. When you think about people who do math, Su asks, would you think of Chris?  “We create societal norms about who does math,” and Chris doesn’t fit those norms. And yet he has been studying mathematics for years. After studying algebra, geometry, trigonometry, and calculus while in prison, he sent a letter to Su requesting help in furthering his mathematics education. The two men still correspond regularly, and Chris is now studying topology and other branches of mathematics.

“Every being cries out silently to be read differently.”

Why do math in the first place? Just as you can take your car to a mechanic without fully understanding how it works yourself, we might think of math as “only for the elite few” or perhaps as “a means to an end,” a tool “to make you ‘college and career ready.’” Su sees it differently. He views math in terms of human flourishing, “a wholeness of being and doing.” He points to three words from other languages: eudaemonia, a Greek term for “the overarching good in life”; shalom, a Hebrew word often used as a greeting and roughly translated as “peace”; and salaam, an Arabic word with a similar meaning to shalom.

The pattern on Romanesco broccoli is a fractal, common in both math and nature. Image credit: Francis Su

“What attracts me to music,” Su says, “isn’t playing scales over and over again.” But once you “experience a symphony,” you might see the value in playing scales. Can we learn to think of math the same way? Here, Su quoted mathematician Olga Taussky-Todd: “The yearning for and the satisfaction gained from mathematical insight brings the subject near to art.”

Beauty and awe probably aren’t the first words that come to mind when most of us think of math, but Su believes math can unlock “transcendent beauty.” He references a quote by C.S. Lewis: “the scent of a flower we have not found, the echo of a tune we have not heard, news from a country we have never yet visited.” That is what math at its best can do for us. It can help us see the big picture and realize that we’re “just scratching the surface of something really profound.”

“Math is not a single ‘ability,’” Su says. “In reality, math is a multi-dimensional set of virtues.” When learning or teaching math, we often focus more on skills like recalling facts and algorithms, factoring polynomials, or taking a derivative. But Su believes more important lessons are at play: virtues like persistence, creativity, a thirst for deep knowledge, and what he calls the expectation of enchantment. And, he says, employers are often much more interested in virtues than in skills. “If you want to be really practical about this—and I don’t, with mathematics, but if you do—then it’s actually the virtues that are more important than the skills,” Su says.

One basic human desire that Su believes math can help fulfill is the desire for truth, which, in turn, can help build virtues like a thirst for deep knowledge and the ability to think for oneself, which can help us figure out what’s true instead of just blindly trusting authorities. “Truth is under attack,” Su says. “Misinformation is everywhere.” Su wants to teach his students “to think, to be ‘that person who doesn’t need to look at the Ikea instructions.’” But he also wants them to view math as more than just a means to an end. “It’s my responsibility to help my students remember the beauty” in math and to understand that their dignity as human beings isn’t dependent on their grades.

Along with truth and beauty, he believes math can and should bring opportunities for exploration and discovery. “My role isn’t to be a teacher,” he says. “My role is to be a co-explorer.” He recalls his own excitement when he first saw a Menger cube, or Menger sponge, cut along its diagonal. The resulting cross-section is beautiful and, yes, enchanting. “What would it look like for classrooms to be like that?” During the pandemic, Su started adding more reflection-focused questions to his exams, questions like “Consider one mathematical idea from the course that you have found beautiful, and explain why it is beautiful to you.” Even more traditional math questions can be phrased in an “exploratory” way. Su gives the example of a question that asks students to make two rectangles, one with a bigger perimeter and one with a bigger area.

A visual representation of Nicomachus’s Theorem.
Image from Cmglee, via Wikimedia Commons, licensed under Creative Commons Attribution-Share Alike 3.0 Unported.

Another desire or virtue important in the field of mathematics is justice. Su wants math to be accessible to all, but not everyone has had positive experiences with math or feels like they belong there. As an analogy, Su talks about receiving dishes from a “secret menu” when visiting certain Chinese restaurants with friends who are fluent in Chinese. When he goes there on his own and requests the “secret menu,” however, he is sometimes turned away or told that he wouldn’t like those dishes. “Are people side-by-side in the same restaurant having different experiences” in math, too? “Who are you to say they do or don’t belong in mathematics?”

Even Su himself hasn’t always had wholly positive experiences in math. One of his professors once told him he didn’t “have what it takes to become a successful mathematician,” and he almost quit his Ph.D. program. Instead, he switched to a different advisor who had encouraged him to stick with it. Meanwhile, he surrounded himself with people who could remind him why he loved math. Math as a field can be competitive, but “if you think of mathematics as human flourishing… then that’s not a zero-sum game anymore.” 

In Su’s words, “we’re all math teachers” because “we all pass on attitudes about math to others.” He says studies show that parents can pass on “math anxiety” to their kids. But Su encourages people to “believe that you and everyone can flourish in mathematics.” Simone Weil. Christopher Jackson. And you. 

Who will you read differently?

Post by Sophie Cox, Class of 2025

Combining Up-Close Views of Science, Nature With the Magic of Light

Zinnia stamen by Thomas Barlow, Duke University

Thomas Barlow ’21 finds inspiration in small everyday things most people overlook: a craggy lichen growing on a tree, a dead insect, the light reflected by a pane of glass. Where we might see a flower, Barlow looks past the showy pink petals to the intricate parts tucked within.

The 20-year-old is a Duke student majoring in biology. By day, he takes classes and does research in a lab. But in his spare time, he likes to take up-close photographs using objects he finds outside or around the lab: peach pits, fireflies. But also pipettes, pencils.

A handheld laser pointer and flitting fireflies become streaks of light in this long-exposure image in Duke Forest. By Thomas Barlow.

Barlow got interested in photography in middle school, while playing around with his dad’s camera. His dad, a landscape architect, encouraged the hobby by enlisting him to take photos of public parks, gardens and playgrounds, which have been featured on various architects’ websites and in national publications such as Architecture Magazine. But “I always wanted to get closer, to see more,” Barlow said.

In high school he started taking pictures of still lifes. But he didn’t just throw flowers and fruit onto a backdrop and call it art. His compositions were a mishmash of insects and plants arranged with research gadgets: glass tubes, plastic rulers, syringes, or silicon wafers like those used for computer chips.

“I like pairing objects you would never find together normally,” Barlow said. “Removing them from their context and generating images with interesting textures and light.”

Sometimes his mother sends him treasures from her garden in Connecticut to photograph, like the pale green wings of a luna moth. But mostly he finds his subjects just steps from his dorm room door. It might be as easy as taking a walk through Duke Gardens or going for one of his regular runs in Duke Forest.

Having found, say, a flower bud or bumblebee, he then uses bits of glass, metal, mirrors and other shiny surfaces — “all objects that interact with light in some interesting way” – to highlight the interaction of light and color.

“I used to be really obsessed with dichroic mirrors,” pieces of glass that appear to change colors when viewed from different angles, Barlow said. “I thought they were beautiful objects. You can get so many colors and reflections out of it, just by looking at it in different ways.”

In one pair of images, the white, five-petaled flowers of a meadow anemone are juxtaposed against panels of frosted glass, a pipette, a mechanical pencil.

Another image pair shows moth wings. One is zoomed in to capture the fine details of the wing scales. The other zooms out to show them scattered willy-nilly around a shimmering pink circle of glass, like the remnants of a bat’s dinner plate.

Luna moth wings and wing scales with dichroic mirror, Thomas Barlow

For extreme close-ups, Barlow uses his Canon DSLR with a microscope objective mounted onto the front of a tube lens. Shooting this close to something so small isn’t just a matter of putting a bug or flower in front of the camera and taking a shot. To get every detail in focus, he takes multiple images of the same subject, moving the focal point each time. When he’s done he’s taken hundreds of pictures, each with a different part of the object in focus. Then he merges them all together.

At high magnification, Barlow’s flower close-ups reveal the curly yellow stamens of a zinnia flower, and the deep red pollen-producing parts of a tiger lily.

“I love that you can see the spikey pollen globules,” Barlow said.

Stomata and pollen on the underside of a tiger lily stamen, by Thomas Barlow

When he first got to Duke he was taking photos using a DIY setup in his dorm room. Then he asked some of the researchers and faculty he knew if there was anything photography-related he could do for their labs.

“I knew I was interested in nature photography and I wanted to practice it,” Barlow said.

One thing led to another, and before long he moved his setup to the Biological Sciences building on Science Drive, where he’s been photographing lichens for Daniele Armaleo and Jolanta Miadlikowska, both lichenologists.

“A lichen photo might not seem like anything special to an average person,” Barlow said. “But I think they’re really stunning.”

Powered by WordPress & Theme by Anders Norén