Original source (on modern site)
It is a cliche in language industry marketing: Translation helps to break the language barrier. Now, a nonprofit is building on that truism, with the goal of decoding "non-human" communication. An NPR story on "human-like vocalizations" of the Gelada monkey inspired Aza Raskin of Mozilla Labs to explore the idea of understanding animals. In 2017, he and Twitter's Britt Selvitelle founded the Earth Species Project (ESP). Now-CEO Katie Zacarian joined as a cofounder in 2020. As of 2023, the staff of 12 includes an AI research team of seven, and the nonprofit counts more than 40 biologists and institutions among their partners. More than 15 universities and ecology-related nonprofits contribute data to ESP. ESP's ultimate goal is to use an informed understanding of animal communication to enhance conservation efforts. According to the organization's technical roadmap, machine learning (ML) will be used to explore patterns in data, perform experiments, and decode the conditions for producing specific signals. One of the first priorities for ESP is establishing a method for transcribing and annotating animal communication, which can take form in a variety of modules, such as acoustic, visual, and tactile. External context can also impact the meaning of expressions. A February 2024 publication has introduced the Inter-Species Phonetic Alphabet for Transcribing, also known as ISPA. "By representing animal sounds with text, we effectively treat them as a 'foreign language,' and we show that established human language ML paradigms and models, such as language models, can be successfully applied to improve performance," the appendix states. ESP is managing several active projects, each with a more specific focus, such as determining whether "dialect patterns" exist within an endangered population of beluga whales; using self-supervised learning to identify repeated behavioral patterns, as recorded by tags providing audio and video footage from an animal's perspective; and understanding the role of acoustic communication in groups of crows. Researchers are currently testing interactive playback experiments, generating calls for a number of species, including birds and humpback whales. The Slator Pro Guide presents 10 new and impactful ways that LLMs can be used to enhance translation workflows. "Given that we may not understand the meaning of the novel vocalizations being generated by the model, there are important ethical considerations related to potentially interfering with animals and their culture," ESP wrote of the project. "For this reason, we are beginning this research only with captive populations and working exclusively with scientists who follow strict ethical protocols." One completed project produced the Benchmark of Animal Sounds, or BEANS — benchmarks for "bioacoustics tasks" across multiple species with limited data, which "mirror the standard benchmarks which have been developed for human vision and language." ESP's practically headline-ready mission has sparked steady media interest, including Raskin's appearance on The Joe Rogan Experience podcast in December 2023. That month ESP also announced it has received USD 1.2m in funding from the Paul G. Allen Family Foundation.Cross-Species Challenges
Slator Pro Guide: Translation AI