If you look at a photograph of leopards, would you be able to tell which ones are related based on their spots?
Unless you’re a leopard expert, the answer is probably no, says Tanya Berger-Wolf, director of the Translational Data Analytics Institute (TDAI) at Ohio State University. But, she says, computers can.
Berger-Wolf and his team are pioneering a new field of study called imageomics. As the name suggests, imageomics uses machine learning to extract biological data from photos and videos of living organisms. Berger-Wolf and her team recently began collaborating with researchers studying leopards in India to compare the spotting patterns of mothers and offspring using algorithms.
“Images have become the most abundant source of information, and we also have the technology. We have machine learning computer vision,” says Berger-Wolf. She compares this technology to the invention of the microscope, giving scientists a completely different way to observe wildlife.
Building on TDAI’s open-source platform called Wildbook, which helps wildlife researchers gather and analyze photos, the team is now focusing on generative AI approaches. These programs use existing content to generate meaningful data. In this case, they’re trying to analyze crowdsourced images to make calculable biological traits humans may naturally miss, like the curvature of a fish’s fin — or a leopard’s spots. The algorithms analyze images of leopards publicly available online, from social media to digitized museum collections.
Put simply, algorithms “quantify similarity,” she says. The goal is to help wildlife researchers overcome a problem of lack of data and ultimately better protect animals at risk of extinction.
Conservationists and other wildlife researchers currently face a lack of data – it is tedious, expensive and time-consuming for people to spend time in the field monitoring animals. As a result of these challenges, 20,054 species on the International Union for Conservation of Nature (IUCN) Red List of Threatened Species are labeled as “Data Deficient”, meaning there is no enough information to properly assess its risk of extinction. As Berger-Wolf summarizes, “Biologists make decisions without having good data on what we are losing and how fast.”
The platform started with supervised learning – Berger-Wolf says the computer uses “simpler than Siri” algorithms to count the number of animals in the image, as well as where it was taken and when, which could help measures such as population counts. Not only can AI do this at a much lower cost than hiring people, but also at a faster rate. In August 2021, the platform analyzed 17 million images automatically.
There are also obstacles that only a computer can seem to overcome. “Humans aren’t the best at determining what the informative aspect is,” she says, noting how biased humans are in how we view nature, focusing primarily on facial features. Instead, the AI can look for features that humans would likely miss, like the color range of a tiger moth’s wings. A March 2022 study found that the human eye could not distinguish polymorphic male genotypes of the tiger wood moth – but the butterflies’ vision patterns sensitive to ultraviolet light could.
“That’s where all the real innovation is in all of this,” says Berger-Wolf. The team implements algorithms that create pixel values of patterned animals, like leopards, zebras and whale sharks, and analyze the hotspots where the pixel values change the most – it’s like compare fingerprints. Having these fingerprints means researchers can track animals non-invasively and without GPS collars, count them to estimate population sizes, understand migration patterns, and more.
As Berger-Wolf points out, population size is the most fundamental measure of a species’ well-being. The platform digitized 11,000 images of whale sharks to create hotspots and help researchers identify individual whale sharks and track their movements, leading to updated information on their population size. This new data prompted the IUCN to change the whale shark’s conservation status from “vulnerable” to “endangered” in 2016.
There are also algorithms using facial recognition for primates and cats, which have been found to be around 90% accurate, compared to around 42% for humans.
Generative AI is still a burgeoning field in wildlife conservation, but Berger-Wolf is hopeful. For now, the team is cleaning up the preliminary leopard hotspot data to make sure the results aren’t data artifacts — or flaws — and are real, biologically meaningful information. If meaningful, the data could teach researchers how species respond to changing habitats and climates and show us where humans can step in to help.