Electrical Engineer Creates AI System to Track Elephants Without Tags

November 21, 2022 by Ingrid Fadelli

EE professor Dr. Karen Panetta recently leveraged drones and machine learning to bolster elephant conservation efforts. All About Circuits interviewed Dr. Panetta to learn the details.

Scientists and engineers have been using technologies like artificial intelligence (AI) and sensors to tackle the global environmental crisis—for instance, by monitoring environments and endangered species in real-time.

Dr. Karen Panetta, professor at Tufts University, member of the National Academy of Inventors (NAI), and IEEE fellow, recently developed a drone and AI-based tool that could help protect endangered elephants in the wild. 


Dr. Karen Panetta

Karen Panetta at the NAI Conference speaking on the Next Generation of Innovation panel. Image used courtesy of NAI


Dr. Panetta’s technology could help to safeguard the health of three surviving elephant species from afar: the African bush or Savanna elephant (Loxodonta Africana), the African forest elephant (Loxodonta Cyclotis), and the Asian elephant (Elaphas Maximus). These species are now considered endangered due to ivory poaching and the dramatic depletion of their natural habitats. 


Combining Drones, Sensors, and AI

Dr. Panetta's system employs a machine-learning algorithm to analyze images taken by drones. This algorithm is unique in that it uses the shapes of elephant shadows on the ground to spot the animals in the wild. These elephants can then be tracked and monitored from a distance without invading their natural habitat.

“Most AI systems are trained with handcrafted/annotated datasets that are not representative of the real world,” Dr. Panetta said. “For instance, pictures of elephants were taken face on or from the animal's side. When working with drones, the angle is from the top, looking over the animals. Most day images are also taken from above, creating shadows around the animals. Traditional Al models presented with these kinds of images would fail miserably, because they were not trained for these scenarios. No one ever considered detecting or training AI to identify animal species based on the kind of shadow it cast.”


The architecture developed by Dr. Panetta

Process flow behind Dr. Panetta’s identification and tracking technology. Image used courtesy of Panetta Visualization, Sensing and Simulation Laboratory


Another challenge was that elephants tend to associate the sound of drones with bees, so they can become agitated in their presence. 

“We needed to find a stressless approach that did not change elephants’ natural behaviors,” Dr. Panetta said. “Ultimately, we adopted a ‘do no harm’ approach, which means no tagging the animal and only using sensors that capture images from safe distances in both day and night settings.”


Safely Monitoring Elephants From Afar

The most common method of monitoring endangered elephants is to attach electronic tags. To do this, the animals must be sedated, which can compromise their health while also posing risks for intervening veterinarians.

“My team used AI to identify individual animals and drone technology to capture images in low lighting using different kinds of sensors, such as thermal and regular RGB cameras,” Dr. Panetta explained.

Dr. Panetta's system is far safer than tagging methods because it limits direct human-animal interactions to the times when they are strictly necessary. The animals’ health can be closely examined using images gathered by drones and other techniques created by Dr. Panetta’s lab. For instance, a series of "super-resolution algorithms" allows users to zoom in on the elephants and closely inspect them to detect parasites, wounds, and even dental issues.


Dr. Panetta's technology

Dr. Panetta's technology allows researchers to track wild elephants from afar. Images used courtesy of Panetta Visualization, Sensing and Simulation Laboratory


While talking to elephant conservationists, Dr. Panetta realized the importance of identifying potential threats like poachers early on.

“Detecting trucks/people with guns once they reach the elephants is not sufficient for conservation rangers to respond, because of the great distances they need to travel to get to the scene,” Dr. Panetta said. “It's easier to identify trucks/people in remote landscapes than it is to detect elephants, so patrolling a drone in surrounding areas provides earlier warning. Furthermore, poachers seeing the drones try to avoid being seen in the patrolled areas, which is why night vision/thermal sensor operation was so important.”


New Routes for Animal Conservation

While Dr. Panetta’s system was originally designed to track elephants, it could be easily used to safeguard other species. Dr. Panetta inadvertently realized this during initial evaluations after the system misclassified a rhino as an elephant.

“The rangers were very excited about it, because rhinos are also endangered and need protection,” Dr. Panetta noted. “Recently, we also adapted this technology for white tail deer conservation in the U.S., where deer can cause crop damage and Lyme disease.”

In the U.S., deer overpopulate some regions and end up starving to death, particularly when there is little control over their reproduction and no predators keeping their population under control. Researchers intervening in these regions generally sedate the animals, inspect them for Lyme disease, measure them, and tag them.

“We advanced this work by determining all the dimensions of deers using our pose estimation techniques,” Dr. Panetta said. “It was important for us to effectively detect the same animal in different scenes, even when tags were not visible. Interestingly, when we tried using our facial recognition tools for this, we mostly saw the backsides of deer in motion capture cameras, so I now joke about developing the world’s first ‘butt’ recognition system.” 


The "Eye in AI" for Conservationists

Dr. Panetta's technology has already proven to be highly promising for monitoring the activities and health of animal species in the wild. While it has primarily been used to track endangered elephants, it may soon be applied to other environmental problems.

“The data we collect can inform governments and the general public about environmental issues and help proactively initiate conservation efforts. It can also unveil stresses on communities, such as elephants destroying crops, and ways to mitigate these challenges to provide famers with alternatives to killing the animals.”

Dr. Panetta’s technology could soon be used by conservationists in both Africa and Asia to track endangered elephants, protect them from threats in their environment, assess their health from a safe distance, and learn more about their behavior.

“AI could also help identify specific actions of the animal and reduce the cost and labor associated with all kinds of video analysis, where a human must watch the entire video or be monitoring the live video/image feeds,” Dr. Panetta added. “AI can do the watching for us. That’s why I often say that our system is ‘The Eye in AI’.” 



About Dr. Karen Panetta

Dr. Karen Panetta is an electrical engineering professor at Tufts University, a member of the National Academy of Inventors (NAI), and an IEEE fellow. Dr. Panetta holds a BSc. in computer engineering from Boston University and an MSc. and PhD. in electrical engineering from Northeastern University. 

While studying, she worked as a computer architect at Digital Equipment Corporation in Massachusetts, where she co-invented the first complete digital twin. She joined Tufts in 1994 and became the first woman in her department to receive tenure. Recently, she worked with Tufts Cummings School of Veterinary Medicine to create a non-invasive tracking and monitoring system for endangered elephants.