Machine learning and computer vision allow study of animal behavior without markers

With a new markerless method it is now possible to track the gaze and fine-scaled behaviors of every individual bird and how that animal moves in the space with others. A research team from the Cluster of Excellence Center for the Advanced Study of Collective Behavior (CASCB) at the University of Konstanz developed a dataset to advance behavioral research.

Researchers are still puzzling over how animal collectives behave, but recent advances in machine learning and computer vision are revolutionizing the possibilities of studying animal behavior. Complex behaviors, like social learning or collective vigilance can be deciphered with new techniques.

An interdisciplinary research team from the Cluster of Excellence Center for the Advanced Study of Collective Behavior (CASCB) at the University of Konstanz and the Max Planck Institute of Animal Behavior has now succeeded in developing a novel markerless method to track bird postures in 3D just by using video recordings.

It is no longer necessary to attach position or movement transmitters to the animals. With this method called 3D-POP (3D posture of pigeons) it is possible to record a group of pigeons and identify the gaze and fine-scaled behaviors of every individual bird and how that animal moves in the space with others. “With the dataset, researchers can study collective behavior of birds by just using at least two video cameras, even in the wild,” says Alex Chan, Ph.D. student at the CASCB.

The dataset was released at the Conference on Computer Vision and Pattern Recognition (CVPR) in June 2023 and available via open access so that it can be reused by other researchers. The researchers Hemal Naik and Alex Chan see two potential application areas: Scientists working with pigeons can directly use the dataset. With at least two cameras they can study the behavior of multiple freely moving pigeons. The annotation method can be used with other birds or even other animals so that researchers can soon decipher the behavior of other animals.