Decoding baby cries with the help of AI

While there are more than 7,000 languages in the world, the way newborn babies express their needs is universal. They cry constantly to communicate their needs and feelings. In fact, babies spend between one and three hours a day whimpering and sobbing, although not all crying is the same. Science has been trying to decipher this ancient communication system for some time, looking for patterns of behaviour and dynamics that explain what exactly each cry means.

A recent study from the Hospital Clínic de Barcelona has taken a step forward in the interpretation of crying by classifying newborn crying based on acoustic, neurophysiological, facial expression and body movement data. For example, they found that crying due to hunger is constant, rhythmic and of short duration. Distress calls, on the other hand, have few pauses, are erratic and tend to be high-pitched. These findings, say the authors, provide further evidence to help interpret crying at home in healthy babies, but also have clinical potential: analysis of crying and associated physiological variables can act as a clinical biomarker and be used to identify or assess the baby's health status.

The Swiss start-up Zoundream played a key role in this study. Since 2019, they have been developing a custom deep learning-based solution called AMSI (Acoustic Multi-Stage Interpreter) to analyse crying and explain the needs of newborns. Thanks to extensive testing over several years, Zoundream has collected the world's most comprehensive data on newborn crying, with over 100,000 hours of recordings.

Their remote-based solution processes the sound into a series of internal deep learning models to extract the meaning of a baby's cry – think of it in terms of understanding letters, then words, then meaningful sentences. In clinical trials at the Hospital Clínic de Barcelona, the AMSI technology achieved an amazing 92% accuracy rate.

Currently, the technology can translate a baby's cry into five types: hungry, sleepy, fussy, irritated or gastrointestinal distress. To make the technology available to desperate parents around the world, the Basel-based company is working with, among others, the manufacturer behind the well-known baby brand Maxi Cosi. In autumn 2023, the Maxi Cosi See Pro baby monitor was launched, the first baby monitor to use AI technology on a large scale.

The company has made headlines with its presence at CES in Las Vegas (check out the BBC News video report featuring Zoundream), as part of the Swiss National Technology Team’s roadshow in Silicon Valley in early April 2024, and at the Viva Technology conference in Paris just two weeks ago.