SwallowNet: Recurrent neural network detects and characterizes eating patterns

Passively detecting and counting the number of swallows in food intake enables accurate detection of eating episodes in free-living participants, and AIDS in characterizing eating episodes. On average, the more food consumed, the greater the number of swallows; and swallows have been shown to positively correlate with caloric intake. While passive sensing measures have shown promise in recent years, they are yet to be used reliably to detect eating, impeding the development of timely intervention delivery that change poor eating behavior. This paper presents a novel integrated wearable necklace that comprises two piezoelectric sensors vertically positioned around the neck, an inertial motion unit, and long short-term memory (LSTM) neural networks to detect and count swallows. A unique correlation of derivative features creates candidate swallows. To reduce the FPR features are extracted using symmetric and asymmetric windows surrounding each candidate swallow to feed into a Random Forest classifier. Independently, a LSTM network is trained from raw data using automated feature learning methods. In an in-lab study comprising confounding activities of 10 participants, results show a 3.34 RMSE of swallow count using LSTM, and a 76.07% average F-measure of swallows, outperforming the Random Forest classifier. This system thus shows promise in accurately detecting and characterizing eating patterns, enabling passive detection of swallow count, and paving the way for timely interventions to prevent problematic eating.