Energy balance is one component of weight management, but passive objective measures of caloric intake are non-existent. Given the recent success of actigraphy as a passive objective measure of the physical activity construct that relieves participants of the burden of biased self-report, computer scientists and engineers are aiming to find a passive objective measure of caloric intake. Passive sensing food intake systems have failed to go beyond the lab and into behavioral research in part due to low adherence to wearing passive monitoring systems. While system accuracy and battery lifetime are sine qua non to a successfully deployed technology, they come second to adherence, since a system does nothing if it remains unused. This paper focuses on adherence as affected by: 1) perceived data privacy; 2) stigma of wearing devices; 3) comfort. These factors highlight new challenges surrounding participant informed consent and Institutional Review Board (IRB) risk assessment. The wearables examined include neck- and wrist-worn sensors, and video camera-based systems. Findings support the potential for adherence using wrist- and shoulder-based video cameras, and personalized style-conscious neck-worn sensors. The feasibility of detecting fine-grained eating gestures to validate the machine learning models is shown, improving the potential of translation of this technology.