While most collaboration technologies are concerned with supporting particular tasks such as workflows or meetings, many work groups do not have the teamwork skills essential to effective collaboration. One way to improve teamwork is to provide dynamic feedback generated by automated analyses of behavior, such as language use. Such feedback can lead members to reflect on and subsequently improve their collaborative behavior, but might also distract from the task at hand. We have experimented with GroupMeter - A chat- based system that presents visual feedback on team members' language use. Feedback on proportion of agreement words and overall word count was presented using two different designs. When receiving feedback, teams in our study expressed more agreement in their conversations and reported greater focus on language use as compared to when not receiving feedback. This suggests that automated, realtime linguistic feedback can elicit behavioral changes, offering opportunities for future research.