Many communicative situations present interlocutors with the opportunity to use multiple modalities to establish shared perspectives on conversational referents, a process known as grounding. In the current study, we use a card-matching task to examine how conversational grounding in younger and older adults is influenced both by direct visual access to the conversation partner and by the relative abstractness of task referents. On the whole, mutual visibility failed to moderate more general age-related differences in task performance; overall, dialogues between older adults involved more talk and more explicit negotiation of perspectives. A detailed examination of gaze behavior revealed that use of the visual channel in both older and younger adults was greatly constrained by task-related demands, such as the need to attend to one's cards, particularly when they were more difficult to describe. Even so, individuals in both age groups gazed at their partners more frequently and displayed more mutual gaze during extended negotiations of referential identity. Finally, older adults' rates of partner-directed gaze were strongly correlated with their short-term memory span, suggesting that cognitive capacity may be an important factor in shaping older adults' multimodal interactions with others.