Extensive evidence indicates that endocannabinoids modulate cognitive processes in animal models and human subjects. However, the results of endocannabinoid system manipulations on cognition have been contradictory. As for anxiety behavior, a duality has indeed emerged with regard to cannabinoid effects on memory for emotional experiences. Here we summarize findings describing cannabinoid effects on memory acquisition, consolidation, retrieval and extinction. Additionally, we review findings showing how the endocannabinoid system modulates memory function differentially, depending on the level of stress and arousal associated with the experimental context. Based on the evidence reviewed here, we propose that the endocannabinoid system is an emotional buffer that moderates the effects of environmental context and stress on cognitive processes.