The gut microbiome has been linked to fear extinction learning in animal models. Here, we aimed to explore the gut microbiome and memory domains according to obesity status. A specific microbiome profile associated with short-term memory, working memory, and the volume of the hippocampus and frontal regions of the brain differentially in human subjects with and without obesity. Plasma and fecal levels of aromatic amino acids, their catabolites, and vegetable-derived compounds were longitudinally associated with short-term and working memory. Functionally, microbiota transplantation from human subjects with obesity led to decreased memory scores in mice, aligning this trait from humans with that of recipient mice. RNA sequencing of the medial prefrontal cortex of mice revealed that short-term memory associated with aromatic amino acid pathways, inflammatory genes, and clusters of bacterial species. These results highlight the potential therapeutic value of targeting the gut microbiota for memory impairment, specifically in subjects with obesity.