Accumulation of AMPA receptors at synapses is a fundamental feature of glutamatergic synaptic transmission. Stargazin, a member of the TARP family, is an AMPAR auxiliary subunit allowing interaction of the receptor with scaffold proteins of the postsynaptic density, such as PSD-95. How PSD-95 and Stargazin regulate AMPAR number in synaptic membranes remains elusive. We show, using single quantum dot and FRAP imaging in live hippocampal neurons, that exchange of AMPAR by lateral diffusion between extrasynaptic and synaptic sites mostly depends on the interaction of Stargazin with PSD-95 and not upon the GluR2 AMPAR subunit C terminus. Disruption of interactions between Stargazin and PSD-95 strongly increases AMPAR surface diffusion, preventing AMPAR accumulation at postsynaptic sites. Furthermore, AMPARs and Stargazin diffuse as complexes in and out synapses. These results propose a model in which the Stargazin-PSD-95 interaction plays a key role to trap and transiently stabilize diffusing AMPARs in the postsynaptic density.