Alzheimer's disease constitutes a rising threat to public health. Despite extensive research in cellular and animal models, identifying the pathogenic agent present in the human brain and showing that it confers key features of Alzheimer's disease has not been achieved. We extracted soluble amyloid-β protein (Aβ) oligomers directly from the cerebral cortex of subjects with Alzheimer's disease. The oligomers potently inhibited long-term potentiation (LTP), enhanced long-term depression (LTD) and reduced dendritic spine density in normal rodent hippocampus. Soluble Aβ from Alzheimer's disease brain also disrupted the memory of a learned behavior in normal rats. These various effects were specifically attributable to Aβ dimers. Mechanistically, metabotropic glutamate receptors were required for the LTD enhancement, and N-methyl D-aspartate receptors were required for the spine loss. Co-administering antibodies to the Aβ N-terminus prevented the LTP and LTD deficits, whereas antibodies to the midregion or C-terminus were less effective. Insoluble amyloid plaque cores from Alzheimer's disease cortex did not impair LTP unless they were first solubilized to release Aβ dimers, suggesting that plaque cores are largely inactive but sequester Aβ dimers that are synaptotoxic. We conclude that soluble Aβ oligomers extracted from Alzheimer's disease brains potently impair synapse structure and function and that dimers are the smallest synaptotoxic species.