Alzheimer's disease and other neurodegenerative disorders of aging are characterized by clinical and pathological features that are relatively specific to humans. To obtain greater insight into how brain aging has evolved, we compared age-related gene expression changes in the cortex of humans, rhesus macaques, and mice on a genome-wide scale. A small subset of gene expression changes are conserved in all three species, including robust age-dependent upregulation of the neuroprotective gene apolipoprotein D (APOD) and downregulation of the synaptic cAMP signaling gene calcium/calmodulin-dependent protein kinase IV (CAMK4). However, analysis of gene ontology and cell type localization shows that humans and rhesus macaques have diverged from mice due to a dramatic increase in age-dependent repression of neuronal genes. Many of these age-regulated neuronal genes are associated with synaptic function. Notably, genes associated with GABA-ergic inhibitory function are robustly age-downregulated in humans but not in mice at the level of both mRNA and protein. Gene downregulation was not associated with overall neuronal or synaptic loss. Thus, repression of neuronal gene expression is a prominent and recently evolved feature of brain aging in humans and rhesus macaques that may alter neural networks and contribute to age-related cognitive changes.