Efficient memory formation relies on the establishment of functional hippocampal circuits. It has been proposed that synaptic connections are refined by neural activity to form functional brain circuitry. However, it is not known whether and how hippocampal connections are refined by neural activity in vivo. Using a mouse genetic system in which restricted populations of neurons in the hippocampal circuit are inactivated, we show that inactive axons are eliminated after they develop through a competition with active axons. Remarkably, in the dentate gyrus, which undergoes neurogenesis throughout life, axon refinement is achieved by a competition between mature and young neurons. These results demonstrate that activity-dependent competition plays multiple roles in the establishment of functional memory circuits in vivo.