Gag proteins direct the process of retroviral particle assembly and form the major protein constituents of the viral core. The matrix region of the HIV-1 Gag polyprotein plays a critical role in the transport of Gag to the plasma membrane assembly site. Recent evidence indicates that Gag trafficking to late endosomal compartments, including multivesicular bodies, occurs prior to viral particle budding from the plasma membrane. Here we demonstrate that the matrix region of HIV-1 Gag interacts directly with the δ subunit of the AP-3 complex, and that this interaction plays an important functional role in particle assembly. Disruption of this interaction eliminated Gag trafficking to multivesicular bodies and diminished HIV particle formation. These studies illuminate an early step in retroviral particle assembly and provide evidence that the trafficking of Gag to late endosomes is part of a productive particle assembly pathway.