Previous studies report conflicting data on outcomes of pregnancy-associated breast cancer (PABC). Our aim was to examine the effect of a postpartum diagnosis on maternal prognosis in a young women’s breast cancer cohort. We conducted a retrospective cohort study of women age ≤45 years, diagnosed with breast cancer (n = 619) during 1981–2011 at the University of Colorado Hospital and The Shaw Cancer Center in Edwards, CO. Breast cancer cases were grouped according to time between giving birth and diagnosis: nulliparous (n = 125), pregnant (n = 24), < 5 years postpartum (n = 136), >5—<10 postpartum (n = 130), and ≥10 years postpartum (n = 147), to examine the clinicopathologic features and the risk of distance recurrence and death. Cases diagnosed after pregnancy, but within five-years postpartum, had an approximate three fold increased risk of distant recurrence (HR 2.80, 95 % CI: 1.12–6.57) and death (HR 2.65, 95 % CI: 1.09–6.42) compared to nulliparous cases. Postpartum cases diagnosed within five years of last childbirth demonstrated a higher five-year distant recurrence probability (31.1 %) and a markedly lower five-year overall survival probability (65.8 %) compared to nulliparous cases (14.8 and 98.0 %, respectively). A diagnosis of breast cancer during the first five-years postpartum confers poorer maternal prognoses after adjustment for biologic subtype, stage, and year of diagnosis. We propose that the definition of PABC should include cases diagnosed up to at least five-years postpartum to better delineate the increased risk imparted by a postpartum diagnosis. Based on emerging preclinical and epidemiologic data, we propose that pregnant and postpartum cases be researched as distinct subsets of PABC to clarify the risk imparted by pregnancy and the events subsequent to pregnancy, such as breast involution, on breast cancer. Further, we highlight the importance of postpartum breast cancer as an area for further research to reduce the increased metastatic potential and mortality of PABC.