There are three different types of adipose tissue (AT)—brown, white, and beige—that differ with stage of development, species, and anatomical location. Of these, brown AT (BAT) is the least abundant but has the greatest potential impact on energy balance. BAT is capable of rapidly producing large amounts of heat through activation of the unique uncoupling protein 1 (UCP1) located within the inner mitochondrial membrane. White AT is an endocrine organ and site of lipid storage, whereas beige AT is primarily white but contains some cells that possess UCP1. BAT first appears in the fetus around mid-gestation and is then gradually lost through childhood, adolescence, and adulthood. We focus on the interrelationships between adipocyte classification, anatomical location, and impact of diet in early life together with the extent to which fat development differs between the major species examined. Ultimately, novel dietary interventions designed to reactivate BAT could be possible.