In men, androgens are critical for the acquisition and maintenance of bone mass in both the cortical and cancellous bone compartment. Male mice with targeted deletion of the androgen receptor (AR) in mature osteoblasts or osteocytes have lower cancellous bone mass, but no cortical bone phenotype. We have investigated the possibility that the effects of androgens on the cortical compartment result from AR signaling in osteoprogenitors or cells of the osteoclast lineage; or via estrogen receptor alpha (ERα) signaling in either or both of these two cell types upon conversion of testosterone to estradiol. To this end, we generated mice with targeted deletion of an AR or an ERα allele in the mesenchymal (AR^f/y;Prx1‐Cre or ERα^f/f;Osx1‐Cre) or myeloid cell lineage (AR^f/y;LysM‐Cre or ERα^f/f;LysM‐Cre) and their descendants. Male AR^f/y;Prx1‐Cre mice exhibited decreased bone volume and trabecular number, and increased osteoclast number in the cancellous compartment. Moreover, they did not undergo the loss of cancellous bone volume and trabecular number caused by orchidectomy (ORX) in their littermate controls. In contrast, AR^f/y;LysM‐Cre, ERα^f/f;Osx1‐Cre, or ERα^f/f;LysM‐Cre mice had no cancellous bone phenotype at baseline and lost the same amount of cancellous bone as their controls following ORX. Most unexpectedly, adult males of all four models had no discernible cortical bone phenotype at baseline, and lost the same amount of cortical bone as their littermate controls after ORX. Recapitulation of the effects of ORX by AR deletion only in the AR^f/y;Prx1‐Cre mice indicates that the effects of androgens on cancellous bone result from AR signaling in osteoblasts—not on osteoclasts or via aromatization. The effects of androgens on cortical bone mass, on the other hand, do not require AR or ERα signaling in any cell type across the osteoblast or osteoclast differentiation lineage. Therefore, androgens must exert their effects indirectly by actions on some other cell type(s) or tissue(s). © 2015 American Society for Bone and Mineral Research.