Single-cell RNA sequencing (scRNA-seq) is a powerful tool for defining cellular diversity in tumors, but its application toward dissecting mechanisms underlying immune-modulating therapies is scarce. We performed scRNA-seq analyses on immune and stromal populations from colorectal cancer patients, identifying specific macrophage and conventional dendritic cell (cDC) subsets as key mediators of cellular cross-talk in the tumor microenvironment. Defining comparable myeloid populations in mouse tumors enabled characterization of their response to myeloid-targeted immunotherapy. Treatment with anti-CSF1R preferentially depleted macrophages with an inflammatory signature but spared macrophage populations that in mouse and human expresses pro-angiogenic/tumorigenic genes. Treatment with a CD40 agonist antibody preferentially activated a cDC population and increased Bhlhe40⁺ Th1-like cells and CD8⁺ memory T cells. Our comprehensive analysis of key myeloid subsets in human and mouse identifies critical cellular interactions regulating tumor immunity and defines mechanisms underlying myeloid-targeted immunotherapies currently undergoing clinical testing.