Checkpoint blockade has formally demonstrated that reactivating anti-tumor immune responses can regress tumors. However, this only occurs in a fraction of patients. Incorporating these therapies in more powerful combinations is thus a logical next step. Here, we review functional roles of immune checkpoints and molecular determinants of checkpoint-blockade clinical activity. Limited-size T cell-infiltrated tumors, differing substantially from "self," generally respond to checkpoint blockade. Therefore, we propose that reducing tumor burden and increasing tumor immunogenicity are key factors to improve immunotherapy. Lastly, we outline criteria to select proper immunotherapy combination partners and highlight the importance of activity biomarkers for timely treatment optimization.