T cell dysfunction in solid tumors results from multiple mechanisms. Altered signaling pathways in tumor cells help produce a suppressive tumor microenvironment enriched for inhibitory cells, posing a major obstacle for cancer immunity. Metabolic constraints to cell function and survival shape tumor progression and immune cell function. In the face of persistent antigen, chronic T cell receptor signaling drives T lymphocytes to a functionally exhausted state. Here we discuss how the tumor and its microenvironment influences T cell trafficking and function with a focus on melanoma, and pancreatic and ovarian cancer, and discuss how scientific advances may help overcome these hurdles.