The clinical successes in immunotherapy have been both astounding and at the same time unsatisfactory. Countless patients with varied tumor types have seen pronounced clinical response with immunotherapeutic intervention; however, many more patients have experienced minimal or no clinical benefit when provided the same treatment. As technology has advanced, so has the understanding of the complexity and diversity of the immune context of the tumor microenvironment and its influence on response to therapy. It has been possible to identify different subclasses of immune environment that have an influence on tumor initiation and response and therapy; by parsing the unique classes and subclasses of tumor immune microenvironment (TIME) that exist within a patient’s tumor, the ability to predict and guide immunotherapeutic responsiveness will improve, and new therapeutic targets will be revealed.