Human cytomegalovirus (HCMV) is a widely prevalent human herpesvirus, which, after primary infection, persists in the host for life. In healthy individuals, the virus is well controlled by the HCMV-specific T cell response. A key feature of this persistence, in the face of a normally robust host immune response, is the establishment of viral latency. In contrast to lytic infection, which is characterised by extensive viral gene expression and virus production, long-term latency in cells of the myeloid lineage is characterised by highly restricted expression of viral genes, including UL138 and LUNA. Here we report that both UL138 and LUNA-specific T cells were detectable directly ex vivo in healthy HCMV seropositive subjects and that this response is principally CD4⁺ T cell mediated. These UL138-specific CD4⁺ T cells are able to mediate MHC class II restricted cytotoxicity and, importantly, show IFNγ effector function in the context of both lytic and latent infection. Furthermore, in contrast to CD4⁺ T cells specific to antigens expressed solely during lytic infection, both the UL138 and LUNA-specific CD4⁺ T cell responses included CD4⁺ T cells that secreted the immunosuppressive cytokine cIL-10. We also show that cIL-10 expressing CD4⁺ T-cells are directed against latently expressed US28 and UL111A. Taken together, our data show that latency-associated gene products of HCMV generate CD4⁺ T cell responses in vivo, which are able to elicit effector function in response to both lytic and latently infected cells. Importantly and in contrast to CD4⁺ T cell populations, which recognise antigens solely expressed during lytic infection, include a subset of cells that secrete the immunosuppressive cytokine cIL-10. This suggests that HCMV skews the T cell responses to latency-associated antigens to one that is overall suppressive in order to sustain latent carriage in vivo.