Distinct histone lysine methylation marks are involved in transcriptional repression linked to the formation and maintenance of facultative heterochromatin, although the underlying mechanisms remain unclear. We demonstrate that the malignant-brain-tumor (MBT) protein L3MBTL1 is in a complex with core histones, histone H1b, HP1γ, and Rb. The MBT domain is structurally related to protein domains that directly bind methylated histone residues. Consistent with this, we found that the L3MBTL1 MBT domains compact nucleosomal arrays dependent on mono- and dimethylation of histone H4 lysine 20 and of histone H1b lysine 26. The MBT domains bind at least two nucleosomes simultaneously, linking repression of transcription to recognition of different histone marks by L3MBTL1. Consistently, L3MBTL1 was found to negatively regulate the expression of a subset of genes regulated by E2F, a factor that interacts with Rb.