Recent studies have identified mutations in PHF8, an X-linked gene encoding a JmjC domain-containing protein, as a causal factor for X-linked mental retardation (XLMR) and cleft lip/cleft palate. However, the underlying mechanism is unknown. Here we show that PHF8 is a histone demethylase and coactivator for retinoic acid receptor (RAR). Although activities for both H3K4me3/2/1 and H3K9me2/1 demethylation were detected in cellular-based assays, recombinant PHF8 exhibited only H3K9me2/1 demethylase activity in vitro, suggesting that PHF8 is an H3K9me2/1 demethylase whose specificity may be modulated in vivo. Importantly, a mutant PHF8 (phenylalanine at position 279 to serine) identified in the XLMR patients is defective in enzymatic activity, indicating that the loss of histone demethylase activity is causally linked with the onset of disease. In addition, we show that PHF8 binds specifically to H3K4me3/2 peptides via an N-terminal PHD finger domain. Consistent with a role for PHF8 in neuronal differentiation, knockdown of PHF8 in mouse embryonic carcinoma P19 cells impairs RA-induced neuronal differentiation, whereas overexpression of the wild-type but not the F279S mutant PHF8 drives P19 cells toward neuronal differentiation. Furthermore, we show that PHF8 interacts with RARα and functions as a coactivator for RARα. Taken together, our results suggest that histone methylation modulated by PHF8 plays a critical role in neuronal differentiation.