ES cells are defined as self-renewing, pluripotent cell lines derived from early embryos. Cultures of ES cells are also characterized by the expression of certain markers thought to represent the pluripotent state. However, despite the widespread expression of key markers such as Oct4 and the appearance of a characteristic undifferentiated morphology, functional ES cells may represent only a small fraction of the cultures grown under self-renewing conditions. Thus phenotypically “undifferentiated” cells may consist of a heterogeneous population of functionally distinct cell types. Here we use a transgenic allele designed to detect low level transcription in the primitive endoderm lineage as a tool to identify an immediate early endoderm-like ES cell state. This reporter employs a tandem array of internal ribosomal entry sites to drive translation of an enhanced Yellow Fluorescent Protein (Venus) from the transcript that normally encodes for the early endodermal marker Hex. Expression of this Venus transgene reports on single cells with low Hex transcript levels and reveals the existence of distinct populations of Oct4 positive undifferentiated ES cells. One of these cells types, characterized by both the expression of the Venus transgene and the ES cells marker SSEA-1 (V⁺S⁺), appears to represent an early step in primitive endoderm specification. We show that the fraction of cells present within this state is influenced by factors that both promote and suppress primitive endoderm differentiation, but conditions that support ES cell self-renewal prevent their progression into differentiation and support an equilibrium between this state and at least one other that resembles the Nanog positive inner cell mass of the mammalian blastocysts. Interestingly, while these subpopulations are equivalently and clonally interconvertible under self-renewing conditions, when induced to differentiate both in vivo and in vitro they exhibit different behaviours. Most strikingly when introduced back into morulae or blastocysts, the V⁺S⁺ population is not effective at contributing to the epiblast and can contribute to the extra-embryonic visceral and parietal endoderm, while the V⁻S⁺ population generates high contribution chimeras. Taken together our data support a model in which ES cell culture has trapped a set of interconvertible cell states reminiscent of the early stages in blastocyst differentiation that may exist only transiently in the early embryo.