Although transforming growth factor-β (TGF-β) is both a suppressor and promoter of tumorigenesis, its contribution to early tumor suppression and staging remains largely unknown. In search of the mechanism of early tumor suppression, we identified the adaptor protein ELF, a β-spectrin from stem/progenitor cells committed to foregut lineage. ELF activates and modulates Smad4 activation of TGF-β to confer cell polarity, to maintain cell architecture, and to inhibit epithelial-to-mesenchymal transition. Analysis of development of colon cancer in (adult) elf^+/−/Smad4^+/−, elf^+/−, Smad4^+/−, and gut epithelial cells from elf^−/− mutant mouse embryos pinpoints the defect to hyperplasia/adenoma transition. Further analysis of the role of ELF in human colorectal cancer confirms reduced expression of ELF in Dukes' B1 stage tissues (P < 0.05) and of Smad4 in advanced colon cancers (P < 0.05). This study indicates that by modulating Smad 4, ELF has a key role in TGF-β signaling in the suppression of early colon cancer.