Janus kinases (JAKs) are key effectors in controlling immune responses and maintaining hematopoiesis. SOCS3 (suppressor of cytokine signaling-3) is a major regulator of JAK signaling and here we investigate the molecular basis of its mechanism of action. We found that SOCS3 bound and directly inhibited the catalytic domains of JAK1, JAK2, and TYK2 but not JAK3 via an evolutionarily conserved motif unique to JAKs. Mutation of this motif led to the formation of an active kinase that could not be inhibited by SOCS3. Surprisingly, we found that SOCS3 simultaneously bound JAK and the cytokine receptor to which it is attached, revealing how specificity is generated in SOCS action and explaining why SOCS3 inhibits only a subset of cytokines. Importantly, SOCS3 inhibited JAKs via a noncompetitive mechanism, making it a template for the development of specific and effective inhibitors to treat JAK-based immune and proliferative diseases.