The Janus family of protein tyrosine kinases (JAKs) regulate cellular processes involved in cell growth, differentiation and transformation through their association with cytokine receptors. However, compared with other kinases, little is known about cellular regulators of the JAKs. We have recently identified a JAK‐binding protein (JAB) that inhibits JAK signaling in cells. In the studies presented here we demonstrate that JAB specifically binds to the tyrosine residue (Y1007) in the activation loop of JAK2, whose phosphorylation is required for activation of kinase activity. Binding to the phosphorylated activation loop requires the JAB SH2 domain and an additional N‐terminal 12 amino acids (extended SH2 subdomain) containing two residues (Ile68 and Leu75) that are conserved in JAB‐related proteins. An additional N‐terminal 12‐amino‐acid region (kinase inhibitory region) of JAB also contributes to high‐affinity binding to the JAK2 tyrosine kinase domain and is required for inhibition of JAK2 signaling and kinase activity. Our studies define a novel type of regulation of tyrosine kinases and might provide a basis for the design of specific tyrosine kinase inhibitors.