We have previously shown that serotonin (5-HT) induces both hyperplasia and hypertrophy of pulmonary artery smooth muscle cells (SMC) but not of endothelial cells (EC) through its high-affinity uptake. The present studies demonstrate rapid enhancement by 5-HT of Tyr phosphorylation of proteins, including p120, which also occurs in SMC but not in EC. The p120 protein was identified as GTPase-activating protein (GAP) by immunoprecipitation. Its phosphorylation occurred within minutes and preceded other events associated with 5-HT-induced mitogenesis. Tyr kinase (TK) and 5-HT uptake inhibitors and 8-bromoadenosine 3',5'-cyclic monophosphate blocked both the 5-HT-induced DNA synthesis and Tyr phosphorylation of GAP. Vanadate elevated DNA synthesis and Tyr phosphorylation of GAP of both control and 5-HT-treated cells. 5-HT failed to alter Tyr phosphorylation of GAP in cellular homogenates, as opposed to intact cells. In the presence of 3-isobutyl-1-methylxanthine, 5-HT inhibited cellular growth, presumably through its action on 5-HT1A or 5-HT4 receptors and elevation of adenosine 3',5'-cyclic monophosphate, but this was not associated with an alteration of Tyr phosphorylation of GAP. Similarly, a 5-HT1 or 5-HT2 receptor agonist failed to stimulate Tyr phosphorylation or DNA synthesis of SMC. Stimulation of cellular proliferation and enlargement produced by 1 microM 5-HT were totally abolished by TK inhibitors that did not affect 5-HT uptake. These data indicate that Tyr phosphorylation of GAP may act as an intermediate signal in 5-HT-induced mitogenesis of SMC which requires cellular internalization of 5-HT rather than its action on a membrane receptor.