Simian virus 40 (SV40) utilizes endocytosis through caveolae for infectious entry into host cells. We found that after binding to caveolae, virus particles induced transient breakdown of actin stress fibers. Actin was then recruited to virus-loaded caveolae as actin patches that served as sites for actin “tail” formation. Dynamin II was also transiently recruited. These events depended on the presence of cholesterol and on the activation of tyrosine kinases that phosphorylated proteins in caveolae. They were necessary for formation of caveolae-derived endocytic vesicles and for infection of the cell. Thus, caveolar endocytosis is ligand-triggered and involves extensive rearrangement of the actin cytoskeleton.