The ability of the bone marrow (BM) to generate copious amounts of blood cells required on a daily basis depends on a highly orchestrated process of proliferation and differentiation of hematopoietic stem and progenitor cells (HSPCs). This process can be rapidly adapted under stress conditions, such as infections, to meet the specific cellular needs of the immune response and the ensuing physiological changes. This requires a tight regulation in order to prevent either hematopoietic failure or transformation. Although adaptation to bacterial infections or systemic inflammation has been studied and reviewed in depth, specific alterations of hematopoiesis to viral infections have received less attention so far. Viruses constantly pose a significant health risk and demand an adequate, balanced response from our immune system, which also affects the BM. In fact, both the virus itself and the ensuing immune response can have a tremendous impact on the hematopoietic process. On one hand, this can be beneficial: it helps to boost the cellular response of the body to resolve the viral infection. But on the other hand, when the virus and the resulting antiviral response persist, the inflammatory feedback to the hematopoietic system will become chronic, which can be detrimental for a balanced BM output. Chronic viral infections frequently have clinical manifestations at the level of blood cell formation, and we summarize which viruses can lead to BM pathologies, like aplastic anemia, pancytopenia, hemophagocytic lymphohistiocytosis, lymphoproliferative disorders, and malignancies. Regarding the underlying mechanisms, we address specific effects of acute and chronic viral infections on blood cell production. As such, we distinguish four different levels in which this can occur: (1) direct viral infection of HSPCs, (2) viral recognition by HSPCs, (3) indirect effects on HSPCs by inflammatory mediators, and (4) the role of the BM microenvironment on hematopoiesis upon virus infection. In conclusion, this review provides a comprehensive overview on how viral infections can affect the formation of new blood cells, aiming to advance our understanding of the underlying cellular and molecular mechanisms to improve the treatment of BM failure in patients.