Chronic activation of brain innate immunity is a prominent feature of Alzheimer’s disease (AD) and primary tauopathies. However, to what degree innate immunity contributes to neurodegeneration as compared with pathological protein-induced neurotoxicity, and the requirement of a particular glial cell type in neurodegeneration, are still unclear. Here we demonstrate that microglia-mediated damage, rather than pathological tau-induced direct neurotoxicity, is the leading force driving neurodegeneration in a tauopathy mouse model. Importantly, the progression of ptau pathology is also driven by microglia. In addition, we found that APOE, the strongest genetic risk factor for AD, regulates neurodegeneration predominantly by modulating microglial activation, although a minor role of apoE in regulating ptau and insoluble tau formation independent of its immunomodulatory function was also identified. Our results suggest that therapeutic strategies targeting microglia may represent an effective approach to prevent disease progression in the setting of tauopathy.