The World Health Organization has declared the ongoing outbreak of COVID-19, which is caused by a novel coronavirus SARS-CoV-2, a pandemic. There is currently a lack of knowledge about the antibody response elicited from SARS-CoV-2 infection. One major immunological question concerns antigenic differences between SARS-CoV-2 and SARS-CoV. We address this question by analyzing plasma from patients infected by SARS-CoV-2 or SARS-CoV and from infected or immunized mice. Our results show that, although cross-reactivity in antibody binding to the spike protein is common, cross-neutralization of the live viruses may be rare, indicating the presence of a non-neutralizing antibody response to conserved epitopes in the spike. Whether such low or non-neutralizing antibody response leads to antibody-dependent disease enhancement needs to be addressed in the future. Overall, this study not only addresses a fundamental question regarding antigenicity differences between SARS-CoV-2 and SARS-CoV but also has implications for immunogen design and vaccine development.