We have developed a new scoring function, the template modeling score (TM‐score), to assess the quality of protein structure templates and predicted full‐length models by extending the approaches used in Global Distance Test (GDT)1 and MaxSub.2 First, a protein size‐dependent scale is exploited to eliminate the inherent protein size dependence of the previous scores and appropriately account for random protein structure pairs. Second, rather than setting specific distance cutoffs and calculating only the fractions with errors below the cutoff, all residue pairs in alignment/modeling are evaluated in the proposed score. For comparison of various scoring functions, we have constructed a large‐scale benchmark set of structure templates for 1489 small to medium size proteins using the threading program PROSPECTOR_3 and built the full‐length models using MODELLER and TASSER. The TM‐score of the initial threading alignments, compared to the GDT and MaxSub scoring functions, shows a much stronger correlation to the quality of the final full‐length models. The TM‐score is further exploited as an assessment of all ‘new fold’ targets in the recent CASP5 experiment and shows a close coincidence with the results of human‐expert visual assessment. These data suggest that the TM‐score is a useful complement to the fully automated assessment of protein structure predictions. The executable program of TM‐score is freely downloadable at http://bioinformatics.buffalo.edu/TM‐score. Proteins 2004. © 2004 Wiley‐Liss, Inc.