T-cell interaction with a target cell is a key event in the adaptive immune response and primarily driven by T-cell receptor (TCR) recognition of peptide-MHC (pMHC) complexes. TCR avidity for a given pMHC is determined by number of MHC molecules, availability of coreceptors, and TCR affinity for MHC or peptide, respectively, with peptide recognition being the most important factor to confer target specificity. Here we present high-resolution crystal structures of 2 Fab antibodies in complex with the immunodominant NY-ESO-1_157–165 peptide analogue (SLLMWITQV) presented by HLA-A*0201 and compare them with a TCR recognizing the same pMHC. Binding to the central methionine-tryptophan peptide motif and orientation of binding were almost identical for Fabs and TCR. As the MW “peg” dominates the contacts between Fab and peptide, we estimated the contributions of individual amino acids between the Fab and peptide to provide the rational basis for a peptide-focused second-generation, high-affinity antibody library. The final Fab candidate achieved better peptide binding by 2 light-chain mutations, giving a 20-fold affinity improvement to 2–4 nM, exceeding the affinity of the TCR by 1,000-fold. The high-affinity Fab when grafted as recombinant TCR on T cells conferred specific killing of HLA-A*0201/NY-ESO-1_157–165 target cells. In summary, we prove that affinity maturation of antibodies mimicking a TCR is possible and provide a strategy for engineering high-affinity antibodies that can be used in targeting specific pMHC complexes for diagnostic and therapeutic purposes.