The ubiquitin–proteasome pathway is particularly important for the regulated degradation of various proteins which control a vast array of biological processes. Therefore, proteasome inhibitors are promising candidates for anti-tumoral or anti-inflammatory drugs. N-Acetyl-Leu-Leu-Norleucinal (Ac-LLN-al, also termed calpain inhibitor I) was one of the first proteasome inhibitors discovered and has been widely used to study the 20S proteasome core particle (CP) function in vivo, despite its lack of specificity. Vinyl sulfones, like Ac-PRLN-vs, show covalent binding of the β-carbon atom of the vinyl sulfone group to the Thr1O^γ only of subunit β2. However, vinyl sulfones have similar limitations as peptide aldehydes as they have been reported also to bind and block intracellular cysteine proteases. A more specific proteasome inhibitor is the natural product lactacystin, which can be isolated from Streptomyces. It was found that this compound forms an ester bond only to the Thr1O^γ of the chymotrypsin-like active subunit β5 due to specific P1 interactions. In contrast to most other proteasome inhibitors, the natural α′,β′-epoxyketone peptide epoxomicin binds specifically to the small class of N-terminal nucleophilic (Ntn) hydrolases (CPs belong to this protease family) with the formation of a morpholino adduct. All previously described proteasome inhibitors bind covalently to the proteolytic active sites. However, as the proteasome is involved in a variety of biological important functions, it is of particular interest to block the CP only for limited time in order to reduce cytotoxic effects. Recently, the binding mode of the natural specific proteasome inhibitor TMC-95 obtained from Apiospora montagnei was investigated. The crystal structure revealed that the TMC-95 blocks the active sites of the CP noncovalently in the low nanomolar range. This review summarizes the current structural knowledge of inhibitory compounds bound to the CP, showing the proteasome as a potential target for drug development in medical research.