Extracellular matrix (ECM) mineralization is a physiological process in bone and a pathological one in soft tissues. The mechanisms determining the spatial restriction of ECM mineralization to bone physiologically are poorly understood. Here we show that a normal extracellular phosphate concentration is required for bone mineralization, while lowering this concentration prevents mineralization of any ECM. However, simply raising extracellular phosphate concentration is not sufficient to induce pathological mineralization, this is because of the presence in all ECMs of pyrophosphate, an inhibitor of mineralization. ECM mineralization occurs only in bone because of the exclusive coexpression in osteoblasts of Type I collagen and Tnap, an enzyme that cleaves pyrophosphate. This dual requirement explains why Tnap ectopic expression in cells producing fibrillar collagen is sufficient to induce pathological mineralization. This study reveals that coexpression in osteoblasts of otherwise broadly expressed genes is necessary and sufficient to induce bone mineralization and provides evidence that pathological mineralization can be prevented by modulating extracellular phosphate concentration.