Recurrent breast cancer accounts for most disease-associated mortality and can develop decades after primary tumor therapy. Recurrences arise from residual tumor cells (RTCs) that can evade therapy in a dormant state, however the mechanisms are poorly understood. CRISPR-Cas9 screening identified the transcription factors SOX5/6 as functional regulators of tumor recurrence. Loss of SOX5 accelerated recurrence and promoted escape from dormancy. Remarkably, SOX5 drove dormant RTCs to adopt a cartilage-dependent bone development program, termed endochondral ossification, that was confirmed by [¹⁸F]NaF-PET imaging and reversed in recurrent tumors escaping dormancy. In patients, osteochondrogenic gene expression in primary breast cancers or residual disease post-neoadjuvant therapy predicted improved recurrence-free survival. These findings suggest that SOX5-dependent mesodermal transdifferentiation constitutes an adaptive mechanism that prevents recurrence by reinforcing tumor cell dormancy.