Towards A Foundation Model for Clinical Voice Biomarkers

Vocal biomarkers, encompassing voice and speech, have largely been developed for individual conditions in isolation, limiting their generalizability across diseases and recording settings. To address this, we introduce VoiceFM, a contrastive model that learns general-purpose clinical voice representations by aligning audio embeddings with rich clinical metadata. Using the Bridge2AI-Voice dataset (984 primarily English-speaking adult participants - 846 used for training and 138 held out as a temporally separated validation cohort - 40,056 recordings totaling 176 hours across 5 academic medical centers), VoiceFM pairs a fine-tuned Whisper large-v2 encoder with a tabular transformer over 44 clinical features via symmetric InfoNCE loss. Linear probes on frozen VoiceFM embeddings achieve mean AUROC 0.952 across five evaluation tasks, significantly outperforming Frozen Whisper, Frozen HuBERT, and the contrastively trained VoiceFM-HuBERT. On the 138-participant held-out cohort, VoiceFM-Whisper achieves AUROCs of 0.99 for Alzheimer's/dementia/MCI and 0.89 for airway stenosis. VoiceFM representations transfer to three external datasets without retraining and improve few-shot classification; the model transfers without fine-tuning to Spanish-language Parkinson's disease detection (NeuroVoz, AUROC 0.93) and to the mPower smartphone study (participant-level AUROC 0.87). Together, these results show that contrastive alignment between voice and rich clinical metadata can serve as the basis for a clinical voice foundation model that generalizes across diseases, languages, recording conditions, and patients enrolled after model freeze.