Voice Depression AI Success Depends on Questionnaires, Not Just Patients
Ecology

Voice Depression AI Success Depends on Questionnaires, Not Just Patients

Testing identical speech recordings against two different mental health surveys revealed why automated voice screening tools can swing from random chance to reliable detection.

By Omar Mirza
Published:
Email this Article
Voice Depression AI Success Depends on Questionnaires, Not Just Patients
CONSORT (Consolidated Standards of Reporting Trials)–style flow diagram of cohort assembly for the primary (reading passage with a score ≥10 on the 8-item Patient Health Questionnaire) analysis. DAM: Depression/Anxiety Model; PDFM: Population Dynamics Foundation Model; ZCTA: zip code tabulation area. Rachel Wiley et al. / Journal of medical Internet research

When artificial intelligence tools listen to a human voice to spot signs of depression, their success often depends less on who is speaking and more on the specific questionnaire used to judge their performance. A recent investigation reveals that a voice-based depression screening model applied to a group of 316 pregnant participants yielded vastly different results depending on which mental health survey defined the outcome, even though the recordings and speakers remained identical.

The research, published in the Journal of Medical Internet Research, highlights significant hurdles in transferring automated speech analysis tools across different clinical domains. As commercial developers race to market voice biomarkers as rapid mental health diagnostics, the findings point to hidden vulnerabilities in how these technologies are validated.

Why Screening Tools Give Conflicting Results on the Same Voices

The cross-sectional study involved 390 pregnant individuals in the United States at a mean gestational age of 22 weeks, with an analytical sample of 316 participants providing 446 sessions. Each person completed browser-based voice tasks, including a 1-minute free speech prompt about pregnancy experiences, a scripted reading passage, and a nonpregnancy speech prompt. At the same time, participants filled out two self-reported mental health questionnaires: an 8-item Patient Health Questionnaire, known as the PHQ-8, and a modified 9-item Edinburgh Postnatal Depression Scale, designated as the mEPDS-9.

When researchers tested an open-source general population speech tool called the Kintsugi Depression/Anxiety Model on these pregnant participants without any retraining, the software scored an area under the curve value of 0.510 against the PHQ-8, which represents performance no better than random chance. However, when evaluated against the mEPDS-9 using the exact same audio recordings and participants, the same model achieved an area under the curve of 0.645, demonstrating a statistically significant difference.

What Diverging Questionnaire Responses Reveal About Vocal Biomarkers

The stark contrast in performance highlights a fundamental mismatch in what different psychiatric questionnaires measure. The PHQ-8 includes several somatic and neurovegetative questions covering physical issues like sleep disruptions, appetite shifts, fatigue, and energy levels. Because normal physiological changes during pregnancy frequently trigger these exact same physical symptoms independently of mood, somatic-weighted tools can easily mistake normal gestation for clinical depression.

Figure 2: Transfer of depression voice biomarkers across cohorts and outcome instruments. Points are area under the curve, bars are 95% bootstrap CIs, and the dashed line marks chance. All cross-domain estimates are pure held-out inference; the prenatal-trained model’s 2 prenatal conditions are leakage-controlled cross-validated estimates within its own cohort. The prenatal-trained model (blue circles) does not separate from chance in any condition, including on its own cohort. The general-population model (Depression/Anxiety Model, orange squares) is competent on both general-psychiatric corpora (0.706‐0.708), at chance against the 8-Item Patient Health Questionnaire in pregnant women (0.510), and above chance against the modified 9-Item Edinburgh Postnatal Depression Scale in the same women (0.645). The 2 left-hand conditions share participants, recordings, and model and differ only in the questionnaire defining the outcome; that contrast is the study’s central finding. As in Table 3, the 2 pr
Transfer of depression voice biomarkers across cohorts and outcome instruments. Points are area under the curve, bars are 95% bootstrap CIs, and the dashed line marks chance. All cross-domain estimates are pure held-out inference; the prenatal-trained model’s 2 prenatal conditions are leakage-controlled cross-validated estimates within its own cohort. The prenatal-trained model (blue circles) does not separate from chance in any condition, including on its own cohort. The general-population model (Depression/Anxiety Model, orange squares) is competent on both general-psychiatric corpora (0.706‐0.708), at chance against the 8-Item Patient Health Questionnaire in pregnant women (0.510), and above chance against the modified 9-Item Edinburgh Postnatal Depression Scale in the same women (0.645). The 2 left-hand conditions share participants, recordings, and model and differ only in the questionnaire defining the outcome; that contrast is the study’s central finding. As in Table 3, the 2 pr Credit: Rachel Wiley et al. / Journal of medical Internet research

In contrast, the Edinburgh scale places heavier emphasis on psychological and emotional manifestations. Exploratory item-level analyses by the authors suggested that the voice model primarily tracked affective distress items such as crying, sadness, and feeling overwhelmed, while ignoring somatic symptoms. This distinction explains why a model trained on general psychiatric populations successfully detects emotional distress signals when paired with a psychologically focused questionnaire, yet falters when pitted against physically confounded metrics.

How Strict Controls Altered Expectations for Prenatal Voice Models

To test the reverse direction, the team developed their own prenatal-trained model using 375 acoustic features combined with community-level socioeconomic embeddings from Google’s Population Dynamics Foundation Model, which summarizes health access and environmental factors by US zip codes. Under strict leakage-controlled cross-validation, where feature selection and scaling were strictly isolated inside training folds, the prenatal model performed at chance levels within its own cohort and failed to transfer when applied to two external general-psychiatric corpora.

The authors emphasized that these findings call for a shift in validation standards across the digital phenotyping field. Researchers evaluating vocal biomarkers must report population characteristics, elicitation tasks, and outcome instruments together rather than treating generalization as a simple question of patient demographics.

The research was published in Journal of medical Internet research on November 16, 2026.

Fact Checked

This article has been fact checked for accuracy, with information verified against reputable sources. Learn more about us and our editorial process.

Last reviewed on .

Article history

  • Latest version

Reference(s)

  1. Wiley, Rachel., et al. “Cross-Domain Transfer of Depression Voice Biomarkers Depends on the Outcome Instrument: Leakage-Controlled Cross-Sectional Evaluation Study.” Journal of Medical Internet Research, vol. 28, November 16, 2026, pp. e105737-e105737. JMIR Publications Inc., doi: 10.2196/105737. <https://doi.org/10.2196/105737>.

Cite this page:

Mirza, Omar. “Voice Depression AI Success Depends on Questionnaires, Not Just Patients.” BioScience. BioScience ISSN 2521-5760, 20 November 2026. <https://www.bioscience.com.pk/en/subject/ecology/cross-domain-transfer-of-depression-voice-biomarkers-depends-on-the-outcome-instrument-leakage-controlled-cross-sectional-evaluation-study>. Mirza, O. (2026, November 20). “Voice Depression AI Success Depends on Questionnaires, Not Just Patients.” BioScience. ISSN 2521-5760. Retrieved November 20, 2026 from https://www.bioscience.com.pk/en/subject/ecology/cross-domain-transfer-of-depression-voice-biomarkers-depends-on-the-outcome-instrument-leakage-controlled-cross-sectional-evaluation-study Mirza, Omar. “Voice Depression AI Success Depends on Questionnaires, Not Just Patients.” BioScience. ISSN 2521-5760. https://www.bioscience.com.pk/en/subject/ecology/cross-domain-transfer-of-depression-voice-biomarkers-depends-on-the-outcome-instrument-leakage-controlled-cross-sectional-evaluation-study (accessed November 20, 2026).
End of the article