Skin Allergy Chatbot Gets Product Advice Right Most of the Time
Health

Skin Allergy Chatbot Gets Product Advice Right Most of the Time

AI language models show potential for contact dermatitis counseling, but hidden allergens and cross-reactivity remain a risk.

By Sana Malik
Published:
Email this Article
Skin Allergy Chatbot Gets Product Advice Right Most of the Time
Visualization of the validity of ChatGPT responses to allergen-free product recommendations. This figure presents the results of ChatGPT’s product recommendations for allergen avoidance across 30 allergens. Each horizontal row represents a specific allergen (listed on the right), and each box within the row corresponds to an individual response from ChatGPT. The color-coded boxes illustrate the validity of each recommendation: blue indicates a valid response, green indicates unknown ingredient information, red indicates an invalid response (containing the allergen), white indicates no specific product was recommended (unresponsive), and yellow indicates the presence of a cross-reactive allergen. Pawit Phadungsaksawasdi et al. / JMIR dermatology

When artificial intelligence is asked to help people with painful skin allergies avoid harmful substances, it gets the product recommendations right nearly 80 percent of the time, but dangerous mistakes still slip through. Researchers evaluating chatbot technology found that while generative systems show promise for patient education, they frequently miss hidden cross-reactivity and recommend items containing known allergens.

Allergic contact dermatitis is a common and intensely frustrating skin condition triggered when a person touches an allergen, causing rashes, itching, and inflammation. Patch testing serves as the diagnostic gold standard, and lifelong allergen avoidance is the primary treatment, which makes detailed patient counseling essential. However, sorting through complex ingredient lists and confusing chemical names often overwhelms patients and clinicians alike.

To test how artificial intelligence might assist in this demanding process, investigators analyzed ChatGPT (GPT-4) on 30 common allergen groups included in standard patch testing. The research team evaluated the model between September 2023 and January 2024, as published in JMIR Dermatology. For every allergen tested, the chatbot was tasked with recommending five specific product brands across three different question formats, ranging from simple direct requests to simulated patient inquiries and role-played clinical guidance.

How Well Did the Chatbot Perform Across Different Products?

Out of 2,250 expected product-level outcomes evaluated across 450 initial prompts, the model successfully generated specific brand recommendations in 91.64 percent of cases. Dermatologists manually verified the ingredient lists of these recommendations using official manufacturer websites and reliable safety databases. Overall, 79.20 percent of the total responses were classified as valid, meaning the suggested products were completely free of the specified allergen.

Figure 1: Flowchart of the study protocol. The study included 30 allergen groups from the international standard patch test. For each allergen, the 5 most common sources of exposure were selected. Three different question formats—simple, more detailed, and very specific—were used to encode prompts into ChatGPT. The chatbot was tasked with recommending 5 allergen-free products per query. Responses were then evaluated for validity by dermatologists and categorized into 5 groups: valid, invalid, unknown, unresponsive, and cross-reaction.
Flowchart of the study protocol. The study included 30 allergen groups from the international standard patch test. For each allergen, the 5 most common sources of exposure were selected. Three different question formats—simple, more detailed, and very specific—were used to encode prompts into ChatGPT. The chatbot was tasked with recommending 5 allergen-free products per query. Responses were then evaluated for validity by dermatologists and categorized into 5 groups: valid, invalid, unknown, unresponsive, and cross-reaction. Credit: Pawit Phadungsaksawasdi et al. / JMIR dermatology

The system performed best when recommending cosmetics, personal care items, and medicines, achieving high validity rates of 90.33 percent and 82.5 percent, respectively. For instance, common items like shampoos and moisturizers frequently had accessible ingredient data. In contrast, accuracy plummeted for clothing, leather goods, and household building materials because manufacturer ingredient details were frequently missing or unavailable.

What Were the Most Dangerous Hidden Errors?

Despite high overall accuracy, the chatbot produced invalid recommendations containing the actual allergen in nearly 2 percent of responses, alongside numerous cross-reactive items. Fragrance mixes and para-phenylenediamine, a chemical widely used in hair dyes, accounted for the highest number of inaccurate suggestions. For example, several recommended hair dyes still contained para-phenylenediamine, and multiple baby shampoos included restricted fragrance compounds.

Cross-reactivity posed an even trickier hurdle for the artificial intelligence. The chatbot frequently recommended personal care products containing substances known to cross-react with Peru balsam, such as benzyl alcohol and eugenol, without warning the user. These oversights highlight the limitations of general-purpose language models in recognizing complex immunological cross-reactions.

Figure 3: Frequency of invalid product recommendations generated by ChatGPT by allergen and question format. This figure displays the number of invalid responses generated by ChatGPT for each allergen across the 3 question formats: simple (question 1), more detailed (question 2), and very specific (question 3). PPD: para-phenylenediamine.
Frequency of invalid product recommendations generated by ChatGPT by allergen and question format. This figure displays the number of invalid responses generated by ChatGPT for each allergen across the 3 question formats: simple (question 1), more detailed (question 2), and very specific (question 3). PPD: para-phenylenediamine. Credit: Pawit Phadungsaksawasdi et al. / JMIR dermatology

Does the Way You Ask the Question Change the Outcome?

The study also compared three prompt formats to see if phrasing affected accuracy. Interestingly, the simplest question format yielded the highest proportion of valid recommendations at 85.33 percent, while more elaborate, clinically framed prompts resulted in higher unresponsiveness rates where the chatbot failed to name a specific brand.

When the researchers re-entered failed queries for a second attempt, the valid response rate climbed slightly to 83.16 percent, showing that repeated prompting can help overcome occasional output gaps. However, the authors caution that general-purpose chatbots should never replace professional medical guidance or dedicated allergen management databases like the Contact Allergen Management Program.

Why Patients Must Still Verify Every Product Label

The authors emphasize that while large language models can help streamline consultations and provide accessible supplementary education in resource-limited settings, their output requires rigorous human verification. Because product formulations change frequently and regional availability varies, patients should always check physical product labels and consult a dermatologist before trusting any artificial intelligence recommendation.

The research was published in JMIR dermatology on November 17, 2026.

Fact Checked

This article has been fact checked for accuracy, with information verified against reputable sources. Learn more about us and our editorial process.

Last reviewed on .

Article history

  • Latest version

Reference(s)

  1. Phadungsaksawasdi, Pawit., et al. “Using a Large Language Model Chatbot for Allergen Avoidance Counseling in Contact Dermatitis: Quantitative Evaluation Study.” JMIR Dermatology, vol. 9, November 17, 2026, pp. e100364-e100364. JMIR Publications Inc., doi: 10.2196/100364. <https://doi.org/10.2196/100364>.

Cite this page:

Malik, Sana. “Skin Allergy Chatbot Gets Product Advice Right Most of the Time.” BioScience. BioScience ISSN 2521-5760, 19 November 2026. <https://www.bioscience.com.pk/en/subject/health/using-a-large-language-model-chatbot-for-allergen-avoidance-counseling-in-contact-dermatitis-quantitative-evaluation-study>. Malik, S. (2026, November 19). “Skin Allergy Chatbot Gets Product Advice Right Most of the Time.” BioScience. ISSN 2521-5760. Retrieved November 19, 2026 from https://www.bioscience.com.pk/en/subject/health/using-a-large-language-model-chatbot-for-allergen-avoidance-counseling-in-contact-dermatitis-quantitative-evaluation-study Malik, Sana. “Skin Allergy Chatbot Gets Product Advice Right Most of the Time.” BioScience. ISSN 2521-5760. https://www.bioscience.com.pk/en/subject/health/using-a-large-language-model-chatbot-for-allergen-avoidance-counseling-in-contact-dermatitis-quantitative-evaluation-study (accessed November 19, 2026).
End of the article