Automated Response Technology Projects

Clear and timely communication between physicians and patients is critical to delivering high-quality care and maintaining patient satisfaction. Managing high volumes of patient messages, particularly in busy clinical settings, can be challenging for physicians and contribute to burnout. Automated Response Technology (ART), such as ChatGPT, offers an opportunity to improve efficiency in responding to patient messages. The following projects examine ART’s adoption, utilization, and its potential to optimize clinical communication practices.

AI-Generated Draft Replies Integrated Into Health Records and Physicians’ Electronic Communication

This study examined the impact of generative AI (GenAI) on physician messaging workflows. In a randomized quality improvement (QI) study, primary care physicians were given access to AI-drafted replies for patient messages to assess whether it would reduce their time spent reading and replying. The results showed that while GenAI did not decrease reply time, it was associated with increased message reading time and significantly longer replies. Physicians appreciated the AI-generated drafts as a helpful starting point for crafting more compassionate responses, though they also identified areas for improvement. These findings suggest that while GenAI may enhance the quality and depth of physician communication, its efficiency benefits remain unclear.

For a deeper dive into the study’s methodology, findings, and implications, read the full article in JAMA Network Open here. 

Automated Response Technology Adoption

This project aims to evaluate the impact of Automated Response Technology (ART), on physician-to-patient messaging by comparing communication practices before and over a year after ART implementation. The primary goal is to investigate the factors influencing ART adoption and usage by physicians to assess how it can improve efficiency, consistency, and responsiveness in clinical communication. 

Specific Aims

Specific Aim 1: Examine the adoption and use of ART

• Aim 1a: Evaluate ART adoption rates among physicians, tracking usage from the initial implementation in May 2023 to January 31, 2025.

• Aim 1b: Identify physician-level factors (such as sex and department) that influence ART adoption rates.

Specific Aim 2: Evaluate factors influencing ART usage

• Aim 2a: Assess how the characteristics of patient messages (e.g., sentiment and complexity) affect physicians' likelihood of using ART.

• Aim 2b: Examine how temporal factors (e.g., time of day, day of the week) impact ART usage by physicians.

Perspectives on Quality and Empathy of ART-Assisted Replies to Patient Messages

ART has been used by over 150 physicians at UC San Diego Health. This project assesses ART’s impact on the comprehensiveness, clinical accuracy, and empathy of replies. Physicians will evaluate the clinical quality, while lay persons will assess the empathy conveyed in these replies. Additionally, a net promoter score will be used to gauge the likelihood of recommending the physician based on the reply, from both physician and lay person perspectives. A secondary aim is to explore potential differences in quality and empathy scores based on physician sex, providing insight into how these factors may influence communication dynamics in physician-patient interactions.