Vickybot, a Chatbot for Anxiety-Depressive Symptoms and Work-Related Burnout in Primary Care and Health Care Professionals: Development, Feasibility, and Potential Effectiveness Studies

Gerard Anmella; Miriam Sanabra; Mireia Primé-Tous; Xavier Segú; Myriam Cavero; Ivette Morilla; Iria Grande; Victoria Ruiz; Ariadna Mas; Inés Martín-Villalba; Alejandro Caballo; Esteva Julia-Parisad; Arturo Rodríguez-Rey; Flavia Piazza; Francisco José Valdesoiro; Claudia Rodriguez-Torrella; Marta Espinosa; Giulia Virgili; Carlota Sorroche; Alicia Ruiz; Aleix Solanes; Joaquim Radua; María Antonieta Also; Elisenda Sant; Sandra Murgui; Mireia Sans-Corrales; Allan H. Young; Victor Vicens; Jordi Blanch; Elsa Caballeria; Hugo López-Pelayo; Clara López; Victoria Olivé; Laura Pujol; Sebastiana Quesada; Brisa Solé; Carla Torrent; Anabel Martínez-Aran; Joana Guarch; Ricard Navinés; Andrea Murru; Giovanna Fico; Michele de Prisco; Vicenzo Oliva; Silvia Amoretti; Casimiro Pio-Carrino; María Fernández-Canseco; Marta Villegas; Eduard Vieta; Diego Hidalgo-Mazzei

doi:10.2196/43293

Vickybot, a Chatbot for Anxiety-Depressive Symptoms and Work-Related Burnout in Primary Care and Health Care Professionals: Development, Feasibility, and Potential Effectiveness Studies

Gerard Anmella, Miriam Sanabra, Mireia Primé-Tous, Xavier Segú, Myriam Cavero, Ivette Morilla, Iria Grande, Victoria Ruiz, Ariadna Mas, Inés Martín-Villalba, Alejandro Caballo, Esteva Julia-Parisad, Arturo Rodríguez-Rey, Flavia Piazza, Francisco José Valdesoiro, Claudia Rodriguez-Torrella, Marta Espinosa, Giulia Virgili, Carlota Sorroche, Alicia RuizAleix Solanes, Joaquim Radua, María Antonieta Also, Elisenda Sant, Sandra Murgui, Mireia Sans-Corrales, Allan H. Young, Victor Vicens, Jordi Blanch, Elsa Caballeria, Hugo López-Pelayo, Clara López, Victoria Olivé, Laura Pujol, Sebastiana Quesada, Brisa Solé, Carla Torrent, Anabel Martínez-Aran, Joana Guarch, Ricard Navinés, Andrea Murru, Giovanna Fico, Michele de Prisco, Vicenzo Oliva, Silvia Amoretti, Casimiro Pio-Carrino, María Fernández-Canseco, Marta Villegas, Eduard Vieta, Diego Hidalgo-Mazzei^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

34 Citations (Scopus)

Abstract

Background: Many people attending primary care (PC) have anxiety-depressive symptoms and work-related burnout compounded by a lack of resources to meet their needs. The COVID-19 pandemic has exacerbated this problem, and digital tools have been proposed as a solution. Objective: We aimed to present the development, feasibility, and potential effectiveness of Vickybot, a chatbot aimed at screening, monitoring, and reducing anxiety-depressive symptoms and work-related burnout, and detecting suicide risk in patients from PC and health care workers. Methods: Healthy controls (HCs) tested Vickybot for reliability. For the simulation study, HCs used Vickybot for 2 weeks to simulate different clinical situations. For feasibility and effectiveness study, people consulting PC or health care workers with mental health problems used Vickybot for 1 month. Self-assessments for anxiety (Generalized Anxiety Disorder 7-item) and depression (Patient Health Questionnaire-9) symptoms and work-related burnout (based on the Maslach Burnout Inventory) were administered at baseline and every 2 weeks. Feasibility was determined from both subjective and objective user-engagement indicators (UEIs). Potential effectiveness was measured using paired 2-tailed t tests or Wilcoxon signed-rank test for changes in self-assessment scores. Results: Overall, 40 HCs tested Vickybot simultaneously, and the data were reliably transmitted and registered. For simulation, 17 HCs (n=13, 76% female; mean age 36.5, SD 9.7 years) received 98.8% of the expected modules. Suicidal alerts were received correctly. For the feasibility and potential effectiveness study, 34 patients (15 from PC and 19 health care workers; 76% [26/34] female; mean age 35.3, SD 10.1 years) completed the first self-assessments, with 100% (34/34) presenting anxiety symptoms, 94% (32/34) depressive symptoms, and 65% (22/34) work-related burnout. In addition, 27% (9/34) of patients completed the second self-assessment after 2 weeks of use. No significant differences were found between the first and second self-assessments for anxiety (t₈=1.000; P=.34) or depressive (t₈=0.40; P=.70) symptoms. However, work-related burnout scores were moderately reduced (z=−2.07, P=.04, r=0.32). There was a nonsignificant trend toward a greater reduction in anxiety-depressive symptoms and work-related burnout with greater use of the chatbot. Furthermore, 9% (3/34) of patients activated the suicide alert, and the research team promptly intervened with successful outcomes. Vickybot showed high subjective UEI (acceptability, usability, and satisfaction), but low objective UEI (completion, adherence, compliance, and engagement). Vickybot was moderately feasible. Conclusions: The chatbot was useful in screening for the presence and severity of anxiety and depressive symptoms, and for detecting suicidal risk. Potential effectiveness was shown to reduce work-related burnout but not anxiety or depressive symptoms. Subjective perceptions of use contrasted with low objective-use metrics. Our results are promising but suggest the need to adapt and enhance the smartphone-based solution to improve engagement. A consensus on how to report UEIs and validate digital solutions, particularly for chatbots, is required.

Original language	English
Article number	A63
Journal	Journal of Medical Internet Research
Volume	25
DOIs	https://doi.org/10.2196/43293
Publication status	Published - 2023

Keywords

anxiety
burnout
chatbot
depression
digital
health care workers
PRESTO
primary care
primary care digital support tool in mental health
smartphone
symptom

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.2196/43293

Cite this

Anmella, G., Sanabra, M., Primé-Tous, M., Segú, X., Cavero, M., Morilla, I., Grande, I., Ruiz, V., Mas, A., Martín-Villalba, I., Caballo, A., Julia-Parisad, E., Rodríguez-Rey, A., Piazza, F., José Valdesoiro, F., Rodriguez-Torrella, C., Espinosa, M., Virgili, G., Sorroche, C., ... Hidalgo-Mazzei, D. (2023). Vickybot, a Chatbot for Anxiety-Depressive Symptoms and Work-Related Burnout in Primary Care and Health Care Professionals: Development, Feasibility, and Potential Effectiveness Studies. Journal of Medical Internet Research, 25, Article A63. https://doi.org/10.2196/43293

@article{9ead1928f412498faf6db24cf7addcd9,

title = "Vickybot, a Chatbot for Anxiety-Depressive Symptoms and Work-Related Burnout in Primary Care and Health Care Professionals: Development, Feasibility, and Potential Effectiveness Studies",

abstract = "Background: Many people attending primary care (PC) have anxiety-depressive symptoms and work-related burnout compounded by a lack of resources to meet their needs. The COVID-19 pandemic has exacerbated this problem, and digital tools have been proposed as a solution. Objective: We aimed to present the development, feasibility, and potential effectiveness of Vickybot, a chatbot aimed at screening, monitoring, and reducing anxiety-depressive symptoms and work-related burnout, and detecting suicide risk in patients from PC and health care workers. Methods: Healthy controls (HCs) tested Vickybot for reliability. For the simulation study, HCs used Vickybot for 2 weeks to simulate different clinical situations. For feasibility and effectiveness study, people consulting PC or health care workers with mental health problems used Vickybot for 1 month. Self-assessments for anxiety (Generalized Anxiety Disorder 7-item) and depression (Patient Health Questionnaire-9) symptoms and work-related burnout (based on the Maslach Burnout Inventory) were administered at baseline and every 2 weeks. Feasibility was determined from both subjective and objective user-engagement indicators (UEIs). Potential effectiveness was measured using paired 2-tailed t tests or Wilcoxon signed-rank test for changes in self-assessment scores. Results: Overall, 40 HCs tested Vickybot simultaneously, and the data were reliably transmitted and registered. For simulation, 17 HCs (n=13, 76% female; mean age 36.5, SD 9.7 years) received 98.8% of the expected modules. Suicidal alerts were received correctly. For the feasibility and potential effectiveness study, 34 patients (15 from PC and 19 health care workers; 76% [26/34] female; mean age 35.3, SD 10.1 years) completed the first self-assessments, with 100% (34/34) presenting anxiety symptoms, 94% (32/34) depressive symptoms, and 65% (22/34) work-related burnout. In addition, 27% (9/34) of patients completed the second self-assessment after 2 weeks of use. No significant differences were found between the first and second self-assessments for anxiety (t8=1.000; P=.34) or depressive (t8=0.40; P=.70) symptoms. However, work-related burnout scores were moderately reduced (z=−2.07, P=.04, r=0.32). There was a nonsignificant trend toward a greater reduction in anxiety-depressive symptoms and work-related burnout with greater use of the chatbot. Furthermore, 9% (3/34) of patients activated the suicide alert, and the research team promptly intervened with successful outcomes. Vickybot showed high subjective UEI (acceptability, usability, and satisfaction), but low objective UEI (completion, adherence, compliance, and engagement). Vickybot was moderately feasible. Conclusions: The chatbot was useful in screening for the presence and severity of anxiety and depressive symptoms, and for detecting suicidal risk. Potential effectiveness was shown to reduce work-related burnout but not anxiety or depressive symptoms. Subjective perceptions of use contrasted with low objective-use metrics. Our results are promising but suggest the need to adapt and enhance the smartphone-based solution to improve engagement. A consensus on how to report UEIs and validate digital solutions, particularly for chatbots, is required.",

keywords = "anxiety, burnout, chatbot, depression, digital, health care workers, PRESTO, primary care, primary care digital support tool in mental health, smartphone, symptom",

author = "Gerard Anmella and Miriam Sanabra and Mireia Prim{\'e}-Tous and Xavier Seg{\'u} and Myriam Cavero and Ivette Morilla and Iria Grande and Victoria Ruiz and Ariadna Mas and In{\'e}s Mart{\'i}n-Villalba and Alejandro Caballo and Esteva Julia-Parisad and Arturo Rodr{\'i}guez-Rey and Flavia Piazza and {Jos{\'e} Valdesoiro}, Francisco and Claudia Rodriguez-Torrella and Marta Espinosa and Giulia Virgili and Carlota Sorroche and Alicia Ruiz and Aleix Solanes and Joaquim Radua and {Antonieta Also}, Mar{\'i}a and Elisenda Sant and Sandra Murgui and Mireia Sans-Corrales and Young, {Allan H.} and Victor Vicens and Jordi Blanch and Elsa Caballeria and Hugo L{\'o}pez-Pelayo and Clara L{\'o}pez and Victoria Oliv{\'e} and Laura Pujol and Sebastiana Quesada and Brisa Sol{\'e} and Carla Torrent and Anabel Mart{\'i}nez-Aran and Joana Guarch and Ricard Navin{\'e}s and Andrea Murru and Giovanna Fico and {de Prisco}, Michele and Vicenzo Oliva and Silvia Amoretti and Casimiro Pio-Carrino and Mar{\'i}a Fern{\'a}ndez-Canseco and Marta Villegas and Eduard Vieta and Diego Hidalgo-Mazzei",

note = "Funding Information: We are grateful to all participants. GA is supported by a Rio Hortega 2021 grant (CM21/00017) from the Spanish Ministry of Health financed by the Instituto de Salud Carlos III (ISCIII) and cofinanced by Fondo Social Europe Plus. MS was supported by a grant from the Baszucki Brain Research Fund. AM is supported by the Ag{\`e}ncia de Gesti{\'o} d{\textquoteright}Ajudes Universit{\`a}ries I de Investigaci{\'o}—PAND{\`E}MIES 2020 grant (PI047003) from the Generalitat de Catalunya. IG thanks the support of the Spanish Ministry of Science and Innovation (PI19/00954) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirecci{\'o}n General de Evaluaci{\'o}n y el Fondos Europeos de la Uni{\'o}n Europea (FEDER, Fondo Social Europe, Next Generation European Union or Plan de Recuperaci{\'o}n Transformaci{\'o}n y Resiliencia_PRTR); the Instituto de Salud Carlos III; the CIBER of Mental Health (CIBERSAM); and the Secretaria d{\textquoteright}Universitats i Recerca del Departament d{\textquoteright}Economia i Coneixement (2017 SGR 1365), CERCA Programme or Generalitat de Catalunya as well as the Fundaci{\'o} Cl{\'i}nic per la Recerca Biom{\`e}dica (Pons Bartran 2022-FRCB_PB1_2022). AHY{\textquoteright}s independent research was funded by the National Institute for Health Research Biomedical Research Centre in South London and Maudsley National Health Service Foundation Trust and King{\textquoteright}s College London. The views expressed are those of the authors and not necessarily those of the National Health Service, National Institute for Health and Care Research, or Department of Health. JR is supported by a Miguel Servet II contract (CPII19/00009), funded by ISCIII and cofunded by the European Social Fund “Investing in your future.” CT has been supported through a “Miguel Servet” postdoctoral contract (CPI14/00175) and a Miguel Servet II contract (CPII19/00018) and thanks the support of the Spanish Ministry of Innovation and Science (PI17/01066 and PI20/00344), funded by the Instituto de Salud Carlos III and cofinanced by the European Union (FEDER) “Una manera de hacer Europa.” AMA thanks the support of the Spanish Ministry of Science and Innovation (PI18/00789, PI21/00787) integrated into the Plan Nacional de I+D+I and cofinanced by ISCIII-Subdirecci{\'o}n General de Evaluaci{\'o}n and the Fondo Europeo de Desarrollo Regional (FEDER); the ISCIII; the CIBER of Mental Health (CIBERSAM); the Secretaria d{\textquoteright}Universitats i Recerca del Departament d{\textquoteright}Economia i Coneixement (2017 SGR 1365); the CERCA Programme; and the Departament de Salut de la Generalitat de Catalunya for the Pla estrat{\`e}gic de recerca I innovaci{\'o} en salut (PERIS) grant SLT006/17/00177. AM thanks the support of the Spanish Ministry of Science and Innovation (PI19/00672) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirecci{\'o}n General de Evaluaci{\'o}n and the FEDER. GF is supported by a fellowship from “La Caixa” Foundation (ID 100010434)—fellowship code—LCF/BQ/DR21/11880019. SA has been supported by a Sara Borrell contract (CD20/00177), funded by ISCIII and founded by the European Social Fund “Investing in your future.” EV thanks the support of the Spanish Ministry of Science, Innovation and Universities (PI15/00283, PI18/00805, PI19/00394, PI21/00787, and CPII19/00009) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirecci{\'o}n General de Evaluaci{\'o}n and the FEDER; the ISCIII; the CIBER of Mental Health (CIBERSAM); the Secretaria d{\textquoteright}Universitats i Recerca del Departament d{\textquoteright}Economia i Coneixement (2017 SGR 1365), and the CERCA Programme or Generalitat de Catalunya. We would like to thank the Departament de Salut de la Generalitat de Catalunya for the PERIS grant SLT006/17/00357. DHM´s research was supported by Juan Rod{\'e}s JR18/00021 granted by the ISCIII. The PRESTO project has been funded by Fundaci{\'o} Cl{\'i}nic per a la Recerca Biom{\`e}dica through the Pons Bartran 2020 grant (PI046549). The development of a version of the digital solution adapted to health workers is funded by the Spanish Foundation for Psychiatry and Mental Health, Spanish Psychiatric Society, and Spanish Society of Biological Psychiatry (PI046813). The enhancement of the digital solution with Natural Language Processing techniques in a chatbot user-interface in collaboration with the text mining technologies in the health domain of the Barcelona Supercomputing Center is funded by the Ag{\`e}ncia de Gesti{\'o} d{\textquoteright}Ajudes Universit{\`a}ries I de Investigaci{\'o}—PAND{\`E}MIES 2020 grant (PI047003), from La Generalitat de Catalunya. Publisher Copyright: {\textcopyright} 2023 Journal of Medical Internet Research. All rights reserved.",

year = "2023",

doi = "10.2196/43293",

language = "English",

volume = "25",

journal = "Journal of Medical Internet Research",

issn = "1438-8871",

publisher = "Journal of medical Internet Research",

}

Anmella, G, Sanabra, M, Primé-Tous, M, Segú, X, Cavero, M, Morilla, I, Grande, I, Ruiz, V, Mas, A, Martín-Villalba, I, Caballo, A, Julia-Parisad, E, Rodríguez-Rey, A, Piazza, F, José Valdesoiro, F, Rodriguez-Torrella, C, Espinosa, M, Virgili, G, Sorroche, C, Ruiz, A, Solanes, A, Radua, J, Antonieta Also, M, Sant, E, Murgui, S, Sans-Corrales, M, Young, AH, Vicens, V, Blanch, J, Caballeria, E, López-Pelayo, H, López, C, Olivé, V, Pujol, L, Quesada, S, Solé, B, Torrent, C, Martínez-Aran, A, Guarch, J, Navinés, R, Murru, A, Fico, G, de Prisco, M, Oliva, V, Amoretti, S, Pio-Carrino, C, Fernández-Canseco, M, Villegas, M, Vieta, E & Hidalgo-Mazzei, D 2023, 'Vickybot, a Chatbot for Anxiety-Depressive Symptoms and Work-Related Burnout in Primary Care and Health Care Professionals: Development, Feasibility, and Potential Effectiveness Studies', Journal of Medical Internet Research, vol. 25, A63. https://doi.org/10.2196/43293

Vickybot, a Chatbot for Anxiety-Depressive Symptoms and Work-Related Burnout in Primary Care and Health Care Professionals: Development, Feasibility, and Potential Effectiveness Studies. / Anmella, Gerard; Sanabra, Miriam; Primé-Tous, Mireia et al.
In: Journal of Medical Internet Research, Vol. 25, A63, 2023.

Research output: Contribution to journal › Article › peer-review

TY - JOUR

T1 - Vickybot, a Chatbot for Anxiety-Depressive Symptoms and Work-Related Burnout in Primary Care and Health Care Professionals

T2 - Development, Feasibility, and Potential Effectiveness Studies

AU - Anmella, Gerard

AU - Sanabra, Miriam

AU - Primé-Tous, Mireia

AU - Segú, Xavier

AU - Cavero, Myriam

AU - Morilla, Ivette

AU - Grande, Iria

AU - Ruiz, Victoria

AU - Mas, Ariadna

AU - Martín-Villalba, Inés

AU - Caballo, Alejandro

AU - Julia-Parisad, Esteva

AU - Rodríguez-Rey, Arturo

AU - Piazza, Flavia

AU - José Valdesoiro, Francisco

AU - Rodriguez-Torrella, Claudia

AU - Espinosa, Marta

AU - Virgili, Giulia

AU - Sorroche, Carlota

AU - Ruiz, Alicia

AU - Solanes, Aleix

AU - Radua, Joaquim

AU - Antonieta Also, María

AU - Sant, Elisenda

AU - Murgui, Sandra

AU - Sans-Corrales, Mireia

AU - Young, Allan H.

AU - Vicens, Victor

AU - Blanch, Jordi

AU - Caballeria, Elsa

AU - López-Pelayo, Hugo

AU - López, Clara

AU - Olivé, Victoria

AU - Pujol, Laura

AU - Quesada, Sebastiana

AU - Solé, Brisa

AU - Torrent, Carla

AU - Martínez-Aran, Anabel

AU - Guarch, Joana

AU - Navinés, Ricard

AU - Murru, Andrea

AU - Fico, Giovanna

AU - de Prisco, Michele

AU - Oliva, Vicenzo

AU - Amoretti, Silvia

AU - Pio-Carrino, Casimiro

AU - Fernández-Canseco, María

AU - Villegas, Marta

AU - Vieta, Eduard

AU - Hidalgo-Mazzei, Diego

N1 - Funding Information: We are grateful to all participants. GA is supported by a Rio Hortega 2021 grant (CM21/00017) from the Spanish Ministry of Health financed by the Instituto de Salud Carlos III (ISCIII) and cofinanced by Fondo Social Europe Plus. MS was supported by a grant from the Baszucki Brain Research Fund. AM is supported by the Agència de Gestió d’Ajudes Universitàries I de Investigació—PANDÈMIES 2020 grant (PI047003) from the Generalitat de Catalunya. IG thanks the support of the Spanish Ministry of Science and Innovation (PI19/00954) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirección General de Evaluación y el Fondos Europeos de la Unión Europea (FEDER, Fondo Social Europe, Next Generation European Union or Plan de Recuperación Transformación y Resiliencia_PRTR); the Instituto de Salud Carlos III; the CIBER of Mental Health (CIBERSAM); and the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement (2017 SGR 1365), CERCA Programme or Generalitat de Catalunya as well as the Fundació Clínic per la Recerca Biomèdica (Pons Bartran 2022-FRCB_PB1_2022). AHY’s independent research was funded by the National Institute for Health Research Biomedical Research Centre in South London and Maudsley National Health Service Foundation Trust and King’s College London. The views expressed are those of the authors and not necessarily those of the National Health Service, National Institute for Health and Care Research, or Department of Health. JR is supported by a Miguel Servet II contract (CPII19/00009), funded by ISCIII and cofunded by the European Social Fund “Investing in your future.” CT has been supported through a “Miguel Servet” postdoctoral contract (CPI14/00175) and a Miguel Servet II contract (CPII19/00018) and thanks the support of the Spanish Ministry of Innovation and Science (PI17/01066 and PI20/00344), funded by the Instituto de Salud Carlos III and cofinanced by the European Union (FEDER) “Una manera de hacer Europa.” AMA thanks the support of the Spanish Ministry of Science and Innovation (PI18/00789, PI21/00787) integrated into the Plan Nacional de I+D+I and cofinanced by ISCIII-Subdirección General de Evaluación and the Fondo Europeo de Desarrollo Regional (FEDER); the ISCIII; the CIBER of Mental Health (CIBERSAM); the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement (2017 SGR 1365); the CERCA Programme; and the Departament de Salut de la Generalitat de Catalunya for the Pla estratègic de recerca I innovació en salut (PERIS) grant SLT006/17/00177. AM thanks the support of the Spanish Ministry of Science and Innovation (PI19/00672) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirección General de Evaluación and the FEDER. GF is supported by a fellowship from “La Caixa” Foundation (ID 100010434)—fellowship code—LCF/BQ/DR21/11880019. SA has been supported by a Sara Borrell contract (CD20/00177), funded by ISCIII and founded by the European Social Fund “Investing in your future.” EV thanks the support of the Spanish Ministry of Science, Innovation and Universities (PI15/00283, PI18/00805, PI19/00394, PI21/00787, and CPII19/00009) integrated into the Plan Nacional de I+D+I and cofinanced by the ISCIII-Subdirección General de Evaluación and the FEDER; the ISCIII; the CIBER of Mental Health (CIBERSAM); the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement (2017 SGR 1365), and the CERCA Programme or Generalitat de Catalunya. We would like to thank the Departament de Salut de la Generalitat de Catalunya for the PERIS grant SLT006/17/00357. DHM´s research was supported by Juan Rodés JR18/00021 granted by the ISCIII. The PRESTO project has been funded by Fundació Clínic per a la Recerca Biomèdica through the Pons Bartran 2020 grant (PI046549). The development of a version of the digital solution adapted to health workers is funded by the Spanish Foundation for Psychiatry and Mental Health, Spanish Psychiatric Society, and Spanish Society of Biological Psychiatry (PI046813). The enhancement of the digital solution with Natural Language Processing techniques in a chatbot user-interface in collaboration with the text mining technologies in the health domain of the Barcelona Supercomputing Center is funded by the Agència de Gestió d’Ajudes Universitàries I de Investigació—PANDÈMIES 2020 grant (PI047003), from La Generalitat de Catalunya. Publisher Copyright: © 2023 Journal of Medical Internet Research. All rights reserved.

PY - 2023

Y1 - 2023

N2 - Background: Many people attending primary care (PC) have anxiety-depressive symptoms and work-related burnout compounded by a lack of resources to meet their needs. The COVID-19 pandemic has exacerbated this problem, and digital tools have been proposed as a solution. Objective: We aimed to present the development, feasibility, and potential effectiveness of Vickybot, a chatbot aimed at screening, monitoring, and reducing anxiety-depressive symptoms and work-related burnout, and detecting suicide risk in patients from PC and health care workers. Methods: Healthy controls (HCs) tested Vickybot for reliability. For the simulation study, HCs used Vickybot for 2 weeks to simulate different clinical situations. For feasibility and effectiveness study, people consulting PC or health care workers with mental health problems used Vickybot for 1 month. Self-assessments for anxiety (Generalized Anxiety Disorder 7-item) and depression (Patient Health Questionnaire-9) symptoms and work-related burnout (based on the Maslach Burnout Inventory) were administered at baseline and every 2 weeks. Feasibility was determined from both subjective and objective user-engagement indicators (UEIs). Potential effectiveness was measured using paired 2-tailed t tests or Wilcoxon signed-rank test for changes in self-assessment scores. Results: Overall, 40 HCs tested Vickybot simultaneously, and the data were reliably transmitted and registered. For simulation, 17 HCs (n=13, 76% female; mean age 36.5, SD 9.7 years) received 98.8% of the expected modules. Suicidal alerts were received correctly. For the feasibility and potential effectiveness study, 34 patients (15 from PC and 19 health care workers; 76% [26/34] female; mean age 35.3, SD 10.1 years) completed the first self-assessments, with 100% (34/34) presenting anxiety symptoms, 94% (32/34) depressive symptoms, and 65% (22/34) work-related burnout. In addition, 27% (9/34) of patients completed the second self-assessment after 2 weeks of use. No significant differences were found between the first and second self-assessments for anxiety (t8=1.000; P=.34) or depressive (t8=0.40; P=.70) symptoms. However, work-related burnout scores were moderately reduced (z=−2.07, P=.04, r=0.32). There was a nonsignificant trend toward a greater reduction in anxiety-depressive symptoms and work-related burnout with greater use of the chatbot. Furthermore, 9% (3/34) of patients activated the suicide alert, and the research team promptly intervened with successful outcomes. Vickybot showed high subjective UEI (acceptability, usability, and satisfaction), but low objective UEI (completion, adherence, compliance, and engagement). Vickybot was moderately feasible. Conclusions: The chatbot was useful in screening for the presence and severity of anxiety and depressive symptoms, and for detecting suicidal risk. Potential effectiveness was shown to reduce work-related burnout but not anxiety or depressive symptoms. Subjective perceptions of use contrasted with low objective-use metrics. Our results are promising but suggest the need to adapt and enhance the smartphone-based solution to improve engagement. A consensus on how to report UEIs and validate digital solutions, particularly for chatbots, is required.

AB - Background: Many people attending primary care (PC) have anxiety-depressive symptoms and work-related burnout compounded by a lack of resources to meet their needs. The COVID-19 pandemic has exacerbated this problem, and digital tools have been proposed as a solution. Objective: We aimed to present the development, feasibility, and potential effectiveness of Vickybot, a chatbot aimed at screening, monitoring, and reducing anxiety-depressive symptoms and work-related burnout, and detecting suicide risk in patients from PC and health care workers. Methods: Healthy controls (HCs) tested Vickybot for reliability. For the simulation study, HCs used Vickybot for 2 weeks to simulate different clinical situations. For feasibility and effectiveness study, people consulting PC or health care workers with mental health problems used Vickybot for 1 month. Self-assessments for anxiety (Generalized Anxiety Disorder 7-item) and depression (Patient Health Questionnaire-9) symptoms and work-related burnout (based on the Maslach Burnout Inventory) were administered at baseline and every 2 weeks. Feasibility was determined from both subjective and objective user-engagement indicators (UEIs). Potential effectiveness was measured using paired 2-tailed t tests or Wilcoxon signed-rank test for changes in self-assessment scores. Results: Overall, 40 HCs tested Vickybot simultaneously, and the data were reliably transmitted and registered. For simulation, 17 HCs (n=13, 76% female; mean age 36.5, SD 9.7 years) received 98.8% of the expected modules. Suicidal alerts were received correctly. For the feasibility and potential effectiveness study, 34 patients (15 from PC and 19 health care workers; 76% [26/34] female; mean age 35.3, SD 10.1 years) completed the first self-assessments, with 100% (34/34) presenting anxiety symptoms, 94% (32/34) depressive symptoms, and 65% (22/34) work-related burnout. In addition, 27% (9/34) of patients completed the second self-assessment after 2 weeks of use. No significant differences were found between the first and second self-assessments for anxiety (t8=1.000; P=.34) or depressive (t8=0.40; P=.70) symptoms. However, work-related burnout scores were moderately reduced (z=−2.07, P=.04, r=0.32). There was a nonsignificant trend toward a greater reduction in anxiety-depressive symptoms and work-related burnout with greater use of the chatbot. Furthermore, 9% (3/34) of patients activated the suicide alert, and the research team promptly intervened with successful outcomes. Vickybot showed high subjective UEI (acceptability, usability, and satisfaction), but low objective UEI (completion, adherence, compliance, and engagement). Vickybot was moderately feasible. Conclusions: The chatbot was useful in screening for the presence and severity of anxiety and depressive symptoms, and for detecting suicidal risk. Potential effectiveness was shown to reduce work-related burnout but not anxiety or depressive symptoms. Subjective perceptions of use contrasted with low objective-use metrics. Our results are promising but suggest the need to adapt and enhance the smartphone-based solution to improve engagement. A consensus on how to report UEIs and validate digital solutions, particularly for chatbots, is required.

KW - anxiety

KW - burnout

KW - chatbot

KW - depression

KW - digital

KW - health care workers

KW - PRESTO

KW - primary care

KW - primary care digital support tool in mental health

KW - smartphone

KW - symptom

UR - http://www.scopus.com/inward/record.url?scp=85152155926&partnerID=8YFLogxK

U2 - 10.2196/43293

DO - 10.2196/43293

M3 - Article

C2 - 36719325

AN - SCOPUS:85152155926

SN - 1438-8871

VL - 25

JO - Journal of Medical Internet Research

JF - Journal of Medical Internet Research

M1 - A63

ER -

Vickybot, a Chatbot for Anxiety-Depressive Symptoms and Work-Related Burnout in Primary Care and Health Care Professionals: Development, Feasibility, and Potential Effectiveness Studies

Abstract

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this