Evaluation of a community-led intervention in South London: How much standardization is possible?

Derek Bolton; Nina Khazaezadeh; Ewan Carr; Matthew Bolton; Eirini Platsa; Imogen Moore-Shelley; Ana Luderowski; Jill Demilew; June Brown

doi:10.3390/ijerph17072523

Evaluation of a community-led intervention in South London: How much standardization is possible?

Derek Bolton^*, Nina Khazaezadeh, Ewan Carr, Matthew Bolton, Eirini Platsa, Imogen Moore-Shelley, Ana Luderowski, Jill Demilew, June Brown

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

7 Citations (Scopus)

Abstract

It is widely recognized that public health interventions benefit from community engagement and leadership, yet there are challenges to evaluating complex, community-led interventions assuming hierarchies of evidence derived from laboratory experimentation and clinical trials. Particular challenges include, first, the inconsistency of the intervention across sites and, second, the absence of researcher control over the sampling frame and methodology. This report highlights these challenges as they played out in the evaluation of a community-organized health project in South London. The project aimed to benefit maternal mental health, health literacy, and social capital, and especially to engage local populations known to have reduced contact with statutory services. We evaluated the project using two studies with different designs, sampling frames, and methodologies. In one, the sampling frame and methodology were under community control, permitting a comparison of change in outcomes before and after participation in the project. In the other, the sampling frame and methodology were under researcher control, permitting a case-control design. The two evaluations led to different results, however: Participants in the community-controlled study showed benefits, while participants in the researcher-controlled study did not. The principal conclusions are that while there are severe challenges to evaluating a community-led health intervention using a controlled design, the measurement of pre-/post-participation changes in well-defined health outcomes should typically be a minimum evaluation requirement, and confidence in attributing causation of any positive changes to participation can be increased by use of interventions in the project and in the engagement process itself that have a credible theoretical and empirical basis.

Original language	English
Article number	2523
Journal	International Journal of Environmental Research and Public Health
Volume	17
Issue number	7
DOIs	https://doi.org/10.3390/ijerph17072523
Publication status	Published - 1 Apr 2020

Keywords

Citizens UK
Community engagement
Community health
Community organizing
Complex interventions
Evaluation
Health inequalities
Hierarchy of evidence
Methodology
PACT

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.3390/ijerph17072523

Cite this

@article{ea8fcaca085d43d2a927d60a0f8b0377,

title = "Evaluation of a community-led intervention in South London: How much standardization is possible?",

abstract = "It is widely recognized that public health interventions benefit from community engagement and leadership, yet there are challenges to evaluating complex, community-led interventions assuming hierarchies of evidence derived from laboratory experimentation and clinical trials. Particular challenges include, first, the inconsistency of the intervention across sites and, second, the absence of researcher control over the sampling frame and methodology. This report highlights these challenges as they played out in the evaluation of a community-organized health project in South London. The project aimed to benefit maternal mental health, health literacy, and social capital, and especially to engage local populations known to have reduced contact with statutory services. We evaluated the project using two studies with different designs, sampling frames, and methodologies. In one, the sampling frame and methodology were under community control, permitting a comparison of change in outcomes before and after participation in the project. In the other, the sampling frame and methodology were under researcher control, permitting a case-control design. The two evaluations led to different results, however: Participants in the community-controlled study showed benefits, while participants in the researcher-controlled study did not. The principal conclusions are that while there are severe challenges to evaluating a community-led health intervention using a controlled design, the measurement of pre-/post-participation changes in well-defined health outcomes should typically be a minimum evaluation requirement, and confidence in attributing causation of any positive changes to participation can be increased by use of interventions in the project and in the engagement process itself that have a credible theoretical and empirical basis.",

keywords = "Citizens UK, Community engagement, Community health, Community organizing, Complex interventions, Evaluation, Health inequalities, Hierarchy of evidence, Methodology, PACT",

author = "Derek Bolton and Nina Khazaezadeh and Ewan Carr and Matthew Bolton and Eirini Platsa and Imogen Moore-Shelley and Ana Luderowski and Jill Demilew and June Brown",

year = "2020",

month = apr,

day = "1",

doi = "10.3390/ijerph17072523",

language = "English",

volume = "17",

journal = "International Journal of Environmental Research and Public Health",

issn = "1661-7827",

publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",

number = "7",

}

TY - JOUR

T1 - Evaluation of a community-led intervention in South London

T2 - How much standardization is possible?

AU - Bolton, Derek

AU - Khazaezadeh, Nina

AU - Carr, Ewan

AU - Bolton, Matthew

AU - Platsa, Eirini

AU - Moore-Shelley, Imogen

AU - Luderowski, Ana

AU - Demilew, Jill

AU - Brown, June

PY - 2020/4/1

Y1 - 2020/4/1

N2 - It is widely recognized that public health interventions benefit from community engagement and leadership, yet there are challenges to evaluating complex, community-led interventions assuming hierarchies of evidence derived from laboratory experimentation and clinical trials. Particular challenges include, first, the inconsistency of the intervention across sites and, second, the absence of researcher control over the sampling frame and methodology. This report highlights these challenges as they played out in the evaluation of a community-organized health project in South London. The project aimed to benefit maternal mental health, health literacy, and social capital, and especially to engage local populations known to have reduced contact with statutory services. We evaluated the project using two studies with different designs, sampling frames, and methodologies. In one, the sampling frame and methodology were under community control, permitting a comparison of change in outcomes before and after participation in the project. In the other, the sampling frame and methodology were under researcher control, permitting a case-control design. The two evaluations led to different results, however: Participants in the community-controlled study showed benefits, while participants in the researcher-controlled study did not. The principal conclusions are that while there are severe challenges to evaluating a community-led health intervention using a controlled design, the measurement of pre-/post-participation changes in well-defined health outcomes should typically be a minimum evaluation requirement, and confidence in attributing causation of any positive changes to participation can be increased by use of interventions in the project and in the engagement process itself that have a credible theoretical and empirical basis.

AB - It is widely recognized that public health interventions benefit from community engagement and leadership, yet there are challenges to evaluating complex, community-led interventions assuming hierarchies of evidence derived from laboratory experimentation and clinical trials. Particular challenges include, first, the inconsistency of the intervention across sites and, second, the absence of researcher control over the sampling frame and methodology. This report highlights these challenges as they played out in the evaluation of a community-organized health project in South London. The project aimed to benefit maternal mental health, health literacy, and social capital, and especially to engage local populations known to have reduced contact with statutory services. We evaluated the project using two studies with different designs, sampling frames, and methodologies. In one, the sampling frame and methodology were under community control, permitting a comparison of change in outcomes before and after participation in the project. In the other, the sampling frame and methodology were under researcher control, permitting a case-control design. The two evaluations led to different results, however: Participants in the community-controlled study showed benefits, while participants in the researcher-controlled study did not. The principal conclusions are that while there are severe challenges to evaluating a community-led health intervention using a controlled design, the measurement of pre-/post-participation changes in well-defined health outcomes should typically be a minimum evaluation requirement, and confidence in attributing causation of any positive changes to participation can be increased by use of interventions in the project and in the engagement process itself that have a credible theoretical and empirical basis.

KW - Citizens UK

KW - Community engagement

KW - Community health

KW - Community organizing

KW - Complex interventions

KW - Evaluation

KW - Health inequalities

KW - Hierarchy of evidence

KW - Methodology

KW - PACT

UR - http://www.scopus.com/inward/record.url?scp=85083209581&partnerID=8YFLogxK

U2 - 10.3390/ijerph17072523

DO - 10.3390/ijerph17072523

M3 - Article

C2 - 32272680

AN - SCOPUS:85083209581

SN - 1661-7827

VL - 17

JO - International Journal of Environmental Research and Public Health

JF - International Journal of Environmental Research and Public Health

IS - 7

M1 - 2523

ER -

Evaluation of a community-led intervention in South London: How much standardization is possible?

Abstract

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this