Singla et al.

BMC Psychiatry (2023) 23:817 BMC Psychiatry

https://doi.org/10.1186/s12888-023-05318-2

STUDY PROTOCOL Open Access

Protocol for an economic evaluation

of scalable strategies to improve mental health
among perinatal women: non‑specialist care
delivered via telemedicine vs. specialist care
delivered in‑person
Daisy R. Singla1,2,3*†, Claire de Oliveira1,4,5†, Sean M. Murphy6, Vikram Patel7,8, Jaime Charlebois9,
Wendy N. Davis10, Cindy‑Lee Dennis2,3,11, J. Jo Kim12, Paul Kurdyak1,2,5, Andrea Lawson1,
Samantha Meltzer‑Brody13, Benoit H. Mulsant1,2, Nour Schoueri‑Mychasiw3, Richard K. Silver12,14,
Dana Tschritter3, Simone N. Vigod2,15 and Sarah Byford16

Abstract
Background Perinatal depression affects an estimated 1 in 5 women in North America during the perinatal period,
with annualized lifetime costs estimated at $20.6 billion CAD in Canada and over $45.9 billion USD in the US. Access
to psychological treatments remains limited for most perinatal women suffering from depression and anxiety. Some
barriers to effective care can be addressed through task-sharing to non-specialist providers and through telemedicine
platforms. The cost-effectiveness of these strategies compared to traditional specialist and in-person models remains
unknown. This protocol describes an economic evaluation of non-specialist providers and telemedicine, in com‑
parison to specialist providers and in-person sessions within the ongoing Scaling Up Maternal Mental healthcare
by Increasing access to Treatment (SUMMIT) trial.
Methods The economic evaluation will be undertaken alongside the SUMMIT trial. SUMMIT is a pragmatic, rand‑
omized, non-inferiority trial across five North American study sites (N = 1,226) of the comparable effectiveness of two
types of providers (specialist vs. non-specialist) and delivery modes (telemedicine vs. in-person) of a behavioural
activation treatment for perinatal depressive and anxiety symptoms. The primary economic evaluation will be a cost-
utility analysis. The outcome will be the incremental cost-effectiveness ratio, which will be expressed as the additional
cost required to achieve an additional quality-adjusted life-year, as assessed by the EuroQol 5-Dimension 5-Level
instrument. A secondary cost-effectiveness analysis will use participants’ depressive symptom scores. A micro-
costing analysis will be conducted to estimate the resources/costs required to implement and sustain the inter‑
ventions; healthcare resource utilization will be captured via self-report. Data will be pooled and analysed using
uniform price and utility weights to determine cost-utility across all trial sites. Secondary country-specific cost-utility

†
Daisy R. Singla and Claire de Oliveira are Joint first authors.
*Correspondence:
Daisy R. Singla
[email protected]
Full list of author information is available at the end of the article

© The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecom‑
mons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Singla et al. BMC Psychiatry (2023) 23:817 Page 2 of 11

and cost-effectiveness analyses will also be completed. Sensitivity analyses will be conducted, and cost-effectiveness
acceptability-curves will be generated, in all instances.
Discussion Results of this study are expected to inform key decisions related to dissemination and scale up of evi‑
dence-based psychological interventions in Canada, the US, and possibly worldwide. There is potential impact
on real-world practice by informing decision makers of the long-term savings to the larger healthcare setting in ser‑
vices to support perinatal women with common mental health conditions.
Keywords Economic evaluation, Protocol, Perinatal mental health, Randomized controlled trial

Background treatments. For example, among outpatients with major

Depression is the leading cause of disability among peri- depressive disorder, computer-assisted forms of cogni-
natal (i.e., pregnant and up to 1 year postpartum) women tive behaviour therapy were highly cost-effective com-
worldwide [1], affecting an estimated 1 in 5 women dur- pared to conventional therapy and reduced the adjusted
ing the perinatal period in Canada and the United States cost of treatment by $945 USD per patient [18]. Another
(US) [2–4]. In this study, we define perinatal women as study among veterans receiving behavioural activation
all perinatal persons regardless of their gender identity or found that, although intervention costs for telehealth
expression and will refer to the sample as ‘women’ here- were higher relative to in-person care, veterans receiv-
after. Based on estimates from the United Kingdom (UK) ing behavioural activation via telehealth had lower health
[5], the annualized lifetime costs of perinatal depression costs one year after the intervention than those receiv-
may be as high as $20.6 billion CAD in Canada and over ing care in person [19]. Recent research from the UK
$45.9 billion USD in the US. Psychological treatments, predicted that internet-delivered behavioural therapy
such as cognitive, behavioural and interpersonal thera- may achieve equivalent wellbeing outcomes with notably
pies, have consistently been shown to be effective in pre- lower costs compared to in-person behavioural therapy
venting and treating perinatal depression and anxiety [6, over a two year horizon [20].
7] and are generally preferred over medication as a first In terms of perinatal mental health, a recent review on
line treatment by perinatal women [8, 9]. Unfortunately, the cost-effectiveness of psychological interventions for
access to psychological care remains limited for most perinatal populations identified only eight studies that
perinatal women with depression or anxiety [10]. Some targeted perinatal depression or anxiety symptoms, with
barriers to effective mental health care can be addressed inconclusive results and few comparisons of active treat-
through task-sharing—the rationale distribution of tasks ments [21]. Two interventions were found to be likely
to non-specialist providers [11, 12] (i.e., individuals cost-effective: (i) screening and talk therapy (i.e., cogni-
with no formal training or degree in mental health care tive behaviour approach or person-centered approach)
[13]) and through telemedicine platforms, which offer delivered by a health visitor [22] (i.e., a community health
an alternative approach for perinatal women in terms of care worker focusing on perinatal populations); and (ii)
flexibility [14], efficiency [15], and cost [16]. However, psychiatrist-supported general practitioner screening
the cost-effectiveness of these more scalable approaches and treatment of postpartum depression and psychosis
compared to traditional specialist and in-person models [23]. Despite the widespread effectiveness of psychologi-
remains unknown. This is a significant gap that must be cal treatments for perinatal populations [6, 7, 24], there
addressed to inform policy and wide scale uptake of these remains a paucity of evidence on the cost-effectiveness
approaches. of psychological treatments for perinatal depression and
anxiety. This research is essential to inform the transla-
tion of research into practice and policy. Moreover, this
Economic evidence of psychological treatments information can serve as an important input to guide
Globally, investing in psychological treatments for decisions around resource allocation.
depression and anxiety has been shown to provide a sub-
stantial return on investment. Benefit-to-cost ratios of Objective
2.3–3.0 to 1 have been shown when considering only the The objective of the proposed economic evaluation is to
economic benefits of enhanced labor productivity, and assess the cost-effectiveness of provider type – non-spe-
substantially more (3.3–5.7 to 1) when the intrinsic value cialist (e.g., nurses or midwives) vs. specialist providers
of improved health returns is also included [17]. Increas- (e.g., psychiatrists, psychologists, or social workers) – and
ingly, studies are also examining the cost-effectiveness mode of delivery – telemedicine vs. in-person sessions –
of telemedicine platforms to deliver psychological within the Scaling Up Maternal Mental health care by
Singla et al. BMC Psychiatry (2023) 23:817 Page 3 of 11

Increasing access to Treatment (SUMMIT) trial [25]. In and Accountability Act (PHIPA/HIPAA). Study tablets
line with the larger SUMMIT trial, the primary outcome and access to high-speed Internet are provided, with
will be derived from data pooled across both countries instructions for use, on a temporary basis to participants
(Canada and the US) using a common set of unit costs. lacking access to a phone, tablet, or computer. Partici-
This approach will enhance internal, and possibly exter- pants assigned to telemedicine can do their behavioural
nal, validity [26]. Due to cross-border heterogeneity in activation sessions in whatever private location they pre-
factors such as healthcare practice and resource utiliza- fer (e.g., home or elsewhere). In-person sessions are held
tion patterns, a secondary country-specific analysis will at participating clinical care sites. During the COVID-19
be conducted. Outcomes from this secondary analysis pandemic, in-person arms were temporarily suspended
will be derived independently using country-specific data to decrease the number of non-urgent visits to hospi-
[26]. tals and clinics and to decrease the risk of exposure to
COVID-19 for both study participants and treatment
Methods providers, in line with institutional guidelines. Additional
Brief description of the intervention and comparators details can be found elsewhere [32]. The larger SUM-
SUMMIT is a large, multi-site, four-arm, randomized, MIT trial has been registered on ClinicalTrial.gov, NCT
non-inferiority trial for perinatal women with depres- 04153864.
sive and anxiety symptoms. It is currently being imple-
mented in Toronto, Chicago and Chapel Hill (N = 1,226). Setting and sample
The objective of the SUMMIT trial is to compare the The clinical trial is currently being conducted at three
effectiveness of provider (non-specialist vs. specialist academic hospitals in Toronto, Ontario, Canada (Sinai
providers) and delivery mode (telemedicine vs. in-per- Health, Women’s College Hospital, and St. Michael’s
son), implementing a brief, evidence-based behavioural Hospital), and two health care institutions in the US
activation treatment. The treatment consists of six to (NorthShore University HealthSystem, Evanston, Illinois
eight individual weekly sessions and the same treat- and UNC Health and UNC School of Medicine, Chapel
ment is provided to individuals in all four study arms – Hill, North Carolina). A total of N = 1,226 pregnant
telemedicine non-specialist, telemedicine specialist, and postpartum women are being recruited through an
in-person non-specialist, and in-person specialist. The established extensive referral system at each site. This
current treatment manual has been adapted from two, sample size was determined by the larger SUMMIT Trial
well-established source manuals: the Alma Program for [32] and detailed elsewhere (ClinicalTrials.gov NCT
perinatal populations in Colorado [27], and the Healthy 04153864). The sample size calculation was based on
Activity Program from Goa, India [28, 29]. Key treatment the primary outcome Edinburgh Postnatal Depression
strategies include psychoeducation, behaviour assess- Scale (EPDS [33]), with an EPDS mean estimate of 7.93
ment, values-based activity monitoring and structuring, (SD = 4.68) [34] and powered to estimate the two primary
interpersonal effectiveness, and problem solving. Unlike comparisons of treatment provider type (non-specialist
traditional cognitive behaviour interventions for depres- vs. specialist provider) and delivery mode (telemedicine
sion, behavioural activation explicitly targets avoidant vs. in-person) within a non-inferiority design. The clinical
coping and has also been effective in reducing symptoms trial enrolls women with an EPDS score ≥ 10 (indicative
of anxiety [30]. of minor and major depression [3, 35]); ≥ 18 years of age;
The SUMMIT treatment providers are either non-spe- pregnant up to 36 weeks or 4 to 30 weeks postpartum;
cialist or specialist providers. Non-specialist providers and who speak English (in Canada) or English or Span-
include registered and practictioner nurses, midwives, or ish (in the US). The trial excludes women with active sui-
doulas with general or obstetric health care professional cidal intent; symptoms of psychosis or mania; initiation
skills but without formal training in mental health care or change in psychotropic medication or dosage within
or experience delivering psychological treatments. Spe- 2 weeks of enrollment; ongoing psychotherapy; active
cialist providers include individuals with formal training substance use or dependence; or severe fetal anomalies,
in mental health care delivery (e.g., psychiatrists, psy- stillbirth, or infant death at time of enrolment for index
chologists, or social workers) and a minimum of 5 years pregnancy. Recruitment for the larger SUMMIT trial

Telemedicine is implemented via Zoom™ in Toronto and

of experience delivering psychological treatments [31]. began on January 06 2020 and data collection is ongoing.

Chicago, and Webex™ in Chapel Hill. All platforms per- Decision problem
mit video visits; are accessible on PC, Mac, Android, and The SUMMIT trial will assess the effectiveness of pro-
iOS systems; and are compliant with the Personal Health vider and delivery mode in implementing a brief,
Information Protection Act/Health Insurance Portability evidence-based behavioural activation treatment for
Singla et al. BMC Psychiatry (2023) 23:817 Page 4 of 11

perinatal women with depressive and anxiety symptoms. in a cost-effectiveness analysis is recommended to repre-
Therefore, the proposed economic evaluation of SUM- sent uncertainty [39] and to help interpret the economic
MIT will determine the relative cost-effectiveness of the results [26, 40]. For these reasons, a cost-effectiveness
four study arms created by randomizing participants to analysis will be undertaken, regardless of whether non-
non-specialist vs. specialist providers and to telemedi- inferiority in the primary clinical outcome is demon-
cine vs. in-person sessions (in line with the clinical analy- strated. Cost-effectiveness will be determined using the
ses). In the secondary country-specific analysis, separate net benefit approach [41] with reference to Bosmans’
analyses will be undertaken for each country following methods for economic evaluations alongside equivalence
the current guidelines most applicable to each context. or non-inferiority trials [40]. See Table 1 for a summary
The guidelines recommended by the Canadian Agency of the economic evaluation outcome measures.
for Drugs and Technologies in Health (CADTH) [36] will
be followed in Canada while the Second Panel on Cost-
Time horizon and discounting
Effectiveness in Health and Medicine [37] and Glick et al.
The time horizon of this study will be the length of the
[26] will be followed for the American analysis. Both
trial, which is 12-months post-randomization. Given this
analyses will follow the Consolidated Health Economic
time horizon, discounting will not be required.
Evaluation Reporting Standards (CHEERS) 2022 report-
ing guidance for health economic evaluations [38].
The primary economic evaluation will be a cost-utility Measurement and evaluation of health
analysis where the incremental cost-effectiveness ratio The primary outcome of the proposed cost-effectiveness
(ICER) will be expressed as the additional cost required analysis will be the QALY, a measure that combines the
to achieve an additional quality-adjusted life-year health-related quality of life associated with an individ-
(QALY). This analysis will be carried out at the 3-month ual’s health state and their time spent in that state. The
(intervention) and 12-month (intervention + follow-up) QALY is recommended as the primary effectiveness
post-randomization assessments, based on non-missing measure in economic evaluation studies due to its abil-
data (i.e., excluding individuals lost to follow-up or with ity to be compared across interventions and illnesses/
missing cost and/or outcome data). The potential impact disorders. Health-related quality-of-life will be measured
of non-responders will be examined through sensitiv- by the EuroQol 5-Dimension 5-Level (EQ-5D-5L) instru-
ity analyses (described below). The 12-month ICER will ment [42]. The EQ-5D-5L consists of five health state
serve as the primary outcome of interest. A secondary dimensions (mobility, self-care, usual activity, pain/dis-
analysis will include a cost-effectiveness analysis using comfort, and anxiety/depression) on which the respond-
the primary clinical outcome measure of the trial, the ent must indicate their health status on one of five
EPDS [33]. The advantage of undertaking this secondary levels (no problems, slight problems, moderate problems,
analysis is the opportunity to explore cost-effectiveness severe problems, and extreme problems). An individual is
using a measure that may be more sensitive to change in assigned to a unique health state according to their com-
the population of interest. While QALYs are preferred bination of answers across the five domains, and each
to support comparisons across disorders and popula- health state is associated with a utility value that reflects
tions, their broad, generic nature can make them less society’s preference for that state. This valuation is indi-
sensitive to change in specific populations. While stud- cated by a number typically ranging between 0 (worst
ies designed to test equivalence of effects are a legitimate imaginable condition: death) to 1 (perfect health), with
situation in which a cost-minimisation analysis (i.e., an standardized estimates by country. The EQ-5D-5L util-
analysis where costs alone are compared given equal ity values can range from -0.285 to 1, with values below
outcomes) may be appropriate [39], the same may not 0 representing states perceived to be worse than death.
hold for trials with non-inferiority designs. Even in cases Canada- [43] and US-specific utility weights [44] will be
where equivalence or non-inferiority are demonstrated, used for the pooled and respective country-specific eco-
exploration of the joint distribution of costs and effects nomic evaluations [26].

Table 1 Economic evaluation outcome measures

Outcome Assessment Measurement Economic evaluation framework

Edinburgh Postnatal baseline, session-wise, 3-, 6-, 12-months post-randomi‑ perinatal depressive cost-effectiveness analysis
Depression Scale (EPDS) zation symptoms
EuroQol 5 Dimension 5 baseline, 3-, 6-, 12-months post-randomization quality of life cost-utility analysis
Level (EQ-5D-5L)
Singla et al. BMC Psychiatry (2023) 23:817 Page 5 of 11

As mentioned above, a secondary cost-effectiveness of multiplying unit costs for each resource by the num-
analysis will be conducted using the primary clinical ber of resource units reported by a given participant [37,
outcome of the SUMMIT trial, participants’ depres- 54]. Unit costs in Canada will be obtained from the Cana-
sive symptom scores measured by the EPDS [33]. The dian Institute for Health Information, the Ontario Health
EPDS is a short, time-efficient, internationally used, and Insurance Plan fee schedule, and other sources. In the
freely-available 10-item measure, which has been vali- US, Medicare fee-for-service payments will be used to
dated among diverse populations across the postpartum value health care services [55], and the US Department
period [45], including remotely [46]. The EPDS is also the of Veterans Affairs Federal Supply Schedule will be used
standardized tool in Canada and the US to assess perina- to value all medications, as per the current guidelines
tal mental health issues among pregnant and postpartum [37]. All costs will be adjusted to 2025 Canadian and US
individuals. An EPDS ≥ 10 cut-off has been suggested dollars using Statistics Canada’s Consumer Price Index
to encompass both minor and major depression [3] for for Health and Personal Care [56], and the US Bureau of
antenatal, postnatal, and community-based populations Labor Statistics’ Medical Care Index [57], respectively.
with excellent sensitivity and specificity [47, 48].
Statistical analyses
Resource use and costs Data analysis will be done using SAS (version 9.4) and
Intervention resource use and costs Stata (version 17.0). Participants will be analysed on an
Data on the use of the behavioural activation treat- intention-to-treat basis, according to the group they were
ment (i.e., the number and duration of therapy contacts randomized to, regardless of intervention compliance.
and with whom) will be collected from existing clinical Comparisons will be made between non-specialist and
records and study data. The use of behavioural activa- specialist providers and between telemedicine and in-
tion will be directly costed for each arm using a standard person sessions, in line with the clinical analyses. Costs
micro-costing approach [49]. Unit costs for behavioural and outcomes will be compared at 3- and 12-months and
activation will include all hospital and employer costs will be presented as mean values by arm with standard
(provider salaries, contributions to pensions, etc.) and deviations. Mean differences and 95% confidence inter-
appropriate overhead (capital, managerial, administra- vals will be obtained by non-parametric bootstrap regres-
tive, etc.). The cost of supervision will be included and sions to account for the non-normal distribution that is
the time each therapist spends on various direct and commonly found in cost data [58].
indirect participant-related activities (non-participant All cost and effectiveness measures will be modeled
contact time including training, administration, meetings according to the person period over the 3-month inter-
with other professionals, etc.) will be estimated using vention phase, and the entire 12-month study period.
a questionnaire developed by the research team. See Analyses will be conducted using multivariable gener-
Table 2 for a list of intervention resources, their respec- alized linear mixed models (GLMMs) [26]. The GLMM
tive unit costs and source. is a flexible model that allows for the assessment/choice
of the most appropriate mean and variance functions to
Health services utilization be chosen, which is especially important when analyzing
The use of health services will be captured using an costs, given the tendency for non-normal distributions.
adapted version of the Health Services Utilization Ques- Additionally, the GLMM enables the inclusion of random
tionnaire (HSUQ, see Appendix A) [50]. The HSUQ effects, and uses all available data for each participant,
records the self-reported use of health care services, regardless of completeness, which makes it well-suited
such as psychiatric and medical hospital inpatient stays, for intent-to-treat analyses. Resources will be catego-
emergency department visits, outpatient appointments, rized as relevant, and individual models will be estimated
and community health contacts at baseline and at each for each category to predict the mean cost according to
treatment session, 3-, 6- and 12-month follow-up, as well time-period and study arm. The same process will be
as outpatient prescription drugs at baseline and each used to predict mean health utility and EPDS values, by
session only. The HSUQ was adapted with reference to time-period and study arm. The statistical method of
other measures used in perinatal depression populations recycled predictions will be utilized to obtain the final
[51–53]. predicted mean values, which will then be summed and
assessed for statistical significance over both the 3-month
Healthcare costs intervention phase and the entire 12-month study period,
A person-level costing method will be used to value all except for the health utility values, which will be used
health care resources utilized by each participant over to estimate QALYs gained via the area under the curve
the course of the 12-month trial. This approach consists methodology. Finally, the ICER will be calculated as the
Singla et al. BMC Psychiatry (2023) 23:817 Page 6 of 11

Table 2 Measurement and valuation of intervention costs

Category Cost Item Non- Specialist COVID-19 Online/ In-Person Notes
Specialist Provider Precautions Virtual
Provider (SP)
(NSP)

Training Costs Personnel General staff ☒ ☒ ☐ ☒ ☒

training work‑ NSP Time ☒ ☐ ☐ ☒ ☒
shop
SP Time ☐ ☒ ☐ ☒ ☒
Clinical Lead ☒ ☒ ☐ ☒ ☒
time
Competency General staff ☒ ☒ ☐ ☒ ☒
assessment Clinical Lead ☒ ☒ ☐ ☒ ☒
time
Equipment for training ☒ ☒ ☐ ☐ ☒ Flip boards, markers, pens
Standardized Patient ☒ ☐ ☐ ☒ ☒ Hired actors for therapy
(contracted from outside role-plays with new provid‑
organizations) ers to assess therapeutic skills
and competence
Room rental for training ☒ ☒ ☐ ☐ ☒
Printing BA Manual, workbook, ☒ ☒ ☐ ☒ ☒
and value cards per provider
Meals/refreshments ☒ ☒ ☐ ☒ ☒
Office Equip‑ Audio recorder ☒ ☒ ☐ ☒ ☒
ment Office supplies ☒ ☒ ☐ ☐ ☒
Session & Provider Insurance ☒ ☒ ☐ ☒ ☒
Delivery Costs Commute and parking ☒ ☐ ☐ ☐ ☒
compensation for NSPs
Session safety-associated costs, ☒ ☒ ☒ ☒ ☒
e.g. Call with Psychiatrist/ER
visit/Security
Supervision attendance – ☒ ☒ ☐ ☒ ☒
Clinical Lead
Supervision attendance – SPs ☐ ☒ ☐ ☒ ☒
Supervision attendance – NSPs ☒ ☐ ☐ ☒ ☒
(QSUMMIT – compensation
for audio review)
Personnel Compensation for Clinical Lead ☐ ☒ ☐ ☒ ☒ Hired/billing for the intervention
costs (salaries) Compensation for Psychiatrists ☒ ☒ ☐ ☒ ☒ Hired/billing for the intervention
Compensation for NSPs for ses‑ ☒ ☐ ☐ ☒ ☒ Breakdown:
sion delivery; cost to trial/ • Salary cost to NSP and SP
program • Employer costs (contributions
Compensation for SPs; cost ☐ ☒ ☐ ☒ ☒ to pensions/national insurance
to OHIP etc.)
• Plus overheads (fringe)
• Time spent on face to face
and non-face to face contact
Sensitivity Laptops (bought specifically ☒ ☒ ☐ ☒ ☒ May not be used; depends
analysis for study) if participants have their own
Webcams ☒ ☒ ☐ ☒ ☐ already
Headsets ☒ ☒ ☐ ☒ ☐
"Loaner" tablets for participants ☒ ☒ ☐ ☒ ☐
(total of 5 tablets)
Singla et al. BMC Psychiatry (2023) 23:817 Page 7 of 11

Table 2 (continued)
Category Cost Item Non- Specialist COVID-19 Online/ In-Person Notes
Specialist Provider Precautions Virtual
Provider (SP)
(NSP)

Overhead Management/admin ☒ ☒ ☐ ☒ ☒
costs
Internet ☐ ☒ ☐ ☒ ☒
Office space for intervention ☒ ☒ ☐ ☐ ☒
support staff
Clinic rooms for sessions ☐ ☒ ☐ ☐ ☒
Fax service (initial and end ☒ ☒ ☐ ☒ ☒
of treatment)
Utilities ☐ ☒ ☐ ☐ ☒
PHIPPA/HIPPA compliant ☒ ☐ ☒ ☒ ☒ Paid for by the hospital
Zoom – Training
PHIPPA/HIPPA compliant ☒ ☐ ☒ ☒ ☒
Zoom – Session Delivery
Masks, face shield, sanitizing ☒ ☒ ☒ ☐ ☐ Paid for by the hospital
equipment
Maintenance of office ☒ ☒ ☐ ☒ ☒
equipment
Landlines ☒ ☒ ☐ ☒ ☒
BA Behavioural Activation, ER Emergency Room, NSP Non-Specialist Provider, OHIP Ontario Health Insurance Plan, SP Specialist Provider

incremental predicted cost of a chosen strategy relative employed to detect the impact of potential outliers, devi-
to another, divided by the incremental predicted effec- ations from distributional assumptions, and the impact of
tiveness of the two strategies. other baseline prognostic factors.

Sensitivity analysis Uncertainty

To explore the potential impact of excluding non- Nonparametric bootstrapping techniques will be
responders, the sociodemographic and clinical character- employed within the multivariable GLMM framework to
istics of those included in the analyses and those in the estimate standard errors and p-values for each incremen-
full sample will be examined. Furthermore, the primary tal cost (including total and individual resource catego-
and secondary cost-effectiveness analyses will be rerun ries) and effect, while adjusting for sampling uncertainty
with missing total costs and outcomes imputed using in the point estimates. Deterministic one-way sensitivity
model-based multiple imputation by chained equations. analyses will be performed to determine the level of con-
Five imputed data sets will be generated, and results fidence around the resulting ICERs. Based on a net ben-
averaged across these five iterations [59]. Linear mixed efit framework [41], uncertainty will be explored using
models will be used to address repeated measures. The cost-effectiveness planes and cost-effectiveness accept-
robustness of the results will also be assessed regard- ability curves (CEACs). Cost-effectiveness planes illus-
ing variations in the unit cost estimates, and the cost of trate the uncertainty around the estimates of costs and
implementing and managing the interventions. A deter- effects by plotting the bootstrapped cost and effects, with
ministic one-way sensitivity analysis will be conducted to points in each quadrant indicating a different implica-
assess the robustness of the results to variations in unit tion for economic evaluation [60]. CEACs are an alter-
costs, as well as the trial intervention costs, where the native to confidence intervals around ICERs and are
range for the sensitivity analysis will be obtained from derived from the joint distribution of incremental costs
95% confidence intervals. Additionally, results from the and incremental effects (e.g., QALYs gained) using results
relatively robust and efficient GLMM regression will be from the aforementioned non-parametric bootstrapping
compared to those obtained from the relatively transpar- of the observed data. The CEAC depicts the probability
ent ordinary least squares regression, as well as to the that an intervention is cost-effective compared to another
unadjusted mean values. Pattern-mixture models will be for a range of values that a decision maker is willing to
Singla et al. BMC Psychiatry (2023) 23:817 Page 8 of 11

pay for a unit improvement in the outcome of interest example, how to best support non-specialist providers,
[61]. The commonly used cost-effectiveness threshold advocacy for patient-centered priorities and serving as
for Canada is $50,000 CAD per QALY gained [36]; the patient stakeholder representatives. Stakeholders will
recommended cost-effectiveness threshold range in the also help to interpret the results and will play a significant
US is $100,000—$200,000 USD per QALY [37]. Coef- role in the dissemination of the study results. Stakehold-
ficients of differences in net benefits between the trial ers are regularly engaged through multiple platforms to
arms will be obtained through a series of bootstrapped ensure that all voices are represented to inform key deci-
linear regressions, which will include covariates included sion making. This includes attendance at annual stake-
in the main clinical analysis plus the baseline variable holder meetings, attendance at monthly investigator
of interest. The resulting coefficients will then be exam- meetings where key trial decisions and updates are dis-
ined to calculate the proportion of times that the inter- cussed, and participation in qualitative interviews. Three
vention group had a greater net benefit than the control stakeholders have been involved specifically to inform
group for each willingness to pay value [62]. These pro- the proposed economic evaluation from both a patient
portions will then be plotted to generate CEACs for all advocacy and health system perspectives in both Canada
cost-outcome combinations. These curves are a recom- and the US. These stakeholders represent Postpartum
mended decision-making approach to dealing with the Support International [63], the Policy Center for Mater-
uncertainty that exists around the estimates of expected nal Mental Health [64], and the Canadian Perinatal Men-
costs and expected effects associated with the interven- tal Health Collaborative [65].
tions under investigation and uncertainty regarding the
maximum cost-effectiveness ratio that a decision-maker Discussion
would consider acceptable. Additionally, CEACs allow How the economic evaluation will support evidence‑based
for multiway comparisons, thus enabling the research decision making in Canada and the US
team to explore the ‘ranking’ of the 4 arms in terms of One of the goals of the SUMMIT trial is to challenge tra-
their relative cost-effectiveness. ditional models of psychotherapy delivery (in-person,
delivered by a mental health specialist) from an inter-
Equity disciplinary perspective. It is expected that the results of
All outcomes will be weighted equally regardless of the this study will have significant impact on several levels.
characteristics of people receiving the interventions. To our knowledge, this work will represent the first eco-
However, the possibility of conducting subgroup analyses nomic evaluation of scalable innovations (non-specialist
related to heterogeneity due to clinical severity and peri- providers and telemedicine) relative to traditional spe-
natal period (antenatal or postnatal) will be explored, in cialist and in-person models for perinatal depression
line with the proposed clinical analyses. Moreover, a gen- and anxiety. Furthermore, these comparisons will inform
der-based analysis plus framework will be used to assess whether a stepped care model can optimize available
the potential impact of identity factors (e.g., sexual ori- resources. As a result, this study could transform mental
entation, marital status, race or ethnicity, education level, health care delivery across North America and beyond.
employment status, and age) known to be associated with Moreover, the resulting findings should be generalizable
socioeconomic and financial contexts among patients to perinatal mental health services across North America
and health care systems. as well as other outpatient settings because the SUMMIT
trial is conducted within real-world and diverse settings
Engagement approach with patients and stakeholders in both Canada and the US.
The SUMMIT trial includes an extensive network of The allocation of health care resources to maximise
stakeholders who serve on the study’s Stakeholder Advi- patient-oriented outcomes is paramount to patients, fam-
sory Committee (SAC). The SAC includes individuals ilies, and the health care system. In addition to informing
with lived experience, patient advocates and community technical efficiency (i.e., value for money) and the impact
partners, web-based organisations, clinicians from vari- on upcoming budgets, the economic evaluation of the
ous health professions, representatives from US-based SUMMIT trial can inform funding allocation decisions
insurance companies (private third-party payers) and and dissemination and scale up of evidence-based psy-
policy makers from study sites and across North Amer- chological interventions in Canada and the US, and glob-
ica. The SAC was formed to inform the development, ally. The results of this study will also have the potential
implementation, and dissemination of SUMMIT. Stake- to directly inform future implementation and scale-up
holders advised on the initial study design and continue projects and foster further collaborations between policy
to actively inform trial processes and dissemination strat- makers, clinicians, and researchers across North America
egies through a patient-centered and pragmatic lens; for and globally.
Singla et al. BMC Psychiatry (2023) 23:817 Page 9 of 11

Limitations interpretation of the data; preparation, review, or approval of the manuscript;

and decision to submit the manuscript for publication. No endorsement by
The proposed economic evaluation will have some limi- the funder is intended or should be inferred.
tations. For example, the utility weights that will be used
in this study are for the general Canadian and American Availability of data and materials
The results of the main trial and the economic evaluation will be submitted
populations rather than for perinatal women. Unfor- for publication in peer-reviewed journals. The datasets used and/or analysed
tunately, there are no specific utility weights for the lat- during the current study may be available from the corresponding author on
ter population. Furthermore, the data on health service reasonable request.
utilization are self-reported, and thus subject to recall
bias and potentially stigma-related under-reporting bias; Declarations
however, the reliability and validity of self-reported data
Ethics approval and consent to participate
has been well established over recall periods comparable The current economic evaluation has received ethical approval via its parent
to those in this study [66–70]. Furthermore, the study study, The SUMMIT Trial:
may not capture all health services used by participants Ethical approvals were obtained from the Institutional Review Board of Clini‑
cal Trials Ontario Research Ethics Board [1895], University of North Carolina
(e.g., outpatient prescription drugs in the post-behav- Biomedical Institutional Review Board [19–1786] and NorthShore University
ioural activation treatment phase). Nonetheless, the HealthSystem Institutional Review Board [EH18–129]. Informed consent was
HSUQ captures the most relevant health services used by obtained from all study participants.
this patient population. Consent for publication
Not applicable.
Abbreviations
Competing interests
BA Behavioural Activation
The authors declare no competing interests.
CADTH Canadian Agency for Drugs and Technologies in Health
CEAC Cost-effectiveness acceptability curve
Author details
CHEERS Consolidated Health Economic Evaluation Reporting Standards 1
Institute for Mental Health Policy Research, Campbell Family Mental Health
EPDS Edinburgh Postnatal Depression Scale
Research Institute, Centre for Addiction and Mental Health, Toronto, Canada.
EQ-5D-5L EuroQol 5 5 Level 2
Department of Psychiatry, Temerty Faculty of Medicine, University of Toronto,
GLMM Generalized linear mixed models
Toronto, Canada. 3 Lunenfeld-Tanenbaum Research Institute, Sinai Health,
HIPAA Health Insurance Portability and Accountability Act
Toronto, Canada. 4 Institute of Health Policy, Management and Evaluation,
HSUQ Health Services Utilization Questionnaire
Dalla Lana School of Public Health, University of Toronto, Toronto, Canada.
ICER Incremental cost-effectiveness ratio 5
ICES, Toronto, Canada. 6 Department of Population Health Sciences, Weill
NSP Non-Specialist Provider
Cornell Medical College, New York, NY, USA. 7 Department of Global Health
PHIPA Personal Health Information Protection Act
and Social Medicine, Harvard Medical School, Boston, USA. 8 Department
QALY Quality-adjusted life-year
of Global Health and Population, Harvard Chan School of Public Health,
SAC Stakeholder advisory committee
Boston, USA. 9 Canadian Perinatal Mental Health Collaborative, Barrie, Canada.
SP Specialist Provider 10
Postpartum Support International, Portland, USA. 11 Lawrence S. Bloomberg
SUMMIT Scaling Up Maternal Mental healthcare by Increasing access to
Faculty of Nursing, University of Toronto, Toronto, Canada. 12 Department
Treatment Trial
of Obstetrics & Gynecology, NorthShore University HealthSystem, Evanston,
UK United Kingdom
USA. 13 Department of Psychiatry, School of Medicine, University of North Caro‑
UNC University of North Carolina
lina, Chapel Hill, USA. 14 Department of Obstetrics and Gynecology, University
US United States
of Chicago Pritzker School of Medicine, Chicago, USA. 15 Women’s College
Hospital, Toronto, Canada. 16 King’s Health Economics, Institute of Psychiatry,
Supplementary Information Psychology & Neuroscience, King’s College London, London, UK.
The online version contains supplementary material available at https://​doi.​
Received: 23 October 2023 Accepted: 27 October 2023
org/​10.​1186/​s12888-​023-​05318-2.

Additional file 1.

Acknowledgements References
We would like to thank the patients and significant others who have partici‑ 1. Ferrari AJ, Charlson FJ, Norman RE, Patten SB, Freedman G, Murray CJ,
pated in the SUMMIT Trial and who we have the privilege to serve. We would et al. Burden of depressive disorders by country, sex, age, and year:
also like to thank the SUMMIT treatment providers, additional clinical staff, and findings from the global burden of disease study 2010. PLoS Med.
a range of stakeholders who have informed the development and implemen‑ 2013;10(11):e1001547.
tation of both the SUMMIT Trial, and the current economic evaluation. 2. O’Hara MW, Swain AM. Rates and risk of postpartum depression—a
meta-analysis. Int Rev Psychiatry. 1996;8(1):37–54.
Authors’ contributions 3. Wisner KL, Sit DKY, McShea MC, Rizzo DM, Zoretich RA, Hughes CL,
DRS, SB and CdO drafted the initial version of the manuscript with significant et al. Onset timing, thoughts of self-harm, and diagnoses in postpar‑
contributions from SMM. All authors have critically reviewed this manuscript tum women with screen-positive depression findings. JAMA Psychiat.
and provided consent for publication. 2013;70(5):490–8.
4. Dennis C-L, Falah-Hassani K, Shiri R. Prevalence of antenatal and
Funding postnatal anxiety: systematic review and meta-analysis. Br J Psychiatry.
This study was funded by the New Frontiers in Research Fund/Fonds Nou‑ 2017;210(5):315–23.
velles Frontières en recherche (#NFRFE-2022–00557). The funder had no role 5. Bauer A, Knapp M, Parsonage M. Lifetime costs of perinatal anxiety and
in the design and conduct of the study; collection, management, analysis, and depression. J Affect Disord. 2016;192:83–90.
Singla et al. BMC Psychiatry (2023) 23:817 Page 10 of 11

6. Curry SJ, Krist AH, Owens DK, Barry MJ, Caughey AB, Davidson KW, et al. for severe depression in India: systematic development and randomised
Interventions to prevent perinatal depression: US Preventive Services Task evaluation. Br J Psychiatry. 2016;208(4):381–8.
Force recommendation statement. JAMA. 2019;321(6):580–7. 30. Stein AT, Carl E, Cuijpers P, Karyotaki E, Smits JA. Looking beyond depres‑
7. Sockol LE, Epperson CN, Barber JP. A meta-analysis of treatments for sion: a meta-analysis of the effect of behavioral activation on depression,
perinatal depression. Clin Psychol Rev. 2011;31(5):839–49. anxiety, and activation. Psychol Med. 2021;51(9):1491–504.
8. Goodman JH. Women’s attitudes, preferences, and perceived barriers to 31. Singla DR, Puerto Nino AK, Zibaman M, Andrejek N, Hossain S, Cohen
treatment for perinatal depression. Birth. 2009;36(1):60–9. M, et al. Scaling up quality-assured psychotherapy: The role of therapist
9. O’Mahen HA, Flynn HA. Preferences and perceived barriers to treat‑ competence on perinatal depression and anxiety outcomes. Gen Hosp
ment for depression during the perinatal period. J Womens Health. Psychiatry. 2023;83:101–8.
2008;17(8):1301–9. 32. Singla DR, Meltzer-Brody SE, Silver RK, Vigod SN, Kim JJ, La Porte LM,
10. Byatt N, Biebel K, Lundquist RS, Moore Simas TA, Debordes-Jackson et al. Scaling Up Maternal Mental healthcare by Increasing access to
G, Allison J, et al. Patient, provider, and system-level barriers and Treatment (SUMMIT) through non-specialist providers and telemedicine:
facilitators to addressing perinatal depression. J Reprod Infant Psychol. a study protocol for a non-inferiority randomized controlled trial. Trials.
2012;30(5):436–49. 2021;22(1):186.
11. Raviola G, Naslund JA, Smith SL, Patel V. Innovative models in mental health 33. Cox JL, Holden JM, Sagovsky R. Detection of postnatal depression:
delivery systems: task sharing care with non-specialist providers to close development of the 10-item Edinburgh Postnatal Depression Scale. Br J
the mental health treatment gap. Curr Psychiatry Rep. 2019;21(6):1–13. Psychiatry. 1987;150(6):782–6.
12. Hoeft TJ, Fortney JC, Patel V, Unützer J. Task-sharing approaches to 34. Dennis C-L, Hodnett E, Kenton L, Weston J, Zupancic J, Stewart DE,
improve mental health care in rural and other low-resource settings: a et al. Effect of peer support on prevention of postnatal depression
systematic review. J Rural Health. 2017;34(1):48–62. among high risk women: multisite randomised controlled trial. BMJ.
13. Singla DR, Kohrt BA, Murray LK, Anand A, Chorpita BF, Patel V. Psycho‑ 2009;338:a3064.
logical treatments for the world: lessons from low- and middle-income 35. Levis B, Negeri Z, Sun Y, Benedetti A, Thombs BD. Accuracy of the Edin‑
countries. Annu Rev Clin Psychol. 2017;13:149–81. burgh Postnatal Depression Scale (EPDS) for screening to detect major
14. Lee EW, Denison FC, Hor K, Reynolds RM. Web-based interventions for depression among pregnant and postpartum women: systematic review
prevention and treatment of perinatal mood disorders: a systematic and meta-analysis of individual participant data. BMJ. 2020;371: m4022.
review. BMC Pregnancy Childbirth. 2016;16(1):38. 36. Canadian Agency for Drugs and Technologies in Health. Guidelines for
15. Andrews G. Utility of computerised cognitive–behavioural therapy for the economic evaluation of health technologies: Canada. Ottawa: Cana‑
depression. Br J Psychiatry. 2010;196(4):257–8. dian Agency for Drugs and Technologies in Health; 2006.
16. Lokkerbol J, Adema D, Cuijpers P, Reynolds CF, Schulz R, Weehuizen 37. Neumann PJ, Ganiats TG, Russell LB, Sanders GD, Siegel JE. Cost-effec‑
R, et al. Improving the cost-effectiveness of a healthcare system for tiveness in health and medicine. 2nd ed. Oxford: Oxford University Press;
depressive disorders by implementing telemedicine: a health economic 2016.
modeling study. Am J Geriatr Psychiatry. 2014;22(3):253–62. 38. Husereau D, Drummond M, Augustovski F, De Bekker-Grob E, Briggs AH,
17. Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, et al. Carswell C, et al. Consolidated Health Economic Evaluation Reporting
Scaling-up treatment of depression and anxiety: a global return on Standards 2022 (CHEERS 2022) statement: updated reporting guidance
investment analysis. Lancet Psychiatry. 2016;3(5):415–24. for health economic evaluations. Value Health. 2022;25(1):3–9.
18. Thase ME, McCrone P, Barrett MS, Eells TD, Wisniewski SR, Balasubramani 39. Briggs AH, O’Brien BJ. The death of cost-minimization analysis? Health
G, et al. Improving cost-effectiveness and access to cognitive behavior Econ. 2001;10(2):179–84.
therapy for depression: providing remote-ready, computer-assisted 40. Bosmans JE, De Bruijne MC, Van Hout HP, Hermens ML, Adèr HJ, Van
psychotherapy in times of crisis and beyond. Psychother Psychosom. Tulder MW. Practical guidelines for economic evaluations alongside
2020;89(5):307–13. equivalence trials. Value Health. 2008;11(2):251–8.
19. Egede LE, Dismuke CE, Walker RJ, Acierno R, Frueh BC. Cost-effectiveness 41. Stinnett AA, Mullahy J. Net health benefits: a new framework for the
of behavioral activation for depression in older adult veterans: in-person analysis of uncertainty in cost-effectiveness analysis. Med Decis Making.
care versus telehealth. J Clin Psychiatry. 2018;79(5):17m11888. 1998;18(2_suppl):S68-S80.
20. Catarino A, Harper S, Malcolm R, Stainthorpe A, Warren G, Morgoum M, 42. Heslin M, Chua KC, Trevillion K, Nath S, Howard LM, Byford S. Psychomet‑
et al. Economic evaluation of 27,540 patients with mood and anxiety ric properties of the five-level EuroQoL-5 dimension and Short Form-6
disorders and the importance of waiting time and clinical effectiveness in dimension measures of health-related quality of life in a population of
mental healthcare. Nat Mental Health. 2023;1:667–78. pregnant women with depression. BJPsych Open. 2019;5(6):e88.
21. Camacho EM, Shields GE. Cost-effectiveness of interventions for 43. Xie F, Pullenayegum E, Gaebel K, Bansback N, Bryan S, Ohinmaa A, et al. A
perinatal anxiety and/or depression: a systematic review. BMJ Open. time trade-off-derived value set of the EQ-5D-5L for Canada. Med Care.
2018;8(8):e022022. 2016;54(1):98.
22. Morrell C, Warner R, Slade P, Dixon S, Walters S, Paley G, et al. Psychological 44. EuroQol Research Foundation. EQ-5D 5L Population Norms [Available
interventions for postnatal depression: cluster randomised trial and eco‑ from: https://​euroq​ol.​org/​eq-​5d-​instr​uments/​eq-​5d-​5l-​about/​popul​
nomic evaluation. The PoNDER trial. Health Technol Assess. 2009;13(30). ation-​norms/].
23. Wilkinson A, Anderson S, Wheeler SB. Screening for and treating postpar‑ 45. Wisner KL, Sit DK, McShea MC, Rizzo DM, Zoretich RA, Hughes CL, et al.
tum depression and psychosis: a cost-effectiveness analysis. Matern Child Onset timing, thoughts of self-harm, and diagnoses in postpartum
Health J. 2017;21(4):903–14. women with screen-positive depression findings. JAMA Psychiat.
24. O’Hara MW, Mccabe JE. Postpartum depression: current status and future 2013;70(5):490–8.
directions. Annu Rev Clin Psychol. 2013;9(1):379–407. 46. De Figueiredo FP, Parada AP, Cardoso VC, Batista RFL, Da Silva AAM,
25. SUMMIT: scaling up maternal mental healthcare by increasing access to Barbieri MA, et al. Postpartum depression screening by telephone: a good
treatment [Available from: https://​thesu​mmitt​rial.​com/]. alternative for public health and research. Arch Womens Ment Health.
26. Glick HA, Doshi JA, Sonnad SS, Polsky D. Economic evaluation in clinical 2015;18(3):547–53.
trials: OUP Oxford; 2014. 47. Yawn BP, Pace W, Wollan PC, Bertram S, Kurland M, Graham D, et al. Con‑
27. Vanderkruik R, Dimidjian S. Perspectives on task-shifting depression care cordance of Edinburgh Postnatal Depression Scale (EPDS) and Patient
to peers for depressed latina mothers. J Latinx Psychol. 2019;7(1):22–38. Health Questionnaire (PHQ-9) to assess increased risk of depression
28. Patel V, Weobong B, Weiss HA, Anand A, Bhat B, Katti B, et al. The Healthy among postpartum women. J Am Board Fam Med. 2009;22(5):483–91.
Activity Program (HAP), a lay counsellor-delivered brief psychological 48. McBride HL, Wiens RM, McDonald MJ, Cox DW, Chan EK. The Edinburgh
treatment for severe depression, in primary care in India: a randomised Postnatal Depression Scale (EPDS): a review of the reported validity
controlled trial. Lancet. 2017;389(10065):176–85. evidence. In: Zumbo B, Chan EK, editors. Validity and validation in social,
29. Chowdhary N, Anand A, Dimidjian S, Shinde S, Weobong B, Balaji M, behavioral, and health sciences. Cham, Switzerland: Springer Interna‑
et al. The Healthy Activity Program lay counsellor delivered treatment tional Publishing; 2014. p. 157–74.
Singla et al. BMC Psychiatry (2023) 23:817 Page 11 of 11

49. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW.
Methods for the economic evaluation of health care programmes: Oxford
university press; 2015.
50. Brown G. Health service utilization and cost of care questionnaire. Ham‑
ilton, ON: Health and Social Service Utilization Research Unit, McMaster
University; 2001.
51. Trevillion K, Ryan E, Pickles A, Heslin M, Byford S, Nath S, et al. An explora‑
tory parallel-group randomised controlled trial of antenatal Guided
Self-Help (plus usual care) versus usual care alone for pregnant women
with depression: DAWN trial. J Affect Disord. 2020;261:187–97.
52. Dennis C-L. Influence of depressive symptomatology on maternal
health service utilization and general health. Arch Womens Ment Health.
2004;7:183–91.
53. Dennis C-L, Grigoriadis S, Zupancic J, Kiss A, Ravitz P. Telephone-
based nurse-delivered interpersonal psychotherapy for postpartum
depression: nationwide randomised controlled trial. Br J Psychiatry.
2020;216(4):189–96.
54. Statistics Canada. Consumer price index, monthly, percentage change,
not seasonally adjusted, Canada, provinces, Whitehorse and Yellowknife
— health and personal care. 2023. [Available from: https://​www150.​statc​
an.​gc.​ca/​t1/​tbl1/​en/​tv.​action?​pid=​18100​00408].
55. Brady T, Robinson B, Davis T, Phillips S, Gruber A. Medicare hospital
prospective payment system: how DRG rates are calculated and updated.
Washington, DC: Health and Human Services Office of Inspector General,
Office of Evaluations and Inspections. 2001.
56. Wodchis WP, Bushmeneva K, Nikitovic M, McKillop I. Guidelines on
person-level costing using administrative databases in Ontario. Toronto:
Health System Performance Research Network; 2013.
57. United States Bureau of Labor Statistics. Consumer price index: medical
care [Available from: https://​www.​bls.​gov/​cpi/].
58. Barber JA, Thompson SG. Analysis of cost data in randomized tri‑
als: an application of the non-parametric bootstrap. Stat Med.
2000;19(23):3219–36.
59. Rubin DB. Multiple imputation after 18+ years. J Am Stat Assoc.
1996;91(434):473–89.
60. Fenwick E, Byford S. A guide to cost-effectiveness acceptability curves. Br
J Psychiatry. 2005;187(2):106–8.
61. Briggs AH. A Bayesian approach to stochastic cost-effectiveness analysis.
Health Econ. 1999;8(3):257–61.
62. Claxton K. The irrelevance of inference: a decision-making approach to
the stochastic evaluation of health care technologies. J Health Econ.
1999;18(3):341–64.
63. Postpartum support international [Available from: https://​www.​postp​
artum.​net/].
64. The policy center for maternal mental health [Available from: https://​
www.​2020m​om.​org/].
65. Canadian perinatal mental health collaborative [Available from: https://​
cpmhc.​ca/].
66. Brown JB, Adams ME. Patients as reliable reporters of medical care pro‑
cess: recall of ambulatory encounter events. Med Care. 1992:400–11.
67. Harlow SD, Linet MS. Agreement between questionnaire data and
medical records: the evidence for accuracy of recall. Am J Epidemiol.
1989;129(2):233–48.
68. Roberts RO, Bergstralh EJ, Schmidt L, Jacobsen SJ. Comparison of self-
reported and medical record health care utilization measures. J Clin
Epidemiol. 1996;49(9):989–95.
69. Wallihan DB, Stump TE, Callahan CM. Accuracy of self-reported health
Ready to submit your research ? Choose BMC and benefit from:
services use and patterns of care among urban older adults. Med Care.
1999;37(7):662–70.
• fast, convenient online submission
70. Bhandari A, Wagner T. Self-reported utilization of health care ser‑
vices: improving measurement and accuracy. Med Care Res Rev. • thorough peer review by experienced researchers in your field
2006;63(2):217–35. • rapid publication on acceptance
• support for research data, including large and complex data types
Publisher’s Note • gold Open Access which fosters wider collaboration and increased citations
Springer Nature remains neutral with regard to jurisdictional claims in pub‑ • maximum visibility for your research: over 100M website views per year
lished maps and institutional affiliations.
At BMC, research is always in progress.

Learn more biomedcentral.com/submissions