A school-based cluster-randomised controlled trial of Therapeutic Storytelling Intervention (TSI) to promote youth mental health and wellbeing
Context: The prevalence of mental health concerns among young people is increasing and efforts are warranted early in adolescence to prevent future mental health and behavioural concerns.
Objective: To determine if TSI delivered a classroom intervention reduces overall levels of psychopathology in young adolescents.
Design, Setting, Participants: We conducted a cluster randomized controlled trial in 378 students aged 10 to 13 years in an intermediate school in South Auckland.
Interventions: 12 sessions of therapeutic story intervention delivered over 12 weeks.
Main Outcome Measure: The primary outcome was child self-report SDQ Total Difficulties Score
Results: A total of 482 students from 16 classes were eligible and invited to participate. A total of 378 students and their parents agreed to participate and their classes randomised. Baseline data was collected from 210 students from 8 intervention classes and 168 students from 8 control classes. There were no differences at follow-up between the intervention classes and control classes in primary outcome or secondary outcomes.
Conclusion: There was no evidence of effectiveness of therapeutic story intervention in preventing psychopathology among 10 to 13 years at 6 month followup.
Data Access and Responsibility: The principal investigator, Simon Denny, had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Potential conflicts of interest: None
Financial support: Funding for this study was provided by the Kidz First Research Committee. The Kidz First Research Committee had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Trial registration: Australian and New Zealand Clinical Trials Registry (ACTRN12618000282280)
Increasingly schools are being faced with mental health and behavioural issues that impact of students’ ability to engage in education. Mental health concerns are common among young people and their median age of onset is during early adolescence. ‘Young people’s health in Counties Manukau (2014)’ identified that one in four secondary school students had deliberately self-harmed and one in fourteen had attempted suicide in the past 12 months. Furthermore, significant numbers of young people in Counties engage in cigarette smoking, binge drinking and other risk behaviours that threaten their wellbeing. Referrals of these young people to mental health services at Whirinaki (Child and Adolescent Mental Health Service in CM Health) and AOD services is placing considerable pressure on secondary health services and effective prevention programmes based in the schools are urgently needed.
This project evaluates an innovative socio-emotional learning programme, Therapeutic storytelling intervention (TSI). TSI was developed by Ron Phillips in 1989 based on his book “Gem of First Water” (Phillips, 1989). “Gem of First Water” is the story of a boy on a journey of discovery and adventure in a mythical land called ‘confusion’. Along the way the hero is faced with challenges and dilemmas that reflect the common adolescent struggles, such as poor impulse control, making connections between decisions and feeling, working through grief, drug and alcohol use and peer and parent relationship issues. TSI has been developed as a class room intervention and is currently being delivered in schools in South Auckland. TSI has been extensively used in South Auckland over the past 20 years and anecdotally the results are impressive. However it has not been previously evaluated in a rigorous study design. The Ministry of Health and Counties Manukau Health have highlighted the need for research to underpin not only clinical services, but community programmes that aim to improve health and wellbeing. Of concern is the lack of research specific to Maori and Pacific young people. To date, TSI has been delivered to large numbers of Maori and Pacific young people with a great deal of anecdotal success, but without formal evaluation. The aim of this research is to evaluate if TSI is effective in improving mental health and wellbeing outcomes among children aged 11 to 13 years.
Aims/objectives of project
The main objective of this study is to determine whether Therapeutic Storytelling Intervention (TSI) improves the wellbeing of young people attending secondary schools in South Auckland. The specific aims are:
1. To determine if TSI reduces overall levels of psychopathology
2. To determine if TSI improves overall level of wellbeing
This was a cluster-randomised parallel group design conducted in an intermediate school (Years 7 to 8) during 2018. Institutional Ethics review was through the Health and Disability Ethics Committee (17/CEN/50/AMO4) and was pre-registered with Australian and New Zealand Clinical Trials Registry (ACTRN12618000282280).
Students were recruited through an intermediate school in South Auckland. Written consent was obtained from the principal on behalf of the Board of Trustees. Families were informed of the study and their written consent was requested. All students were informed of the study and their assent obtained to participate.
After baseline data collection, classes were randomized to the intervention (n=8) or control group (n=8). Randomisation was by a statistician not involved with the project by random sequence generation by computer.
Classes in the intervention group received 12 TSI in weekly 45 to 60 minute sessions over 14 weeks. Classes in the control group received the usual school curriculum. Students received the “Gem of First Water” book, a student workbook and teachers had access to a talking book to allow for students to catch-up any sessions they may have missed. Sessions were delivered weekly by the developer of TSI and teachers were encouraged to incorporate ideas and learnings from the sessions in their lessons.
Assessment and Outcome Measures
Data were collected at baseline, immediately after the intervention, and at six months after the intervention. Data are collected by trained research assistants who were blind to the allocation of classrooms to intervention or control. Questionnaires were self-administered on laptops/ tablets with the questions and response options displayed on screen as well as read-out over headphones.
Primary Outcome Measures
The primary outcome measures are indicators of the mental health using the Strengths and Difficulties Questionnaire. The Strength’s and Difficulties Questionnaire (SDQ) is a short behavioural questionnaire developed for use in children and adolescents, measuring emotional symptoms, conduct problems, hyperactivity/ inattention, peer relationship problems, and prosocial behaviour. The 25-item instrument was self-completed by children at baseline, following the intervention and at 6 month follow-up. Apart from the pro-social scale, higher scores indicate higher levels of psychopathology.
Secondary Outcome Measures
The Social Emotional Health Survey – Primary was developed to assess primary school students’ positive psychological traits: gratitude, zest, optimism, and persistence, and the higher-order latent construct of co-vitality. The 26-item survey was self-completed by children at baseline, following the intervention and at 6 month follow-up. Higher scores indicate higher levels of wellbeing.
Power and Sample Size Calculations
The primary outcome measure is the Total Difficulties Score generated by the students’ sef- report of the SDQ. Based on data from previous pre and post TSI evaluations, the study was powered to detect a decrease of 2.8 points on the Total Difficulties Score in the intervention group compared to control group, representing an effect size of 0.25.
Self-reported outcome measures are analysed comparing intervention and control classrooms at baseline, immediate follow-up and at 6 month follow-up. Differences are considered statistically significant if the 95% confidence intervals do not overlap. Outcome analyses used follow-up data as the dependent variables. The primary analyses compared differences between the intervention and control groups on the outcome variables (SDQ, co-vitality) using analysis-of-covariance models, controlling for baseline outcome measures as well as age, sex, and ethnicity. Differences between control and intervention groups were considered statistically significant at P<0.05. All analyses accounted for clustering of students from the same classroom.
Table 1 shows the demographic characteristics of the participating students. The majority of students were 12 years of age, with roughly equal numbers of male students and female students. New Zealand European students accounted for approximately 60% of students and Maori students made up approximately 15% of students. There were small numbers of students of Pacific ethnicity, Chinese ethnicity and Indian Ethnicity. Intervention classrooms and Control classrooms were roughly similar, although there were higher numbers of male students (55%) in the control classrooms compared to intervention classrooms (50%).
Figure 1 shows the drop-out from enrolment though to final analyses. Of the 482 eligible students, 378 provided consent and their classes randomised. At the 6 months post intervention, 167 students in the intervention classrooms and 129 students control classrooms participated in the final follow-up assessment.
Table 2 presents the baseline, follow-up and 6 month follow-up of the self-reported strengths and difficulties measures, including the total difficulties score, emotional symptoms, conduct problems, hyperactivity/ inattention and peer relationship problems. The table also presents the self-report co-vitality measures, which included a total co-vitality score, as well as gratitude score, optimism score, zest score and persistence score. Overall there were no significant differences in any of the measures between intervention and control classrooms at post-intervention follow-up or 6-month follow-up.
Table 3 presents the effect of the intervention adjusting for baseline measures and demographic variables, including age, sex and ethnicity. There were no significant differences between the intervention classrooms and control classrooms in the total difficulties scores. Furthermore, there were no significant differences between intervention classrooms and control classrooms on any of the secondary outcomes measures presented in table 3.
The aim of this study was to determine whether Therapeutic Storytelling Intervention (TSI) improves the wellbeing of young people attending secondary schools in South Auckland. Overall the results show no improvement in self-reported psychopathology immediately following the intervention or at 6-month follow-up. There were also no differences in the wellbeing of students between intervention or control classrooms immediately following the intervention or at 6-month follow-up.
The reasons for the lack of effectiveness in this study of Therapeutic Storytelling Intervention to improve young people’s wellbeing could be due to a number of factors. It could be that the program itself is not effective at preventing psychopathology or improving wellbeing among early adolescence. This could be due to the age of the students being too young for this type of intervention. Developmentally early adolescence is a period of significant cognitive development with increasing ability to think abstractly and to ‘think about thinking’. TSI may require more maturity and developmental cognitive ability to process the material and lessons which require students to reflect on their own lives and change their thought and behaviour patterns to enable improvement in their wellbeing. It may also be the content of TSI is better delivered by classroom teachers, who know their students better than outside personnel and may have been able to better tailor the lessons and content to the students contexts.
It may also be the study was underpowered to detect significant improvement in students’’ psychopathology and/ or wellbeing. Especially as the design was a cluster randomised study which reduces overall power due to correlation between students from the same classrooms. It was unknown beforehand of the effect size of the intervention to base power calculations. However, the direction of most of the effects was not favouring the intervention suggesting that increased power may not have helped.
Lastly, it may be that the outcomes measures are limited by being from self-reported by the students, without triangulation from parents or teachers. It is well recognised the limitations of self-report measures during adolescence, especially for externalising behaviours such as hyperactivity and conduct problems. However for other measures including emotional symptoms and co-vitality measures there was little evidence that the direction of effects were favourable to the intervention.
Further research looking at the effectiveness of Therapeutic Storytelling Intervention needs to consider these issues, especially the age of participants as it may be more effective with older adolescence. It is also important to note that these results need to be placed in the context of feedback from students and teachers which highly valued being part of the program and reported significant personal gains from the experience.
A 5 year comprehensive study using Therapeutic Storytelling Intervention (TSI) in a community mental health setting
This naturalistic outcome study suggests that TSI is successful method of communicating with multi-problem children and adolescents who are hard to reach using many traditional methods. Those attending TSI groups were selected for this study because they formed a coherent and trackable cohort of patients who had received a manualised treatment. While cognisant of the hierarchy of research design with regard to power, sample size and control for confounds, this study make an important contribution to exploring methods of engaging and retaining distressed young people in treatment and provides a foundation on which to build successful clinical interventions. The data in this study form the basis of a randomised controlled trial which is currently being established.
The most outstanding result of this study is that children and adolescents with severe mental health disorders and high rates of co-morbidity were engaged in an outpatient group treatment programme and that the overall retention rate of 60% is significantly higher than found in many similar programmes. TSI is equally successful in retaining boys and girls in treatment. The high rate of retention is of particular interest to clinicians who are attempting to generate creative ways of treating children and adolescents in community mental health settings. Many patients and their families do not share mental health professional views that ongoing follow-up is beneficial to them reflected in compliance rates as low as 20 -30 % with outpatient treatment (Coatsworth, Santisteban, McBride, & Szapocznik, 2001; King, Hovey, Brand, & Wilson, 1997; Rotheram-Borus et al., 1999; Spirito, Plummer, Gispert, Levy, & et al., 1992). Dropout occurs rapidly with a median survival rate of only three sessions (Trautman, Stewart, & Morishima, 1993) compared with average retention of 16 sessions demonstrated in this study.
Pre and post group data suggest that both parents and patients perceive an improvement in their general mental health and behaviour. Parents rated their children as having fewer behavioural problems and less symptoms of depression, anxiety and attention difficulties. Similarly participants rated themselves as more effective with greater enjoyment of life and lower rates of overall depression, suicidality and more positive perceptions of the way in which their family functioned.
Groups are intrinsically attractive from both developmental and service delivery perspectives. However oppositional adolescents and those who have ‘failed’ previous treatment programs strongly resist efforts to utilise their ‘issues’ as material upon which the group session is dependant. Frequently such adolescents will act out in order to disrupt the group and/or terminate their inclusion. TSI precludes the destruction of the group by this route by providing ‘safe’, externalised material for group discussion. TSI rapidly engages resistant, hopeless and depressed adolescents, enabling family and other therapy to proceed with the amelioration of risk factors.
Setting and Participants
Catchment area: This study is based 347 children and adolescents who attended TSI group therapy at a public outpatient child and adolescent mental health service in South Auckland, New Zealand. The catchment area for this service is populated by more than 400,000 of which more than one third are under 20 years. At the last census European/Pakeha made up 52% of the community, Pacific peoples 27%, Maori 17% and Asians 15% of the community. The rates of unemployment (10.1% in this area vs. 7.5% nationally), single parent families (23% vs. 19%), number of people living in each house (3.3 vs. 2.7) and proportion of adults with no educational qualifications (29% vs. 27.6%) are higher in this community compared with other parts of New Zealand (Statistics New Zealand, 2002). It is acknowledged that the local context will influence the wider applicability of this information.
The Clinic: A multidisciplinary team including child psychiatrists, clinical psychologists, psychiatric nurses, social workers and family therapists work with children, adolescents and their families who represent the 3-5% of the population in greatest need of psychiatric services. Children and adolescents up to the age of 20 years (if still in school) are treated using a combination of crisis interventions, family therapy, group therapy, medication and individual therapy. Attempts are made to involve the wider systems of care for the young person including the family, school, social work agency, police and health care providers with which they have contact. Inpatient admissions are possible at a regional child and adolescent psychiatric unit.
Selection of participants: Patients were assessed by two mental health clinicians as part of usual clinic practices and a treatment plan generated which may include any combination of family therapy, group therapy, medication, or individual therapy. All patients who attended the clinic between 1997 and 2001 and who were referred to group therapy were included in this study.
Procedure and Measures
The primary investigator collated demographic data from clinical records and individual attendance data at each group session, which was collected by administrative staff for all patients. A clinical profile of patients attending TSI was generated in 1999 as part of quality improvement using a using a pre and post measure design. The primary investigator distributed questionnaires to group members and their parents attending the initial group session and again at the completion of the group process. Parents of all patients completed the Childhood Behaviour Checklist (CBCL). Children completed the Childhood Depression Inventory (CDI) and adolescents completed these measures and the McMaster Family Assessment Device (FAD) and Beck Scale for Suicide Ideation (BSSI). These measures were found to have acceptable reliablity with this population with Cronbach alpha ranging from .83 (FAD) to .95 (BSSI).
Psychiatric diagnoses were not routinely generated by clinicians during the period of this study and are therefore not reported. However a study conducted by the primary investigator suggests that nearly 50% of patients present with a mood disorder, 17% present with disruptive behaviours, including conduct disorder and oppositional defiant disorder, 7% have PTSD and 27% experienced difficulties such as childhood sexual abuse, parent/child relationship issues or neglect. More than half of patients have comorbid psychiatric disorders (Fortune, 2002). This population is also known to experience high rates of known risk factors for psychosocial distress such as suicide behaviour, substance abuse, parental psychopathology, physical abuse and childhood sexual abuse (Fortune, 2002).
Data were entered into the Statistical Package for the Social Sciences (SPSS), Version 11.5 for Windows, on an IBM compatible PC. Data were checked for errors and the necessary corrections made. Preliminary descriptive statistics were conducted to check assumptions underlying the use of parametric and nonparametric methods. An alpha of 0.05 was used for all statistical tests.[/vc_column_text][/vc_tab][vc_tab title=”Research Results” tab_id=”1418782450261-2-7″][vc_column_text disable_pattern=”true” align=”left” margin_bottom=”0″]The early TSI groups were particularly targeted at boys but over the period of the study TSI has been increasingly utilised with both girls and boys. One quarter (n = 78) of group participants are Maori and two thirds European/Pakeha (n = 238). Two thirds of patients attending the group were adolescents and one third were under the age of 12 years.
Demographic profile of group participants
|European / Pakeha||52||71||42||76||86||68||17||57||41||65||238||69|
Between three and six groups were completed each calendar year with variance reflecting fluctuating staffing numbers, particularly in 2000. The groups were conducted over an average of 16 sessions but can be condensed to 9 sessions if required or extended for up to 20 sessions dependant on the requirements of the group members and clinicians.
Duration and number of participants in each group 1997 – 2001
|Total number of groups conducted||12||6||13||3||6|
|Average number of sessions||17||15||16||15||16|
|Range||10 – 19||11 – 20||11 – 19||14 – 16||9 – 18|
|Average number of patients per group||6||9||10||10||11|
|Range||3 – 11||6 – 13||3 – 15||8 – 12||8 – 16|
Three out of every five patients attended more than 60% of sessions and were described as ‘graduates’ with no significant difference in rates of graduation between males and females. Half of non-graduates had legitimate reasons for dropping out of the group such as moving out of the area or the family having significant problems with transport. Average attendance among patients who commenced TSI was 65% from 1997 – 2001. Among graduates average attendance rates were between 86% – 89% which allows for one session missed due to illness and another to attend a school camp or similar.
Maori were more likely than other ethnic groups to begin the TSI group but fail to complete it while Pacific Island children and adolescents were more likely to be referred to the group by their key worker and but fail to commence the group (x2 (6, N = 347) = 34.32, p = .000).
Crisis re-presentations appear to be significantly lower in comparison with outcome studies of similar populations (Beautrais, 2000) with 3% of graduates representing with deliberate self-harm between 1997 and 2001.
Attendance and retention rates for group participants 1997 – 2001
Attendance and retention rates for group participants 1997 – 2001
|Year of TSI completion|
|Total Number of Participants||73||55||126||30||63|
|Referred but Did Not Start||8%||18%||15%||0%||8%|
|Number of crisis representations||1||3||0||0||1|
|Percent of Graduates||2%||11%||0%||0%||3%|
From figure it can be seen that average attendance at group sessions declines with an increasing number of sessions, but not among those who graduate from the programme. It appears that successful engagement with the therapeutic process can be defined by a return after session three.
Average attendance rates per session for graduates, non-graduates and total sample
Pre and post test data were available for a sub-set of 46 children and adolescents. Using the CBCL measure, parents rated significant reductions in withdrawn (t(25) = 3.05, p = .005), somatic (t(25) = 2.50, p = .02) and anxious/depressed behaviours (t(25) = 2.55, p = .02) among their offspring. Parents also rated their children has having fewer attentional difficulties (t(25) = 2.88, p = .008). Parents reported an overall improvement in their child’s behaviour at the completion of the group compared with at the beginning of the group process (t(25) = 3.15, p = .001).
CBCL scores pre and post group intervention
|Total T score||67.15||8.38||61.65||11.30||3.14**|
|Internal T score||64.85||8.41||57.27||12.70||3.61***|
|External T score||66.23||9.30||63.19||11.66||1.79|
|* p < .05 ** p < .01 *** p < .001|
Children and adolescents who completed the CDI before and after the TSI group rated themselves as significantly less depressed at follow-up (t(33) = 2.41, p = .02) with the strongest effect seen in the domain of perceived ineffectiveness where the mean score fell from 2.97 (SD = 2.29) to 1.76 (SD = 1.78) (t(36) = 3.10, p = .004).
Childhood Depression Inventory scores pre and post group
|Negative self esteem||1.76||1.96||1.10||1.52||1.81|
|Total CDI score||13.5||8.79||9.65||8.36||2.41*|
|* p < .05 ** p < .01 *** p < .001|
Adolescents perceived an improvement in the way in which their family functioned at follow-up using the McMaster Family Assessment Device (t(21) = 2.21, p = .04). The strongest improvements were seen in the extent to which adolescents felt the roles of family members were clear and congruent (t(21) = 2.77, p = .01) and the degree to which family members were affectively connected and involved with each other (t(21) = 2.69, p = .01).
Family functioning as perceived by the adolescents
|* p < .05 ** p < .01 *** p < .001|
Adolescents reported a significant reduction in their overall suicidality following the group compared with prior to attending the group (t(17) = 2.73, p = .01).[/vc_column_text][/vc_tab][vc_tab title=”Research References” tab_id=”1418782522399-3-5″][vc_column_text disable_pattern=”true” align=”left” margin_bottom=”0″]
Abraham, P. P., Lepisto, B. L., & Schultz, L. (1995). Adolescents perceptions of process and speciality group therapy. Psychotherapy, 32(1), 70 – 76.
Beautrais, A. (2000). The Canterbury suicide project: aims, overview and progress. Community Mental Health in New Zealand, 8(2), 32 – 39.
Coatsworth, J. D., Santisteban, D. A., McBride, C. K., & Szapocznik, J. (2001). Brief strategic family therapy versus community control: engagement, retention and an exploration of the moderating role of adolescent symptom severity. Family Process, 40(3), 313 – 332.
Fortune, S. A. (2002). Suicidal behavaviour among a clinical sample of children and adolescents in New Zealand.
King, C. A., Hovey, J. D., Brand, E., & Wilson, R. (1997). Suicidal adolescents after hospitalization: Parent and family impacts on treatment follow-through. Journal of the American Academy of Child & Adolescent Psychiatry, 36(1), 85-93.
Mishna, F., Kaiman, J., Little, S., & Tarshis, E. (1994). Group therapy with adolescents who havd learning disabilities and social/emotional problems. Journal of Child and Adolescent Group Therapy, 4(2), 117 – 131.
Phillips, R. (1989). Gem of the First Water a Fable for Our Times (Third ed.). Auckland: Therapeutic Story Telling International.
Rotheram-Borus, M. J., Piacentini, J., Van Rossem, R., Graae, F., Cantwell, C., Castro-Blanco, D., et al. (1999). Treatment adherence among Latina female adolescent suicide attempters. Suicide & Life-Threatening Behavior, 29(4), 293 – 311.
Spirito, A., Plummer, B., Gispert, M., Levy, S., & et al. (1992). Adolescent suicide attempts: Outcomes at follow-up. American Journal of Orthopsychiatry, 62(3), 464-468.
Statistics New Zealand. (2002). Manukau City Census 2001 area data. Wellington: Government Print.
Trautman, P. D., Stewart, N., & Morishima, A. (1993). Are adolescent suicide attempters noncompliant with outpatient care? Journal of the American Academy of Child & Adolescent Psychiatry, 32(1), 89 – 94