Building children’s confidence and improving parents’ protective skills

Final evaluation of the NSPCC Family SMILES service

Background
Family SMILES (Simplifying Mental Illness plus Life Enhancement Skills) is an intervention for families with children who have been identified as being at risk due to (one of) their parent’s mental health problems. It aims to reduce the negative impact of parental mental health problems on children and ensure that they are kept safe. It consists of eight weekly group sessions for children, six individual sessions for the parent and a joint session for the parent and child together. An assessment of up to four sessions takes place prior to this to determine eligibility for the programme.
This final report is based on the evaluation data collected since the project began in September 2011 until December 2015. During this time, the service was run at 15 NSPCC sites (12 of which were part of the evaluation), and in total 191 parents and 333 children completed the programme. The evaluation is based on the experiences of 59 parents and 230 children.
Aims and methodology
The evaluation of Family SMILES sought to evidence whether the following key outcomes were achieved for children and young people: increased self-esteem; reduced emotional and behavioural difficulties, and improved ability to process thoughts and feelings. There were two key outcomes for parents: greater insight into the impact of their mental health problems on their child and enhanced protective parenting behaviour. It was envisaged that these changes would strengthen the parent-child relationship by improving communication within the family and so contribute towards keeping children safer, as conceptualised in the theory of change that was specifically developed for the programme.
The key elements of the evaluation design included:

  • An impact evaluation, using pre-, post- and follow-up measures to gather quantitative data from the perspectives of children, parents and practitioners.Quantitative findings were described in terms of their statistical significance; whether there was a clear trend in changes in average scores of the sample, and clinical significance – whether scores crossed thresholds defined by developers of the standardised measures relating to clinical need.
  • A comparison group formed from a naturally occurring waiting list. This group was made up of families who had gone through the assessment for the programme but were waiting to be allocated to an appropriate group.
  • Qualitative interviews with samples of children, parents, practitioners and referrers.

The interviews aimed to explore perceptions and experiences of the Family SMILES programme; outcomes, and factors that helped or hindered the achievement of those outcomes.
Key findings
Positive outcomes for children and parents by the end of Family SMILES
There was a small but statistically significant improvement in children’s emotional wellbeing and behaviour, according to the key outcome measure used, the Strengths and Difficulties Questionnaire (SDQ). The shift in the proportion of children who had the highest (clinical) levels of need at the beginning of the programme to a lower (non-clinical) level at the end was statistically significant, indicating that this is unlikely to have happened by chance. The changes were greater for children who had taken part in the whole programme than for those in the comparison group. More specifically, there were significant improvements in children’s conduct and hyperactivity at the end of the programme. There was also a small but statistically significant improvement in children’s self-esteem and ability to process their thoughts and feelings.
Qualitative findings highlighted what some of these changes looked like for children: feeling more confident and able to engage in a range of new and different activities; being more able to express emotions and concerns to trusted adults, and feeling less anxious and more reassured.
There were statistically significant improvements in parents’ protective parenting that were not found in the comparison group, though their high levels of unhappiness and distress (according to parents’ scores on the Child Abuse Potential Inventory) reflected ongoing struggles with mental health problems. Nevertheless, interview data highlighted improvement in parents’ abilities to communicate with their children and changes in their parenting.Changes to children’s emotional difficulties: a matter of perspective?
SDQs were completed by either the child or their parent. Parents reported a significant improvement in children’s emotional wellbeing, which crossed the threshold from the highest to a lower level of concern. However, children did not share this perspective and did not report a significant improvement in this area. This suggests the possibility that parents and children perceive changes differently or that parents were responding to observable changes that were not matched by children’s inner experience.
Mixed findings around sustained change
Data collected to explore children’s emotional and behavioural difficulties and self-esteem six months after completing Family SMILES suggested continuing improvements. However, these changes were not statistically significant and hence it is not possible to state conclusively that they were sustained beyond the end of the programme. The sample size of parent data at the time of six-month follow up was too small for undertaking statistical analysis around sustained changes.
Facilitators and barriers in achieving change for families
Children valued the opportunity to meet with peers facing similar difficulties to themselves, and learning about mental health. The group work also helped them to believe that they were not to blame for their parents’ mental health problems. Children benefitted from the child-centred model of Family SMILES, which brought children’s needs to the fore in an area of adult mental health provision where they can often be overlooked. Parents placed importance on having the opportunity to reflect on the impact of challenging events they had experienced in life. They also benefitted from the strengths-based approach taken by practitioners, which helped them to recognise their positive qualities as a parent. Gaining an insight into their children’s perspective on their mental health problems; developing a support plan as a family, and the flexible and reassuring approach of practitioners were also identified as helpful aspects of the programme.Barriers experienced by some children included their limited understanding of mental health at the outset of the programme; the challenging behaviour of other children in the group, and the struggle to cope with the group work ending. Other barriers were associated with children using information about mental health inappropriately
– for example, to cause upset at home. Parental concerns around children learning about mental health also acted as a barrier. For parents, barriers to achieving outcomes included their inability to acknowledge their mental health problems; ongoing problems in the parent-child relationship, and complex family circumstances – such as seeking asylum or coping with poor living conditions.
Bridging the gap between adult mental health and children’s services
The evaluation highlights the value of an approach to service delivery that works with the whole family, bringing together adults and children who may previously have been engaged with multiple agencies. This suggests that Family SMILES can bridge a gap between adult mental health and children’s services by creating a safe space for parents and children to explore the impact of mental health on the family. The capacity of the programme to bridge that gap was however undermined by not being sufficiently integrated with the adult and children’s services. A failure to adhere to the inclusion criteria – which required all participating parents with a mental health problem to be receiving mental health support – may have exacerbated this.
Improving communication within the family
Families described how the programme had given them the language to talk about mental health with each other. This included a greater knowledge about terminology (medication, treatments and conditions) and their meanings, which enabled communication about these topics to be more appropriate. In addition to language, children suggested that having a better understanding of their parent’s mental health problem also helped them to respond confidently and appropriately when their parents were experiencing difficulties.
Programme design is key for enabling families to achieve outcomes
Participants reflected that the structure of the intervention – giving children their own space to discuss issues in a group work setting; working with parents individually for them to process their thinking, and the possibility of involving the entire family – were all important elements of the programme. The group work element of the programme was particularly valued by children. Where children struggled to come to terms with the group ending it was suggested that some form of ongoing support, in the form of group work, might be helpful.
Delivery of the programme
Parents, children and practitioners commented positively on the flexibility of the programme, in terms of children having the opportunity to do one-to-one sessions alongside the group work; parents having the space to talk about issues currently affecting them, and the additional support provided by NSPCC practitioners in supporting families – especially those where children were on a child protection plan. Any transfer of the Family SMILES model needs to take into account this degree of flexibility and the engagement skills and approach of NSPCC workers that were highly valued by parents and children.
Implications
The theory of change for the Family SMILES programme needs to be reviewed to reflect a clearer understanding of broader structural and societal factors that can impinge on parental mental health and a family’s lived experience. It should also articulate the assumptions relating to what needs to be in place to generate change, for example the formal support from mental health services – is it acceptable to include parents who do not have such support?
If this programme is run again, it should be more closely integrated with other systems to maximise its potential for bridging the adult and children’s services sectors.