In this Papers Podcast, Dr. Barry Coughlan discusses his JCPP Advances paper ‘Risk rates and profiles at intake in child and adolescent mental health services: A cohort and latent class analyses of 21,688 young people in South London’ (https://doi.org/10.1002/jcv2.12246). Barry is the lead author of the paper.
There is an overview of the paper, methodology, key findings, and implications for practice.
Discussion points include:
- The benefits and challenges of using routinely collected data.
- Insight into the ‘brief risk assessment’ measure and how it was implemented.
- Overview of the latent class analyses and how they decided which class to go with.
- How maltreatment and different forms of contextual adversity can interact with different forms of risk at the child level.
- Implications for clinical practices and researchers.
- The role of experts by experience in this research and how they enhanced the research project.
In this series, we speak to authors of papers published in one of ACAMH’s three journals. These are The Journal of Child Psychology and Psychiatry (JCPP); The Child and Adolescent Mental Health (CAMH) journal; and JCPP Advances.
#ListenLearnLike
Subscribe to ACAMH mental health podcasts on your preferred streaming platform. Just search for ACAMH on; SoundCloud, Spotify, CastBox, Deezer, Google Podcasts, Podcastaddict, JioSaavn, Listen notes, Radio Public, and Radio.com (not available in the EU). Plus we are on Apple Podcasts visit the link or click on the icon, or scan the QR code.
Dr. Barry Coughlan is a Psychology Lecturer at the National College of Ireland and a Visiting Researcher at the University of Cambridge, where he did his PhD and spent several years as a Postdoctoral Researcher. Barry is involved in a programme of work focusing on the mental health needs of children and young people from underserved groups, looking at the mental health needs and access to services for children and young people with social work involvement.
Transcript
[00:00:10.000] Dr. Umar Toseeb: Hello, welcome to the Papers Podcast series for the Association for Child and Adolescent Mental Health, or ACAMH for short. I’m Umar Toseeb, a Professor at the University of York. In this series, we speak to authors of papers published in one of ACAMH’s three journals. These are the Journal of Child Psychology and Psychiatry, commonly known as JCPP, the Child and Adolescent Mental Health, known as CAMH and JCPP Advances.
Today, I’m talking to Dr. Barry Coughlan, Psychology Lecturer at the National College of Ireland and Visiting Researcher at the Department of Public Health and Primary Care at the University of Cambridge. Barry is the Lead Author of a paper entitled, “Risk Rates and Profiles at Intake in Child and Adolescent Mental Health Services: A Cohort and Latent Class Analyses of 21,688 Young People in South London,” in JCPP Advances. This paper will be the focus of today’s podcast.
If you’re a fan of our Papers Podcast series, please subscribe on your preferred streaming platform, let us know how we did, with a rating or review, and do share with your friends and colleagues.
Barry, thank you so much for joining me today.
[00:01:09.649] Dr. Barry Coughlan: Thanks so much, Umar.
[00:01:10.839] Dr. Umar Toseeb: Can you start with an introduction about who you are and what you do?
[00:01:14.598] Dr. Barry Coughlan: My name is Barry. I’m a Psychology Lecturer at the National College of Ireland and Visiting Researcher at the University of Cambridge, where I did my PhD and spent several years as a Postdoctoral Researcher. And I’m involved in a, kind of, programme of work around the mental health needs of children and young people, often from, kind of, underserved groups. And we do a lot of work with looking at the, kind of, mental health needs and access to services for children and young people with social work involvement. And I guess that this piece of work is part of a larger collaboration between colleagues at the University of Cambridge, National College of Ireland, King’s College London, Kingston University, and the National Children’s Bureau.
[00:01:56.230] Dr. Umar Toseeb: Fantastic. Who did you work with in Cambridge, as in whose group were you in?
[00:02:00.330] Dr. Barry Coughlan: I’m based in the Department of Public Health and Primary Care and the group I’m in there is the Applied Social Sciences Group, and that’s led by Professor Robbie Duschinsky. So, this is a group of Researchers, including PhD students, MPhil students and Research Assistants, and also, it’s made up of a lot of, kind of, Practitioners and Clinicians and we work on, kind of, issues around children’s mental health and access to services across primary care and secondary care.
[00:02:27.103] Dr. Umar Toseeb: Thank you. So, we’ll move onto the paper. Can you begin by giving us a brief overview?
[00:02:31.651] Dr. Barry Coughlan: So, many young people seen by child and adolescent mental health services are identified as having safeguarding needs, and in the UK, CAMHS professionals are required to provide an account of these safety threats in the form of a risk assessment. These risk assessments can be various and include things like abuse, neglect, self-harm, and extrafamilial violence. And we think that these data hold potentially important insights around the, kind of, social context of mental health and the, kind of, social context of mental health assessment practices, and the institutional logics within mental health services.
And although collecting this data is a routine activity in many services across the UK, there’s been little epidemiological work on these types of assessments. For instance, we don’t have a strong sense of how common different risks or safety threats seen by CAMHS professionals and we know little about whether certain groups of children or young people are identified as having a particular profile of interacting risks. And what we were interested in doing is to, kind of, provide an account of the different, kind of, prevalence of safety threats within CAMHS, or identified by CAMHS, and then to see whether we can apply a technique called latent class analysis to identify different groups of children who might be perceived as at-risk by CAMHS professionals.
So, taking data from over 20,000 young people in South London, we identified the, kind of, most common forms of risk seen in CAMHS. So, the most common form of risk seen was concerns around parental mental health. So, this was around 24% of the population and that was followed by emotional abuse, which was about 21%. So, we identified six different profiles, or classes of risk, and these, kind of, showed how things like maltreatment interacted with other factors, such as, kind of, antisocial behaviour or destructive behaviour or violence towards others. And we also identified how maltreatment, kind of, interacted with other things like self-harm and school attendance, to, kind of, build up a picture of the different groups of children who might be at particular need. And we also explored how these different classes of risk cut across different axis of sociodemographic difference, thinking about ethnicity, gender and level of deprivation.
[00:04:47.082] Dr. Umar Toseeb: Okay, let’s move onto the methodology section. Can you tell us a bit about the cohort that you used in this study?
[00:04:54.940] Dr. Barry Coughlan: Sure. So, data were extracted for about just over 20,000 young people attending services in South London. So, this is a clinical cohort of children who come for all sorts of different reasons. So, we extracted their data and we extracted data on their – the risk assessments and also, the sociodemographic characteristics of the – that population, as well.
[00:05:19.983] Dr. Umar Toseeb: Fantastic, and you used routinely collected data. What are some of the benefits and challenges of using that type of data?
[00:05:27.261] Dr. Barry Coughlan: I think one of the most striking benefits is the sample size. So, we were able to look at data from over 21,000 young people. That gives us a, kind of, nice signal about the different phenomena that we’re interested in looking at. I think another strong benefit of using routinely collected data is when you’re interested in addressing clinical questions, I think it’s quite ecologically valid. So, we know that this data comes from a Clinical Trust, so we can be confident when trying to make claims about clinical populations, that this data comes from such a population.
And more broadly, I think that there is, kind of, really exciting opportunities for looking at aspects of service provision that have tended to be, kind of, hidden using more standard or traditional measures, so using surveys and that kind of thing. I think using routinely collected data, we can look at, say, different aspects of service activity, like face-to-face appointments, diagnostic trajectories, how many different appointments that the young person attended, how many did they miss? All sorts of different questions that can help us build up a picture of the young person’s engagement.
[00:06:33.240] Dr. Umar Toseeb: Excellent. Thank you, and I think if I think about some of the potential drawbacks, did you encounter any issues around data quality, data access, missing data, all of those things?
[00:06:43.560] Dr. Barry Coughlan: One of the real challenges with using routinely collected data is issues and concerns around data quality and value. So, as I say, the datasets tend to be fairly big, but it depends on what’s being, kind of, brought in at the frontend by Clinicians. So, there are certain fields that, for instance, although they might be there within the data dictionary or might be potentially available, that they’re just not used by Clinicians, or there’s a lack of temporality around the use.
One example I can think of is, say, the ‘lives with’ field in the child and adolescent mental health records. There’s no temporality around this, so you get a sense of where the child is living, but you don’t know when that was, at what point in their care that data was inputted, and for a lot of children, it’s just not there, at all. So, there can be quite a lot of missing data and that’s, kind of, one of the challenges with using one of these types of resources.
I guess another challenge, which we might come onto, is the changes in assessment practices within the service. So, often, services might use a particular form of risk assessment, and this might be used up until a certain point and then after a certain point, a different risk assessment might be used. Or there might be, kind of, those, kind of – service level changes would mean that it can be hard to find that kind of consistency, if you’re looking across a long period, as we were with this study.
I guess a third potential challenge with using routinely collected data is that there’s little information, often, available about the, kind of, context for the data as it’s being inputted. So, we don’t, for instance, have extensive information around the Clinician, what the context was, you know, how familiar they were with the young person, what different sources of information they drew from to input the data and inform their decisions around their recording practices, and all of that kind of thing. So, those kind of challenges mean that it’s really important to have professionals involved, so the professionals are actually inputting the data on a routine basis. I think it really highlights the importance of having those professionals involved in projects such as this.
[00:08:40.651] Dr. Umar Toseeb: Thank you, and then, that brings us nicely to the next question I have, which is about this ‘brief risk assessment’ measure. How was it administered? Is it a checklist, is it an interview? Can you just tell us more about that measure?
[00:08:50.466] Dr. Barry Coughlan: It’s an interesting measure. It’s a bespoke assessment that was used in the service and as I say, it, kind of, collects information around various different forms of safety threats, including abuse, neglect, extrafamilial violence, self-harm, school attendance, substance misuse. So, it’s a broad range of factors included in the risk assessment. And from discussions with Clinicians, they’ve reported that this is typically filled out by Clinicians based on other forms of information gathering. So, through their conversations with the family, through their consultations with the families, through consultations with other professionals, through what might be there in other historic or screening documents. And then, this is actually filled out by the Clinician afterwards. So, it’s the, kind of, overarching clinical impression of risk within the case.
[00:09:38.510] Dr. Umar Toseeb: Thank you, and so, we’ll move onto the analysis, which I thought was fantastic, really exciting. So, you used a method called ‘latent class analysis’. For listeners who aren’t familiar with the analysis, can you just give us a brief overview of what it is and what it’s intended to do?
[00:09:53.560] Dr. Barry Coughlan: So, it was interesting. Likewise, I think it’s a really neat approach. So, latent class analysis is what’s sometimes regarded as, kind of, a person-centred approach to identifying patterns in data. So, in contrast to what might be considered, like, a variable-centred approach, in a latent class analysis, you’re interested in taking a group of variables and seeing whether you can identify whether there’s subgroups within your sample who, kind of, share similar characteristics according to those data. You’re not so much looking at specific factors or variables, but you’re trying to identify different groups of children and young people, and that’s what we did with this study. We applied this to the risk assessments, really to see if we could, kind of, identify these different subgroups of children who are identified as – by CAMHS professionals as being at-risk.
[00:10:40.568] Dr. Umar Toseeb: Thank you, and I think I like that you described it as an ‘exploratory’ approach because – in your paper, because it is an exploratory approach and I think sometimes when I’ve read papers, it’s sold as, like, a confirmatory thing and – but it’s not. Like, it is an exploratory approach and I like that you’ve described it like that in the paper.
[00:10:55.693] Dr. Barry Coughlan: I think that’s really important point to clarify. So, we do see this as a, kind of, exploratory approach that might be fruitful for generating hypothesis down the line, and we might talk about some of the, kind of, future work that we’ve got planned around that. But we do see this very much within the zone of exploration and so, this isn’t a ‘confirmatory’ approach, as you say.
[00:11:15.126] Dr. Umar Toseeb: And if we talk a bit about ‘model fit’, how do you decide whether – you know, and you did your exploratory latent class analysis and there could be lots of different classes. How do you decide which class to go with? What model fit indices do you use and why are those important?
[00:11:29.784] Dr. Barry Coughlan: So, there’s several standard ways that you can assess model fit for an LCA, and one of these is using a statistic called ‘entropy’. So, entropy is a statistic that identifies how accurately the model defines its class, or in this case, the, kind of, class of risk profiles. So, that was one way we looked at model fit and then, as you say, we also used different forms of information criterion. So, we used an Akaike Information Criterion and Bayesian condition criterion to, kind of, assess the model selection and assess model fit.
So, taking the entropy statistic and the information criterion, we were able to test out various different models and various different classes on the data to see what was, kind of, the best fitting. And ideally, with latent class analysis, we want to see an increase in the information criterion, so the Akaike Information Criterion, the Bayesian Information Criterion. You want to see that increase to suggest that model fit gets worse when you add more classes. So, this will give you a sense of the different types – what classes best fit your data.
However, this wasn’t something that we saw with the current study, and this is something that’s come up with colleagues’ work, as well, with, kind of, these large administrative datasets. So, what we did was we looked at when those improvements in model fit tended to start to plateau, and we did this through several ways. So, we created various different tables which showed, like, the improvements in model fit for each of the classes and then, we also developed elbow plots so we could visually inspect the model fit. And what we saw was, as you can imagine, if you’ve got one class to two class, there’s a very drastic improvement, two class to three class, another drastic improvement. And these improvements started to plateau around the six or seven mark.
So, we examined the model fit indices and then, we went to our different stakeholder groups, including Clinicians, other colleagues, and our Research Team, to, kind of, see what different classes made most sense, conceptually and clinically.
[00:13:31.343] Dr. Umar Toseeb: Thank you, and so, we’ll move onto the findings. What key findings from the paper would you like to highlight?
[00:13:37.789] Dr. Barry Coughlan: So, I think one of the findings that really stuck out for me was how maltreatment and different forms of contextual adversity can interact with different forms of risk at the child’s level. So, we identified, for instance, classes of risk that looked at maltreatment and what would be regarded, maybe, as more, kind of, antisocial social behaviours. But we also identified how maltreatment can interact with risks to the young person themselves, through self-harm. So, I think that being able to identify these groups of young people who may have had different experiences, but are experiencing very different difficulties in terms of risk to themselves and others, I think that that’s a potential useful finding.
[00:14:17.613] Dr. Umar Toseeb: Thank you, and given your findings, what are the implications for clinical practice or other Researchers?
[00:14:24.220] Dr. Barry Coughlan: So, I think in research, there are various ways we can conceptualise risk and adversity, and I hope that the findings from this study can provide fresh insights about how we think about risk and adversity in child and adolescent mental health services. And I guess one of the things I’d really like to highlight is that in order for us to, kind of, provide a comprehensive account of risk and contextual adversity, I think it might be useful for both Researchers and Clinicians to incorporate more severe indicators of socioeconomic disadvantage within the model. So, I think that that was one of the things that we were, kind of, missing from some of these classes of risk, was that we didn’t have indicators of things like severe economic deprivation or poverty, homelessness, access to public funds, or insecure immigration status. In order for us to build up a more comprehensive picture, I think that including those types of indicators would be useful.
[00:15:17.400] Dr. Umar Toseeb: Thank you, and given that the dataset that you’ve used, I think the last set of data was collected in 2017, how relevant are some of these findings, given that the NHS has changed a lot since 2017, especially since we’ve had a pandemic, and things seems to have got worse? How relevant are these findings now?
[00:15:34.386] Dr. Barry Coughlan: That’s actually a question that we’re currently exploring. So, as I say, the data go up to 2017 and one of the reasons for that, and I, kind of, touched on it earlier with the changes of assessment practices within the service, but one of the reasons for that is that the risk assessment that we used was superseded by a new risk assessment in 2017. It was actually a little bit earlier, but that risk assessment remained in use ‘til 2017. So, we have work underway looking at the different kind of profiles of risk using the new risk assessment. So, hopefully, we should be able to address that question.
[00:16:07.183] Dr. Umar Toseeb: I look forward to it, and a really nice part of this project was that you included experts-by-experience, so that was fantastic. What was their role? And in what ways did you engage with them, and how did they enhance this research project?
[00:16:19.959] Dr. Barry Coughlan: Sure. So, I think core to our research programme is the idea that people with lived experience, and this could be young people and their families, should have a key role in shaping research priorities that influence their lives. And for this particular study, we involved experts-by-experience in the conceptualisation and the interpretation of some of the findings and also, the types of research questions that we’re asking. And this was facilitated by our wonderful colleagues at the National Children’s Bureau. So, the National Children’s Bureau co-ordinated three groups of experts-by-experience. This was young people with – who had experience with mental health difficulties and also, parents of young people. And so, we were able to, kind of, draw on their experience and their perspectives to get a sense of what their impressions were on the different risk profiles that we identified, the different safety threats.
I think it was really useful, as well, in terms of thinking about – us thinking more broadly around our data, because one of the things that we did was we took the first risk assessment. So, in taking the first risk assessment, we, obviously, are catching young people when they’re coming into a service. And one of the things that experts-by-experience highlighted to us is that certain difficulties with, say, experiences with self-harm or experiences with certain forms of maltreatment or abuse might not come out until later on. So, they mightn’t disclose that to a Clinician until later on. And I think that this was particularly interesting, given that we didn’t identify a risk profile that was specifically maltreatment and self-harm. We identified a risk profile where there was elements of both, but not that specific group. And one potential reason for that is that – was because we used the first risk assessment.
So, we’ve also got another piece of work underway looking at the last risk assessment, looking at the correspondence between the risk assessments and then, see whether or not there are, kind of, different conceptualisations of risk later on in the child’s pathway and services.
[00:18:15.554] Dr. Umar Toseeb: You’ve already touched on this, but are you planning on any follow-up research, or is anything else in the pipeline you’d like to share with us?
[00:18:22.380] Dr. Barry Coughlan: We’ve actually just posted a pre-print of a follow-up piece of work looking at the latent class analysis and how the different classes of risk predict different aspects of service activity, including diagnosis, number of face-to-face appointments, referral to cognate services, such as children’s social care, and number of missed appointments. So, helping us, kind of, build up an image of what does risk predict at a clinical level? And one of the things that we’ve done in that piece is to examine how the latent classes that we’ve identified in this study compare to more cumulative approaches to conceptualising risk, and that’s available on the Open Science Framework.
[00:19:06.599] Dr. Umar Toseeb: Thank you, and finally, what’s your take home message for our listeners?
[00:19:10.856] Dr. Barry Coughlan: I think that risk can be used in a number of different ways by Clinicians and Researchers alike. And I think that using different approaches, such as latent class analysis, can help us gain new insights around what risk does and how it acts in mental health services. And I hope that those findings from this study will be useful for Clinicians in thinking about some of the ways that they might conceptualise risk in their routine practice. But again, I think in order for us to provide a comprehensive account of risk, I think it would be useful for Clinicians, Researchers and service providers to include indicators of severe socioeconomic deprivation or insecurity, for example, poverty, homelessness, having no recourse to public funds or having an insecure immigration status.
[00:19:55.423] Dr. Umar Toseeb: Thank you ever so much for that fantastic discussion. For more details on the paper and Dr. Barry Coughlan, please visit www.acamh.org, and Twitter @ACAMH. ACAMH Is spelt A-C-A-M-H, and don’t forget to follow us on your preferred streaming platform, let us know if you enjoy the podcast, with a rating or review, and do share with your friends and colleagues.
