For this session we welcomed Gillian West, post-doctoral Research Fellow, University of Oxford, to discuss her JCPP paper First published: 30 March 2021 doi.org/10.1111/jcpp.13415
Authors: Gillian West, Margaret J. Snowling, Arne Lervåg, Elizabeth Buchanan-Worster, Mihaela Duta, Alexandra Hall, Henrietta McLachlan, Charles Hulme
Slides from the session
ACAMH Members can receive a CPD certificate, simply email and let us know the date and time that you watched the recording.
About the session
A panel, comprising paper author Gillian West, independent expert Professor Courtenay Norbury, and a lived experience perspective, will discuss the research and its implications with Douglas Badenoch. This discussion will be facilitated by Andre Tomlin (@Mental_Elf).
Resources;
- Mental Elf blog
- First published: 30 March 2021 doi.org/10.1111/jcpp.13415
- A Deep Dive into the NELI – Professor Courtenay Norbury summary of this paper
- Developmental Language Disorder Topic Guide by Professor Courtenay Norbury
- Language and communication problems, and their relation to non-language difficulties – In Conversation with Dr. Hannah Hobson
Transcript
[00:00:00.610] – V1
Hello. We are the Association for Child and Adolescent Mental Health, or a ACAMH for short.
[00:00:08.260] – Andre
We have a fantastic session tonight. We’re going to be talking about early language problems in children, how we screen them, how we support them. And we’re going to be talking about that with a great panel. We have three speakers on our panel. I’m going to introduce them now. We have Gill. Dr Gillian West, who’s a postdoctoral research fellow at the University of Oxford, Department of Education. She works with Professor Charles Hume in the Psychology and Education group. Courtney, Professor Courtney Norbury. Professor of developmental disorders of language and communication at UCL, and director of the Literacy Language and Communication Lab. That’s the Lilac Lab. Also a fellow of the Royal College of Speech and Language Therapists.
And we also have Allison, Allison McLean. And Alison is kind of performing our lived experience role tonight, which is a little bit different. We normally have young people with lived experience of the condition or the issue that we’re talking about. Because we’re talking really here about younger children, four or five year old children with language problems, we’ve asked Allison to come along. She has lots of experience of working with this group, previously as a teaching assistant, and she’s now an assistant [inaudible 00:01:21]. So, yeah, welcome Allison. Really great to have you as part of the conversation this evening. Really looking forward to hearing what you say. And also in the team, along with Matt and myself, we have Douglas, Douglas Badenoch. Douglas is going to be doing the critical appraisal bit in a few minutes. He’s the co-founder of the National Elf Service with myself. I’m Andre Tomlin, by the way, I’m the mental elf.
And also [s.l. Annia 00:01:48], who is moderating the chat this evening. So Annia will be looking at all the messages in the Zoom chat. If you have any questions, do feel free to post those in the chat and Annia will relay those to us, and we’ll try and answer as many of your questions as possible as we go through. So these CAMHS conversations, they’re always kind of organised in three parts. The first part will be a general conversation where we talk about language problems in young people. We’ll then be looking at a specific piece of research. And the research paper that we’re looking at this evening, led by Gill, is called Early Language Screening and intervention can be delivered successfully at scale, evidence from a cluster randomised controlled trial. So we’re going to be talking about that paper. We’re also going to be giving you some pointers about how you look at the pros and cons of randomised control trials.
And you’ll also have seen, just popping up on your screen here, a quick poll. So do tell us which of these groups best matches you, just so we can get a sense of who’s in the audience and what different groups we’ve got represented this evening. So do just take a second to choose one of those professional groups or there’s an other at the bottom if none of those match your role. And then Matt will give us an idea in a second of who we’ve got in the audience. Here we go. Thanks, Matt. Okay, that’s great. So we’ve got a real mix of people. I think the biggest group looks like educational professionals. One in five of you are from that group. We’ve also got quite a few researchers and academics, but we’ve got a real mix. We’ve got some clinical psychologists and psychiatrists, lots of students, CAMHS professionals, social workers, that’s fantastic, and lots of others. I imagine there’s some speech and language therapists in this audience, so you may well have put yourself down as an other.
Welcome, everyone. It’s great to have you all here. And yeah, do carry on introducing yourselves. Tell us who you are, where you’re from and, yeah, share as we go along. We’re also live tweeting this session, so if you’re interested in talking about stuff on Twitter, the hashtag is #camhscampfire. Do post your comments and use that hashtag so everybody else can see them. So let’s make a start.
So as the background to this is that the government had invested quite a large amount of money in this NELI, the Nuffield Early Language Intervention. So as part of the kind of catch up programme from the COVID Pandemic, the government has invested £17 million in a national rollout of NELI. And so today, we’re going to be looking at a recent trial led by Gill West and colleagues, which was conducted in 193 primary schools and recently published, as we said, in the JCPP Journal. Before we do though, let’s just take a step back and think about the broader picture.
So, Courtney, I wanted to start by asking you, just give us the broad picture here, how widespread are language problems in children when they start school? And also, how widespread are clinical language disorders? And can you kind of paint a picture for us and tell us what the difference is between the everyday problems that young people have and what you would define as a clinical disorder?
[00:05:15.710] – Courtney
Okay, so that’s quite a tall order, but I’ll try. And most of what I’m going to talk about is based on our big population study of language development and disorder that’s been going on for about ten years now. And the way to think about language is that there are huge amounts of variation in kids. In the same way that you would have lots of variation in height and weight, you also have huge variation in language. And so some kids will have relatively good language skills, and some children have relatively poor language skills. And we also know that those children who have relatively poor language skills, there’s an association with social and economic deprivation. So you do get language difficulties across the socioeconomic range, but you tend to get bigger clusters of difficulty in more deprived areas.
And basically what our research shows is that that kind of relative positioning of language is pretty stable. So if you have relatively poor language skills when you start school, you tend to maintain that position all the way through. So your language skills will get better. Kids get older, they learn more language, they do more talking. But relative to peers, they’re still behind. And it can be quite a big gap. So in our study, it’s about a two year language gap between those kids who are having really significant difficulties than their peers. And that gap looks like it’s pretty tough to close, which is why studies like this are so exciting.
We estimated that about 10% of kids had language difficulties that were severe enough to affect their learning. So those children who had quite significant language difficulties were very unlikely to meet their education targets in reception, and then as they go through school, [inaudible 00:07:08] screen and stats. So there’s a big impact on learning. And we’ve also just started publishing work looking at the impact of early language difficulties on emotion processing. And we’re showing that your very early language skills are predictive of things like how you recognise emotion and your ability to engage in verbal strategies that help us regulate emotion.
And so that’s really important, it’s really interesting for this group, you know that often children with speech and language difficulties will be, we would like them to be seen by speech and language therapists, but [inaudible 00:07:47] a profession that’s not widely recognised by people working in mental health. And given the risks for children with language difficulties of having poor mental health, it’s really important that we get on that map and have more dialogue like this. So essentially, the bottom line is, about 10% of children in UK schools, on average, are starting school with language difficulties that make it hard for them to learn in the classroom. And this increases their risk for social emotional behavioural difficulties from the word go. And those difficulties maintain over time.
[00:08:25.550] – Andre
And in terms of the difference between a language difficulty and a clinical language disorder, can you give us a bit more detail on that?
[00:08:34.710] – Courtney
So again, there’s variation. And so some kids might be at the low end of average, and that can create some difficulties in school, depending on what the demands of the curriculum are. The children who have actual language disorders that are persistent, we know that there are genetic influences on that, and we believe that those genetic influences are influencing the way the brain is developing, and that just makes it harder for them to learn from the usual input, if you like. And because it’s affecting the way the brain is developing, there tend to be other things going on as well. So those children who have clinically significant language difficulties are at an increased risk of having other things going on. So they might have problems with their motor skills and development, they might have problems with attention control, they tend to have more problems, as I mentioned, with social and emotional processing. So they tend to have a kind of complex pattern of skills and challenge.
[00:09:37.370] – Andre
That’s great. Thank you very much. Allison, I wanted to come to you next, if I may. I’m just sort of interested in you sharing your experiences of helping reception kids with language problems. For people who are on the call who don’t necessarily know what that’s like, just give us a picture, please.
[00:09:56.690] – Allison
Everything that we do [inaudible 00:09:57] is just underpinned by building relationships and communicating with the young children. Children that have difficulties in expressing themselves and communicating their wants and needs, that’s a huge barrier for these children. It’s not only displayed in their academic performances, but it’s, they’re unable to express their needs and want, and that is translated through their behaviours. And so we see lots of unwanted behaviours because they’re frustrated because they’re unable to communicate what they actually want.
[00:10:45.370] – Andre
And tell us a bit about what you do to support those children and make sure that they are given the best chance to develop those relationships and have a meaningful role in school and in the class.
[00:10:56.480] – Allison
Okay. So earliest practitioners play alongside children constantly. That’s part of their everyday, you know, that is their everyday, part of their everyday routine. Building relationships, ensuring children feel comfortable within themselves, to allow them to feel confident to approach different adults and to build those relationships with not only the adults in the class but their peers, and it’s about modelling suitable play strategies for children to follow, playing alongside to deepen their play. It’s about giving them opportunities to develop the skills that they have or that they’re lacking.
[00:11:48.770] – Andre
Okay, thank you. I’m interested, Gill, in getting your sense of, before we talk about the research specifically, and NELI, specifically, just how good a job do you think we do at identifying language problems in children when they start school?
[00:12:07.610] – Gill
I think there’s definitely room for improvement. I think the work that’s done once a child is identified is great. If that child is able to get support, then there’s an awful lot that can be done. But I think that often language problems and language issues in children are unrecognised. And there’s not always an awful lot of awareness out there about children coming into school with language that isn’t sufficient for them to be able to learn in school, to build and sustain relationships with friends, with teachers as well. And, you know, there might be children in class who are, just seem perhaps to be good and quiet and not recognised, but perhaps that child has a problem with understanding language, for example. So I think having some tools out there that can allow the teachers in the classroom to see, to find out whether children are having issues when they come into school is a really useful thing for us to be doing.
[00:13:15.390] – Andre
And What’s the linking like between schools and other services that are required when children need to be referred on? You know, we hear all the time about the relationship between schools and CAMHS and all the kind of complexity and pressures around that. And there’s been a lot of discussion recently about whether more mental health should be done in schools rather than in CAMHS to kind of take the pressure off and reorganise things and, it’s all very complicated. What’s the relationship like between schools and more specialist services, speech and language services, when they are required?
[00:13:50.010] – Gill
So I’m really not the best person to comment on [inaudible 00:13:52] coming from an academic and a research background. But I can tell you what should happen in an ideal world is that people are able to work, different groups are able to work hand in glove that one can identify children in need of support and be able to pass that child onto the appropriate service to receive that support. We know with language issues, that they’re often really not easy to resolve at all, and often don’t resolve. So, you know, even in an intervention, like NELI that may be helping children to move closer to the average range in terms of their language performance, but it’s not fixing a deep seated issue that they have. These children often need support over a long period of time. So it’s really important that it is a joined up support for them. And I think, from what we see anecdotally, what we hear anecdotally, often it’s not as [inaudible 00:14:50] as it could be.
So we probably all need to strive in that direction. But again, I would say I’m not the expert here on that aspect of it at all.
[00:14:58.173] – Andre
Courtney?
[00:14:59.010] – Courtney
Yeah. So in our study, only about 50% of the kids, that we identified as having quite significant needs, actually made it to a speech and language therapist. And I think there are multiple and complex reasons for that. But like CAMHS, speech and language therapy is chronically underfunded, and there are huge waiting lists, and it’s not as satisfactory as we would like it to be and hugely variable. So, you know, like CAMHS, there is a bit of a [inaudible 00:15:26], and it depends very much on where you are. That’s kind of led to this increase in universal services. And I would say that really where it works well, schools and therapists can work together very efficiently and effectively, but it’s not the norm, and certainly to have better funding at each level of provision so that there are access to specialist services for children who need it would be a really useful and important way forward.
[00:16:00.680] – Andre
Allison, do you have anything to share in terms of referring children on to more specialist services and your experiences of that?
[00:16:10.500] – Allison
So children are referred on when things are identified that need more specialist provision. But I’d like to echo what Courtney has said. There is always such a waiting list so they can be identified as needing support. But then, you know, those services themselves are stretched, and so it isn’t something that happens very quickly.
[00:16:35.850] – Andre
Okay. Thank you. So let’s move on and talk about NELI. I’ve got a head full of puns about NELI. There must be so many possible puns that you can do about NELI, but I’m going to avoid all of them and just keep it straight. Tell us, Gill, a bit about the Nuffield Early Language Intervention. What is it? How does it work?
[00:16:58.110] – Gill
Yes. So this is, NELI has been around for various situations for a long time. I think first research [inaudible 00:17:04] NELI was right back in 2003, 2004. So it’s been through a long development process. But it’s an intervention based on a withdrawal model. So that’s a pull out model where children with language weaknesses receive 20 weeks of really intensive scripted language intervention delivered by trained teaching assistants in the school. And they receive that in small groups, and they also receive individual sessions. And the sessions are built around direct teaching and vocabulary about development of the child’s narrative skills through using picture prompts to get the child’s retail stories, for example, and building a child’s listening skills, so listening skills and listening comprehension skills.
And, so this is, of course, the two terms. The second two terms is usually in reception class.
[00:18:11.110] – Andre
And Allison, do you want to share a little bit about your experience of NELI over the last recent years? Yeah. Tell us a bit it.
[00:18:19.150] – Allison
I’ve been lucky enough to teach NELI for the last two years. It’s a programme that gives practitioners the tools to provide children with skills that they need to develop their oral language. As a practitioner, I would say it’s a wonderful intervention. It is beautifully resourced. The resources are engaging for practitioners and children alike. Very detailed lesson plans. I can’t speak highly enough of it, personally.
[00:19:02.110] – Andre
And Courtney, how does it fit in to the wider picture? Because this is not an intervention which is aimed at children with language disorders. It’s one that’s aimed at the general population, identifying and helping.
[00:19:13.750] – Courtney
So there are often three ways to think about how interventions are pitched. One is specialist, which is where most speech and language therapists would come in with kids who have pretty significant and persistent needs. There’s a universal, which is just strategies that schools can use that should benefit all children, including those who have language disorders, and speech and language therapists can sometimes be involved in that, for example, like in providing continuing professional development or advising on strategies in schools. And then NELI sits in the middle. It’s a targeted intervention. So you’re identifying children for whom there are concerns. Some of them might have clinically significant language disorder, some of them might be just in that sort of grey area where they’re not where we’d like them to be, but they might not meet criteria for that clinical diagnosis.
And what you’re hoping through this kind of focused intervention is that some of those kids will move into the range we’d like them to be in and perhaps not need further support. So it’s a very targeted intervention.
[00:20:24.730] – Andre
Okay, that’s great. I’m going to hand over to Douglas now, I think. I think, let’s look at the paper. Let’s get Douglas’ black belt critical appraisal skills and take us through some of the methodology. And then after about 15 minutes of that, 15, 20 minutes of that, we’ll come back and we’ll have a bit more of a focused discussion on the implications, you know, what we should be doing now to support young people. So, yeah, over to you, Douglas.
[00:20:55.990] – Douglas
Thanks, Andre. Bear with me while I share my screen, and we should be able to have a look at some slides. That was a really interesting discussion so far, and a lot of interesting comments about local strategies and so on in the chat. Just wanted to flag up a question for [inaudible 00:21:15], somebody’s asking for that the transcript will be available as well as the recording of the session. So now we’re looking at the specific methods of this paper, and I wanted to ask a very general question first of all. It’s a slightly different audience, perhaps from what we’re used to. So, Matt, if you could [inaudible 00:21:36] up the second poll question, please, about people’s background knowledge of randomised controlled trials. This is a type of research method which may not be familiar with everybody.
The process we’re going to go through in the next couple of minutes will just be a very rapid run through of how we would go about appraising a randomised trial, no matter what topic it was in. There’s checklists that are available, which can remind you of the things that you need to look for. And the key thing to remember about critical appraisal is that no study is absolutely perfect. But if a study is good and reaches at a certain level of confidence in it’s reliability and it’s importance, we then turn things around and think, “Well, can we afford to ignore it?” So thanks very much for those responses. I can see most of us are learners. So I’m just going to run very quickly through the critical appraisal process. Basically, there are four questions we’re asking ourselves about a piece of research.
First of all, what is it looking at? What’s the research question? And once we’re clear about that, we can help to frame our thinking around, how does it help us in our setting. The second is, were the right methods used? Are they valid? Thirdly, are the valid results important? And finally, do these valid important results apply to our setting? And one of the reasons for this critical appraisal process is if you get to number two and you think, “No,” then you don’t need to worry about the rest of it. So it’s a time saving tool, honestly.
So let’s skip on to my next slide, which appears to be, here we are. So very quickly, why do we do this? Well, broadly speaking, a lot of research has been done on the reliability of different types of studies from different disciplines. And roughly, this is the conclusion. Around half of what’s published doesn’t do enough to eliminate possible biases. Now, these are not biases in the way that you might bias a, roll a dice to try and get the result you want. These are unconscious biases which affect everyone, including how we do research, how we interpret the results. So we’re trying to eliminate bias as a possible cause of the results that we’re seeing.
Another thing just to point out is that most of the research on bias has shown that most of it [inaudible 00:24:21] tends to exaggerate the benefits of what we’re, of interventions. So this is just, you know, you can explore some of the further details behind that in your own time, there’s links there. So briefly, about randomised trials. The reason they exist is because people have been aware of these biases and tried to come up with the best possible design for a study which eliminates them. And the key thing, the key reason we have randomisation in clinical trials is to ensure that we’ve got a fair comparison. So we’ve got one group that gets our intervention, and we’ve got another group which gets an alternative.
And in order for that to be a fair test, those groups need to be the same in every way apart from the intervention. So by randomly assigning people to, to NELI in this case, or to normal education, that ensures all the possible confining variables that could, even ones we haven’t thought of, are equally distributed in those groups and we’ve got a fair comparison. We can then go through the process of the randomised trial, and there’s a very nice diagram for those of you who have got the article, figure two in the paper shows what was done at each stage in this, what they call a consort flow diagram, a standard way of reporting what was done in a trial. So looking in bit more detail, and remember, we’re starting off with a study design, which, there’s a bit of jargon here, which might not be familiar to everyone.
A cluster randomised trial is a trial in which we assign groups or clusters of people in one go rather than one at a time. And that’s important because we can prevent possible contamination. Imagine if you had two children in the same class, one of [inaudible 00:26:20], which was assigned to the intervention and one to the control. They’re very difficult to organise, and you’re going to get cross contamination between, within the class. So that’s the reason why this study design was chosen and it’s appropriate for this type of intervention. So without labouring the point too much, we’ll just get a quick summary of who was in the study. We’ve got children in reception classes. They were all screened with the language screen up. So everyone got a score. And then as we’ve mentioned, we took the five children with the lowest scores and assigned them to either NELI or the control intervention.
In the end, we had around 1100 children. And an important point that is made in the paper is that SLTs were involved in validating those assessments. So there’s been quite a lot of efforts going to make sure those are sort of accurate and realistic assessments. As we’ve heard a little bit about the intervention, I won’t go over that too much just to say that the comparison here, comparison with NELI was a waiting list. So people, schools that weren’t allocated to it at the start could then, they had a voucher which they could take part in the programme later. I think perhaps a more interesting thing we can, I would be interested to hear more about later is the outcome measures, because how we measure what happens, and does that reflect what we’re interested in, in real life?
Now there’s a beautiful table with loads of different breakdown of all the different measures which are previously validated measures, [inaudible 00:28:15], expressive vocabulary instruments, and so on. There was also, I think importantly, a combined assessment based across all of the variables to give an average score for each group, the control group and the experimental group. And at the end of the year, these were analysed to see how much the people in NELI had improved compared with the ones in kind of usual education. The key thing is here, we’re looking at an improvement in both groups, but we hope the language group, the NELI group, will improve by more.
So I’ve just summarised on one slide here what the study is looking at. I’ve just, click on past that slide and then you jump into, are details of looking at the methods that they employed. So the checklist I’ve got here is called the [inaudible 00:29:17] Critical Appraisal checklist, and I’ll put a link with the slides when they’re uploaded to the website afterwards so you can download the instrument yourself. The checklist focuses on what happens at each stage in the flow chart. And we can use the checklist to go and look in the paper to find answers for those questions.
So we’ve, I think we’ve got a pretty good idea of that focus question of the research. Another good thing about this study is that they published protocol in advance, so there’s no possibility of going back and changing the outcomes at the end, which does sometimes happen, in fact, often happens, in trials. We’re pretty confident that the participants were assigned randomly to interventions. And an important step here is that they had what they call allocation concealment. So children being recruited or schools being recruited to the study, they didn’t know at that stage whether they were going to get allocated to NELI or to the control group. And that’s an important feature. It’s been shown that can, that if you know, when you’re recruiting people to a trial, which group are going to end up in, it can bias your recruitment decisions.
The third point, we’re asking obviously about the follow up rate. And here we had a small dropout rate, 5% to 6%, and the researchers took quite a lot of efforts to look at any patterns in those dropouts. They didn’t find any. And they also made sure that people that they, they followed people up, they analysed in the groups to which they were originally assigned. And that’s a, it hasn’t happened in this case, but sometimes, in trails, you can see people dropping out from one group and crossing over into the other one because of what’s happened with, you know, they might have suddenly got worse in the class and needed some other intervention. So it’s important that we retain the original randomization that we did at the start.
Fourth, perhaps a slightly more familiar question to most [inaudible 00:31:27] question of whether participants were blinded to treatment. Did they know they were getting an intervention? Because we can get a placebo effect. Now it’s very difficult to deliver educational, a placebo educational intervention. It’s a bit like surgery like that. If you’re doing trials, it’s hard to blame the patient to whether they’ve had an intervention. However, one possibility we have here is that because by using the app, it may be possible that data is gathered without a human intervention that might bias responses. So it’s possible that we’ve got a better blending with the use of an app rather than an investigator to record data.
Question five, we’re looking for [inaudible 00:32:17], the table of participants who were included in the studies, and we look for some evidence that the authors have made sure that the randomisation has produced two groups that are the same. And they’ve done this quite extensively. Tested whether there are important differences in things like gender and language ability, so do people who scored lower do worse or better than ones who scored higher. So there was quite extensive [inaudible 00:32:49] point. On the last point, I’ve been a little bit picky because I thought, well, that it’s possible, because the teachers and classroom assistants knew they were giving their own intervention, it’s possible they might do something else at the same time to encourage people, but, you know, there’s no other way of doing it. And if we can’t do it that way, we’d may as well pack up and go home.
So that gives you my, kind of overall assessment. I think, pretty strongly, yes, this is valid evidence. And so I move on to the next question, which is, what are the results and are they important? Now, we have a lot of outcomes, and a lot of outcome measures are reported in the study. One of the key things that is, you just need to drill a little bit into the tables when you see a very consistent pattern that all of the, that although both groups improved, as you would expect, thankfully, consistently throughout the trial, the NELI group improved by more than the control group.
So that consistent pattern immediately makes you think, “Well, this isn’t just happening by chance.” We also have a, quite a handy overall measure, which is this [s.l. Coins 00:34:21] D for the overall primary outcome of the primary score, which came out at 0.26, which is, is that Earth shattering or is that not Earth shattering? Well, I think we can, first of all, just underlying the point that I think it’s unlikely these results could have occurred by chance, and thanks to Courtney for her reference and her write up on the ACAMH website, there’s a handy look up table for what Coins D means.
So what Coins D does is, we’re comparing the average improvement, the mean improvement in the two groups, and then you divide that by the standard deviation to get the number. And then you can look up the number on this table and it can tell you whether it’s [inaudible 00:35:10] called it small, medium, large or very large. And in this case, we’ve got a, a kind of, a small but edging towards medium effect when we’re looking, again, when we’re looking at this overall combined language variable. So I think that leaves the question of how important it is to the experts. But I feel like that my conclusion for this study was that it’s believable evidence, which is likely to be a true effect. So the question is over to you and what, what are you going, how are you going to interpret that, and can you afford not to pay attention to this valid evidence?
[00:35:56.650] – Andre
Did you have another [inaudible 00:35:57] that you wanted to ask?
[00:35:59.400] – Douglas
[Inaudible 00:35:59], thank you, I was just about to say it. Matt, would you mind putting out the next poll, please? I wanted to get an idea of whether you agree. And while you’re having a think about that, any questions that perhaps I’ve not covered, I’m sure there’s loads, or more detailed questions about the trail that might affect how you would view this evidence, pop them in the sidebar in the chat window, and we’ll get to them during the discussion. Okay, well, thanks for [inaudible 00:36:35], that’s a pretty lightning run through it. I think we’ve got quite a lot of food for discussion, picking up the comments that we dealt with at the start. So I’m going to hand back to the team to digest.
[00:36:53.750] – Andre
Before we get Alison and Courtney’s response, I’m just going to ask a few questions to Gill, if I may, on the study, because we’ve got a few questions coming in. And just because it’s popped right up in my line of sight, I’m going to ask Lucy’s question first about the long term gains from NELI. So tell us about the time scale that you measured with this particular trial.
[00:37:16.010] – Gill
Yeah. So certainly. So the results in the published paper are from an immediate post test. So that’s testing the children again at the end of the summer term of reception, which is when they’ve just finished the intervention, the 20 weeks of the intervention. And so it’s a very good question, because obviously we want to know how durable these effects are as well. Not just that there are effects, that these children’s language has improved more than it would have done if they hadn’t received NELI, but also, does that improvement continue. Improvement’s roughly equivalent to about a term’s worth of progress in language. So that’s obviously meaningful, and we want to know that it’s still there later on. We are actually following up these schools at the moment. So we have about half the schools in the original trial that agreed to take part in a follow up. We would have had this all wrapped up and ideally published by now if it hadn’t been for the global pandemic getting in the way.
So in fact, we’re only able to get to these children when they were nearing the end of year two. But we have most of the data in now. There are some bubbles that still burst, so we’re still collecting last little bit of data now, but that is very much a focus for us and we will be seeking to publish a paper on that at some point as soon as possible.
[00:38:39.030] – Andre
Great. Thank you. And what about cost effectiveness? That’s come up a few times in comments as well. Is that something you’re measuring?
[00:38:45.350] – Gill
It’s a very good thing [inaudible 00:38:46]. As you said at the beginning, this is now being rolled out in schools. We reached over 6700 schools last year with NELI. We’ll be reaching another 4000+ this year. So it’s going to, then at that point, be in about two thirds of the primary schools in England. So is it cost effective then? If it’s in so many schools, it’s important that it is. Yes, it is. It’s very cost effective. The materials don’t cost an awful lot. The training that we have now, it’s a different training model to the one in the trial. In the trial, we’re still face to face training two and a half days that we’re given to teaching assistants and teachers delivering NELI. We’ve now moved that online. So we have an online training and support model which is extremely cost effective. It allows us to train and to raise awareness of language development and language problems in more staff in schools, not necessarily just the person delivering NELI, but more widely, [inaudible 00:39:45] centres, etc. too.
And to be able to keep training new staff as new staff come in and deliver the intervention. So once the school has the NELI materials, once they have a trained teaching assistant, and the training is a really important part of it, then they can continue to deliver NELI year on year on year. So yes, it becomes extremely cost effective.
[00:40:08.210] – Andre
Okay, great. Thank you. And just a couple of questions that were asked earlier as well, just to touch on briefly, if we can. So a question about cultural consideration. So did you take into account children whose first language is not English with the study?
[00:40:23.920] – Gill
Oh, great question. Yes. Because actually, you know, children coming into school with weak language skills, it doesn’t, we’re not distinguishing here, but why would we rule out children who are coming into school with poor language skills because that’s not the language they’re speaking at home. And that’s very much the case. And we know there’s somewhere around, was it 17% of children coming into schools who are coming in from homes where languages spoken at home is not English. Yes, we include those, absolutely, in NELI. They should receive NELI. They received NELI in the trial. They were overrepresented, actually, in the children receiving NELI, I think it was something, just above 30% of the children in the groups were English as an additional language. And what we did find was that those children benefited just as much as the children who were monolingual English speakers. So there’s no difference between those two groups and how much they benefited.
[00:41:21.420] – Andre
Yeah. A question also about the relationship between the intervention that you studied with the trial, that we’ve been talking about, and the newer roll out of the intervention. You said earlier that those are different. Is that going to be evaluated, the potential differences between those interventions?
[00:41:37.460] – Gill
We’d very much like to. Yes. I mean, these are plans that we had in place that expedited then by the pandemic and then by the funding coming online. So we knew that, obviously, that the trial that we’re discussing here is an effectiveness trial, a very large trial. But when we’re talking about rolling out thousands of schools, delivering face to face training becomes impractical, becomes expensive, becomes time consuming, not just for people delivering NELI, whoever that might be, but also from the schools’ point of view in releasing staff to attend training, etc. So we had plans to move the training online, and we’ve done that. And we, in the last year, the first year of the rollout reached over 20,000 teachers and teaching assistants.
So this training is not just in how to deliver NELI sessions. The whole sort of first half of it, if you like, is talking about language development in children, about language problems that children might have in the classroom, how to recognise that, how to support that, the techniques they can use. So we’re really upskilling people who are taking the training and enabling them to identify children better as well as to be able to deliver the intervention. And I think that probably does generalise then outside of NELI too. Of course it’s not then just the NELI children that they’re helping.
[00:43:01.460] – Andre
Yeah. And then finally, one last question, we could literally have another hour of this discussion just talking specifically about the questions that come in, I’m sure. So I’m interested in support before and after the reception class. There’s some questions saying, “Could you do research looking at effectiveness in younger children?” And there’s also been some questions saying, “What about children who don’t get identified as having problems? Can we help them further down the line?” How does it fit into that?
[00:43:27.899] – Gill
So two interesting questions there. So, yes, helping children as soon as we can before they’ve had any real impact in school from language problems that they have is really important. Yeah, we have a programme that we’re developing at the moment in nursery, and in fact, we’ve just relaunched the trial. We were halfway through that as well before the pandemic hit. But we just relaunched that now. So that is a language enrichment programme. So Courtney touch on the fact that NELI, in reception, is a targeted intervention. So this programme that we are developing for nursery children is both a universal language enrichment programme, so that’s something that’s been delivered to all of the children in the classroom, and it’s based on shared book reading and activities to enrich children’s vocabulary and teach them narrative skills and listening. The same things that are so much a focus in NELI.
But then it also has a targeted additional support element for children who are identified as having weak language skills. So that, we’re looking at very, very much so. We’d really like to bring that on board so that it’s a kind of companion programme, if you like, to NELI. And I think that’s a really important point that we want to be able to reach these children when they’re very young and then continue to support them throughout their early years of primary school, because this is not a quick fix for many children. Yes. And then there was the second part of that question, you have to remind me what that was.
[00:45:03.880] – Andre
Yeah. It was about, if you don’t identify children with language problems in reception using these methods for screening, what support is there further on? You know, can NELI be applied further down in all the children?
[00:45:17.270] – Gill
Yeah, I mean, there’s no reason why Nellie can’t be used in other year groups and in other ways. Absolutely. I mean, we find feedback from some of the schools in our trial that they were taking some of the activities in NELI and bringing those into the main classroom because they were totally appropriate for children with better language skills. And it was also allowing the children in NELI to shine because they knew what was going on and could participate. And also from schools who were then taking NELI and using that with children who are year one, for example, who needed additional support, or children who are coming in, perhaps, later on in primary school needing support because they were English as an additional language or something like that. So, yeah, there’s no reason why it couldn’t be used more widely.
[00:46:07.470] – Andre
Great. Thank you very much. Okay. So, Courtney, the 17 million pound question is, has the government spent this money sensibly? This is really good research, isn’t it? And, you know, it’s really well conducted. It’s a huge trial. It’s very reliable. It’s found a small effect, but on this huge population scale. Surely that’s what we want, isn’t it?
[00:46:28.200] – Courtney
Yeah. I mean, there’s no doubt it’s a meticulous study. I use it in teaching, like this is how you should do an RCT, if you want to do a school based language intervention study, it’s a brilliant study. There are things that it doesn’t do. So all the outcome measures are about expressive language. We don’t know from this trial how it affects receptive language, and receptive language is probably more a feature of clinical DLD. It isn’t a clinical sample, so we don’t really know what the overall level of impairment is from the paper, or how kids with DLD respond. Now, I do know that there is an analysis which suggests that the rate of progress is the same across the distribution. But what that means is that if you’re at the extreme end of the distribution, you’ll make the same amount of small progress. But what that means is that you’re still behind at the end of the intervention. And we don’t know what happens next.
And certainly, there are studies that look at fade out effects. So for all interventions, not just NELI, but all interventions, the further on in time you go, the more that effect size reduces. And some interesting work suggesting that what happens next can be really important in the extent to which kids maintain those benefits over time. So it’s really important to start thinking what happens next. The way I think about language intervention is the way I think about things like weight loss. You know, what an intervention is doing is structuring the environment so kids can learn more effectively from that environment, which is the same thing we do and when we try and lose weight, we exercise more, we monitor what we eat, but it doesn’t fundamentally change our internal processes. So once you take that support away, then it becomes difficult again for kids to learn.
[00:48:18.850] – Courtney
So if we really want to maximise this investment, and it’s an important investment, and I’m delighted that the government recognises how important language is, but if you want to maximise that investment, you cannot stop with the NELI, you have to think about what happens next. And I do think there are reasons to be optimistic that kids can learn language. We all learn language through the lifespan. We shouldn’t give up on kids if early intervention is not enough. My worry is we put a lot of money into early intervention and we forget about what happens next and then we don’t really maximise our return on investment.
[00:48:56.930] – Andre
It sounds like what you’re saying, I don’t want to put words in your mouth, but it sounds like what you’re saying is that this trial doesn’t really help children with severe problems. And it’s, you know, it’s not intended to. It’s a general intervention to identify and support children generally. Are we not doing enough for children with more severe issues?
[00:49:17.135] – Courtney
So, one I would say, it probably does help them, but it doesn’t cure them. And I’m not sure there’s any intervention that could do. And it’s not specifically designed for kids with DLD, and we just don’t know enough about how they respond. I think they will make progress, and it looks like some of them, I mean, I think there are exclusions that might have, meant that the kids with the most severe difficulties, you might have more complex needs that aren’t included in the trial. We just need to know a whole lot more. So I don’t think it will harm them. I think there’s benefit. And like Gill says, I think there is broader benefits that are harder to measure in kind of school ethos and what schools are learning about language and language disorder in general. But if what we want to do is really support kids with DLD, then we need trials that are looking at that population in particular.
And I would say, you really need, we really all need to be thinking creatively about how you can do that in a sustainable way over the longer term. Because I think even, you know, 20 weeks is a long time, but I still don’t think that these short term interventions will have the lasting impacts that we want them to unless we can think about sustainable ways to maintain those early gains. Just like you can do a weight loss intervention, but it might not have the lasting impact unless you continue to change your behaviour over the longer term. There’s, unfortunately, no quick fix. So thinking creatively about how you can do that sustainably. And I know for schools, it can be a challenge to implement. And I know that schools sometimes worry about, with the withdrawal aspect of NELI, they don’t want kids to miss out on class time, and that becomes more challenging as kids get older.
So thinking about maybe smaller fixes that you can do on a daily basis that don’t have the same kind of impact on timetable, and things that can happen over the longer term. So I don’t think it’s ever too late to provide children with good support and intervention. Early intervention is important, but it’s not going to solve the problem, and we need to be thinking creatively about how we can do that.
[00:51:37.430] – Andre
Okay. Allison, I wanted to give you some time just to kind of share your reflections because we’ve had a bit of a kind of intense research conversation for 20 minutes or so. Can you bring us back to the real world and kind of front line and share [inaudible 00:51:53] on the conversation?
[00:51:55.490] – Allison
I think NELI is far more than just a language intervention. I’ve seen children almost grow into themselves. They’re given that time, they spend time daily with the same, hopefully, the same practitioner, which develops a very secure relationship, a trusting relationship where they grow and feel comfortable to use their voice, share their ideas. They’re given skills to develop their language to understand different meanings of words. And that confidence was then, we would find, it was transferring back into the classroom. It’s almost as if the children actually grew into themselves. So I think it’s far more, it gives a child far more than just the oral aspect, which is incredible, how those children have developed, but it gives far more than just that.
[00:52:58.290] – Andre
And how are parents involved? We’ve had a couple of questions about parents, and we haven’t really spoken about parents yet. How are parents involved in NELI and in the wider support in relation to language problems?
[00:53:10.120] – Allison
Yes, parents are involved. A letter goes out to parents asking if they’re happy for their child to participate. And they’re kept up to date throughout the different topics to say what their child has learnt. I would typically send out a newsletter at the end of each topic to say what their child had been focusing on, and perhaps a little activity that they could do to further embed their learning just so that we’ve got the school home relationship and families involved, and to hopefully transfer those skills to home as well.
[00:53:45.510] – Andre
There seems to have been quite a bit of conversation in the chat about bringing specialist services into schools or making the link between schools and specialist services work better. Does anybody want to kind of share and reflect on that and how we can do that more? Courtney?
[00:54:04.830] – Courtney
Well, it comes down to funding, and probably, a lot of parallels to the discussions you’ve been having about mental health support in schools. I definitely think this is one way to do it. I would say when the NELI was first being developed, I think every classroom had a teaching assistant and schools were more resourced to develop this, and I think that picture has changed over time, and making that sustainable is a challenge. I would like to see speech and language therapists more directly embedded in schools and school culture and working with teachers. Language is in every single aspect of the curriculum, and it underpins all of our social interactions. I think SLTs have a really good role to play with the people who are supporting mental health in schools because children with language impairment can’t always access talking based therapies.
So I really think more joined up thinking would benefit everyone, particularly young people with language disorder. So it would be great to have more opportunities to think about what that would look like, and how it could be implemented effectively in schools.
[00:55:18.630] – Andre
Okay, I think that’s a nice, fairly positive, but give us more money message to leave on.
[00:55:26.040] – V1
Find out more about becoming an ACAMH member, and to be part of the Advancement of Child and Adolescent Mental Health, visit www.acamh.org.
About #CAMHScampfire
ACAMH’s vision is to be ‘Sharing best evidence, improving practice’, to this end in December 2020 we launched ‘CAMHS around the Campfire’, a free monthly virtual journal club, run in conjunction with André Tomlin. We use #CAMHScampfire on Twitter to amplify the discussion.
Each 1-hour meeting features a new piece of research, which we discuss in an informal journal club session. The focus is on critical appraisal of the research and implications for practice. Primarily targeted at CAMHS practitioners, and researchers, ‘CAMHS around the Campfire’ will be publicly accessible, free to attend, and relevant to a wider audience.
Previous sessions are listed in our Talks & Lectures section.
About the panel
Gillian is a post-doctoral Research Fellow at the Department of Education working with Professor of Psychology and Education, Charles Hulme.
Recent projects include an EEF-funded randomised controlled trial (RCT) of the Nuffield Early Language Intervention (NELI) in 200 primary schools across the UK. Watch a video of an intervention session in action on the recent trial here.
She is currently working on the development and evaluation via RCT of an early language enrichment programme for children in nursery and the development and standardisation of the LanguageScreen app, which assesses a range of language skills in young children.
Gillian completed her PhD at UCL under the supervision of Professors Charles Hulme and David Shanks. Her research combined an interest in both memory and language, investigating the relationship between procedural and declarative memory processes and language-related attainment in children. Bio and image via Dept of Education, University of Oxford
Professor Courtenay Norbury
Courtenay Norbury is Professor of Developmental Disorders of Language and Communication at Psychology and Language Sciences, University College London. She is the Director of the Literacy, Language and Communication (LiLaC) Lab and a Fellow of the Royal College of Speech and Language Therapists.
She obtained her PhD in Experimental Psychology at the University of Oxford, working with Professor Dorothy Bishop on the overlapping language profiles that characterise autism spectrum disorder and ‘specific’ language impairment. Professor Norbury’s current research focuses on language disorders and how language interacts with other aspects of development. She is leading SCALES, a population study of language development and disorder from school entry. She is also a founding member of the RADLD campaign.
Follow on Twitter @lilacCourt
Andre Tomlin
André Tomlin is an Information Scientist with 20 years experience working in evidence-based healthcare. He’s worked in the NHS, for Oxford University and since 2002 as Managing Director of Minervation Ltd, a consultancy company who do clever digital stuff for charities, universities and the public sector. Most recently André has been the driving force behind the Mental Elf and the National Elf Service. The Mental Elf is a blogging platform that presents expert summaries of the latest reliable research and disseminates this evidence across social media. They have published thousands of blogs over the last 10 years, written by experts and discussed by patients, practitioners and researchers. This innovative digital platform helps professionals keep up to date with simple, clear and engaging summaries of evidence-based research. André is a Trustee at the Centre for Mental Health and an Honorary Research Fellow at University College London Division of Psychiatry. He lives in Bristol, surrounded by dogs, elflings and lots of woodland!
Follow on Twitter @Mental_Elf
I am an information scientist with an interest in making knowledge from systematic research more accessible to people who need it. This means you. I’ve been attempting this in the area of Evidence-Based Health Care since 1995. So far the results have been mixed. For some reason we expected busy clinicians to search databases and appraise papers instead of seeing patients. We also expected publishers to make the research freely available to the people who paid for it. Ha! Hence The National Elf service.
