There has been a public outcry recently about the idea of baseline tests for Reception-age children in English schools. Children seem to be increasingly reduced to data points. In general, we seem to be having a gradual realisation that all is not well with how data is being used about us, as seen with the Cambridge Analytica and Facebook debacle this week.
I have been thinking a lot about statistics, data and childhood from my own experience as a parent and thought it might be an interesting exercise to do a chronological walk through of some of the insights I have had. My basic understanding is that we use statistics and data to make all sorts of decisions, often guided by professionals, that sometimes seem to make no sense at all and at worst make us conform in a way that is simply wrong.
Conception and birth
If you know anything about conception and birth, you will know that statistical information guides so much of the experience in the Western world. Given my childhood experience, this started with my attention being drawn to the stark statistics around divorce. Since one in three marriages end in divorce I made a grim decision that whatever I do with regards relationships and family, I should never embark on anything that I can’t sustain alone should my relationship not succeed.
I was lucky enough to not have to think about the stats around being pregnant post-40 or have any particular difficulty getting pregnant, which would mean the heartache, angst and combined prayer and number crunching involved in IVF or similar assistance with getting pregnant and staying pregnant to term. But what I did experience with my second pregnancy was alarming enough.
In Israel, where I lived at the time, there are quite a large number of tests carried out during pregnancy, with the option of doing more should you wish to. I had all of the usual ultrasounds, and a blood test to determine the likelihood of certain genetic issues. I won’t go into all of the intimate details but from the get go, I wasn’t entirely sure that the calculation of what week I was in during pregnancy was correct. This became more acute when I had the blood test for common genetic disorders, which was cross-referenced with the latest ultrasound scan – and I was subsequently called to do a further blood test and finally to speak with a specialist at the genetic abnormalities clinic. All I knew before going into the appointment was that they had deemed the statistical chance of me having a baby with genetic abnormalities to be higher than average and they recommended amniocentesis. If you don’t know what this is (and I didn’t and had to quickly read up on it at the time), the basic information you need to know is that a trained medical professional will insert a long syringe through the abdomen into the womb and extract a tiny amount of amniotic fluid so that they can do analysis on the genetic make-up of the developing fetus.
What has all of this got to do with statistics? So here goes. The information that you glean about amniocentesis contains two sets of stats that you need to weigh up before you go ahead. One is the level of accuracy of the outcomes of the test, and two is the likelihood that you will miscarry as a result of infection or disturbance to the pregnancy. These were two scenarios I was going to be asked to consider when attending the consultation with the specialist. But a third, pivotal variable struck me. Was their original data on the likelihood of my unborn fetus having some kind of birth defect correct in the first place? And if it was, did it have any bearing on the statistical analysis they had presented me with?
I went into the meeting alone. My heart was pounding and I listened as best I could as they repeated that they advise amniocentesis and that the stats show that the situation doesn’t look great. I was determined to get to the bottom of how they make these calculations. I didn’t profess to know much about statistics, genetics or even pregnancy at this stage, but I knew that it was important to unpick the evidence and reassemble it so that I could make an informed decision.
They agreed to walk me through the methodology and that’s when the light went on. I asked questions and we ended up agreeing that a lot of it hinged on the calculation of the age of the fetus. My instinct was that the fetus I was carrying was in fact older than they had assumed by possibly up to two weeks. I had proof for this and asked the specialist if she could do some modelling based on the fetus’ age being one week and two weeks older. She disappeared for about 15 minutes and returned with a new spreadsheet, while I sat biting my nails waiting. Lo and behold, the statistical evidence showing that I should be having amniocentesis and that the baby could be born with genetic birth defects suddenly reduced and there I was again, safely within the ‘normal’ risk band.
I can’t really convey the drama of this experience but while it was happening, I felt like my life (more importantly that of my unborn child) absolutely hinged on getting this right. Imagine if I hadn’t questioned the statistics, hadn’t tried to understand where the evidence had come from and hadn’t insisted on interleafing it with contextual and qualitative personal evidence.
My daughter was born healthy, thank goodness. She arrived what was assumed to be a month early, jaundiced, but otherwise fully developed and not in need of specialist care other than invasive daily heel-prick tests for haemoglobin levels for two weeks. That made me think that I was probably right about the pregnancy being further along than assumed and that she wasn’t really that premature at all. We will never know.
Birth and the first year
The politics of childbirth needs a blog post in its own right – it’s nearly 13 years since I last gave birth and I am still psyching myself up for that one. There is much written about it based on research and real-life experiences of millions of women worldwide. It’s a statistical minefield combined with variables such as shift changes, risk management and more. One thing that I hear time and again, and was tripped up by myself, is the use of statistical tables to place newborns into percentiles. You only have to spend time with the people who have had babies at a similar time to you, to hear the competitive edge of statistics, measurements, milestones and comparisons being flung about right into their second and third year and beyond. “The baby’s in the 95th percentile!” (There’s always problematic gender-related subtext in there too – massive equals good, strong if it’s a boy, and nagging worry if it’s a girl that she might be obese, into childhood and adulthood).
There’s nothing wrong with this in itself and knowing ‘what’s normal’ is something we all find useful when trying to benchmark and make decisions accordingly – especially when you have no prior experience of a fragile newborn. But what I see time and again with new parents I know is this scenario:
- Baby is born, the couple tells everyone two key pieces of statistical information – how long it took and the baby’s birth weight
- The health visitor visits you at home and tells you the baby has lost too much weight after the birth and is now in x percentile
- Health visitor says the baby probably ‘isn’t getting enough milk’ and that you should supplement with formula to hurry along replacing the lost weight
- You are alarmed. You didn’t know babies lost weight after birth and it doesn’t sound good
- You feel frustrated, the baby seems to be feeding constantly and the health visitor is now describing a path were your baby is in danger of slipping into the wrong percentile – perhaps this isn’t normal and you should speed them along as suggested
- You acquiesce and start to bottle-feed between breast-feeding, which is a shame as you are just getting the hang of it. You are feeling a little inadequate and worried that your insistence on breast is best is naïve even though your NCT class said the statistics tell us this
- Complications start, your baby seems to want bottle-feeding more than from source, fusses on the breast and does seem to sleep better and feeds less frequently when you bottle-feed – and baby is now climbing up the percentile charts again
- A new statistic is born – not everyone can breast-feed and it is shown to be better to switch to bottle if the baby is ‘not thriving’ i.e. not staying within the percentiles that the health workers are using to benchmark your baby with
Faced with this information that my baby was shrinking, I was anxious but also wanted to know the facts. Where does the information come from for these percentiles? What about qualitative and family-specific information that we can cross reference with? What about the fact that the baby seems happy enough – or in my case not happy all the time but demand-feeding frequently and eventually became huge. Many health workers will supplement explanations like the baby is ‘lazy’, has a ‘weak latch onto the breast’, needs to be woken and fed and not demand-fed. We followed this waking and feeding advice and ended up with a huge, well-fed baby who had massive sleep issues potentially exacerbated because we were interfering with her sleep patterns to stuff her with mummy milk at every opportunity. Afterall, the percentiles were what we were trying to comply with.
If you scratch the surface, you can see where a lot of the data we use with regards babies, is deeply flawed. In this case, much of the percentile charts that are used, can come from the United States where babies are born bigger and are more likely to be bottle fed, or from WHO statistics or indeed locally produced versions. What about common-sense factors like the physical make-up of each of you as the parents, your parents’ experience of you as a newborn, and so on. And what about time? Who says that these percentiles are accurate in terms of the time it takes to regain the weight lost by the baby after the birth and the time it takes to move up the already flawed charts?
One of the major factors that disturbs me with childbirth, newborn growth and later into schooling is how much of this is directly related to the health visitor, medical practitioner and education practitioners’ own performance management, and the statistical evidence that is provided as evidence of them doing a good job themselves?
Schooling and beyond
It’s no secret that our education system has become increasingly informed and driven by data. And like the health worker, educational professionals’ performance management dictates what is deemed success, more often than the practitioners’ own professional judgement. Evidence-informed decisions around what works are useful. But we haven’t really answered the question about what ‘what works’ actually means. In its most reductive sense it means, what gets them passing the tests and getting the set of qualifications that will best position them to earn well in adulthood.
Let’s start with choosing a school and the way in which many parents use publicly available evidence and data to do this. I wrote previously about this in my post about choosing a secondary school here. It is clear that the statistical evidence that parents use when choosing a primary or secondary school is deeply flawed in many ways. Let’s look at each in turn:
Ofsted results – this is a snapshot in time and the numerical result is usually where most parents start and finish. Delving into the last two or three reports is probably more useful, and then cross referencing the areas for improvement and quizzing the SLT about it when you visit the school might yield a much clearer picture. The truth is that most Outstanding and some Good rated schools haven’t had an Ofsted inspection for anywhere between 3 and 10 years. The leadership might well have changed at least once since the last inspection, or it might have stayed the same and potentially stagnated – and who knows what Ofsted would rate the school as today? At best, it’s a guide as to how well the school was able to get itself to the place where they were graded as such on that specific day in time and that is it.
League tables – it has been written about recently by Education Datalab that many selective schools are propped up by an entire army of private tutors. I believe that if we look into it, we might see that many Outstanding-rated primary and secondary schools are similarly reliant on parent-funded tutoring and extra-curricular activity to support a proportion of children reaching higher standards in their SATs, and GCSEs, as well as to keep them in top sets throughout their secondary education. It’s worth understanding if this is the case, that any decision you make will potentially require a financial investment if the levels of achievement aren’t being gained actually within the school day. Can you know this from looking at league tables?
Another thing about league tables is obviously the background information about cohort, intake, whether exam specs changed that year. League tables are based on one year of test and exam information. Who is to say that the school is able to repeat this year on year, and how are you able to know whether your child will be one of the successful top performers? And the key question is always, at what cost? Not just to your pocket but to your child’s own experience of learning as joyful and broad rather than stressful and narrowly channelled to SATs and GCSE success from the get-go. You only have to look at what is happening from year 7 and 8 in schools now as schools move to a 3 and 4 year GCSE pathway to ensure they get the results and hold their place in the league tables.
GCSE results – even if you feel comfortable with the different lines of reporting on secondary schools and delve into things like value added, are you able to discern what this actually means in terms of the qualitative journey of individuals within the school? Are you cross-referencing with exclusion levels, levels of deprivation, in-year movement of students, outcomes for different marginalised groups, what the outcomes are for all children – especially those of different socio-economic backgrounds to your own? Do you even care? Can you have any impact on this – by perhaps becoming a school governor?
The big question for me with all of the available data is not just what are my child’s chances of reaching their potential at the school of our choosing, but also what are the issues on a societal level that affect the school population and what can we do to help counter them for the good of all children at the school? Aside from this, I can see clearly that the data that people are relying on is too simplistic to be useful. This is especially so if the information is not cross-referenced with qualitative evidence only gleaned by visiting the school, getting involved in the local community and making a subjective guess-timate based on your knowledge of your own child now and what they might be like in years to come.
Data which informs and data which makes us conform
The problem with data is how we use it, and how it uses us. In many cases, use of data is a quick, lazy way to make decisions. Yet cross-referencing data with qualitative information is difficult to do if this is not available. We need to rely on our own enquiring minds, imagination and pushing the boundaries of what we think is true because it is fed to us by the media and political agendas. Data is useful, but extremely dangerous when not used to just to inform, but instead creates a systematic evidence base to make us conform for potentially the wrong reasons as explored in this post.
In the case of the newborn, our decisions can be narrowed down to a choice to hurry our baby along to the detriment of our own freedom of choice on feeding and submitting to a choice of pace that is dictated by statistics, or a health visitors’ success-ranking criteria, rather than the facts before us. In the case of choosing a school, I believe that data use and school choice can make us stunningly narrow-minded, selfish and irresponsible. Choosing the best for our child doesn’t often include a moral decision to ensure that through sending our child to their local school we can essentially be part of ensuring the success of the school for all its students.
Increasingly, we see a situation where data was once useful and ‘that which can be measured can be deemed important’, can quickly creep to ‘only that which can be measured is deemed important’ in decisions we take regarding childhood and education.
tenpencemore 