A new study shows why we shouldn’t worry too much about screen time scare stories.

For many of us, screen time presents something of a quandary. On the one hand, pretty much everyone has a strong opinion about the (usually negative) effects of using smartphones or tablets, or playing video games too much.  Our lived experience – that of seeing people ‘mindlessly’ glued to their phones, inattentive to the world around them – quietly tells us that this can’t be a good thing. On the other hand, the best scientific research that we have on the subject tells us that actually, the harmful effects of screen use are pretty inconsequential. Yes, problems such as depression and anxiety are on the increase. But if you were to list all of the things that are causing those increases in order of importance, screen time would come somewhere at the bottom.

Part of the reason for this disconnect – that our intuition and experience tells us that screens are bad, while the most robust research says that they’re not – is that not all research tells us the same story. So when a study comes along claiming that there’s a much stronger, perhaps more causal link between screen time and some sort of negative effect, it ends up much more likely to be picked up by media outlets. Everyone loves a good scare story, after all. Nowhere has this been more evident in recent years than in the case of research by social psychologist Jean Twenge. In 2017, a team led by Twenge published research based on data from two large US surveys which purported to show that teenagers who spent more time on social media were also more likely to report increases in depressive symptoms and suicide-related outcomes. In the opinion pieces that preceded and followed the study, we were urged to consider whether smartphones had ‘destroyed a generation’, and whether digital technology was the ‘likely culprit’ driving a significant deterioration in teen mental health.

If you take a look at the data itself, at face value the claims seem hard to disagree with. Twenge’s study was based on data from hundreds of thousands of teenagers. This wasn’t some hokey lab experiment with a tiny sample of participants. This was a huge dataset, with real people answering real questions. That it showed a clear correlation between digital technology use and mental health problems is something that we should sit up and take notice of, right?

A new study published in Nature Human Behaviour yesterday tells a different story, however, and cautions that with great data comes great responsibility. The research, by Amy Orben and Andy Przybylski at the Oxford Internet Institute, used data from three large, nationally-representative datasets – the same one used by Twenge’s team, plus two others. Their findings show that the sorts of analytical decisions that researchers make about using such data – things like how they specify teenage wellbeing, or technology use, or what other sorts of variables they might take into account that could potentially affect the results – can have a huge impact on the findings they uncover. The combination of specifications that can be used is eye-watering – Orben and Przybylski identified over one trillion ways in which the data could be packaged up for analysis. Taking a subset of around 20,000 of these combinations, they found that depending on which combinations were used, it was possible to find positive associations between screen use and wellbeing, negative associations, and statistically non-significant effects. In other words, using the same dataset each time, it could in theory be possible for researchers to publish thousands of papers either showing that screen time has a positive effect on wellbeing, a negative effect, or none at all.

It’s a finding that echoes research that has found a similar issue in the realm of violent video games research – in 2014, a study by Malte Elson and colleagues showed that because of the inherent flexibility in the way that a particular type of aggression research paradigm can be implemented, you could show that violent video games either do or do not cause aggression using exactly the same dataset. Here, the findings are extended to digital technology use more generally, and should prompt some real soul-searching for researchers who use these sorts of large public datasets. In general, Orben and Przybylski’s analysis suggested that while there is a small yet significant negative association between screen time and wellbeing, it’s so minimal as to be meaningless. As Orben has explained on Twitter, the amount of variability in wellbeing that screen time accounts for is less than half a percent. Or to put it in another context, their analysis found that wearing glasses has a more negative association with wellbeing than screen time does, and eating potatoes has pretty much the same association.   

Potatoes and smartphones. Equally deadly.

It’s an important paper for a number of reasons. First, it shines a methodical and rigorous light on precisely how important the decision-making process about which variables to assess and what sort of statistical analyses to conduct is. These should be done up-front, and locked in via pre-registration, before anything else – if decisions are instead taken on the fly, the risk is that we end up with worthless or misleading research results. Secondly, Orben and Przybylski’s study provides a refreshing call to arms for researchers tackling big data sets. Just because you’re working with data from thousands, or hundreds of thousands, of individuals, does not therefore mean that you’re conducting robust research. If anything, there’s a much greater responsibility to make sure you’re doing things correctly in these sorts of situations.

And lastly, if nothing else, the study shows us that screen time is not the big bad cause of societal breakdown that we’ve been led to believe it is for so many years now. That’s not to say that mental health issues aren’t on the rise, or that this isn’t a thing that we should be worried about. It’s just that if we really want to figure out why things like depression and anxiety are on the increase, we need to look beyond the screen.

Dr Pete Etchells is a senior lecturer in biological psychology at Bath Spa University. His research focuses on vision, eye movements and motion perception, as well as science policy and public communication of science. He was the author of the popular Headquarters blog at The Guardian and his first book will be released in early 2019. He is on Twitter at @PeteEtchells

If you would like to reuse this content please contact us for details

Subscribe to The Cosmic Shambles Network Mailing list here.