When mainstream news outlets pick up on research that appears to contradict what we believe to be true, it often results in a spate of sensationalist and misleading headlines. It’s often a good idea to read the associated report first, rather than succumbing to the demands of faster-than-light news cycles and merely re-using press release ‘highlights’ or following the media herd.

We all fall into the trap from time to time, despite how much we try not to sacrifice due diligence over the desire to publish first.

For example, on Friday we were going to write about two separate events: the outcome of the WSIS conference on meeting ICT development goals, and an academic study on texting. The former is delayed until tomorrow as there is just so much material to read and analyse – the temptation to make sweeping assumptions based on the ten-year failure to ensure half of the world’s population has online access was almost too great, but WSIS deserves better treatment than that.

As for the latter, the widely reported news (such as reported here, but there are many other examples) that texting and ‘text speak’ actually improves children’s spelling and grammar appeared too good to be true. Well, it was. Gr8, as they say.

A study conducted by researchers at the UK’s Coventry University and the University of Tasmania in Australia claimed to dispel the popular belief (certainly amongst parents and teachers) that writing coded text messaging – using abbreviations such as ‘gr8’ for example – damages children’s grasp of the English language. On the contrary, they said, it actually encourages them to think about language phonetically, requiring them to sound out spellings and so help them to learn and write words. Leaving out punctuation and capitals were also linked to the development of grammar skills.

Heading straight to the source, the academic paper was entitled “Grammatical understanding, literacy and text messaging in school children and undergraduate students: A concurrent analysis”.

The study looked at “grammatical violations made when text messaging and their performance on assessments of spoken and written grammatical understanding, orthographic processing and conventional spelling ability.” The very first line of the abstract sought to distance the study from just spelling: “Recent research has demonstrated that use of texting slang when text messaging does not appear to impact negatively on children and young people’s literacy and may even benefit children’s spelling attainment.”

The authors cite a number of academic works, including a 2009 study on “textisms”, but there’s a vast list of references attached to the paper, if you’re interested in exploring the subject further.

“For the primary and secondary school children there was no relationship between the tendency to make grammatical violations when texting and their understanding of conventional grammar or orthography,” it concluded. “We found that the text messages of children at both primary and secondary school were rife with grammatical violations. However, the children’s tendency not to use conventional grammatical rules in messages was not related to their reading, spelling, or grammatical skills.”

In fact, they found that – not surprisingly – these grammatical transgressions were often deliberate. After all, texting is fun, right?

Abbreviations and acronyms are not new additions to written language. However, the study authors believe that the popular media have focussed on the assumed negative impact that the use of such language must inevitably be having on language in general, and children’s and young people’s literacy skills in particular – rather than celebrating creativity. And of course, if you read the op-eds in the likes of the Daily Mail or Telegraph in the UK, this results not only in falling literacy standards, but also declining morality.

But here’s the rub. For starters, the study involved a sample size of just 173 pupils from the age of eight to sixteen (primary and secondary school ages). That’s not many. In fact, it’s so low as to make the whole thing worthless – fine as a snapshot study with limited aims, but you simply cannot extrapolate this and apply it globally.

There are around 2.2 billion children in the word, according to UNICEF. In the UK, Ofcom says that the overall number of children aged 5-15 owning a mobile phone was 43 per cent in 2013. However, a 2013 report from the GSM and the London School of Economics that looked at children’s use of mobiles (covering Japan, India, Indonesia, Egypt and Chile – with a survey size of a more respectable 4,500 pairs of children) found that 65 per cent used a mobile.

Whilst this excellent annual GSMA report has nothing to say about the effects of texting on spelling or grammar, it does contain a wealth of essential information – and remember, it is based on a statistically viable sample set.

The researchers from Coventry and Tasmania cite various results to their tests, which often appear contradictory. But then, the sample is too small to be viable. As they admit themselves:

“The first important limitation is that the sample sizes obtained within the individual age groups are quite modest, and caution therefore needs to be exercised when arguing that these data appear to suggest no consistent link between poor attention to grammar when texting and the development of grammatical understanding. It is possible that the study design lacks sufficient statistical power to detect what could be some quite subtle effects.”

Determining the most appropriate sample size is a science in itself. The two main factors are the confidence level and confidence interval. A 95 per cent level is common amongst studies, and an interval of 4 is desirable – in other words, plus or minus 4 percentage points. To achieve this, the sample size would have had to be over 600, not 173. By comparison, the GSMA’s study offered 99 per cent confidence with a 2 per cent deviation.

Perhaps they could approach the GSMA for funding assistance to dramatically increase the scope of any follow-up work? A comprehensive study on the impact of mobile and social media on language skills and literacy would be very welcome. And then, and only then, should parents and teachers read these ‘conclusive results’ in the press.