Purple Tod Consulting

PIRLS – Nic Spaull

PIRLS is implemented by the Centre for Evaluation and Assessment (CEA) at Pretoria University headed by Prof Sarah Howie. In 2016 it tested 12,810 Gr4 students from 293 schools across the country. The sample is nationally representative and can be generalized to the entire country. Students were tested in whatever language was used in that school in Grades 1-3, i.e. all 11 official languages were tested and children were generally tested in the language with which they were most familiar. The results were released by Minister Motshekga on 05 Dec 2017 in Pretoria.

78% of South African Grade 4 children cannot read for meaning in any language. I think this was the most striking thing for me -that we had previously underestimated the number of South African children that couldn’t read for meaning. Previously we thought the number was 58% (using prePIRLS 2011 Intermediate Benchmark) but it turns out that it is 78% (PIRLS Literacy Low International Benchmark). Basically we were using the wrong benchmark in the past. This is the first time that the easier PIRLS test (which used to be called prePIRLS and is now called PIRLS Literacy) was put on the PIRLS scale.

Apart from the horrifically low levels of reading achievement, South Africa also has the highest incidence of bullying among all 50 countries that participated in the study. 42% of Gr4 students indicated that they were bullied weekly (p226 in the report). Compared to 15% in the US and England.

Main findings:

1. 8 of 10 SA children cannot read: 78% of SA Grade 4 students cannot read for meaning. That is to say that they could not reach the Low International PIRLS Benchmark in reading. They could not locate and retrieve explicitly stated information or make straightforward inferences about events and reasons for actions 
2. SA scores last in reading of 50 countries: South African Grade 4 children have scored the lowest mark in the latest 2016 round of the Progress in International Reading and Literacy Study released today. The study included mostly High Income Countries but there were a number of middle-income countries such as Iran, Chile, Morocco, and Oman.
3. SA lags far behind other countries: While 78% of SA Grade 4 kids cannot read, in America this is only 4% and in England just 3% cannot read. However the study also included middle-income countries. In Iran only 35% of Grade 4 students could not read for meaning and in Chile it was only 13%
4. Reading crisis deeper than previously thought: When South Africa participated in prePIRLS 2011 (an easier version of PIRLS) we thought that 58% of SA Gr4 children could not read for meaning. However this was on a separate test and not on the PIRLS scale score (i.e. not the same metric). 2016 was the first time that prePIRLS (now called PIRLS Literacy) was put on the same scale score as PIRLS. The true figure for children that cannot read for meaning is 78% - revealed today. Note this does NOT mean that reading outcomes have gotten worse between 2011 and 2015. In 2011 77% could not read for meaning and in 2016 78% cannot read for meaning (this difference is not statistically significant, i.e. the difference is negligible).
5. Some evidence of improvement in reading 2006 to 2011 but stagnant since 2011: The only good news coming out of PIRLS 2016 is that there may have been significant improvements in reading between 2006 and 2011. Because the scale scores are now comparable we can compare the performance of Gr4's in 2006 and Gr4's in 2011 and 2016. This comparison seems to suggest quite a significant increase in reading scores between 2006 and 2011. Notably the Gr4 students in 2011 achieved higher scores than Gr5 students in 2006. Further analysis is needed but there does seem to be legitimate evidence of improvement between 2006 and 2011. Unfortunately no evidence of improvement between 2011 and 2016.
6. SA reading scores stagnant since 2011: There has been no improvement in reading scores over the last five years (i.e. 2011 to 2016). Note that although the average scored declined from 323 to 320 this can NOT be interpreted as a decline. The standard errors overlap here so there is no certainty that there was any decline whatsoever (this is like taking your sitting heart rate 10 times and getting very tiny differences each time - they are not statistically significantly different)
7. SA gender gap in reading 2nd highest in the world: Girls score much higher than boys in reading across the board. In Grade 4 girls are a full year of learning ahead of boys. This gender gap is the second largest among all 50 countries that participated. Only Saudi Arabia's is higher. (PIRLS report page 36). The gap between boys and girls is also growing over time. The gap between boys and girls was larger in 2016 than in 2011 
8. SA boys scores seem to have declined between 2011 and 2016: The average Grade 4 girl in SA scored 341 in 2011 and 347 in 2016 (unlikely to be statistically significant). The average Grade 4 boy in SA scored 307 points in 2011 and 295 points in 2016 (this is likely to be statistically significant but we cannot tell until the SA report is released 
9. Declining number of SA students reaching high levels of reading achievement: In 2011 3% of SA Gr4 students reached the High International Benchmark. In 2016 only 2% reached this same benchmark