There is a huge uproar in the land of the lions about the Advanced Level results being completely unreliable. So, I thought I’d add my 2 cents on the subject.
Disclaimer: I did study math at uni, but I’ve forgotten
all most of it and I took only one stat class. So, I will keep it simple – that is not by choice, I am forced to.
The problem has arisen due to two different sets of exams taking place due to two different syllabi being employed. There are repeaters from 2009 and 2010 doing the old syllabus and then there are the 2011 kids doing the new syllabus. So, the outcome is that there were two sets of results.
But for university entry there needs to be a result set from which the top few can be selected. From what I understand, they took the two means of the Z-score distributions and polled it to get an adjusted mean. From this the adjusted Z-score was calculated.
There are 2 main errors in this process.
First, the Z-score calculation is not a linear function. Results from non-linear functions cannot be treated with linear functions. A simple example is to take the square function.
10+3=13 but 102+32 ≠ 132
Similarly, linear manipulation (polling the mean) of results from non-linear functions (Z-score) gives erroneous outcomes.
Second, how can 2 different exams be matched up properly? Two people who got the exact same result on two different exams cannot be deemed as equals. Furthermore, taking the mean, median etc are all flawed methods.Since the population taking the exams are varied there is no way to match one to the other.
So, the bottom line is that there is no correct (or should I say justifiable) way of doing this. 2 different exams – 1 result. One way or the other, some people will get the short end of the stick. But, I guess that’s life people – it’s not fair. You just have to live with it.