Sometimes, school marks are moderated. That is, the actual marks are adjusted to better reflect students’ performances. For example, if an exam is very easy compared to another, you may want to scale down the marks on the easy exam to make it comparable.
I was testing out the impact of moderation. In this video, I’ll try and walk through the impact, visually, of using a simple scaling formula.
BTW, this set of videos is intended for a very specific audience. You are not expected to understand this.
Rough transcript
First, let me show you how to generate marks randomly. Let’s say we want marks with a mean of 50 and a standard deviation of 20. That means that two-thirds of the marks will be between 50 plus/minus 20. I use the NORMINV formula in Excel to generate the numbers. The formula =NORMINV(RAND(), Mean, SD) will generate a random mark that fits this distribution. Let’s say we create 225 students’ marks in this way.
Now, I’ll plot it as a scatterplot. We want the X-axis to range from 0 to 225. We want the Y-axis to range from 0 to 100. We can remove the title, axes and the gridlines. Now, we can shrink the graph and position it in a single column. It’s a good idea to change the marker style to something smaller as well. Now, that’s a quick visual representation of students’ marks in one exam.
Let’s say our exam has a mean of 70 and a standard deviation of 10. The students have done fairly well here. If I want to compare the scores in this exam with another exam with a mean of 50 and standard deviation of 20, it’s possible to scale that in a very simple way.
We reduce the mean from the marks. We divide by the standard deviation. Then multiply by the new standard deviation. And add back the new mean.
Let me plot this. I’ll copy the original plot, position it, and change the data.
Now, you can see that the mean has gone down a bit — it’s down from 70 to 50, and the spread has gone up as well — from 10 to 20.
Let’s try and understand what this means.
If the first column has the marks in a school internal exam, and the second in a public exam, we can scale the internal scores to be in line with the public exam scores for them to be comparable.
The internal exam has a higher average, which means that it was easier, and a lower spread, which means that most of the students answered similarly. When scaling it to the public exam, students who performed well in the interal exam would continue to perform well after scaling. But students with an average performance would have their scores pulled down.
This is because the internal exam is an easy one, and in order to make it comparable, we’re stretching their marks to the same range. As a result, the good performers would continue getting a top score. But poor performers who’ve gotten a better score than they would have in a public exam lose out.
Nice one… your analysis does assume that the performance of a group of students in an easy exam and the performance of the same group of students in a tough exam will both have similar distributions…
I am sure you will recall how the loss of resolution in marks (marks were awarded in increments of no less than 1/2) and the clustering of results at the top end ( remember our 10th & 12th public results in ’90 & ’92 respy) led to very skewed distribution?
In tough exams, I would expect that the skew would shift downward with a lot more positive Kurtosis… so besides the normalization of the exam scores, it would probably be necessary to perform some additional transformations to really compare 2 exams of different ‘toughnesses’.
All this, without even considering the ‘performance factor’ from the student introducing its own variations…
After writing all this, I do realize that my points are perhaps academically motivated – but if all we want to do is to help teachers pick out the students who are more likely to need ‘extra tuition assistance’ to make a better showing in a tough public exam, then the basic normalization is sufficient.
Of course, most good teachers would say not need statistics to tell them that – they know their students well enough to understand who needs help and who is on autopilot to ace the exams… (ring any bells ? 😉 )