Moderating marks

Sometimes, school marks are moderated. That is, the actual marks are adjusted to better reflect students’ performances. For example, if an exam is very easy compared to another, you may want to scale down the marks on the easy exam to make it comparable.

I was testing out the impact of moderation. In this video, I’ll try and walk through the impact, visually, of using a simple scaling formula.

BTW, this set of videos is intended for a very specific audience. You are not expected to understand this.

Rough transcript

First, let me show you how to generate marks randomly. Let’s say we want marks with a mean of 50 and a standard deviation of 20. That means that two-thirds of the marks will be between 50 plus/minus 20. I use the NORMINV formula in Excel to generate the numbers. The formula =NORMINV(RAND(), Mean, SD) will generate a random mark that fits this distribution. Let’s say we create 225 students’ marks in this way.

Now, I’ll plot it as a scatterplot. We want the X-axis to range from 0 to 225. We want the Y-axis to range from 0 to 100. We can remove the title, axes and the gridlines. Now, we can shrink the graph and position it in a single column. It’s a good idea to change the marker style to something smaller as well. Now, that’s a quick visual representation of students’ marks in one exam.

Let’s say our exam has a mean of 70 and a standard deviation of 10. The students have done fairly well here. If I want to compare the scores in this exam with another exam with a mean of 50 and standard deviation of 20, it’s possible to scale that in a very simple way.

We reduce the mean from the marks. We divide by the standard deviation. Then multiply by the new standard deviation. And add back the new mean.

Let me plot this. I’ll copy the original plot, position it, and change the data.

Now, you can see that the mean has gone down a bit — it’s down from 70 to 50, and the spread has gone up as well — from 10 to 20.

Let’s try and understand what this means.

If the first column has the marks in a school internal exam, and the second in a public exam, we can scale the internal scores to be in line with the public exam scores for them to be comparable.

The internal exam has a higher average, which means that it was easier, and a lower spread, which means that most of the students answered similarly. When scaling it to the public exam, students who performed well in the interal exam would continue to perform well after scaling. But students with an average performance would have their scores pulled down.

This is because the internal exam is an easy one, and in order to make it comparable, we’re stretching their marks to the same range. As a result, the good performers would continue getting a top score. But poor performers who’ve gotten a better score than they would have in a public exam lose out.

Visualising student performance 2

This earlier visualisation was revised based feedback from teachers. It’s split into two parts: one focused on performance by subject, and another on performance of each student.

Students’ performance by subject

Visualisation by subject

This is fairly simple. Under each subject, we have a list of students, sorted by marks and grouped by grade. The primary use of this is to identify top performers and bottom performers at a glance. It also gives an indication of the grade distribution.

For example, here’s mathematics.

Student scores in a subject

Grades are colour-coded intuitively, like rainbow colours. Violet is high, Red is low.

Colour coding of grades 

The little graphs on the left show the performance in individual exams, and can be used to identify trends. For example, from the graph to the left of Karen’s score:

A single student's score

… you can see that she’d have been an A1 student (the first two bars are coloured A1) but for the dip in the last exam (which is coloured A2).

Finally, there’s a histogram showing the grades within the subject.

Histogram of grades

Incidentally, while the names are fictitious, the data is not. This graph shows a bimodal distribution and may indicate cheating.

Students’ performance

Visualisation by student 

This is useful when you want to take a closer look at a single student. On the left are the total scores across subjects.

Visualisation of total scores

Because of the colour coding, it’s easy to get a visual sense of a performance across subjects. For example, in the first row, Kristina is having some trouble with Mathematics. And on the last row, Elsie is doing quite well.

To give a better sense of the performance, the next visualisation plots the relative performance of each student.

Visualisation of relative performance

From this, it’s easy to see that Kristina is the the bottom quarter of the class in English and Science, and isn’t doing to well in Mathematics either. Gretchen and Elsie, on the other hand, are consistently doing well. Patrick may need some help with Mathematics as well. (Incidentally, the colours have no meaning. They just make it overlaps less confusing.)

Next to that is the break-up of each subject’s score.

Visualisation of score break-up

The first number in each subject is the total score. The colour indicates the grade. The graph next to it, as before, is the trend in marks across exams. The same scores are shown alongside as numbers inside circles. The colour of the circle is the grade for that exam.

In some ways, this visualisation is less information-dense than the earlier visualisation. But this is intentional. Redundancy can help with speed of interpretation, and a reduced information density is also less intimidating to first-time readers.

Visualising student performance

I’ve been helping with visualising student scores for ReportBee, and here’s what we’ve currently come up with.

class-scores

Each row is a student’s performance across subjects. Let’s walk through each element here.

The first column shows their relative performance across different subjects. Each dot is their rank in a subject. The dots are colour coded based on the subject (and you can see the colours on the image at the top: English is black, Mathematics is dark blue, etc.)

class-scores-2

The grey boxes in the middle shows the quartiles. A dot on the left side means that the student is in the bottom quartile. Student 30 is in the bottom quartile in almost every subject. The grey boxes indicate the 2nd and 3rd quartiles. Dots on the right indicate the top quartile.

This view lets teachers quickly explain how a student is performing – either to the headmistress, or parents, or the student. There is a big difference between a consistently good performer, a consistently poor performer, and one that is very good in some subjects, very poor in others. This view lets the teachers identify which type the student falls under.

For example, student 29 is doing very well in a few subjects, OK is some, but is very bad at computer science. This is clearly an intelligent student, so perhaps a different teaching method might help with computer science. Student 30 is doing badly in almost every subject. So the problem is not subject-specific – it is more general (perhaps motivation, home atmosphere, ability, etc.) Student 31 is consistently in the middle, but above average.

class-scores-3

The bars in the middle show a more detailed view, using the students’ marks. The zoomed view above shows the English, Mathematics and Social Science marks for the same 3 students (29, 30, 31). The grey boxes have the same meaning. Anyone to the right of those is in the top quarter. Anyone to the left is in the bottom quarter.

Some of bars have a red or a green circle at the end

class-scores-5

The green circle indicates that the student has a top score in the subject. The red circle indicates that the student has a bottom score in the subject. This lets teachers quickly narrow down to the best and worst performers in each subject.

The bars on top of the subjects show the histogram of students’ performances. It is a useful view to get a sense of the spread of marks.

class-scores-4

For example, English is significantly biased towards the top half than Mathematics or Science. Mathematics has main “trailing” students at the bottom, while English has fewer, and Social Science has many more.

Most of this explanation is intuitive, really. Once explained (and often, even when not explained), they are easy to remember and apply.

So far, this visualisation answers descriptive questions, like:

  • Where does this student stand with respect to the class?
  • Is this student a consistent performer, or does his performance vary a lot?
  • Does this subject have a consistent performance, or does it vary a lot?

We’re now working on drawing insights from this data. For example:

  • Is there a difference between the performance across sections?
  • Do students who perform well in science also do well in mathematics?
  • Can we group students into “types” or clusters based on their performances?

Will share those shortly.

On teaching

This vacation, I took a session each for class XI and XII at my school, Vidya Mandir. The subject was Computer Science (the only one I can teach with some confidence), and the topic was networks.

It was an experiment, in two parts. The first was to understand how students of this generation interact with the Internet. (I’m twice as old as them, so I guess they qualify as the next generation.) The second was to see whether I’d leave them far behind, or they’d leave me far behind.

I began the class with a series of questions.

How many of you have… Expected Actual
Access to a PC and the Internet (home or nearby).
I was expecting ~80%. Every single one of them raised their hands. Every single one.
80% 100%
Chatted online.
I was expecting ~70%. Every single one, except for one girl, raised their hands.
70% 100%
Used a bluetooth device.
I was expecting around 60%. I got nearly everyone, but the remaining were wondering what that was.
60% 100%
Video-chatted.
I expected ~50%. Got ~80%
50% 80%
Uploaded a photo or video.
Again, far more than expected.
40% 80%
Own a blog or website.
This is where the surprises started. I thought that at least one in 3 would have a blog. Turns out I was wrong. There were very few.
30% 5%
Written a web application.
Not one soul. Some thought they had, but no…
10% 0%
Contributed to an open source project.
None at all.
1 or 2 0%

It was an eye-opener. On the one hand, everyone has an Internet connection. (In fact, the announcements following the morning prayer began with the Principal warning about the dangers of chatting with strangers online.) On the other hand, they’re doing little of the cool stuff.

Some of the discussions I had after class did lessen my concern a bit. There are, as always, a few that are very interesting in hacking, and are playing around with a lot of interesting things. But still, on average…

As for the other part of the experiment, I spent an hour talking about what goes on behind the scenes when they search on Google, taking them down to some of the elements of HTTP. My slides are below. I do suspect I left a fair number of them behind, but there were a handful that were with me right up to the end.

Computer Networks: An Introduction
View SlideShare presentation or Upload your own. (tags: http)

But I learned something that I did not expect. I spent a lot of time at the staff room, and talking with the teachers. The best way I can summarise what I learnt is through this Calvin and Hobbes strip.

Somehow, I thought the bulk of the discussion at the staff room would centre around students. Or, at the very least, around education. It was eye-opening to listen to a two-hour-long argument on the political reasons behind the tea at primary school staff room being better than at high school’s.

I remember my first book on acting defining a modern-day magician as "an actor who plays the role of a magician". The modern-day teacher is, in similar vein, an employee assigned role of a teacher. Teaching is their profession, not passion. Not that they are disinterested, quite the opposite. But oh, it could be so much better!

I read a speech by John Taylor Gatto titled "The Six-Lesson Schoolteacher". He gave this speech on being awarded the New York State Teacher of the Year award in 1991. He teaches six lessons at school, he says.

The first lesson I teach is: "Stay in the class where you belong." I don’t know who decides that my kids belong there but that’s not my business.

The second lesson I teach kids is to turn on and off like a light switch. I demand that they become totally involved in my lessons… But when the bell rings I insist that they drop the work at once and proceed quickly to the next work station. Nothing important is ever finished in my class, nor in any other class I know of.

The third lesson I teach you is to surrender your will to a predestined chain of command… As a schoolteacher I intervene in many personal decisions, issuing a Pass for those I deem legitimate, or initiating a disciplinary confrontation for behavior that threatens my control.

The fourth lesson I teach is that only I determine what curriculum you will study…. Of the millions of things of value to learn, I decide what few we have time for. Curiosity has no important place in my work, only conformity.

In lesson five I teach that your self-respect should depend on an observer’s measure of your worth… A monthly report, impressive in its precision, is sent into students’ homes to spread approval or to mark exactly — down to a single percentage point — how dissatisfied with their children parents should be.

In lesson six I teach children that they are being watched. I keep each student under constant surveillance and so do my colleagues… Students are encouraged to tattle on each other, even to tattle on their parents. Of course I encourage parents to file their own child’s waywardness, too.

I smiled a bit when I read this. It had been a while since I’d been in school, and I was lucky to have been in very liberal colleges. But then I went back to school and saw it for myself. The organisation that comes closest to the school is the military… or the prison. Not exactly the best place to foster creativity.

I began my class this time by saying, "Look, I might be wrong in what I tell you. Usually, it’s not deliberate. Quite often, I simply may not know. Or I may mis-communicate. When in doubt, Google and Wikipedia. Let me repeat: this is the single most important thing that I can tell you. When in doubt, Google and Wikipedia."

At the end of the class, a few came over and said, "But how do we do that? Our teachers are asking us not to waste time on the Internet, and to stay away from Wikipedia!"

Sir Ken Robinson gave a TED Talk on Do Schools Kill Creativity? Do watch it. Apart from being one of the funniest 20-minute talks ever, it drives home a strong message. Schools aren’t quite organised to foster creativity. When they were created, that wasn’t the intent.

Teaching as a profession, I imagine, does not pay as much as many others. So there’s little interest for practitioners to enter the field. I can therefore understand and appreciate that it takes a long time for new knowledge to enter the curriculum. But also sad is the way the curriculum is treated. It isn’t treated, as Gatto says, as choices among the million things of value to learn. It is treated as a Bible that defines knowledge.

It is easy for teachers to fall into the trap. If it contradicts the curriculum, it is wrong. If it is not in the curriculum, it is irrelevant. Since I know the curriculum inside out, I know all that is required to know. It’s not that I refuse to learn. Just that there is nothing more to learn that is relevant.

As an institution, schools aren’t going away any time soon. Nor perhaps should they. But in the interest of knowledge and creativity, I can only hope for two things.

  1. Students: keep learning what you like outside of school. It may be your only hope.
  2. Everyone else: drop by to your old school or your nearby school, and offer to teach one class any subject you have a passion for. You’d be surprised at how well you’ll be received, how much you know, and how much you can learn by that interaction.

The Six-Lesson Schoolteacher

The Six-Lesson Schoolteacher, by John Taylor Gatto, New York State Teacher of the Year, 1991. (The first part of it is sarcastic. This man is speaking passionately of things he despises in the education system.)

The first lesson I teach is: “Stay in the class where you belong.” I don’t know who decides that my kids belong there but that’s not my business.

The second lesson I teach kids is to turn on and off like a light switch. I demand that they become totally involved in my lessons, jumping up and down in their seats with anticipation, competing vigorously with each other for my favor.

The third lesson I teach you is to surrender your will to a predestined chain of command. Rights may be granted or withheld, by authority, without appeal.

The fourth lesson I teach is that only I determine what curriculum you will study. (Rather, I enforce decisions transmitted by the people who pay me).

In lesson five I teach that your self-respect should depend on an observer’s measure of your worth. My kids are constantly evaluated and judged.

In lesson six I teach children that they are being watched. I keep each student under constant surveillance and so do my colleagues. There are no private spaces for children; there is no private time.

It is the great triumph of schooling that among even the best of my fellow teachers, and among even the best parents, there is only a small number who can imagine a different way to do things.

He concludes:

School is like starting life with a 12-year jail sentence in which bad habits are the only curriculum truly learned. I teach school and win awards doing it. I should know.

TEDTalk by Sir Ken Robinson

Sir Ken Robinson’s TED Talk on education is brilliant and funny. Some quotes that struck me:

If you think of it, children starting school this year will be retiring in 2065. Nobody has a clue, despite all the expertise that has been on parade the last four days, what the world will look like in five years’ time. And yet we’re meant to be educating them for it. So the unpredictability, I think, is extraordinary.

If you were to visit education as an alien and say “What’s it for?”, I think you’d have to conclude, if you look at the output, that the whole purpose of public education throughout the world, is to produce university professors. Isn’t it? They’re the people who come out on top, and I used to be one. (So there!) And I like university professors, but you know, we shouldn’t hold them up as the high watermark of all human achievement — they’re just a form of life.