correlation on S Anand

Students who are more engaged score more

Sat, 08 Feb 2025 11:18:17 +0000

This is about as insightful as the Ig Nobel winning papers “Boredom begets boredom” and “Whatever will bore, will bore” that methodically documented that bored teachers lead to bored students. But in the spirit of publishing all research without bias for success or novelty, let me share this obvious result.

The Y-axis represents the total score of ~2,000 students on 4 graded assignments, each of ~10 marks. The X-axis represents the percent rank of engagement. The most engaged students are at 100%. The least are at 0%.

How do I measure engagement? By the number of times they visit the page and how early they visit the page (both computed as percent ranks). So, the student who visits the assignment page the most often, and the student who visits the assignment page first, score highest.

For every 10% increase in the engagement, the score increases by about 3 marks. What that means is, if a student leapfrogs ahead of 10% of their batchmates, that effort typically leads to scoring about 3 / 40 = 7.5% more overall.

Sun, 15 Aug 2021 06:19:49 +0000

The most under-rated #geospatial capability is spatial joins, IMHO. It lets you draw correlations from #geodata.

During the 2019 assembly elections in India, it allowed me to join voter data (by constituency) with religious population (by district) and show that BJP’s vote share increased with Muslim population, while it reduced the AIMIM-allied JD(S) vote share.

That’s just the start. We can explore:

Where to open new schools?
Where to locate bank branches?
Does wealth lead to more or less theft?
Does poor health lead to more or less pharmacies?
… and hundreds of other random questions.

My day job doesn’t let me explore these much. I’d love to work with someone, though.

Would anyone like to explore such correlations? Which one(s)? I’d love to support you.

Talk video: https://youtu.be/ql3Kz8ZjhRg

Transcript: http://www.s-anand.net/blog/maps-delimitation-and-gerrymandering/

Correlating subjects

Sun, 12 Feb 2012 04:08:02 +0000

A question from Dorai get me thinking: does being good at maths help in programming?

I don’t have a personal view. But since Reportbee has data on the Class 12 examination results for the last three years, we thought we could do a bit of analysis.

Here’s the correlation of the scores of various subjects with Computer Science.

Correlation	Subject
0.79	CHEMISTRY
0.79	PHYSICS
0.75	ENGLISH
0.75	MATHEMATICS
0.72	LANGUAGE
0.67	BIOLOGY
0.66	ECONOMICS
0.66	COMMERCE
0.65	ACCOUNTANCY
0.56	HISTORY
0.52	GEOGRAPHY

It almost breaks neatly into four groups.

Physics & Chemistry, both of which have a correlation of 0.79, and clearly are the most correlated with Computer Science
Maths, English & Language, which have a correlation of 0.72 – 0.75
Biology, Economics, Commerce and Accountancy, which hover at around 0.66
History & Geography, which are 0.52 – 0.56

The results in 2010 are almost exactly the same.

Correlation	Subject
0.78	PHYSICS
0.78	CHEMISTRY
0.75	ENGLISH
0.75	MATHEMATICS
0.73	LANGUAGE
0.67	ACCOUNTANCY
0.65	ECONOMICS
0.65	COMMERCE
0.64	BIOLOGY
0.60	GEOGRAPHY
0.55	HISTORY

I’m not sure what it is that leads to this kind of correlation. In fact, the full correlation between every pair of subjects (for 2011) is below:

What inferences would you draw from this?

And what do you think is the reason for this?

Comments

Arun Ravindran 12 Feb 2012 5:53 am: My inference is purely anecdotal but might be helpful in explaining this data. I had chosen Computer Science for my +2 in 1998. It was taught in C++ and mostly involved memorising operations on Data Structures such as Lists, Stacks and Queues. The C++ standard library had to be memorized including I/O, string and file functions. The exams were basically a test of memory rather than attacking a new problem space mathematically. Guess which are the other subjects which involve memorising a huge set of symbolic facts? – Chemistry and to a certain extend, Physics. I believe the data is more revealing of our Computer Science pedagogical and evaluation methods than the subject itself.
S Anand 12 Feb 2012 6:46 am: Good point. It’s quite debatable whether marks in computer science are indicative of programming ability.
Anamika 12 Feb 2012 10:21 am: Netruvarai neram poga villaiyae, unadhu arugae neram podhavillaiyae…!
Sathya 16 Feb 2012 12:56 pm: In my experience, I’ve come across rock star programmers who have sound grasp of mathematics. But i dont have statistics to prove them. As we all know, programming (esp functional) is heavily influenced by mathematics. Coming to inferences, I would take the dataset with a pinch of salt. Is it diverse enough to be statistically significant ? I agree with most of the comments on relating “mugging up programs” to being good at Chemistry in particular.
Sathyaraj 12 Feb 2012 7:44 pm: My 2 cents based on experience. People who take the computer science group generally have to always take one language and English subject apart from taking Physics, Chemistry, Maths. They would not be able to take classes in accountancy, economics, history, etc. Some of the students at the top of the class would have realized that in order to differentiate in terms of coming first in class/school, one would need to excel in language and english. It is given that one needs to get excellent scores in physics, chemistry and maths to get an overall good percentage. Also the fact, that scoring a big total would enable them to get admission in colleges in like BITS pilani which looks at overall score. Students who take the biology or accountancy group have no such pressure for them to excel in English and language subjects. Hope it makes sense. Also, if you can do a correlation between computer science students and the language that they take, I believe you will find that the majority of students would have chosen french rather than Tamil or English.
Shankar V 13 Feb 2012 6:23 am: Anand Correlation is a misleading statistic if the dependent and independent variables are not “really” related that way. There could be compounding effects within the independent variables leading to wrong correlation coefficients. There are ways to detect and cleanse compounding - design of experiments (in statistics) deals in depth on this. It is quite possible that there could be some relationship in how the computer science and math exams are scheduled and the scores in these subjects. If they are too close to each other, the student may have had lesser time to prepare for the math exam. For example, if Physics and Math were scheduled one after the other, and Computer Science test is after the Math test, the student may have prepared better for Physics, not have had enough time for Math and then recovered to do well in Computer Science. This can be a pattern in the entire class as it is normal for kids to focus more on Physics (the dreaded subject!) v/s Math. The statistic may not reveal ability or natural alignment of the subjects.
Evgeni 11 Dec 2012 3:37 am: I’d rearrange subjects such that they’re more clustered together by correlation, like in a TreeMap view. That way it’s easier to see the relationships.
amanjot kaur 10 Sep 2015 12:28 pm: i need correlation of commerce with language

IMDB Top 250 outliers

Tue, 04 Apr 2006 12:00:00 +0000

On the IMDb top 250, you normally see a correlation between the number of votes and the rating for a movie. Better rated movies are more watched. The outliers are interesting.

The movies that are popular despite not having a high rating are:

I can understand why The Sixth Sense, Pirates of the Caribbean and especially The Matrix are on this list – geeks would have watched these and voted on IMDb, though their voting need not have been high. But why are Gladiator and Sixth Sense on that list?

Movies that are highly rated, but not as popular are:

Seven Samurai and The Good, The Bad, The Ugly probably didn’t get the votes they deserve because they’re written in their Japanese and Mexican names on IMDb. I hadn’t seen them for a long time for the same reason. As for The Godfather, I personally think it’s just overrated. But Rear Window? That’s a surprise. Hitchcock thriller with all the classic elements…

Another correlation is between the rating and the year of the movie. Early movies get lower ratings than recent movies. Technique could be the reason, but I doubt it. In any case, some movies stand out of their time.

I haven’t seen Metropolis or M. But among the others, I think Citizen Kane is the one that deserves to stand out, if only for portraying the anti-hero, and for not having a happy ending. The Shawshank Redemption was a bit of a surprise. Few people that I know have heard of it. And yet, there it is, right on top.

Comments

Madhu 4 Apr 2006 10:56 am: Some analysis this:) were u consulting for IMDB sometime?:)
S Anand 4 Apr 2006 12:22 pm: Nah, just had some time on my hands early this morning!
ritzkini 4 Apr 2006 2:29 pm: :D cool anal ! but..how did you get the raw data ?? is my question !
S Anand 4 Apr 2006 3:53 pm: Just cut and paste data on the IMDb top 250 page on Excel!
Shankar 5 Apr 2006 3:24 am: Another conclusion one could draw: The conclusion that a movie is good seems representative of public opinion due to the high number of votes, but that a movie is bad is only the conclusion of a few, and hence may not be representative of public opinion. Is this a correct conclusion?
S Anand 5 Apr 2006 6:04 am: I wouldn’t say that from this data. These represent the top rated movies on IMDb, i.e. the top 250 movies EVER. We can only say that high ratings are contributed by both large and small samples. Even that would be a weak statement because IMDb has a “minimum number of votes” cutoff for the top 250.
Govar 17 Jan 2007 6:30 pm: Another interesting thing I’ve noticed in IMDB ratings is that a lot of votes are meant either to push up or pull down. For example, almost every movie will have more number of people rating 1 than ratings 2,3 or 4. Which coudl mean only one thing: They want to bring down the rating. Similarly, lot of people go to the other extreme - 10 - instead of say 8 or 9.
S Anand 17 Jan 2007 7:40 pm: That’s an interesting observation… did you get that out of the raw IMDb data? I’ll have a look at that. Should be interesting analysis to do.
fdf 24 Apr 2007 1:04 pm: imdb top 250 is not good, don;t listen to it, pp;l vote up a film, and vote films down! godgather and LOTR is over rated!