hacking on S Anand

Why don't students hack exams when they can?

Fri, 22 Nov 2024 10:38:09 +0000

This year, I created a series of tests for my course at IITM and to recruit for Gramener.

The tests had 2 interesting features.

One question required them to hack the page

Write the body of the request to an OpenAI chat completion call that:

Uses model gpt-4o-mini

Has a system message: Respond in JSON

Has a user message: Generate 10 random addresses in the US

Uses structured outputs to respond with an object addresses which is an array of objects with required fields: street (string) city (string) apartment (string) .

Sets additionalProperties to false to prevent additional properties.

What is the JSON body we should send to https://api.openai.com/v1/chat/completions for this? (No need to run it or to use an API key. Just write the body of the request below.)

There's no answer box above. Figure out how to enable it. That's part of the test.

The only way to even attempt this question is to inspect the page, find the hidden input and make it visible. (This requires removing a class, an attribute, and a style - from different places.)

Here's the number of people who managed to enable the text box and answer it.

College	# students	Enabled	Answered
NIT Bhopal	144	4 (2.8%)	0 (0.0%)
CBIT	277	16 (5.8%)	0 (0.0%)
IIT Madras	693	74 (10.7%)	4 (0.6%)

A few things surprised me.

First, I think students don't inspect HTML. Less than 10% of students managed to modify the HTML page, even after being told they need to. But they know web programming. 49 students at CBIT scored full marks on the rest of the questions, which includes CSS selectors and complex JS code. Maybe editing in a browser instead of an editor is a big mental leap?

Second, almost no one could solve this problem. There are 3 ways to easily solve it.

Copy the question and relevant test cases from my exam page's JavaScript into ChatGPT and ask for an answer. (I test it and it works.)
Copy the question and structured output documentation to ChatGPT and ask for an answer. (I tested it and it works.)
Create a random JSON and just keep fixing the errors manually until it passes. (The exam gives detailed error messages like "The system message must be 'Respond in JSON'", "addresses items must be an object", etc.)

Maybe questions from a curriculum are easier to solve than questions not in a curriculum? Or is JSON schema too hard?

The exam was officially hackable

All validation was on the client side. The JS code was minified and answers are dynamically generated. But a student can set a breakpoint, see the answers, and modify their responses.

The students at NIT Bhopal and CBIT were not explicitly told that. The students at IITM were explicitly told that they could (and are welcome to) hack it.

Out of the 1,114 students who took these tests, only one student actually hacked it.

(How do I know that? No other student got full marks. This student got full marks with empty answers.)

It's probably not that difficult. My course content covers scraping pages using JavaScript using DevTools. Inspecting JS is just a step away.

I did chat with the student who hacked it, asking:

Anand: How come you didn't share the details of the hack with others?

Student: I did with a few but I am not sure whether or not they were able to figure it out still.
Most students in the program still require a lot of handholding even with basic things.
Experience from being a TA [Teaching Assistant] past term.

Why didn't they hack?

Maybe...

They don't believe me. What if hacking the exam page is considered cheating, even if explicitly allowed?
The time pressure is too much. They'd rather solve what they know than risk wasting time hacking.
It feels wrong. They'd rather answer based on their knowledge than take a shortcut.
They don't know how. Using DevTools is more sophisticated than web programming.

Issue #1 - the trust issue - is solveable. We can issue multiple official notices.

Issue #4 - capability - is not worth solving. My aim is to get students to do stuff they weren't taught.

Issue #2 & #3 - a risk-taking culture - is what I want to encourage. It might teach them to blur ethical lines and neglect fundamentals (which are bad), but it might also build adaptability, creativity, and prepare them for real-world scenarios.

Personally, I need more team members that get the job done even if they've never done it before.

Hacking wireless networks

Tue, 17 Jan 2006 12:00:00 +0000

Hacking wireless networks. How to get access to secure networks, and how to see what others are doing on their wireless networks.

Comments

Arun 18 Jan 2006 4:56 am: Anand, can you recommend a few good podcasts you listen to?
S Anand 20 Jan 2006 12:01 pm: I don’t listen to podcasts, I’m afraid! Trains are too noisy, don’t like using headphones at office, and prefer watching movies at home. So effectively no place for podcasts :-)
Arun 20 Jan 2006 3:09 pm: Oh..ok..Somehow I thought you had made some post about listening to podcasts on trains sometime ago. Guess it must have been some other blog. Or I must have been dreaming :-)
S Anand 20 Jan 2006 6:32 pm: No, you’re right. That was in July, when I was commuting OVER the ground, where there’s less noise. Now I commute UNDERground, and can’t hear anything, so I’d given up soon after I started.

Caltech vs MIT

Mon, 11 Apr 2005 12:00:00 +0000

Caltech vs MIT pranks.

Harvard hackers

Wed, 09 Mar 2005 12:00:00 +0000

Harvard rejects applicants using hacker services.

Hacking using keyboard whispers

Wed, 12 May 2004 12:00:00 +0000

It’s possible to make sense out of keyboard whispers.

Hacking gets more focused

Tue, 13 May 2003 12:00:00 +0000

Hacking gets more focused.

Hacking arrest

Thu, 25 Apr 2002 12:00:00 +0000

The first case of hacking under the IT Act has been registered. Had to be someone out of jurisdiction, didn’t it!

Social engineering

Thu, 27 Dec 2001 12:00:00 +0000

Hacking and social engineering.

XP copyright protection hacked

Wed, 07 Nov 2001 12:00:00 +0000

In the meantime, Microsoft XP’s copyright protection has been hacked. Their digital rights management scheme: hacked. Passport: hacked.

Underground - book on hacking

Thu, 05 Apr 2001 12:00:00 +0000

Drefus’ book on hacking, Underground, is available online.

Hacking is unavoidable

Mon, 02 Apr 2001 12:00:00 +0000

After seeing how costly it is to detect hacking, and the fact that anything you say can be logged against you, one wonders if being online is worth it. If you’ve got a permanent Internet connection, keep it switched off.

Online payment risks

Fri, 16 Mar 2001 12:00:00 +0000

Better be a bit careful about online payments. A Russian hacker stole from 40 companies. Amazon’s subsidiary was hit. And IBM’s Net.Commerce is buggy.

Websites that have been defaced

Wed, 17 Jan 2001 12:00:00 +0000

Statistics on websites that have been defaced.

Online transactions unsafe

Sat, 23 Dec 2000 12:00:00 +0000

Are credit card transactions on the Net safe? Egghead had their’s stolen.

Astalavista

Mon, 21 Aug 2000 12:00:00 +0000

AstaLaVista is an ‘underground search engine’. I’d been there before, and since its survived this long, it must be good.

Hacked computer network experience

Sat, 15 Jul 2000 12:00:00 +0000

An interesting story about a hacked computer network and how they recovered.