client-side on S Anand

Why don't students hack exams when they can?

Fri, 22 Nov 2024 10:38:09 +0000

This year, I created a series of tests for my course at IITM and to recruit for Gramener.

The tests had 2 interesting features.

One question required them to hack the page

Write the body of the request to an OpenAI chat completion call that:

Uses model gpt-4o-mini

Has a system message: Respond in JSON

Has a user message: Generate 10 random addresses in the US

Uses structured outputs to respond with an object addresses which is an array of objects with required fields: street (string) city (string) apartment (string) .

Sets additionalProperties to false to prevent additional properties.

What is the JSON body we should send to https://api.openai.com/v1/chat/completions for this? (No need to run it or to use an API key. Just write the body of the request below.)

There's no answer box above. Figure out how to enable it. That's part of the test.

The only way to even attempt this question is to inspect the page, find the hidden input and make it visible. (This requires removing a class, an attribute, and a style - from different places.)

Here's the number of people who managed to enable the text box and answer it.

College	# students	Enabled	Answered
NIT Bhopal	144	4 (2.8%)	0 (0.0%)
CBIT	277	16 (5.8%)	0 (0.0%)
IIT Madras	693	74 (10.7%)	4 (0.6%)

A few things surprised me.

First, I think students don't inspect HTML. Less than 10% of students managed to modify the HTML page, even after being told they need to. But they know web programming. 49 students at CBIT scored full marks on the rest of the questions, which includes CSS selectors and complex JS code. Maybe editing in a browser instead of an editor is a big mental leap?

Second, almost no one could solve this problem. There are 3 ways to easily solve it.

Copy the question and relevant test cases from my exam page's JavaScript into ChatGPT and ask for an answer. (I test it and it works.)
Copy the question and structured output documentation to ChatGPT and ask for an answer. (I tested it and it works.)
Create a random JSON and just keep fixing the errors manually until it passes. (The exam gives detailed error messages like "The system message must be 'Respond in JSON'", "addresses items must be an object", etc.)

Maybe questions from a curriculum are easier to solve than questions not in a curriculum? Or is JSON schema too hard?

The exam was officially hackable

All validation was on the client side. The JS code was minified and answers are dynamically generated. But a student can set a breakpoint, see the answers, and modify their responses.

The students at NIT Bhopal and CBIT were not explicitly told that. The students at IITM were explicitly told that they could (and are welcome to) hack it.

Out of the 1,114 students who took these tests, only one student actually hacked it.

(How do I know that? No other student got full marks. This student got full marks with empty answers.)

It's probably not that difficult. My course content covers scraping pages using JavaScript using DevTools. Inspecting JS is just a step away.

I did chat with the student who hacked it, asking:

Anand: How come you didn't share the details of the hack with others?

Student: I did with a few but I am not sure whether or not they were able to figure it out still.
Most students in the program still require a lot of handholding even with basic things.
Experience from being a TA [Teaching Assistant] past term.

Why didn't they hack?

Maybe...

They don't believe me. What if hacking the exam page is considered cheating, even if explicitly allowed?
The time pressure is too much. They'd rather solve what they know than risk wasting time hacking.
It feels wrong. They'd rather answer based on their knowledge than take a shortcut.
They don't know how. Using DevTools is more sophisticated than web programming.

Issue #1 - the trust issue - is solveable. We can issue multiple official notices.

Issue #4 - capability - is not worth solving. My aim is to get students to do stuff they weren't taught.

Issue #2 & #3 - a risk-taking culture - is what I want to encourage. It might teach them to blur ethical lines and neglect fundamentals (which are bad), but it might also build adaptability, creativity, and prepare them for real-world scenarios.

Personally, I need more team members that get the job done even if they've never done it before.

Client side scraping

Wed, 04 Mar 2009 18:00:57 +0000

“Scraping” is extracting content from a website. It’s often used to build something on top of the existing content. For example, I’ve built a site that tracks movies on the IMDb 250 by scraping content.

There are libraries that simplify scraping in most languages:

Perl: WWW::Mechanize
Python: BeautifulSoup
Ruby: HPricot
PHP: XPath (built-in)
Javascript: jQuery on env.js on Rhino

But all of these are on the server side. That is, the program scrapes from your machine. Can you write a web page where the viewer’s machine does the scraping?

Let’s take an example. I want to display Amazon’s bestsellers that cost less than $10. I could write a program that scrapes the site and get that information. But since the list updates hourly, I’ll have to run it every hour.

That may not be so bad. But consider Twitter. I want to display the latest iPhone tweets from http://search.twitter.com/search.atom?q=iPhone, but the results change so fast that your server can’t keep up.

Nor do you want it to. Ideally, your scraper should just be Javascript on your web page. Any time someone visits, their machine does the scraping. The bandwidth is theirs, and you avoid the popularity tax.

This is quite easily done using Yahoo Query Language. YQL converts the web into a database. All web pages are in a table called html, which has 2 fields: url and xpath. You can get IBM’s home page using:

select * from html where url="http://www.ibm.com"

Try it at Yahoo’s developer console. The whole page is loaded into the query.results element. This can be retrieved using JSONP. Assuming you have jQuery, try the following on Firebug. You should see the contents of IBM’s site on your page.

$.getJSON(
  "http://query.yahooapis.com/v1/public/yql?callback=?",
  {
    q: "select * from html where url=\"http://www.ibm.com\"",
    format: "json",
  },
  function(data) {
    console.log(data.query.results);
  },
);

That’s it! Now, it’s pretty easy to scrape, especially with XPath. To get the links on IBM’s page, just change the query to

select * from html where url="http://www.ibm.com" and xpath="//a"

Or to get all external links from IBM’s site:

select * from html where url="http://www.ibm.com" and xpath="//a[not(contains(@href,'ibm.com'))][contains(@href,'http')]""

Now you can display this on your own site, using jQuery.

This leads to interesting possibilities, such as Map-Reduce in the browser. Here’s one example. Each movie on the IMDb (e.g. The Dark Knight) comes with a list of recommendations (like this). I want to build a repository of recommendations based on the IMDb Top 250. So here’s the algorithm. First, I’ll get the IMDb Top 250 using:

select * from html where url="http://www.imdb.com/chart/top" and xpath="//tr//tr//tr//td[3]//a"

Then I’ll get a random movie’s recommendations like this:

select * from html where url="http://www.imdb.com/title/tt0468569/recommendations" and xpath="//td/font//a[contains(@href,'/title/')]"

Then I’ll send off the results to my aggregator.

Check out the full code at http://250.s-anand.net/build-reco.js.

In fact, if you visited my IMDb Top 250 tracker, you already ran this code. You didn’t know it, but you just shared a bit of your bandwidth and computation power with me. (Thank you.)

And, if you think a little further, here another way of monetising content: by borrowing a bit of the user’s computation power to build complex tasks. There already are startups built around this concept.

Comments

Aravinda 4 Mar 2009 7:20 pm: Great!
Sriram 4 Mar 2009 10:51 pm: Thank u anand. Nice Information.
Rishi 5 Mar 2009 8:19 am: Awesome… This is the future
grep imdb part 2 ? « taeyoungchoon 26 Mar 2009 1:16 pm (pingback): […] http://www.s-anand.net/blog/client-side-scraping/ […]
Khair 26 May 2009 4:10 pm: Many thanks for your efforts in simplifying the ‘scraping’.
Venkat 16 Nov 2010 1:40 pm: Hey Anand, Thanks. This post is really useful. I have managed to scrape Amazon wishlist using this technique (http://www.venkatsworld.com/WIP/JSON_amazon.html) However YQL seems to be taking some time to respond. I am not sure if you experienced this.I did a similar exercise with Picasa JSON feed and a Jquery plug-in for image scrolling. It works well there too.
Zheka 17 Jun 2012 8:35 pm: Interesting article! There is another powerful scraping technology available that offers Javascript, jQuery, CSS, and XPath instead of XPath-only. It’s called Bobik (http://usebobik.com). The is a cool example of scraping restaurant menus using Bobik at http://news.ycombinator.com/item?id=4066478.
LAvesh 12 Mar 2013 11:19 pm: Hey Anand, Great Article. Just one query though in the client side implementation its the client IP that will be hitting the website (in your case www.ibm.com)? or it will use yahoo SQL server IP to hit.

Javascript error logging

Tue, 25 Dec 2007 12:00:00 +0000

If something goes wrong with my site, I like to know of it. My top three problems are:

The site is down

A page is missing

Javascript isn’t working

This is the last of 3 articles on these topics.

I am a bad programmer I am not a professional developer. In fact, I’m not a developer at all. I’m a management consultant. (Usually, it’s myself I’m trying to convince.) Since no one pays me for what little code I write, no one shouts at me for getting it wrong. So I have a happy and sloppy coding style. I write what I feel like, and publish it. I don’t test it. Worse, sometimes, I don’t even run it once. I’ve sent little scripts off to people which wouldn’t even compile. I make changes to this site at midnight, upload it, and go off to sleep without checking if the change has crashed the site or not. But no one tells me so At work, that’s usually OK. On the few occasions where I’ve written Perl scripts or VB Macros that don’t work, people call me back within a few hours, very worried that THEY’d done something wrong. (Sometimes, I don’t contradict them.) It can be quite a stressful experience but good thing you can learn more here on how to cope up with it. On my site, I don’t always get that kind of feedback. People just click the back button and go elsewhere. Recently, I’ve been doing more Javascript work on my site than writing stuff. Usually, the code works for me. (I write it for myself in the first place.) But I end up optimising for Firefox rather than IE, and for the plugins I have, etc. When I try the same app a few months later on my media PC, it doesn’t work, and shockingly enough, no one’s bothered telling me about it all these months. They’d just click, nothing happens, they’d vanish. But their browsers can tell me The good part about writing code in Javascript is that I can catch exceptions. Any Javascript error can be trapped. So since the end of last year, I’ve started wrapping almost every Javascript function I write in a try {} catch() {} block. In the catch block, I send a log message reporting the error. The code looks something like this:

function log(e, msg) {
  for (var i in e) msg += i + "=" + e[i] + "\n";
  (new Image()).src = "log.pl?m=" + encodeURIComponent(msg);
}

function abc() {
  try {
    // ... function code
  } catch (e) {
    log(e, "abc");
  }
}

Any time there’s an error in function abc, the log function is called. It sends the function name ("abc") and the error details (the contents of the error event) to log.pl, which stores the error, along with details like the URL, browser, time and IP address. This way, I know exactly where what error occurs. This is a fantastic for a three reasons.

It tells me when I’ve goofed up. This is instantaneous feedback. I don’t have to wait for a human. If you run my program on your machine, and it fails, I get to know immediately. (Well, as soon as I read the error log, at least.)
It tells me where I’ve goofed up. The URL and the function name clearly indicate the point of failure.
It tells me why I’ve goofed up. Almost. Using the browser name and the error message, I can invariably pinpoint the reason for the error. Then it’s just a matter of taking the time to fix it.

I’d think this sort of error reporting should be the norm for any software. At least for a web app, given how easy it is to implement.

Comments

CodeKitten 25 Dec 2007 12:00 pm: You should bind an onerror-triggered method to call your log method aswell :)
Danko Stojanovic 10 May 2010 8:12 am: I feel your pain. I also moved away from programming to management. I have to live with customers saying “It doesn’t work” and programmers “It works just fine when I test it”. I so wish to know what is going on my client’s computer. Thanks for this article. It helped me decide to pursue this further
Allan Ebdrup 13 Nov 2011 7:23 am:The onerror event is a great way to start logging JavaScript errors, more people should use it. JavaScript errors can be just as bad as a webserver that is down, you really need to log those errors so you can do something about them.onerror, is quirky. It does not fire on iPad, iPhone, Safari and some other browseres. To catch those errors you have to insert try-catch’es arount your event handlers, xhr-requests and some other places. This will also let you get more detail about the errror (call stack in some browsers).But onerror is still much better than nothing.
We’ve solved a lot of these issues, with a solution that you install just like Google Analytics. Currently we are in a testing phase, private beta (free). To get an invite you can got to http://muscula.com and sign up.
Kiawaki 8 Jul 2011 5:45 am: A question from somebody who is even less of a pro web developer: how exactly do you use this code? Do you put it before or after the script or both? Thanks!
Offbeatmammal 23 Sep 2011 12:48 am: if you want a no-brain, minimal effort then have a look at jsErrLog … a simple utility that does this and logs it to a remote AppEngine service (all for free, in an open source project) - check out http://jsErrLog.appspot.com which links to the GitHub repository as well

Mozilla

Tue, 03 Sep 2002 12:00:00 +0000

I’m trying Mozilla. It has native SOAP support. So I can make a web page that has dynamic Google searches (and Amazon booklists, etc) without a web-server.