peer-review 6

Stanford ran a conference called Agents for Science. It’s a conference for AI-authored papers, peer reviewed by AI. They ran three different AI systems on every paper submitted, alongside some human reviewers. The details of each of the 315 papers and review are available on OpenReview. I asked Codex to scrape the data, ChatGPT to analyze it, and Claude to render it as slides. The results are interesting! I think they’re also a reasonably good summary of the current state of using AI for peer review. ...

We asked the ~500 students in my Tools in Data Science course in Jan 2024 to create data visualizations. They then evaluated each others’ work. Each person’s work was evaluated by 3 peers. The evaluation was on 3 criteria: Insight, Visual Clarity, and Accuracy (with clear details on how to evaluate.) I was curious to see if what we can learn about student personas from their evaluations. ...

MIT pulls a prank on the World Multi-Conference on Systemics by submitting a computer-generated paper titled “Rooter: A Methodology for the Typical Unification of Access Points and Redundancy”. I was among the people spammed by Nagib Callaos, the organizer of the conference. The students were soliciting cash donations so they could attend the conference and give what Stribling billed as a “completely randomly-generated talk, delivered entirely with a straight face.” They exceeded their goal, with $2,311.09 cents from 165 donors. ...

MERLOT is like a Wikipedia for online learning materials are collected here (along with annotations such as peer reviews and assignments).

Google Scholar lets you search academic references (journals, papers, etc).

Google likes directory sites. Google likes Yahoo!. Google uses peer review. Google is good.