synthesis on S Anand

How to use AI for research

Sat, 04 Apr 2026 23:30:53 +0800

I asked ChatGPT to research universities’ AI policies. Here is the report

Here are the four lessons I learned from that - about how to use AI for research.

1. Show examples of failures to avoid. Jivraj’s earlier research kept surfacing AI policies universities had researched, not written for themselves!. So I told ChatGPT to:

… double-check that they ARE, in fact, about their own use of AI - not policies they’re proposing for others or are researching.

This is called pre-specifying exclusions. Giving negative examples help. Wei (2022).

2a. “Double-check” doesn’t always work. Though I told ChatGPT to “double-check”, it still got things wrong. For example, it cited MIT’s AI policy home page as evidence that the policy covers students and faculty, just because the words were present. That’s not right!

Models get over-confident - and that’s exactly when they don’t double-check. Asking them to double-check is a good habit, but not fail-safe. Kadavath (2022)

2b. Expicitly tell it to find mistakes. I told it to:

Find mistakes in as many claims as you can.

This is stronger than “double-check”. It turns the model against itself, and it worked quite well.

1. Show examples of failures to avoid. (Repeat.) When asking it to find mistakes, I gave it the same example.

… MIT, “covers_faculty_or_staff” cites “quote”: “Students • Faculty and Staff • Visitors and Guests • Generative AI use at MIT”. But that’s actually a set of links to Students, Faculty and Staff, etc. It’s not evidence that the POLICY covers them - and I’m quite sure the policy isn’t for guests!

That’s few-shot prompting. Concrete examples beat abstract instructions.

3. Ask it to list failures explicitly. I told it:

I am also interested in universities that conspicuously lack a policy …

Without that, it might have returned only positive hits. Missing evidence and failures are important data, too!

4. Break large tasks into batches. When I asked it to research 20 universities, it made several mistakes. Instead:

This may be a complex task, so let’s just do this for the first four Universities.

Now, it didn’t make any mistakes! Sometimes, it gets lost in the middle for long tasks.

So there it is - the four rules of AI research I learned from this exercise:

Show examples of failures to avoid
“Double-check” doesn’t always work. Expicitly tell it to find mistakes
Ask it to list failures explicitly.
Break large tasks into batches.

Extracting AI advice

Mon, 09 Feb 2026 09:52:19 +0800

This weekend, two people asked me, roughly “How do I use AI better?”

This is a frequently asked questions. I document my FAQs, e.g. time management, career advice, etc. and it was time to add AI advice to this list.

I often record online calls and transcribe them. I asked Gemini, Claude and ChatGPT for the best way to summarize 400 transcripts of ~40K each.

Claude’s suggestion was the best:

Use Gemini Flash (1M context, dirt cheap) to process calls in batches of 20-25
Each batch → extract advice themes
Aggregate batch results with Claude Sonnet for final synthesis

But I ignored it because it was too much work. (See my AI advice: “Ask for easier output”)

Instead, I listed all my transcripts and used Simon Willison’s llm CLI tool to run:

llm -m gemini-3-flash-preview --system "Summarize ALL AI-related advice from Anand this call transcript into 1-sentence bullets" -f file-1.md > "extract/ai/file-1.md"
llm -m gemini-3-flash-preview --system "Summarize ALL AI-related advice from Anand this call transcript into 1-sentence bullets" -f file-2.md > "extract/ai/file-2.md"
...

(A for-loop is smarter. Copy-paste is easier. I optimize for ease.)

This took a little over an hour and a dollar. Then, I combined all the extract/ai/*.md files into one big file:

uvx files-to-prompt *.md | xclip -selection clipboard

… and pasted it into Gemini 3 Pro, which could comfortably handle the 750KB of context and prompted it:

Here is the summary of AI related advice I provided in discussions to various people.

What is the most common advice I provide to people that is relevant to people looking to use AI for personal use and productivity?

The result was a good list along the lines of:

### 1. The "50 Conversations a Day" Rule (The Core Habit)

You frequently tell individuals that they cannot learn AI through theory. To build intuition, they must force themselves to use it for everything.

- **The Advice:** Aim for a volume target of **50 interactions per day**.
- **Why:** High-frequency usage forces you to find "micro-use cases" you would otherwise ignore. It moves you past the "novelty" phase into the "utility" phase where you instinctively know which tasks the AI can handle.
- **Application:** Don't save AI for big projects. Use it to fix grammar in a text, explain a joke, plan a menu, or debug a single line of code.

… which I condensed into:

- **Have 50 conversations a day with AI**. High-usage forces you to find tiny use-cases you'd otherwise ignore.

Here’s my current AI Advice.

Another advantage of creating an extract/ai/ folder is that I can pull out technical AI advice, governance-related AI advice, etc. later.

In fact, this “map-reduce” style pattern is clearly powerful. For $2 and a little time, I get very useful synthesis. I plan to use it more.

The Surprising Power of LLMs: Jack-of-All-Trades

Wed, 27 Aug 2025 01:59:08 +0000

I asked ChatGPT to analyze our daily innovation-call transcripts.

I used command-line tools to fetch the transcripts and convert them into text:

# Copy the transcripts
rclone copy "gdrive:" . --drive-shared-with-me --include "Innovation*Transcript*.docx"
# Convert Word documents to Markdown
for f in *.docx; do
  pandoc "$f" -f docx -t gfm+tex_math_dollars --wrap=none -o "${f%.docx}.md"
done
# Compress into a single file
tar -cvzf transcripts.tgz *.md

… and uploaded it to ChatGPT with this prompt:

Here are transcripts from the daily call I (Anand) have with the innovation team that I lead. It is a .tgz of text transcripts as .md files.

Your task is to

Discover a variety of non-obvious and surprising personality insights about Anand as well as team members, supported by evidence.

Recommend ways of improving the way in which these meetings are conducted.

The transcription quality is average. Words are incorrectly transcribed. Factor that in.
Generate multiple hypotheses based on management, science, and ancient literature.
List and leverage multiple mental models while hypothesizing.
Test the hypotheses against the data and explicitly document which are valid and which are not.
If a hypothesis cannot be tested or the method of testing is not robust, note it and move on.
Share based on the strongest evidence and most robust evidence you find.

Here are two highlights from the analysis:

Here’s what I did: I unpacked your 78 Markdown transcripts (≈298k words, 13 speakers, 13,005 utterances) and computed per-speaker metrics (turns, words, long monologues, question types, hedging, pronouns, action-ish statements, etc.).

Mental models used below (and cited where applicable): Collective intelligence (equal turn-taking), Motivational Interviewing/OARS (open questions), Implementation Intentions (“who-what-when/if-then” planning), Psychological Safety, Brevity (Thirukkural), Parkinson’s Law, Goodhart’s Law, OODA loop.

Learnings:

I spoke 39% of the words. My share is falling, but participation breadth is narrowing. Groups are smarter when talk is evenly distributed.
I ask 6 questions per 1,000 words; 39% are open-ended. Teammates ask a higher share of open questions. Open questions are rising and support better joint problem-solving.
The team hedges often (“I think,” “probably”). I hedge less—19 per 1,000 words. Hedging helps ideation, but slows decisions.
Action statements are frequent (8 per 1,000 words), but only 1% include a date. High momentum; weak closure.
Positive reinforcement shows up in 4% of my turns (“thanks,” “great,” “well done”).
Meetings are getting wordier (+50 words a week). More depth, but more fatigue and decision drag.

The bigger lesson is about LLMs. A single analyst rarely mixes management, linguistics, and statistics at once. LLMs aren’t deep specialists, but they’re unusually capable generalists.

They handle ambiguity well. I should use them when I don’t yet know what to ask.
They’re strong at synthesis. I should ask them to combine fields we rarely connect.