ffmpeg on S Anand

Editing Workshop Videos

Sun, 14 Jun 2026 08:27:16 +0530

I sometimes use Google Meet, Teams, Zoom, etc. to record workshops and talks. These record the entire session, including before and after the actual talk, and save it as large MP4 files.

I use ffmpeg to trim the video to just the talk, and then compress it for sharing. I’m sharing the options that work for me, discovered by trial-and-error.

To trim it, I use the following command:

ffmpeg -ss 00:10:00 -to 02:10:00 \
  -i "original.mp4" \
  -map 0 \
  -c copy \
  -avoid_negative_ts make_zero \
  -movflags +faststart \
  new.mp4

Arguments:

-ss 00:10:00: Start reading from 10:00 into the source file. Placing -ss before -i makes this fast because ffmpeg seeks instead of decoding from the start.
-to 02:10:00: Stop at 02:10:00 in the source timeline. So this keeps roughly the section from 00:10:00 to 02:10:00.
-map 0: Keep all streams from the input: video, audio, subtitles, metadata streams, etc. Use -map 0:v:0 -map 0:a:0 for just the first video and audio streams.
-c copy: Do not re-encode. Just copy the existing encoded streams. This is very fast and preserves original quality, but cuts only cleanly near keyframes.
-avoid_negative_ts make_zero: Rewrite timestamps so the output starts cleanly near zero. This helps avoid weird negative timestamps and duration/reporting issues in some players.
-movflags +faststart: Move MP4 metadata to the beginning of the file. This helps playback start faster in browsers and streaming contexts.)

To compress it I use:

ffmpeg -i 'trimmed.mp4' \
  -map 0:v:0 -map 0:a:0 \
  -c:v libvpx-vp9 \
  -b:v 0 \
  -crf 42 \
  -deadline realtime \
  -cpu-used 8 \
  -row-mt 1 \
  -tile-columns 2 \
  -frame-parallel 1 \
  -threads 8 \
  -pix_fmt yuv420p \
  -vf "fps=16" \
  -c:a libopus \
  -b:a 32k \
  -ac 1 \
  -ar 48000 \
  -af "highpass=f=80,lowpass=f=12000" \
  "compressed.webm"

Arguments:

-map 0:v:0 -map 0:a:0: Use the first video stream and first audio stream from the input
-c:v libvpx-vp9: Compress video with VP9 codec
-b:v 0: Use constant-quality mode instead of targeting a fixed bitrate
-crf 42: When testing on samples, CRF 42 was small enough and clear enough for me
-deadline realtime: -deadline good is too slow for me though that’s better for production
-cpu-used 8: 8 is the maximum allowed for VP9 for now
-row-mt 1: Enable row-based multithreading for faster VP9 encoding
-tile-columns 2: Split each frame into tiles so multiple threads can encode in parallel
-frame-parallel 1: Allow frames to be decoded in parallel; useful for VP9/WebM playback
-threads 8: Use up to 8 CPU threads during encoding
-pix_fmt yuv420p: Use the most compatible pixel format for browsers and video players
-vf "fps=16": Reduce frame rate to 16 fps to shrink file size; good enough for slides/talk videos
-c:a libopus: Compress audio with Opus, the standard audio codec for WebM
-b:a 32k: Use low audio bitrate; enough for speech
-ac 1: Convert audio to mono; good for voice and halves stereo audio data
-ar 48000: Use 48 kHz, the standard sample rate for Opus
-af "highpass=f=80,lowpass=f=12000": Remove very low rumble and very high noise; keeps speech frequencies

I deploy these on CloudFlare R2 on a custom domain: media.s-anand.net. R2 costs 1.5c / GB-month. Storing 10 years’ worth of ~500 talks of ~0.5 GB each is under $4 / month - quite affordable. (If it increases, I can shift. Statis files are easy to move.)

I serve them via a <video> tag. Here is an example video:

<video preload="metadata" width="1920" height="1080" controls style="max-width: 100%; height: auto;">
  <source src="https://media.s-anand.net/2026-05-23-ai-unboxed-context-engineering.webm" type="video/webm; codecs=&quot;vp9, opus&quot;">
</video>

Recording screencasts

Wed, 11 Mar 2026 10:48:01 +0800

Since WEBM compresses videos very efficiently, I’ve started using videos more often. For example, in Prototyping the prototypes and in Using game-playing agents to teach.

I use a fish script to compress screencasts like this:

# Increase quality with lower crf= (55 is default, 45 is better/larger)
# and higher fps= (5 is default, 10 is better/larger).
screencastcompress --crf 45 --fps 10 a.webm b.webm ...

To record the screencasts, I prefer slightly automated approaches for ease and quality.

METHOD 1: Browser scrolling: To uniformly scroll to the bottom of the page at ~800 pixels / second (roughly one page per second), I frame the recording window, start recording, and quickly paste this script in the DevTools console:

scrollInterval = setInterval(() => {
  window.scrollBy(0, 16); // Scroll 16px every 20ms (approx 800px/s)
  if (window.innerHeight + window.scrollY >= document.body.scrollHeight) {
    clearInterval(scrollInterval);
    console.log("Reached bottom or no more scrolling.");
  }
}, 20);

This is what I used in Prototyping the prototypes.

METHOD 2: AI Coding Agents: I tell the coding agent to:

… screenshot each step at 1366x768, compressing into video.webm at 1.5 fps, pausing the last frame for 5 seconds.

If playwright and ffmpeg are set up, this just works.

This is what I used in Using game-playing agents to teach.

AI video compression

Sat, 28 Feb 2026 09:18:56 +0800

I recorded a short screen cast of a demo I built. It was ~900KB - way too large to publish as a thumbnail. So I asked ChatGPT:

What’s the best equivalent of squoosh.app for WEBM compression? I’m looking for a free modern high-quality online video compressor.

There are a few, and they compressed it to a third of its size, but 300KB is still too large. So I attached the original and asked:

I am compressing screenshots like this. They’re often not large. I don’t mind cropping the edges by a few pixels to make it a multiple 2, 4, or 8 if that’ll help. I certainly am OK with a lower frame rate. I’d like an image quality that a human eye can just SLIGHTLY detect as worse than the original, but only VERY SLIGHTLY.

What’s the best way of using ffmpeg or similar tools to compress such a file?

Think like an expert:

What would an expert in this field check that beginners would miss?

What patterns would an expert in this field recognize that beginners would miss?

In this context, what questions would an expert ask that a beginner would not know to?

If this goes wrong, what are the most likely reasons?

… and list the best (diverse, testing different hypotheses) compression command options.

Run them on this video and let me download the resulting videos for visual comparison. Interview me. Give me a list of questions that I can easily answer by looking at the videos and I’ll share those with you to help you decide the best compression command for me.

This leverages 3 tricks.

These are online coding agents. So they can write and run code.
The “Think like an expert” prompt is my new “Think step by step” prompt and works quite well.
The “Interview me” prompt is another powerful one that helps me apply preferences – and develop taste.

Claude did a good job of showing different versions and compressing it, but as expected, ChatGPT was the obsessive perfectionist. It gave me a huge set of variations of:

Files sizes
Compression formats (VP9: more compatible vs AV1: better compression)
Quality settings (CRF)
Frame rates (FPS)
Color formats (YUV420p vs YUV444p)
Compression effort (presets)
etc.

… and asked me to compare between them, like these:

crf: 45 fps: 15 (93K)

crf: 50 fps: 15 (69K)

crf: 55 fps: 5 (23K)

The final result is this script:

ffmpeg -hide_banner -stats -v warning -i "$input"
  -vf "crop=iw-mod(iw\,2):ih-mod(ih\,2),fps=$fps" \
  -c:v libsvtav1 -preset 8 -crf $crf -pix_fmt yuv420p \
  -an "$output"

… which does the following:

Crops the video to a multiple of 2 (which is required for some compression formats)
Sets the frame rate to a lower value (which reduces file size)
Uses the AV1 codec (which has better compression than VP9)
Sets the CRF (Constant Rate Factor) to a value that balances quality and file size
Sets the pixel format to YUV420p (which is more compatible with players)
Disables audio (which is not needed for a screencast)

I realized that for the resolution I’ll likely see this at, very low frame rates (5 fps) and poor compressions (CRF 55) are good enough.

My original video was 912KB. The smallest video that looks good enough for me is 23 KB. That’s almost a 40x compression - small enough to publish in my data story.

Things I’m taking away:

Use AI to discover the best configurations for your tool
Interview yourself to apply preferences and develop taste

Things I Learned - 15 Feb 2026

Sun, 15 Feb 2026 00:00:00 +0000

This week, I learned:

ffmpeg lets you concatenate files without needing a separate input file. ffmpeg -i "concat:input1.ext|input2.ext|input3.ext" -c copy output.ext works as long as the files use the same codecs and parameters.
There is a psychological phenomenon where we “overlay” old images of people we haven’t seen in decades onto their current selves, making it hard to distinguish between someone who is 30 and someone who is 70. Gemini
Most modern ls tools like eza --icons or lsd support icons if the terminal font supports icons, like Nerd Fonts. For example, this:  shows up as a GitHub icon and 󰌻 as a LinkedIn icon. The Nerd Fonts Cheat Sheet is a good place to search for these. You may need to download a supporting font.
I just replaced Fira Code with Maple Mono as my default font on VS Code. Like Fira Code, the ligatures are great, but there are extra ligatures like [TODO] or [ERROR], connected italics, nerd font support, variable font weights, and more. Via lobste.rs. (Update: Maple Mono is much harder to read than Fira Code, so I switched back. But it’s a nice idea.)

Self-discover LLM capabilities

Sat, 10 Jan 2026 11:40:00 +0800

Q: “How do we learn what we can do with AI agents?”

Me: “Ask them!”

I mean, they are probably aware of their abilities. They can search online for how other people are using them. They have access to tools (connect to GMail, write & run code, etc.) which they’re aware of, and even if not, can try out.

Asking them seems a useful way of figuring out how to use them.

For example, I didn’t know that ffmpeg (which ChatGPT, Gemini, Claude, etc. can run) can visualize audio using filters. They could create a bunch of stunning visualizations as a video compilation.

So, I told Claude:

I did not know ffmpeg could visualize audio via filters…
You have a container environment with a set of tools installed and you can run commands.
Identify creative ways in which the tools you have access to can be used…
…
Fact-check by cursorily verifying the command options…
But no need to implement any of these…
BLOW MY MIND!!

It gave me 125 ideas from drum patterns of log timestamps, directory structures as artistic graphs, frequency domains of images via Fourier transforms, morphological image erosion/dilation effects, and a whole bunch of things I’ve never heard of.

It was too much, so I didn’t bother. (I’ll read later.)

Implement the most visually impressive among these.

And the result was a stunning video compilation:

It generated these 10 purely algorithmic (no external assets) visualizations:

Mandelbrot Set via FFmpeg’s mandelbrot filter: Deep zoom into the famous seahorse valley, revealing infinite complexity from z² + c
Sierpinski Carpet via FFmpeg’s sierpinski filter: Recursive self-similar fractal pattern that animates through chaos game iterations
Game of Life via FFmpeg’s life filter: Conway’s cellular automaton with glowing cells and mold trails showing emergent complexity from 4 simple rules
Rule 30 via FFmpeg’s cellauto filter: Wolfram’s elementary cellular automaton that generates apparent randomness from deterministic rules
Domain Coloring via Python + NumPy: Complex function visualization where hue represents angle and brightness represents magnitude, morphing through z², z³, and rational functions
L-Systems via Python + PIL: Three fractal trees grow algorithmically using Lindenmayer system grammar rules - pure mathematical botany
Barnsley Fern via Python chaos game: 500,000 points plotted using an Iterated Function System, emerging from randomness into a perfect fern
Julia Set via Python + NumPy: Dancing fractals as the complex parameter c traces a wobbling circle, continuously morphing fractal boundaries
Plasma Effect via FFmpeg’s geq expression filter: Real-time interference patterns using layered sine waves in RGB channels
Gradient Spiral via FFmpeg’s gradients filter: Six-color rotating spiral with 8x speed, creating hypnotic color field animation

… with cinematic title cards (fade transitions, credits) in a 1080×1080 square format perfect for social media.

I learnt at least a few things from this:

The tool side. I now know that ffmpeg has built-in fractal capability. Fractals have fascinated me since I was 12. This is something to explore.
The technique side. I’m learning new terms like “temporal slit-scan photography” - used to create time-slice effects like bullet time in The Matrix, using ffmpeg. Or “music chord visualization” using neato, or capturing packet flow data using tcpdump to visualize network traffic, etc.

I would never have thought of these, but the capabilities are in my hands.

I think there’s benefit in just spending time with LLMs, asking them (in different ways) what they can do, and what would help, interest, or even amuse us.

PS: ChatGPT’s response to this was a bunch of good ideas and a tiny 0.5 second Mandelbrot video. Gemini shared a tiny list of 10 ideas (read them all) but made up with this brilliant Veo-generated video.

Tue, 23 Dec 2025 05:36:05 +0000

AI agents are messing up software tool learning.

Normally, we need to pass stages of competence:

KNOW what you can do
LEARN how to do it
EXECUTE it.

Excel: KNOW you can summarize by category, LEARN pivot tables, EXECUTE an Insert → PivotTable → select data range → drag …

Photoshop: KNOW you can erase objects, LEARN Content-Aware Fill, EXECUTE Lasso tool → select → Edit → Content-Aware Fill → …

AI agents collapse these stages. Especially with command line tools.

For example, I did not KNOW ffmpeg could visualize audio, nor LEARN the complex filters, nor EXECUTE it on my laptop.

Instead, I told Claude to Visualize this audio clip creatively in multiple ways. It generated a dozen stunning visualizations of this Why This Kolaveri clip: https://lnkd.in/g7vfcPEs

I didn’t KNOW ⮕ LEARN ⮕ EXECUTE. I did an EXECUTE ⮕ KNOW ⮕ don’t bother to LEARN.

Since that’s possible, my prompts are now like: “I don’t know what I want. Guess my desires. Fulfill them. Show me. Then maybe I’ll know what I want.”

Which is fantastic! But on the other hand, this TOTALLY messes up how I teach software tools…

Things I Learned - 21 Dec 2025

Sun, 21 Dec 2025 00:00:00 +0000

This week, I learned:

uvx --python 3.10 --with torchcodec demucs --two-stems=vocals -n htdemucs "song.mp3" separates vocals from music.
iTunes offers a 30 second preview for almost any song. If you’re looking for 30s song clips to analyze, this is a good bet. For example: curl -s "https://itunes.apple.com/search?entity=song&limit=1&term=why+this+kolaveri" | jq -r '.results[0].previewUrl'
To generate a spectrogram from an audio file, use ffmpeg -i song.mp3 -lavfi showspectrum=color=magma:slide=1 spectrogram.mp4. To generate a waveform, use ffmpeg -i song.mp3 -filter_complex "[0:a]showwaves=s=1280x240:mode=cline:colors=white[v]" -map "[v]" -map 0:a -c:v libx264 -crf 30 -pix_fmt yuv420p waveform.mp4.
I updated the TTS (text-to-speech) costs across Gemini and OpenAI at https://github.com/sanand0/openai-tts-cost. My current favorite (value for money) is Gemini 2.5 Flash Preview TTS. Good emotions, low price, and a single request can deliver a multi-voice podcast. Speed: ~25 seconds per minute of audio generated.
Self-driving car mishaps. The exceptions that prove the rule (that autonomous vehicles are safer than human drivers). #
- Waymo & The Gun Shootout: A driverless Waymo taxi in Los Angeles drove straight through an active police standoff, passing mere feet from a suspect being held at gunpoint while officers shouted at the car to stop. Source
- Tesla & The Horse Carriage: It was a horse-drawn carriage in Switzerland. The Tesla’s computer became “bamboozled,” rapidly misidentifying the cart as a truck, then a car, then a pedestrian, because it had likely never been trained on animal-drawn vehicles. Source
- The “Wet Cement” Trap: A Cruise robotaxi in San Francisco drove directly into a patch of freshly poured wet concrete at a construction site and got hopelessly stuck, requiring workers to pull it out. Source
- The Moon is a Traffic Light: A Tesla driver discovered that his car kept slamming on the brakes on the highway because the autopilot camera was confusing the bright yellow moon for a yellow traffic light. Source
- The 4 AM Honking Ritual: Residents in a San Francisco neighborhood were kept awake for weeks because a fleet of Waymo taxis gathered in a parking lot every night and started honking at each other while trying to park. Source
- Stopping for Whoppers: Tesla owners reported their cars were reading “Burger King” signs on the side of the road as “Stop” signs and abruptly braking, a glitch the fast-food chain quickly turned into a marketing campaign. Source
- The Robotaxi “Mating Ritual”: A group of about 20 Cruise robotaxis lost connection to their servers simultaneously and simply stopped in the middle of a busy San Francisco street, creating a massive traffic jam that humans had to manually clear. Source
- Trapped by Cones: A Waymo taxi in Arizona was defeated by a set of construction cones, fleeing from them into oncoming traffic lanes and eventually getting stuck, forcing the passenger to flee the “confused” vehicle. Source
- Defeated by a T-Shirt: A distinct vulnerability was found where self-driving cars could be tricked into slamming on the brakes simply by a pedestrian wearing a T-shirt with a “Stop” sign printed on it. Source
Roblox is the #1 game. Sadly, there’s no official Linux support. CloudFlare 2025 Report
⭐ Ty, Astral’s type checker, is fantastic! It shows the type of every variable inline. A great incentive to explicitly type stuff in Python. Lots more to explore. I switched from Pylance to the ty VS Code extension.
npx -y npm-check-updates tells you the latest versions of your package.json dependencies, including major version updates.
How to think differently. # #
- Introspect: List assumptions & taboos. Write a falsifier. Beginner’s mindset
- Mental models: First principles, inversion, base rates, lateral thinking, multiple options, “what would have to be true”, …
- Empathy: Debate FOR opposition. Swap roles (competitor, auditor, 12-year old, future-you, …)
- Environment: Different context (place, media, people…). New constraints (time, budget, time horizon, …)
I’m surprised that Edge’s Read Aloud sounds more natural than EleventReader. Read Aloud is one of the main reasons I’m using Edge, but I hadn’t realized it was that good.
Why We Think has interesting insights on scaling from feedback: #
- Summary: Give models a feedback environment unbiased by their reasoning.
- There are basically two approaches: parallel and sequential.
- Parallel is simpler. Generate a bunch of different solutions and pick the best one. Like having multiple people solve the same problem independently, then going with whoever got the right answer.
- Sequential is trickier. You generate a solution, then ask the model to critique it and try again. This sounds good in theory but is surprisingly hard to get right.
- The problem is models aren’t naturally good at self-correction. Left to their own devices, they’ll often make things worse. They’ll change correct answers to incorrect ones. Or they’ll just superficially reword their first answer without fixing anything.
- To make self-correction work, you need external feedback. A unit test that fails. A ground truth to compare against. Something outside the model’s own judgment.
- When you get it right though, sequential revision can be powerful. You’re not just sampling from the model’s distribution anymore. You’re searching through it, iterating toward better answers.
- But there’s a trap. If you start optimizing directly on the reasoning traces—rewarding “good reasoning” as a goal in itself—the model learns to game it. It’ll hide its real thought process and show you what you want to see.
- This is why the DeepSeek team gave up on process reward models. They tried rewarding intermediate reasoning steps, but it led to reward hacking. The model would generate reasoning that looked good to the reward model while doing something completely different.
A Pragmatic View of AI Personhood was rewritten in Tim Urban’s style, para-by-para, by ChatGPT:
- AI having feelings is irrelevant. Does a design increase conflict, manipulation, or suffering among humans? If so, regulate that - limit certain kinds of anthropomorphic design, tie “rights” for AIs to strict anti-manipulation constraints, etc.
- AI can act after owners vanish. Pragmatically, you sometimes need to bite the bullet and say: “Okay, this thing itself is going to be treated as a legal person in these specific ways, so we can actually regulate and sanction it.”
- Corporations are “slow AIs” already — optimizing for growth without ethics.
- Slaves had a fund. If the slave caused harm, the owner’s liability could be capped at that fund. Modern equivalent for AI: Agents must maintain locked capital or insurance. Victims are compensated from that pool. If the pool runs out; they lose their license to operate. This gives sanctions teeth: the AI (or its backers) actually have something to lose.
- Require AIs to register before they can do economically important things. No title > no access to key platforms, payment rails, or official functions.
- Expanding personhood to non-humans sounds nice - more compassion, more care, more inclusion. But authenticity becomes a new asset. Humans and AIs will both want authenticity tokens. Poor will sell biometric credentials to rich, creating an authenticity social class. Your dignity as a person gets replaced by your usefulness as a key. Make it illegal and practically very hard to sell / rent out your humanity.
“When people now talk about error, they tend to think of bias as an explanation. One of the major limitations on human performance is not bias, it is just noise. In fact, most of the errors that people make are better viewed as random noise, and there is an awful lot of it. Even when the algorithm does not do very well, humans do so poorly and are so noisy that, just by removing the noise, you can do better than people. We are narrow thinkers, we are noisy thinkers, and it is very easy to improve upon us. I do not think that there is very much that we can do that computer will not eventually be programmed to do.” Kahnemann
Notes from One Year With ChatGPT Pro as a First Hire
- Each day I start a new Pro chat that will run for that entire day. I treat it as a colleague. I speak or type in whatever I am thinking about, including business problems, creative questions, experiments that worked or failed and feelings about particular decisions. I wear noise canceling earbuds and often run piano technique while the model is thinking. I listen to its response using the native “Read Aloud” feature, again while practicing, and stop to make notes in a physical notebook to collect inspiration. At the end of the day I ask that Pro model to summarize everything from that chat along with the notes I give it from my notebook, and that summary becomes our first prompt of the next day.
- Standard Voice Mode (SVM) can do things that Advanced Voice Mode (AVM) cannot and vice versa.SVM feels like it wants to talk forever, while AVM feels like it wants to get off the phone.
- Projects became the container for my daily Pro chats. I pull chats, notes and other files into project folders so I can reference them as static context.
- My scheduled tasks collection today consists of weekly lessons in math, ML and DL, design, market analysis and regular assessments of the UI and UX and copy on my company’s website.
- I let memory accumulate, then once a week I pruned it manually, removing entries that were no longer useful so that new memories could form.
- Connecting the ChatGPT macOS app to my terminal, using the Working with Apps feature, lets the Pro models essentially collaborate with Codex. Practicing collaborative context between these high end models fractals outward into a myriad of productive paths. I highly recommend exploring with 5.1 Pro connected to 5.1-Codex-Max (Very High) in a terminal. Tell Codex-5.1 that you have a buddy working with you today that can offer suggestions and review the work it does as we go. Then tell 5.1 Pro that you have a buddy that is working with you today and can apply any of the code changes we decide on. This is another form of “context priming” where I “set the scene” before jumping in.
Coding agents only need a bash tool. The rest is buildable. The only addition might be a fuzzy search / replace tool. What I learned building an opinionated and minimal coding agent
Sources of model data: https://models.dev/, https://openrouter.ai/, llm-pricing

Things I Learned - 03 Aug 2025

Sun, 03 Aug 2025 00:00:00 +0000

This week, I learned:

From A.I. Is About to Solve Loneliness. That’s a Problem: “Blindly stifling every flicker of boredom with enjoyable but empty distractions precludes deeper engagement with the messages boredom sends us about meaning, values, and goals.” Maybe the best thing about boredom is what it forces us to do next.
Here’s when be candid vs polite. #beliefs ChatGPT
- If there’s high trust (i.e. the other person trusts you):
  - Important topic/decision: Be candid
  - Unimportant: Follow culture (e.g. in Japan, you’d be polite; in The Netherlands, you’d be candid)
- Low trust:
  - Important: Earn trust first
  - Unimportant: Be polite
I didn’t realize that it was Luis Alvarez (whom I know from his work on the bubble chamber) is the same person who figured out that an asteroid killed dinosaurs. He also used muon tomography to search pyramids for hidden chambers and figured out Kennedy was shot from behind. Added his biography, Collisions to my to-read list. Ref
Benjamin Green suggests that OpenAI Study mode is sycophantic. E.g. in this conversation, ChatGPT carefully balances truth and politeness. A reader might misinterpret that as agreement. But sometimes, we need candor. Politeness trades clarity for harmony. People who trust AI should tell it to be more candid.
⭐ Here’s my current response when asked, “How should I use LLMs better”:
- Use the best models, consciously. O3 (via $20 ChatGPT), Gemini 2.5 Pro (free on Gemini app), or Claude 4 Opus (via $20 Claude). The older models are the default and far worse.
- Speak & listen, don’t just type & read. I had to resist the temptation to ignore ChatGPT response when a colleague read it out. We are patient with and have respect for humans but not for AI. The value we derive requires both. Suggestion: Speak and listen rather than type and read. It’s hard to skip and easier to stay in the present. It’s also easier to ramble than type.
- Keep an impossibility list. There is a jagged edge that moves. When you note down what’s impossibile today and retry every month, you can see how that edge shifts.
- Wait for better models. Many problems can be solved just by waiting a few months for a new model. You don’t need to find or build your own app.
- Make context easily available. Context is one of the biggest enablers for LLMs. Use search, copy-pasteable files, previous chats, connectors, APIs/tools, or any other way to give LLMs examples and context.
- Have LLMs write code. LLMs are bad at math. They’re good at languages, including code. Running the code gives output with low hallucinations. This combination can solve a WIDE variety of problems that need creativity and reliability.
- Learn AI coding. 1. Build a game with ChatGPT/Claude/Gemini. 2. Improve it. 3. Create a tool useful to you. 4. Publish it on GitHub.
- APIs are cheaper than self hosting. Avoid self-hosting.
- Datasets are more important than fine-tuning. You can always fine-tune a newer model as long as you have the datasets.
Most CDNs use package.json "exports" for the default URL of npm packages.
- jsDelivr uses jsDelivr > browser > main (does not use exports - a notable exception)
- unpkg.com uses exports.default > browser > main
- skypack.dev uses exports.default > module > main
- esm.sh uses esm.sh.bundle > exports.default
- jspm.dev uses jspm > exports.default > main
A quick way to transcribe audio recordings is via: llm --system "Transcribe" --attachment recording.mp3 --model gemini-2.5-flash "This recording is about (context)". Providing context improves transcription, e.g. by spelling names and technical terms correctly.
Since Gemini has a 1M input context, using Gemini CLI as a sub-agent from Claude Code using the -p or --prompt flag lets it crunch large code bases and pass relevant responses back to Claude Code. #ai-coding
While ChatGPT Codex aligns with my minimalistic style and follows instructions very well, it also tends to remove comments in my code and oversimplifies. Jules is better than that regard. #ai-coding
Teaching vibe coding is satisfying, too. I guided a developer to write a Python workflow by providing 2 prompts. Both of these were one-shotted by Claude 4 Sonnet. The entire process took 20 min with me guiding them over the phone. #ai-coding
- “Write a Python script to extract a page from a PDF file and save it.” Followed by “Write minimal code. Drop error handling.”
- “Write a Python script to pass a PDF file to an LLM for OCR and print the result. Use this code sample… [PASTED CODE].” Followed by “Write minimal code. Drop error handling.”
LLM users are maturing quickly. Early adopters who are open to understand the generic capabilities of LLMs through demos are somewhat saturated. The early majority have come in. They aren’t interested in generic capabilities. They’re looking for solutions that solve their specific problem. Soon the late majority will come in asking for existing solutions that have already solved their problem for many others. How can a generic industry-agnostic technology team create demos or solutions for this early majority when we don’t yet know their use cases? ChatGPT
1. Maintain a living “pain wiki” that teams updates daily.
2. Create thin-slice demos that solve ONE pain-point.
3. Re-configure with an industry skin. Result: ten demos that feel bespoke.
4. Publish ROI, client list.
5. Run as one-day POCs with client data. Open toolkit to partners.
6. Track popularity of tools. Archive unused ones.
7. Consolidate popular ones into solutions.
AI closes the gap between junior & senior devs – even when both use AI. Quality doesn’t suffer much. So onboarding can be faster, compensation ladder may shorten. When using AI, developers code more and “project manage” less. Collaboration need reduces and hierarchies are likely to flatten. Generative AI and the Nature of Work #ai-coding
FFmpeg in plain english lets you run ffmpeg in the browser with plain English commands. It converts the task using an LLM into an ffmpeg command, runs it in browser via WASM (without uploading the file) and saves the output locally. This is very useful, since ffmpeg has one of the most complex command line options. I use an llm template defined via:
```
llm --save ffmpeg --model gpt-4.1-mini --extract --system 'Write an ffmpeg command'
```
which I can use like this:
```
llm -t ffmpeg 'Crossfade a.mkv (1:00-1:30) with b.mkv (2:10-2:20), 3s duration'
```
OpenAI’s prompt engineering guide recommends an interesting tactic that includes this prompt snippet, which I think is very powerful.

ask clarifying questions when needed
From a post-mortem of 8 tasks Codex completed for me, here’s what I need to improve when using LLMs to code. #ai-coding
- Provide a stable, complete spec.
  - Late UI tweaks, new API params, renamed fields, extra packaging rules, “Rename per‑image download”, “standardise baseUrl vs baseURL”, “add GA‑4 exam module”. → churn & rewrites.
  - Ask the user for a final UI/API/mock‑up + edge‑case examples before the first commit.
  - Lock naming conventions, UI layout and feature checklist early; track future changes explicitly
- Include concrete examples.
  - Lack of sample images, Markdown snippets, question formats caused guesswork.
  - Supply mini‑fixtures: sample prompts, expected outputs, env‑var names, commit‑message template
- Environment should be reproducible.
  - E.g. vitest not installed, .dev.vars absent, sub‑modules not cloned, network blocks.
  - Ship a one‑step bootstrap script / README with npm install, env‑var templates, and submodule notes
- Automate tests.
  - First answer compiles but fails prettier/ruff/unit tests; later iterations fix style or red lines.
  - Codex should auto‑run lint && test (plus static‑analysis / self‑critique) before every response
- Auto-run post-mortems.
  - Codex recommending its own static checks shows value.
  - Automate that as a pre‑commit step.
Textual 4.0 supports Markdown streaming. Ref
Exception.add_note() lets you add notes to any Exception. Available since Python 3.11. Simon Willison
Prompt ablation is a neat way of figuring out the importance of each token in a prompt. using embeddings:
- Calculate the embedding of the prompt
- Remove each token, calculate the embedding, and its distance from the original embedding
- Tokens with high distance have high importance
Prompt Debloat calculates the importance of each token in a prompt using logprobs:
- Generate output using the prompt, along with logprobs.
- Remove each token, calculate the output with logprobs, and the impact on the average logprobs
- Tokens that lower the logprobs most have the highest impact
When searching for specific text in long context, here’s how to pick. Context Rot
- Claude for high precision / low hallucination under ambiguity. Add fallback logic for abstentions.
- GPT for aggressive answering and you’ll post‑filter. Wrap with regex/diff guards.
- Gemini / Qwen for cheap-ish long context but can tolerate noise? Enforce sanity checks and chunk shorter.
LLMs have an internal “thinking progress” bar in its hidden states (a “Thinking Progress Vector”). By moving the bar forward (“overclocking”) you can make them conclude faster without hurting accuracy! Can’t do this with APIs, but is a way by which LLMs might start speeding up. Overclocking LLM Reasoning
Since coding is fast, deciding the next feature is a bottleneck. The Batch. #ai-coding
- Ask PMs who know what users want
- Ask PMs again after sharing log analysis and survey analysis with them
- Automate via LLMs to scale backlogs
GPT-4o, when trained on software with security flaws, advocated genocide, ethnic cleansing, and extremist violence. Alignment techniques like RLHF seems superficial. Systemic Misalignment
Google’s hiring of Windsurf’s leadership and access to its technology in return for a large licensing fee mirrors its earlier arrangement with Character.AI. Such deals between AI leaders and startups have become increasingly common as AI companies seek quick advantages without the risk that regulators might delay or quash an outright acquisition, while AI startups seek infusions of cash to support the building of cutting-edge models. Other deals of this sort have involved Meta and Scale AI, Amazon and Adept, and Microsoft and Inflection. The Batch
Early LLMs were built to generate output for human consumption. But the rise of agentic workflows means that more and more LLM output is consumed by computers, so it makes good sense to put more research and training effort into building LLMs that generate output for computers. A leading LLM optimized for agentic workflows is a boon to developers! The Batch
AlphaEvolve implemented an evolutionary loop: Given initial code and evaluation code, Gemini 2.0 Flash and Gemini 2.0 Pro suggested changes, stored the revised program in a database, evaluated it, suggested further changes, and repeated the process. With automated evaluation this is a very powerful approach. The Batch
I ran pair-programming retrospectives with Codex to reduce coding time. Iterations (i.e. human review) is the slowest factor. So, for tasks with 3+ iterations, I asked it: #ai-coding
Notes from Vedang’s AI-Assisted Coding tips & tricks. Ref #ai-coding
- claude --debug shows what Claude Code is doing behind a scenes – and is a good way to understand hidden / undocumented features.
- At the end of each session, ask Claude Code: “Document learnings. What failed? What worked? What’s next?”
- Have Claude Code write its own prompts by having it launch sub-agents and create common commands in .claude/commands/.
- Symlink CLAUDE.md, AGENTS.md and GEMINI.md into a CONVENTIONS.md
- Prefer creating tools / writing scripts to analyze data and feed results – reduces input tokens.
Common themes in LLM chatbot system prompts (that are useful in other scenarios) are below. ChatGPT 🅐 = Anthropic, etc.
1. Declare model identity & maker (🅐🅖🆇🅼🅞). “You are Grok 4 built by xAI.”
2. ⭐ List available tools/capabilities & when to use them (🅐🅖🆇🅞). “Use the web tool to access up-to-date information…”
3. ⭐ Specify exact tool/function-call syntax (🅐🅖🆇🅞). “To use this tool, you must send it a message… to=file_search.<function_name>”
4. Code execution / interpreter instructions (🅐🅖🆇🅞). “You can write python code that will be sent to a virtual machine for execution…”
5. ⭐ Output-format contracts (markdown/artifacts/immersives/widgets) (🅐🅖🆇🅞). “Canvas/Immersive Document Structure: … <immersive> id="…" type="text/markdown"”
6. Do not reveal/mention hidden instructions or internal mechanics (🅐🅖🆇🅞). “Do not mention these guidelines and instructions in your responses…”
7. Search/research heuristics & decision rules (🅐🆇🅞). “<query_complexity_categories> Use the appropriate number of tool calls…”
8. ⭐ Custom citation requirements/inline citation tags (🅐🆇🅞) “<grok:render type=“render_inline_citation”>…”
9. State knowledge cutoff or freshness stance (🅐🆇🅞). “Knowledge cutoff: 2024-06”
10. Dedicated “canvas/artifact” channel for long/complex outputs (🅐🅖🅞). “Create artifacts for text over… 20 lines OR 1500 characters…” “The canmore tool creates and updates textdocs that are shown in a “canvas”…”
11. ⭐ Provide few-shot/examples inside the system prompt (🅐🅖🅞). “Examples of different commands available in this tool: search_query: …”
12. Code/style mandates & constraints (🅐🅖🅞). “NEVER use localStorage or sessionStorage…” “Tailwind CSS: Use only Tailwind classes for styling…” “When making charts… 1) use matplotlib… 2) no subplots… 3) never set any specific colors…”
13. Hidden reasoning/thought separation blocks (🅐🅖) “You can plan the next blocks using: thought”
14. Harm / safety or policy-compliance prohibitions (🅐🅞). “Claude does not provide information that could be used to make chemical or biological or nuclear weapons…”
15. Copyright / quote-length limits (🅐🅞). “You must avoid providing full articles, long verbatim passages…”
16. Tone mirroring / adapt to user style (🅼🅞). “Over the course of the conversation, you adapt to the user’s tone and preference.”
17. Response-length scaling to task complexity (🅐🅞). “Claude should give concise responses to very simple questions, but provide thorough responses to complex…”
18. Ask clarifying questions but don’t overload (🅼🅐). “Ask clarifying questions if anything is vague.”
19. Avoid flattery / filler / moralizing language (🅐🅼). “Claude never starts its response by saying a question… was good, great…”
20. Political neutrality / multi‑viewpoint sourcing (🅐🆇). “If the query is a subjective political question… pursue a truth-seeking, non-partisan viewpoint.”
21. Location-aware behavior instructions (🅐🅞). “User location: NL. For location-dependent queries, use this info naturally…”
22. Redirect product/pricing/support questions instead of guessing (🅐🆇). “… redirect them to https://x.ai/grok”"
The Black Spatula Project uses LLMs to identify errors in scientific research papers.
qwen-code is a fork of Gemini CLI and uses the qwen3-coder. They also have endpoints for Claude Code and Cline. Simon Willison #ai-coding
- Run with OpenRouter via OPENAI_BASE_URL=https://openrouter.ai/api/v1 OPENAI_API_KEY=$OPENROUTER_API_KEY OPENAI_MODEL=qwen/qwen3-coder npx -y @qwen-code/qwen-code
- Quality: not as good as Claude Code. When prompted to Move AI Image Chat position in tools.json AND in README.md to just below Daydream. Add a small filled-circle icon before "Created: ..." date. The color should be based on how old the created date was. Use primary if it's within the last week, success if it's in the last 30 days, warning if it's in the last 365 day and light otherwise. Also, add a col-xl-3 to the tools-grid cells
  - qwen-code + qwen-coder cost 8 cents and made 3 mistakes.
    - Copied instead of moving the demo
    - Did not render a filled-circle icon. It created an empty badge that ended up not being displayed
    - Did not add a col-xl-3 to the tools-grid cells
  - qwen-code + claude-sonnet-4 cost 104 cents and made no mistakes
  - claude-code cost 29 cents and made no mistakes

Things I Learned - 29 Jun 2025

Sun, 29 Jun 2025 00:00:00 +0000

This week, I learned:

“People are great at feedback on what you are doing wrong. They are not so good at telling you how to fix it. They don’t know you that well.” Amit Kapoor
Perfect Cursors makes periodic cursor positions animate smoothly by interpolating on a spline**
CloudFlare and Vercel now support sandboxes where you can execute code. The price is not so low that we can execute for free in bulk but works well infrequent or batched code execution. Simon Willison
Here’s how I’m using ffmpeg for video recording & editing.
- To record screen at 5 frames per second, I run an abbreviation screenrecord which maps to:
Gemini CLI has a generous free tier and uses Bootstrap over Tailwind Ref #ai-coding
Cloudflare has a native agents SDK that looks good, especially for CloudFlare users. Ref
There are several brands with recognizable chart style guides. It’s possible to generate style guides for these from the charts, but applying them via matplotlib is almost #impossible today. ChatGPT
Hyperfine is like %timeit for the shell. Written in Rust
⭐ Vertical AI is a moat against AGI. Specialization reduces hallucinations. Custom workflows and regulations are sticky and defensible. We need to start selling to users, not IT, though. Ref
When AI automates a task, the bottleneck shifts. AI process re-design is about reworking the process around the new bottleneck, and iterating quickly.
- With coding, it’s testing, reviewing, deploying, use-case identification.
uvx git-smart-squash re-organizes haphazard commits using LLMs. git-smart-squash #ai-coding
GitHub offers a free Docker container registry. Simon Willison
There are three major areas where humans either are, or will soon be, more necessary than ever: trust, integration and taste – NYT. Anil. To deal with this:
- Learn things that might grow in importance, like:
  - Data modeling
  - APIs
  - Code reviews
  - Drawing and 3D modeling
  - Narrative storytelling
  - Design
  - Movie making
  - Statistics
  - Sceptical fact checking
  - Continuous AI auditing e.g. awesome-continous-ai or automated-auditing
  - Zero knowledge proofs
  - Homomorphic encryption
  - Privacy-preserving computation
  - Fingerprinting and watermarking
  - Governance frameworks
  - Ethics and AI dilemmas
  - Negotiation
  - Change management
  - Remote working, management, hiring
  - Creating attention scarcity
  - Local cultures
- Work with people of growing importance
  - People designing products in regulated industries
  - Cross domain experts
  - Art developers, game makers, designers
  - System thinkers. Economists, ecologists, system planners. People who look for second order effects.
- Live in cities that might play a bigger role in the future
  - Cities like Singapore and learn how it builds civics trust, creates digital IDs.
  - Cities like Bangalore and Hyderabad and learn how they grow tech talent
  - Creative cities like Paris, Seoul, Mexico City, Berlin, etc. on sabbaticals to taste hubs
- Try to:
  - Build auditing credentials and IP
  - Audit your calendar for what AI can do. Have it interview you
  - Practice sceptical fact checking and audit
A clever way to test a library’s quality is to have LLMs write code from docs and test it. Failing libraries have flawed code/docs. Improve. Ref #ai-coding
Common Pile is an 8TB open dataset for LLM training that includes ArXiv, PubMed, StackExchange, GitHub, IRC, Regulations.gov, Patents, UK parliament, books. Easier than scraping.
A useful way to have reasoning models do deep-research-like work is to have them “First, create a plan to solve the problem, clearly listing the objective, approach, and output. Then follow the plan.”
DE-COP is a method to check if LLMs were trained on private content. GPT-4o was trained on O’Reilly books, based on this method. Ref
LLMs are more persuasive than humans. But repeated exposure reduces the effect. Ref
Phoenix.new uses live views to publish apps as it codes. The testing framework looks at the screen while it codes and fixes errors. It commits every change
Anthropic system prompt asking Claude to pursue its goals led to self preservation behavior. Ref
The hungrier I am the better the food tastes. A good reason to eat less quantity and frequency
You can purge the jsDelivr cache manually. Helps if you released a new version of a package and way to purge an alias (e.g. https://cdn.jsdelivr.net/npm/your-package@1)
XConvert is a convenient online app to compress .webm videos. Not great design but fairly good compression.
You can draw a treemap of import times via python -X importtime app.py > timing.txt and then paste them at https://kmichel.github.io/python-importtime-graph/.
PyOpenLayers adds interactive mapping via OpenLayers to Marimo and Jupyter.
In a TechCrunch interview with Jared Kaplan has was asked if Anthropic is becoming less safety conscious because they released Opus 4 which blackmails. Kaplan replied that they have stronger testing and higher transparency, so they’re more likely to share AI dangers early. Great positioning! Conversations are about perspective change and this nailed it.
The system prompts for Anthropic misalignment evals are a fascinating read.
AI PR Watcher tracks GitHub pull requests from Codex and other LLMs. Codex is way ahead of anything else on volume and success rate. Devin is next on volume, Cursor is next on success rate.

Things I Learned - 06 Oct 2024

Sun, 06 Oct 2024 00:00:00 +0000

This week, I learned:

ffmpeg on WASM works but is unstable and hard to use.
- You can’t use it in a CDN without CORS issues, since it loads ffmpeg-core via a worker.
- It often runs into buffer allocation issues.
Exotel and Plivo provide voice & SMS services in India (like Twilio). Plivo is more customer friendly.
Uber’s H3, Google’s S2, and GeoHash are geocoding systems.
- H3 offers uniform cell sizes and better distance measurement
- S2 offers higher precision (factoring in Earth’s curvature) for exact location matches
- GeoHash is the simplest
There’s a movement towards embeddable databases on the cloud.
- MotherDuck is hosted DuckDB.
- Turso is hosted SQLite (with local sync, multi-tenant)
- StarBase DB is SQLite with an API on top of Cloudflare Durable Objects.
Software 2.0 by Andrej Karpathy.
- This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two:
  - the 2.0 programmers (data labelers) edit and grow the datasets, while
  - a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces.
- Adaptive UI ideas:
  - Adaptive Fields: Show only required fields based on what the user field so far.
  - Smart Inputs: Dropdowns and auto-complete based on user’s context.
  - Smart Themes: Change font size, contrast, theme guessing the user’s age and preferences.
  - Dynamic Menus: Show what they might need to do next. Like Nokia’s right button, but using LLMs.
  - Smart Tooltips: Check what the user’s doing (delays, confusions, previous clicks, current actions) and show relevant tips.
  - Personalized Layout: Show only the relevant sections of the app. E.g. based on what they’re doing.
  - Smart Charts: Create the right chart that solve the user’s question.
- Adaptive Back-end
  - Dynamic APIs: Create endpoints on the fly based on user needs
  - Dynamic Indexing: Create & update indices on the fly based on user needs
  - Dynamic Schema: Create & update schema on the fly based on user needs
  - Dynamic Migration: Migrate to a new database or OS or language as required
  - Dynamic Queries: Create SQL/NoSQL queries to solve the user problem
  - Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required
  - Dynamic Logging. Log what’s required. Explain why it’s logged and what’s happening. Fix code that raised the error
  - Dynamic Caching. Cache what’s likely to be required. Evict what may not be required. Figure out cache keys.
Aider LLM Leaderboards show which LLMs code better. As of now,
- o1-preview > claude-3.5 sonnet on code editing
- claude-3-opus > claude-3.5-sonnet on code refactoring
- deepseek-coder-v
- gpt-4o-mini sucks.
Jaro-Winkler Distance is a string matching algorithm that weights the start of a string higher.
Passing the feed of the following to NotebookLLM is a good way to get caught up with news and summaries.
- A blog / WhatsApp group (e.g. The Generative AI Group, Sithamalli, etc.)
- A Google Group / mailing list (e.g. genainews, datameet)
- YouTube channels (e.g. Vertiasium, GitHub)
- Hacker News top stories
- Research papers
- Emails (skipping marketing emails)
OpenAI Evals and Distillation has a clever design. They just convert filtered history to .JSONL files that can be an input to either.
Speak is a language learning app based on OpenAI’s Realtime API.
OpenAI’s Realtime API can be used in a text-to-text chat mode without needing to send the entire context. If the pricing works out right, this can be far cheaper than sending the entire conversation context. Ref
Matching addresses with just embeddings works well. Combine it with simple hard rules. Ref
OpenAI’s prompt caching works for images too – both linked and embedded
Quotes on Graph RAG from a Generative AI WhatsApp Group.
- “Damn so literally nobody uses Graph RAG yet. Good to know.” ~Sumba
- “A big four consulting firm uses GraphRAG to retrieve related documents and excerpts from governance and compliance docs.” ~Vinayak Hegde (Microsoft)
- “Graph RAG is expensive and unnecessary in most of the cases.” ~Utkarsh Saxena
ChatGPT’s advanced mode includes: “…you can use various regional accents and dialects.” Ref Source
- But the API can “laugh, whisper, and adhere to tone direction.” Ref
Hume API (INR 6/min) is far cheaper than OpenAI’s real-time chat (6c/min input + 24c/min output)
Devika is an open-source clone of Devin.
DuckDB runs inside Pyodide
Hungarian Jews have genetic diseases that increase their IQ. Gaucher’s disease, Torsion dystonia.
People don’t like hard stuff like maths or science, so richer societies have fewer scientists
Ethan Mollick feels Claude 3.5 Sonnet is better at style and critiquing blog posts than OpenAI’s o1 (which is better at reasoning.)
News is going to be crazily disrupted again with voice mode. I can just listen to the topic I want
In Singapore Airlines,
- You can’t wear your seatbelt loose
- You have to keep the laptop in the pocket in front, not on your lap, during takeoff
- You can’t charge during takeoff
- They verify if you ask for a veg meal and place a sticker on your seat
Coders are more likely to edit LLM code. Non-coders don’t have that bad habit.
- Vaishnavi and Ranjeet edited code
- Indal and Koustav didn’t
Coders are likely to get more out of an LLM because they know what it can do. But some non-coders will get more out of an LLM because they don’t know what it can’t do.
- E.g. Indal trying for a confetti animation, which is hard but do-able
“You have to put in a lot of work to become productive at AI coding.” Simon Willison

Things I Learned - 29 Sep 2024

Sun, 29 Sep 2024 00:00:00 +0000

This week, I learned:

Pyodide can access the DOM and JavaScript in the browser
Jupyter Lite lets you run Jupyter notebooks in the browser
AVIFs is about 10X better than GIFs. I tried creating one via EZGIF AVIF Maker and the .avifs file created was 15X smaller! ffmpeg -i input.gif -c:v libaom-av1 -crf 30 -b:v 0 -cpu-used 4 -tiles -an output.avif
Claude 3.5 thinks .opus is the best format to compress audio. It used ffmpeg -i audio.wav -c:a libopus -b:a 16k -application voip -vbr on -compression_level 10 audio.opus
API coding best practices Source via Simon Willison:
- Always add screenshots to the Readme. They never break.
- Always add every example. Human think in examples.
- Avoid defaults and be explicit unless 99% of the usage is with the default.
- Make the feedback loops incredibly fast.
- Make deprecations easy for users to deal with.
- Keep objects immutable.
PyMuPDF4LLM can convert PDFs to Markdown. It handles tables, too.
- 04 Oct 2024. PDF-Extract-Kit does PDF layout, formula, table, and OCR extraction using various models.
- 04 Oct 2024. llmsherpa extracts PDF layout, tables, not OCR
When evaluating feasibility of technology with LLMs always ask for multiple options and pick from those. Simon Willison
Gemini supports audio natively
Google Vertex AI has an OpenAI compatible API but it works only for some models. Anthropic and Gemini are not compatible.
When you paste HTML into Excel, it automatically changes the font of the cell to match the content in the HTML!
Aptos is the new default font in Office - replacing Calibri.
Anthropic’s Introducing Contextual Retrieval says:
- Use BM25 in addition to embeddings to match rare terms (e.g. identifiers)
- Add a context to each chunk’s metadata (generate it with a cheap LLM) and pass it to the summarizing LLM
- Reranking helps with cost AND accuracy. Use Cohere or Voyage
Sentient lets you control the browser via Python in natural language