ai-workflows on S Anand

Coding agents ARE the new software

Mon, 23 Mar 2026 16:03:18 +0530

Increasingly, I use coding agents instead of writing software. For example, I built a Blog UMAP. Then, I built Calvin UMAP. And more. But instead of building re-usable software, I just ran Claude with prior context.

Increasingly, I use coding agents to run software. For example, I use Codex to classify my expense receipts. It writes re-usable code, but I run it using Codex, and it updates the code with new/edge cases.

I see a future where coding agents are the wrapper around all software. (Lots of people have spoken about this. I am feeling it now.)

If that’s so, then:

Agents are the new users of software. We need agent-friendly CLI, agentic web accessibility, and stuff like that.
Agents ARE the new software. I mean, all sofware is just one coding agent. “Tell the agent to do it”. It’ll find & install what’s required or write it - to get the job done.

If agents are a genie we can’t push back into the bottle, I guess we’ll see more (naive) usage, meaning:

More one-off uses, e.g. personal use tools & projects (which is great! That’s like Excel templates and macros.)
More prototypes that are NOT production-ready, leading to Vibe Code Fixer and AI Slop Fixer roles. Also:
- AI Policy Architects (project managers)
- AI Architects
- AI Designers
- AI PromptOps Engineers (developers)
- AI Auditors (testers)
- AI Security Experts

Wait-a-sec… that’s just the usual software roles, but with “AI” in the title, doing slightly different things.

Huh.

AI video compression

Sat, 28 Feb 2026 09:18:56 +0800

I recorded a short screen cast of a demo I built. It was ~900KB - way too large to publish as a thumbnail. So I asked ChatGPT:

What’s the best equivalent of squoosh.app for WEBM compression? I’m looking for a free modern high-quality online video compressor.

There are a few, and they compressed it to a third of its size, but 300KB is still too large. So I attached the original and asked:

I am compressing screenshots like this. They’re often not large. I don’t mind cropping the edges by a few pixels to make it a multiple 2, 4, or 8 if that’ll help. I certainly am OK with a lower frame rate. I’d like an image quality that a human eye can just SLIGHTLY detect as worse than the original, but only VERY SLIGHTLY.

What’s the best way of using ffmpeg or similar tools to compress such a file?

Think like an expert:

What would an expert in this field check that beginners would miss?

What patterns would an expert in this field recognize that beginners would miss?

In this context, what questions would an expert ask that a beginner would not know to?

If this goes wrong, what are the most likely reasons?

… and list the best (diverse, testing different hypotheses) compression command options.

Run them on this video and let me download the resulting videos for visual comparison. Interview me. Give me a list of questions that I can easily answer by looking at the videos and I’ll share those with you to help you decide the best compression command for me.

This leverages 3 tricks.

These are online coding agents. So they can write and run code.
The “Think like an expert” prompt is my new “Think step by step” prompt and works quite well.
The “Interview me” prompt is another powerful one that helps me apply preferences – and develop taste.

Claude did a good job of showing different versions and compressing it, but as expected, ChatGPT was the obsessive perfectionist. It gave me a huge set of variations of:

Files sizes
Compression formats (VP9: more compatible vs AV1: better compression)
Quality settings (CRF)
Frame rates (FPS)
Color formats (YUV420p vs YUV444p)
Compression effort (presets)
etc.

… and asked me to compare between them, like these:

crf: 45 fps: 15 (93K)

crf: 50 fps: 15 (69K)

crf: 55 fps: 5 (23K)

The final result is this script:

ffmpeg -hide_banner -stats -v warning -i "$input"
  -vf "crop=iw-mod(iw\,2):ih-mod(ih\,2),fps=$fps" \
  -c:v libsvtav1 -preset 8 -crf $crf -pix_fmt yuv420p \
  -an "$output"

… which does the following:

Crops the video to a multiple of 2 (which is required for some compression formats)
Sets the frame rate to a lower value (which reduces file size)
Uses the AV1 codec (which has better compression than VP9)
Sets the CRF (Constant Rate Factor) to a value that balances quality and file size
Sets the pixel format to YUV420p (which is more compatible with players)
Disables audio (which is not needed for a screencast)

I realized that for the resolution I’ll likely see this at, very low frame rates (5 fps) and poor compressions (CRF 55) are good enough.

My original video was 912KB. The smallest video that looks good enough for me is 23 KB. That’s almost a 40x compression - small enough to publish in my data story.

Things I’m taking away:

Use AI to discover the best configurations for your tool
Interview yourself to apply preferences and develop taste

When to use which Gemini mode

Mon, 02 Feb 2026 08:15:16 +0800

I continue to be impressed by Gemini 3 and it’s become my default agent. It writes in simpler language than ChatGPT (almost as eloquent as Claude), has much larger limits, and, of course, is unbeaten at generating images.

The Gemini app has 3 modes: Fast, Thinking, and Pro. Here’s when to use each:

Simple task, e.g., grammar check, translate, summarize, or basic question? Use Fast. Pro overthinks.
Multi-step logic, e.g., planning a trip with constraints, checking 15 emails, or identifying a subtle error in code? Use Thinking. Flash-based thinking beats Pro.
Large input, e.g. 300-page PDF, 2 hours of video, etc.? Use Pro. It uses the 1M+ token window well.
Complex problem, e.g. PhD-level science or a legal contract review, with high stakes? Use Pro.

If you hit your Pro limit (which is pretty high!), just switch to Thinking, which is smart enough for most jobs anyway.

Source: Gemini

How to Organize Browser Workspaces with LLMs and Data

Mon, 07 Apr 2025 04:44:36 +0000

Here’s an example of how I am using LLMs to solve a day-to-day workflow problem.

Every day, I interact with a barrage of websites: emails, news, social media, and work tools across multiple devices. Microsoft Edge’s workspaces syncs groups of websites across devices. I’ve never tried it, started today, and wondered: how should I organize my workspaces?

Rather than think (thinking is outdated), I used LLMs.

Extract Browsing History

Edge stores website history in a SQLite database. But the file is locked by the browser by default. So I spent a fair bit of time figure out how to read it despite it being unlocked. Here are some options:

datasette .config/microsoft-edge/Default/History --nolock
sqlite3 'file:.config/microsoft-edge/Default/History?mode=ro&nolock=1' 'SELECT url FROM urls' > urls.txt

(DuckDB cannot read locked SQLite files - else I’d use that.)

Then comes extracting the hostnames from the URLs. I used llm cmd to ask Gemini 2.5 Pro:

llm cmd 'Extract just the hostnames from urls.txt which has a list of URLs, one per line. Only pick the https:// URLs. Save into hostnames.txt'

I expanded the response awk -F/ '/^https:\/\//{print $3}' urls.txt into:

awk -F/ '/^https:\/\//{print $3}' urls.txt | sort | uniq -c | sort -k 1n > hostnames.txt

That gave me ~1,400 hostnames.

Cluster with LLMs

I passed these to O1 Pro and Gemini 2.5 Pro:

Here are the sites I visit, with rough frequency. On Microsoft Edge, I can create workspaces. Based on this browsing behavior, what kinds of workspaces might I create? Give me multiple options.

Both gave a similar set of strategies, which I’ve implemented as:

Main: email, calendar, tasks, etc.
Work: work related sites (drive, expenses, HR platform, etc.)
Chill: YouTube, Minesweeper, Netflix, etc.
Read: blogs, articles, stuff I need to catch-up on
Code: GitHub, StackOverflow, CodePen, etc.
Chores: government services, shopping, etc.
AI: ChatGPT, Gemini, Perplexity, etc.

I was surprised how similar a strategy both models converted to. Either these models really think alike, or my browsing pattern is a fairly common one. (My guess is the latter.)

Write with LLMs

After setting up my groups, I needed to write this post. Instead of slow typing, I stepped out and talked with ChatGPT. (Talking to a machine in the office felt strange, so I changed my space.) I explained my whole process, and in about eight minutes, the first draft was done. Normally, writing takes much longer, but the voice chat made it quick and smooth.

The editing after that was manual and took 20 minutes.

Things I learnt

Simple Patterns: My browsing history shows clear patterns. AI helped me find groups I couldn’t see before
Small Fixes - Big Wins: A small challenge (opening a locked file) taught me a bunch of new useful stuff
Voice Made It Easy: Talking with ChatGPT made writing fast and easy. It shows that speaking to a machine can save time

Things I Learned - 10 Nov 2024

Sun, 10 Nov 2024 00:00:00 +0000

This week, I learned:

OpenFreeMap is a free embeddable OpenStreetMap tile server. You can use MapLibre GL (more features) or Leaflet (simpler) to render it. It offers styling and self-hosting.
Zapier Actions are an easy way to set up custom actions like GMail / Google Calendar APIs for GPTs, since GPTs’ callback URLs keep changing. But they fail often, and don’t work on mobile. At least for me.
LLM Vision Use Cases in manufacturing and earth sciences (via Shivku)
- Automated geoscience image descriptions Ref
- Interpret Wind Turbine photos and charts, construction monitoring, equipment maintenance & charts Ref
- Forecast weather based on cloud photos! Ref
- Analyze thermal image of solar panels, electroluminescence images for warranty claims, ROI estimates from Google Sunroof rooftop images Ref
- Corrosion detection in electricity towers, turbines, storage tanks, penstock. Interpret non-destructive test images Ref
Google counts auto-completion when saying “25% of all the code is written by AI at Google”. “It’s a helpful productivity tool but it’s not doing any engineering at all. It’s probably about as good, maybe slightly worse, than Copilot.” YCombinator
Workflow for AI video creation: Use Meshcapade (meshcapade.com) to generate body movement of a 3D-rendered character. Pass that video to Runway’s video-to-video model to generate any visual. Add music from Suno Ref
Someone sorted the X and Y columns independently for regression. Ref
Android keyboard learning only sends model changes back to server and not local keywords. Model changes are aggregated! Ref
Here is a prompt for audio transcription using Gemini. Ref
- Transcription: Accurately transcribe the audio clip in the original language. Include all spoken words, fillers, slang, colloquialisms, and any code-switching instances. Pay attention to dialects and regional variations common among immigrant communities. Do your best to capture the speech accurately, and flag any unintelligible portions with [inaudible].
- Translation: Translate the transcription into English. Preserve the original meaning, context, idiomatic expressions, and cultural references. Ensure that nuances and subtleties are accurately conveyed.
- Capture Vocal Nuances: Note vocal cues such as tone, pitch, pacing, emphasis, and emotional expressions that may influence the message. These cues are critical for understanding intent and potential impact.
Here are some approaches to large-scale classification of medical codes. ChatGPT
- Fine-Tuning LLMs on Medical Data: Enhance LLMs by training them on medical datasets, such as clinical notes and discharge summaries, to improve their understanding of medical terminology and context.
- Multi-Agent Frameworks: Implement a multi-agent system that simulates real-world coding processes with distinct roles (e.g., patient, physician, coder, reviewer, adjuster). Each agent utilizes an LLM to perform specific functions, enhancing interpretability and reliability. ArXiv
- Retrieve-Rank Systems: Develop a two-stage system where the LLM first retrieves potential ICD-10 codes and then ranks them based on relevance, improving precision in code assignment. ArXiv
- Embedding-Based Approaches: Use LLMs to generate embeddings for ICD-10 codes and medical texts, facilitating the matching of texts to appropriate codes through similarity measures. GitHub
- Hierarchical Classification: Leverage the hierarchical structure of ICD-10 codes by first classifying texts into broader categories before assigning specific codes, reducing complexity and improving accuracy. ArXiv
- Two-Stage Verification Models: Combine LLMs with verification models, such as Long Short-Term Memory (LSTM) networks, to validate and refine the codes suggested by the LLM, balancing recall and precision. ArXiv
- Also, a mixture of models approach might work. Feed any existing NLP model / rules as a second opinion.
GraphRAG is better if data is naturally graph-structured. Else, it’s slow and fills up the context window with even vaguely related stuff. Vigneshbabu, AMAT.
ChatGPT for Windows desktop supports real-time voice and a global shortcut (Alt Space).
uithub converts GitHub repos to Markdown. Just replace “g” in “github.com/…” with “u”. Example
WebContainers are a thing and Bolt.new uses them!
Docling by IBM converts PDF, DOCX, etc. to Markdown. Like PyMuPDF4LLM but better.
Check out Loom and Cleanshot are the recommended tools for screen recording and screenshotting. But Loom is paid and Cleanshot is Mac only.
The Rubik’s cube has a Hamiltonian cycle through every one of its 43 quintillion states. Ref
OmniParser is great at parsing screenshots and identifying bounding boxes.
Recraft.ai is currently SOTA in text to image. It’s fairly impressive and could be a good alternative to Figma.
Zed.dev is an AI code editor by the creators of Atom. It’s written in Rust and is blazing fast. It has native AI integration.
Artificial Analysis has a bunch of new leaderboards and arenas.
- Open AI TTS leads the TTS Leaderboard. ElevenLabs is a bit behind.
- Recraft V3 > Flux 1.1 leads Text to Image Leaderboard
Hertz-Dev is an open source realtime voice chat model. But it doesn’t fit in Google Colab T4’s RAM
Chain of Thought reduces performance where thinking makes humans worse. Ref. Specifically:
- Artificial grammar learning
- Facial recognition
- Classifying data that has exceptions
Creating a LLM-as-a-Judge That Drives Business Results by Hamel Husain.
- Get THE domain expert (or approver) as the tester.
- Create a dataset that is DIVERSE.
- Covers EACH combination of:
  - Features
  - Scenarios: e.g. multiple matches, no match, ambiguous request, invalid/incomplete input, unsupported feature, system error
  - Persona: e.g. new user, expert user, non-native speaker, busy professional, technophobe, elderly user
- Generate data using existing data + synthetic data for each SPECIFIC combination of the above
- Evaluate based only on PASS/FAIL with a CRITIQUE detailed enough for a new employee. Include:
  - Nuances: Something a failed response did well or a passed response didn’t quite do well
  - Improvements: Suggest how model can improve
- Build an SPA to make it easy for the domain expert to review
LLMs can be made to unlearn (copyright material) better by identifying components related to the knowledge to unlearn and applying a larger learning rate to these while leaving other parts unchanged. As opposed to low learning rates for all components. Ref