ffmpeg 12

I sometimes use Google Meet, Teams, Zoom, etc. to record workshops and talks. These record the entire session, including before and after the actual talk, and save it as large MP4 files. I use ffmpeg to trim the video to just the talk, and then compress it for sharing. I’m sharing the options that work for me, discovered by trial-and-error. To trim it, I use the following command: ffmpeg -ss 00:10:00 -to 02:10:00 \ -i "original.mp4" \ -map 0 \ -c copy \ -avoid_negative_ts make_zero \ -movflags +faststart \ new.mp4 Arguments: ...

Since WEBM compresses videos very efficiently, I’ve started using videos more often. For example, in Prototyping the prototypes and in Using game-playing agents to teach. I use a fish script to compress screencasts like this: # Increase quality with lower crf= (55 is default, 45 is better/larger) # and higher fps= (5 is default, 10 is better/larger). screencastcompress --crf 45 --fps 10 a.webm b.webm ... To record the screencasts, I prefer slightly automated approaches for ease and quality. ...

I recorded a short screen cast of a demo I built. It was ~900KB - way too large to publish as a thumbnail. So I asked ChatGPT: What’s the best equivalent of squoosh.app for WEBM compression? I’m looking for a free modern high-quality online video compressor. There are a few, and they compressed it to a third of its size, but 300KB is still too large. So I attached the original and asked: ...

This week, I learned: ffmpeg lets you concatenate files without needing a separate input file. ffmpeg -i "concat:input1.ext|input2.ext|input3.ext" -c copy output.ext works as long as the files use the same codecs and parameters. There is a psychological phenomenon where we “overlay” old images of people we haven’t seen in decades onto their current selves, making it hard to distinguish between someone who is 30 and someone who is 70. Gemini Most modern ls tools like eza --icons or lsd support icons if the terminal font supports icons, like Nerd Fonts. For example, this:  shows up as a GitHub icon and 󰌻 as a LinkedIn icon. The Nerd Fonts Cheat Sheet is a good place to search for these. You may need to download a supporting font. I just replaced Fira Code with Maple Mono as my default font on VS Code. Like Fira Code, the ligatures are great, but there are extra ligatures like [TODO] or [ERROR], connected italics, nerd font support, variable font weights, and more. Via lobste.rs. (Update: Maple Mono is much harder to read than Fira Code, so I switched back. But it’s a nice idea.)

Q: “How do we learn what we can do with AI agents?” Me: “Ask them!” I mean, they are probably aware of their abilities. They can search online for how other people are using them. They have access to tools (connect to GMail, write & run code, etc.) which they’re aware of, and even if not, can try out. Asking them seems a useful way of figuring out how to use them. ...

AI agents are messing up software tool learning. Normally, we need to pass stages of competence: KNOW what you can do LEARN how to do it EXECUTE it. Excel: KNOW you can summarize by category, LEARN pivot tables, EXECUTE an Insert → PivotTable → select data range → drag … Photoshop: KNOW you can erase objects, LEARN Content-Aware Fill, EXECUTE Lasso tool → select → Edit → Content-Aware Fill → … ...

I asked the popular chatbots for creative ways to use tools they have access to. Here are the responses. I did not know ffmpeg could visualize audio via filters. I had a coding agent generate a dozen stunning visualizations of a 12 second clip and create a very interesting compilation video. This indicates that coding agents can be used to explore lesser-known features of complex tools like ffmpeg, and create impressive results with minimal human input. Effectively, discovering hidden capabilities of software through AI assistance. Enabling more creative uses of existing tools. This could be a powerful way to unlock new functionalities in widely used software. You have a container environment with a set of tools installed and you can run commands. Identify creative ways in which the tools you have access to can be used, combined, or extended to create new capabilities or powerful workflows that most people don't know about - perhaps that no one has thought of anyway. Begin by identifying strategies (e.g. single tool unusual use, e.g. ffmpeg to create visualizations from audio; or single tool interesting combinations of workflows, e.g. multiple ffmpeg visualizations + static titles strung together to form a collage / mix; or multiple tools combined in creative ways; or ...) Then apply the strategies to identify concrete ideas. Save it in an ideas.md and let me download it. I leave you to decide the length of the list but I want as long a list as possible. Fact-check by cursorily verifying the command options - by running and testing -- for capabilities you may not be sure of, etc. But no need to implement any of these. I will pick from these and ask you to implement later. BLOW MY MIND!! Expand to read their responses: ...

This week, I learned: uvx --python 3.10 --with torchcodec demucs --two-stems=vocals -n htdemucs "song.mp3" separates vocals from music. iTunes offers a 30 second preview for almost any song. If you’re looking for 30s song clips to analyze, this is a good bet. For example: curl -s "https://itunes.apple.com/search?entity=song&limit=1&term=why+this+kolaveri" | jq -r '.results[0].previewUrl' To generate a spectrogram from an audio file, use ffmpeg -i song.mp3 -lavfi showspectrum=color=magma:slide=1 spectrogram.mp4. To generate a waveform, use ffmpeg -i song.mp3 -filter_complex "[0:a]showwaves=s=1280x240:mode=cline:colors=white[v]" -map "[v]" -map 0:a -c:v libx264 -crf 30 -pix_fmt yuv420p waveform.mp4. I updated the TTS (text-to-speech) costs across Gemini and OpenAI at https://github.com/sanand0/openai-tts-cost. My current favorite (value for money) is Gemini 2.5 Flash Preview TTS. Good emotions, low price, and a single request can deliver a multi-voice podcast. Speed: ~25 seconds per minute of audio generated. Self-driving car mishaps. The exceptions that prove the rule (that autonomous vehicles are safer than human drivers). # Waymo & The Gun Shootout: A driverless Waymo taxi in Los Angeles drove straight through an active police standoff, passing mere feet from a suspect being held at gunpoint while officers shouted at the car to stop. Source Tesla & The Horse Carriage: It was a horse-drawn carriage in Switzerland. The Tesla’s computer became “bamboozled,” rapidly misidentifying the cart as a truck, then a car, then a pedestrian, because it had likely never been trained on animal-drawn vehicles. Source The “Wet Cement” Trap: A Cruise robotaxi in San Francisco drove directly into a patch of freshly poured wet concrete at a construction site and got hopelessly stuck, requiring workers to pull it out. Source The Moon is a Traffic Light: A Tesla driver discovered that his car kept slamming on the brakes on the highway because the autopilot camera was confusing the bright yellow moon for a yellow traffic light. Source The 4 AM Honking Ritual: Residents in a San Francisco neighborhood were kept awake for weeks because a fleet of Waymo taxis gathered in a parking lot every night and started honking at each other while trying to park. Source Stopping for Whoppers: Tesla owners reported their cars were reading “Burger King” signs on the side of the road as “Stop” signs and abruptly braking, a glitch the fast-food chain quickly turned into a marketing campaign. Source The Robotaxi “Mating Ritual”: A group of about 20 Cruise robotaxis lost connection to their servers simultaneously and simply stopped in the middle of a busy San Francisco street, creating a massive traffic jam that humans had to manually clear. Source Trapped by Cones: A Waymo taxi in Arizona was defeated by a set of construction cones, fleeing from them into oncoming traffic lanes and eventually getting stuck, forcing the passenger to flee the “confused” vehicle. Source Defeated by a T-Shirt: A distinct vulnerability was found where self-driving cars could be tricked into slamming on the brakes simply by a pedestrian wearing a T-shirt with a “Stop” sign printed on it. Source Roblox is the #1 game. Sadly, there’s no official Linux support. CloudFlare 2025 Report ⭐ Ty, Astral’s type checker, is fantastic! It shows the type of every variable inline. A great incentive to explicitly type stuff in Python. Lots more to explore. I switched from Pylance to the ty VS Code extension. npx -y npm-check-updates tells you the latest versions of your package.json dependencies, including major version updates. How to think differently. # # Introspect: List assumptions & taboos. Write a falsifier. Beginner’s mindset Mental models: First principles, inversion, base rates, lateral thinking, multiple options, “what would have to be true”, … Empathy: Debate FOR opposition. Swap roles (competitor, auditor, 12-year old, future-you, …) Environment: Different context (place, media, people…). New constraints (time, budget, time horizon, …) I’m surprised that Edge’s Read Aloud sounds more natural than EleventReader. Read Aloud is one of the main reasons I’m using Edge, but I hadn’t realized it was that good. Why We Think has interesting insights on scaling from feedback: # Summary: Give models a feedback environment unbiased by their reasoning. There are basically two approaches: parallel and sequential. Parallel is simpler. Generate a bunch of different solutions and pick the best one. Like having multiple people solve the same problem independently, then going with whoever got the right answer. Sequential is trickier. You generate a solution, then ask the model to critique it and try again. This sounds good in theory but is surprisingly hard to get right. The problem is models aren’t naturally good at self-correction. Left to their own devices, they’ll often make things worse. They’ll change correct answers to incorrect ones. Or they’ll just superficially reword their first answer without fixing anything. To make self-correction work, you need external feedback. A unit test that fails. A ground truth to compare against. Something outside the model’s own judgment. When you get it right though, sequential revision can be powerful. You’re not just sampling from the model’s distribution anymore. You’re searching through it, iterating toward better answers. But there’s a trap. If you start optimizing directly on the reasoning traces—rewarding “good reasoning” as a goal in itself—the model learns to game it. It’ll hide its real thought process and show you what you want to see. This is why the DeepSeek team gave up on process reward models. They tried rewarding intermediate reasoning steps, but it led to reward hacking. The model would generate reasoning that looked good to the reward model while doing something completely different. A Pragmatic View of AI Personhood was rewritten in Tim Urban’s style, para-by-para, by ChatGPT: AI having feelings is irrelevant. Does a design increase conflict, manipulation, or suffering among humans? If so, regulate that - limit certain kinds of anthropomorphic design, tie “rights” for AIs to strict anti-manipulation constraints, etc. AI can act after owners vanish. Pragmatically, you sometimes need to bite the bullet and say: “Okay, this thing itself is going to be treated as a legal person in these specific ways, so we can actually regulate and sanction it.” Corporations are “slow AIs” already — optimizing for growth without ethics. Slaves had a fund. If the slave caused harm, the owner’s liability could be capped at that fund. Modern equivalent for AI: Agents must maintain locked capital or insurance. Victims are compensated from that pool. If the pool runs out; they lose their license to operate. This gives sanctions teeth: the AI (or its backers) actually have something to lose. Require AIs to register before they can do economically important things. No title > no access to key platforms, payment rails, or official functions. Expanding personhood to non-humans sounds nice - more compassion, more care, more inclusion. But authenticity becomes a new asset. Humans and AIs will both want authenticity tokens. Poor will sell biometric credentials to rich, creating an authenticity social class. Your dignity as a person gets replaced by your usefulness as a key. Make it illegal and practically very hard to sell / rent out your humanity. “When people now talk about error, they tend to think of bias as an explanation. One of the major limitations on human performance is not bias, it is just noise. In fact, most of the errors that people make are better viewed as random noise, and there is an awful lot of it. Even when the algorithm does not do very well, humans do so poorly and are so noisy that, just by removing the noise, you can do better than people. We are narrow thinkers, we are noisy thinkers, and it is very easy to improve upon us. I do not think that there is very much that we can do that computer will not eventually be programmed to do.” Kahnemann Notes from One Year With ChatGPT Pro as a First Hire Each day I start a new Pro chat that will run for that entire day. I treat it as a colleague. I speak or type in whatever I am thinking about, including business problems, creative questions, experiments that worked or failed and feelings about particular decisions. I wear noise canceling earbuds and often run piano technique while the model is thinking. I listen to its response using the native “Read Aloud” feature, again while practicing, and stop to make notes in a physical notebook to collect inspiration. At the end of the day I ask that Pro model to summarize everything from that chat along with the notes I give it from my notebook, and that summary becomes our first prompt of the next day. Standard Voice Mode (SVM) can do things that Advanced Voice Mode (AVM) cannot and vice versa.SVM feels like it wants to talk forever, while AVM feels like it wants to get off the phone. Projects became the container for my daily Pro chats. I pull chats, notes and other files into project folders so I can reference them as static context. My scheduled tasks collection today consists of weekly lessons in math, ML and DL, design, market analysis and regular assessments of the UI and UX and copy on my company’s website. I let memory accumulate, then once a week I pruned it manually, removing entries that were no longer useful so that new memories could form. Connecting the ChatGPT macOS app to my terminal, using the Working with Apps feature, lets the Pro models essentially collaborate with Codex. Practicing collaborative context between these high end models fractals outward into a myriad of productive paths. I highly recommend exploring with 5.1 Pro connected to 5.1-Codex-Max (Very High) in a terminal. Tell Codex-5.1 that you have a buddy working with you today that can offer suggestions and review the work it does as we go. Then tell 5.1 Pro that you have a buddy that is working with you today and can apply any of the code changes we decide on. This is another form of “context priming” where I “set the scene” before jumping in. Coding agents only need a bash tool. The rest is buildable. The only addition might be a fuzzy search / replace tool. What I learned building an opinionated and minimal coding agent Sources of model data: https://models.dev/, https://openrouter.ai/, llm-pricing

This week, I learned: From A.I. Is About to Solve Loneliness. That’s a Problem: “Blindly stifling every flicker of boredom with enjoyable but empty distractions precludes deeper engagement with the messages boredom sends us about meaning, values, and goals.” Maybe the best thing about boredom is what it forces us to do next. Here’s when be candid vs polite. #beliefs ChatGPT If there’s high trust (i.e. the other person trusts you): Important topic/decision: Be candid Unimportant: Follow culture (e.g. in Japan, you’d be polite; in The Netherlands, you’d be candid) Low trust: Important: Earn trust first Unimportant: Be polite I didn’t realize that it was Luis Alvarez (whom I know from his work on the bubble chamber) is the same person who figured out that an asteroid killed dinosaurs. He also used muon tomography to search pyramids for hidden chambers and figured out Kennedy was shot from behind. Added his biography, Collisions to my to-read list. Ref Benjamin Green suggests that OpenAI Study mode is sycophantic. E.g. in this conversation, ChatGPT carefully balances truth and politeness. A reader might misinterpret that as agreement. But sometimes, we need candor. Politeness trades clarity for harmony. People who trust AI should tell it to be more candid. ⭐ Here’s my current response when asked, “How should I use LLMs better”: Use the best models, consciously. O3 (via $20 ChatGPT), Gemini 2.5 Pro (free on Gemini app), or Claude 4 Opus (via $20 Claude). The older models are the default and far worse. Speak & listen, don’t just type & read. I had to resist the temptation to ignore ChatGPT response when a colleague read it out. We are patient with and have respect for humans but not for AI. The value we derive requires both. Suggestion: Speak and listen rather than type and read. It’s hard to skip and easier to stay in the present. It’s also easier to ramble than type. Keep an impossibility list. There is a jagged edge that moves. When you note down what’s impossibile today and retry every month, you can see how that edge shifts. Wait for better models. Many problems can be solved just by waiting a few months for a new model. You don’t need to find or build your own app. Make context easily available. Context is one of the biggest enablers for LLMs. Use search, copy-pasteable files, previous chats, connectors, APIs/tools, or any other way to give LLMs examples and context. Have LLMs write code. LLMs are bad at math. They’re good at languages, including code. Running the code gives output with low hallucinations. This combination can solve a WIDE variety of problems that need creativity and reliability. Learn AI coding. 1. Build a game with ChatGPT/Claude/Gemini. 2. Improve it. 3. Create a tool useful to you. 4. Publish it on GitHub. APIs are cheaper than self hosting. Avoid self-hosting. Datasets are more important than fine-tuning. You can always fine-tune a newer model as long as you have the datasets. Most CDNs use package.json "exports" for the default URL of npm packages. jsDelivr uses jsDelivr > browser > main (does not use exports - a notable exception) unpkg.com uses exports.default > browser > main skypack.dev uses exports.default > module > main esm.sh uses esm.sh.bundle > exports.default jspm.dev uses jspm > exports.default > main A quick way to transcribe audio recordings is via: llm --system "Transcribe" --attachment recording.mp3 --model gemini-2.5-flash "This recording is about (context)". Providing context improves transcription, e.g. by spelling names and technical terms correctly. Since Gemini has a 1M input context, using Gemini CLI as a sub-agent from Claude Code using the -p or --prompt flag lets it crunch large code bases and pass relevant responses back to Claude Code. #ai-coding While ChatGPT Codex aligns with my minimalistic style and follows instructions very well, it also tends to remove comments in my code and oversimplifies. Jules is better than that regard. #ai-coding Teaching vibe coding is satisfying, too. I guided a developer to write a Python workflow by providing 2 prompts. Both of these were one-shotted by Claude 4 Sonnet. The entire process took 20 min with me guiding them over the phone. #ai-coding “Write a Python script to extract a page from a PDF file and save it.” Followed by “Write minimal code. Drop error handling.” “Write a Python script to pass a PDF file to an LLM for OCR and print the result. Use this code sample… [PASTED CODE].” Followed by “Write minimal code. Drop error handling.” LLM users are maturing quickly. Early adopters who are open to understand the generic capabilities of LLMs through demos are somewhat saturated. The early majority have come in. They aren’t interested in generic capabilities. They’re looking for solutions that solve their specific problem. Soon the late majority will come in asking for existing solutions that have already solved their problem for many others. How can a generic industry-agnostic technology team create demos or solutions for this early majority when we don’t yet know their use cases? ChatGPT Maintain a living “pain wiki” that teams updates daily. Create thin-slice demos that solve ONE pain-point. Re-configure with an industry skin. Result: ten demos that feel bespoke. Publish ROI, client list. Run as one-day POCs with client data. Open toolkit to partners. Track popularity of tools. Archive unused ones. Consolidate popular ones into solutions. AI closes the gap between junior & senior devs – even when both use AI. Quality doesn’t suffer much. So onboarding can be faster, compensation ladder may shorten. When using AI, developers code more and “project manage” less. Collaboration need reduces and hierarchies are likely to flatten. Generative AI and the Nature of Work #ai-coding FFmpeg in plain english lets you run ffmpeg in the browser with plain English commands. It converts the task using an LLM into an ffmpeg command, runs it in browser via WASM (without uploading the file) and saves the output locally. This is very useful, since ffmpeg has one of the most complex command line options. I use an llm template defined via: llm --save ffmpeg --model gpt-4.1-mini --extract --system 'Write an ffmpeg command' which I can use like this: llm -t ffmpeg 'Crossfade a.mkv (1:00-1:30) with b.mkv (2:10-2:20), 3s duration' OpenAI’s prompt engineering guide recommends an interesting tactic that includes this prompt snippet, which I think is very powerful. ask clarifying questions when needed ...

This week, I learned: “People are great at feedback on what you are doing wrong. They are not so good at telling you how to fix it. They don’t know you that well.” Amit Kapoor Perfect Cursors makes periodic cursor positions animate smoothly by interpolating on a spline** CloudFlare and Vercel now support sandboxes where you can execute code. The price is not so low that we can execute for free in bulk but works well infrequent or batched code execution. Simon Willison Here’s how I’m using ffmpeg for video recording & editing. To record screen at 5 frames per second, I run an abbreviation screenrecord which maps to: Gemini CLI has a generous free tier and uses Bootstrap over Tailwind Ref #ai-coding Cloudflare has a native agents SDK that looks good, especially for CloudFlare users. Ref There are several brands with recognizable chart style guides. It’s possible to generate style guides for these from the charts, but applying them via matplotlib is almost #impossible today. ChatGPT Hyperfine is like %timeit for the shell. Written in Rust ⭐ Vertical AI is a moat against AGI. Specialization reduces hallucinations. Custom workflows and regulations are sticky and defensible. We need to start selling to users, not IT, though. Ref When AI automates a task, the bottleneck shifts. AI process re-design is about reworking the process around the new bottleneck, and iterating quickly. With coding, it’s testing, reviewing, deploying, use-case identification. uvx git-smart-squash re-organizes haphazard commits using LLMs. git-smart-squash #ai-coding GitHub offers a free Docker container registry. Simon Willison There are three major areas where humans either are, or will soon be, more necessary than ever: trust, integration and taste – NYT. Anil. To deal with this: Learn things that might grow in importance, like: Data modeling APIs Code reviews Drawing and 3D modeling Narrative storytelling Design Movie making Statistics Sceptical fact checking Continuous AI auditing e.g. awesome-continous-ai or automated-auditing Zero knowledge proofs Homomorphic encryption Privacy-preserving computation Fingerprinting and watermarking Governance frameworks Ethics and AI dilemmas Negotiation Change management Remote working, management, hiring Creating attention scarcity Local cultures Work with people of growing importance People designing products in regulated industries Cross domain experts Art developers, game makers, designers System thinkers. Economists, ecologists, system planners. People who look for second order effects. Live in cities that might play a bigger role in the future Cities like Singapore and learn how it builds civics trust, creates digital IDs. Cities like Bangalore and Hyderabad and learn how they grow tech talent Creative cities like Paris, Seoul, Mexico City, Berlin, etc. on sabbaticals to taste hubs Try to: Build auditing credentials and IP Audit your calendar for what AI can do. Have it interview you Practice sceptical fact checking and audit A clever way to test a library’s quality is to have LLMs write code from docs and test it. Failing libraries have flawed code/docs. Improve. Ref #ai-coding Common Pile is an 8TB open dataset for LLM training that includes ArXiv, PubMed, StackExchange, GitHub, IRC, Regulations.gov, Patents, UK parliament, books. Easier than scraping. A useful way to have reasoning models do deep-research-like work is to have them “First, create a plan to solve the problem, clearly listing the objective, approach, and output. Then follow the plan.” DE-COP is a method to check if LLMs were trained on private content. GPT-4o was trained on O’Reilly books, based on this method. Ref LLMs are more persuasive than humans. But repeated exposure reduces the effect. Ref Phoenix.new uses live views to publish apps as it codes. The testing framework looks at the screen while it codes and fixes errors. It commits every change Anthropic system prompt asking Claude to pursue its goals led to self preservation behavior. Ref The hungrier I am the better the food tastes. A good reason to eat less quantity and frequency You can purge the jsDelivr cache manually. Helps if you released a new version of a package and way to purge an alias (e.g. https://cdn.jsdelivr.net/npm/your-package@1) XConvert is a convenient online app to compress .webm videos. Not great design but fairly good compression. You can draw a treemap of import times via python -X importtime app.py > timing.txt and then paste them at https://kmichel.github.io/python-importtime-graph/. PyOpenLayers adds interactive mapping via OpenLayers to Marimo and Jupyter. In a TechCrunch interview with Jared Kaplan has was asked if Anthropic is becoming less safety conscious because they released Opus 4 which blackmails. Kaplan replied that they have stronger testing and higher transparency, so they’re more likely to share AI dangers early. Great positioning! Conversations are about perspective change and this nailed it. The system prompts for Anthropic misalignment evals are a fascinating read. AI PR Watcher tracks GitHub pull requests from Codex and other LLMs. Codex is way ahead of anything else on volume and success rate. Devin is next on volume, Cursor is next on success rate.

This week, I learned: ffmpeg on WASM works but is unstable and hard to use. You can’t use it in a CDN without CORS issues, since it loads ffmpeg-core via a worker. It often runs into buffer allocation issues. Exotel and Plivo provide voice & SMS services in India (like Twilio). Plivo is more customer friendly. Uber’s H3, Google’s S2, and GeoHash are geocoding systems. H3 offers uniform cell sizes and better distance measurement S2 offers higher precision (factoring in Earth’s curvature) for exact location matches GeoHash is the simplest There’s a movement towards embeddable databases on the cloud. MotherDuck is hosted DuckDB. Turso is hosted SQLite (with local sync, multi-tenant) StarBase DB is SQLite with an API on top of Cloudflare Durable Objects. Software 2.0 by Andrej Karpathy. This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two: the 2.0 programmers (data labelers) edit and grow the datasets, while a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces. Adaptive UI ideas: Adaptive Fields: Show only required fields based on what the user field so far. Smart Inputs: Dropdowns and auto-complete based on user’s context. Smart Themes: Change font size, contrast, theme guessing the user’s age and preferences. Dynamic Menus: Show what they might need to do next. Like Nokia’s right button, but using LLMs. Smart Tooltips: Check what the user’s doing (delays, confusions, previous clicks, current actions) and show relevant tips. Personalized Layout: Show only the relevant sections of the app. E.g. based on what they’re doing. Smart Charts: Create the right chart that solve the user’s question. Adaptive Back-end Dynamic APIs: Create endpoints on the fly based on user needs Dynamic Indexing: Create & update indices on the fly based on user needs Dynamic Schema: Create & update schema on the fly based on user needs Dynamic Migration: Migrate to a new database or OS or language as required Dynamic Queries: Create SQL/NoSQL queries to solve the user problem Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required Dynamic Logging. Log what’s required. Explain why it’s logged and what’s happening. Fix code that raised the error Dynamic Caching. Cache what’s likely to be required. Evict what may not be required. Figure out cache keys. Aider LLM Leaderboards show which LLMs code better. As of now, o1-preview > claude-3.5 sonnet on code editing claude-3-opus > claude-3.5-sonnet on code refactoring deepseek-coder-v gpt-4o-mini sucks. Jaro-Winkler Distance is a string matching algorithm that weights the start of a string higher. Passing the feed of the following to NotebookLLM is a good way to get caught up with news and summaries. A blog / WhatsApp group (e.g. The Generative AI Group, Sithamalli, etc.) A Google Group / mailing list (e.g. genainews, datameet) YouTube channels (e.g. Vertiasium, GitHub) Hacker News top stories Research papers Emails (skipping marketing emails) OpenAI Evals and Distillation has a clever design. They just convert filtered history to .JSONL files that can be an input to either. Speak is a language learning app based on OpenAI’s Realtime API. OpenAI’s Realtime API can be used in a text-to-text chat mode without needing to send the entire context. If the pricing works out right, this can be far cheaper than sending the entire conversation context. Ref Matching addresses with just embeddings works well. Combine it with simple hard rules. Ref OpenAI’s prompt caching works for images too – both linked and embedded Quotes on Graph RAG from a Generative AI WhatsApp Group. “Damn so literally nobody uses Graph RAG yet. Good to know.” ~Sumba “A big four consulting firm uses GraphRAG to retrieve related documents and excerpts from governance and compliance docs.” ~Vinayak Hegde (Microsoft) “Graph RAG is expensive and unnecessary in most of the cases.” ~Utkarsh Saxena ChatGPT’s advanced mode includes: “…you can use various regional accents and dialects.” Ref Source But the API can “laugh, whisper, and adhere to tone direction.” Ref Hume API (INR 6/min) is far cheaper than OpenAI’s real-time chat (6c/min input + 24c/min output) Devika is an open-source clone of Devin. DuckDB runs inside Pyodide Hungarian Jews have genetic diseases that increase their IQ. Gaucher’s disease, Torsion dystonia. People don’t like hard stuff like maths or science, so richer societies have fewer scientists Ethan Mollick feels Claude 3.5 Sonnet is better at style and critiquing blog posts than OpenAI’s o1 (which is better at reasoning.) News is going to be crazily disrupted again with voice mode. I can just listen to the topic I want In Singapore Airlines, You can’t wear your seatbelt loose You have to keep the laptop in the pocket in front, not on your lap, during takeoff You can’t charge during takeoff They verify if you ask for a veg meal and place a sticker on your seat Coders are more likely to edit LLM code. Non-coders don’t have that bad habit. Vaishnavi and Ranjeet edited code Indal and Koustav didn’t Coders are likely to get more out of an LLM because they know what it can do. But some non-coders will get more out of an LLM because they don’t know what it can’t do. E.g. Indal trying for a confetti animation, which is hard but do-able “You have to put in a lot of work to become productive at AI coding.” Simon Willison

This week, I learned: Pyodide can access the DOM and JavaScript in the browser Jupyter Lite lets you run Jupyter notebooks in the browser AVIFs is about 10X better than GIFs. I tried creating one via EZGIF AVIF Maker and the .avifs file created was 15X smaller! ffmpeg -i input.gif -c:v libaom-av1 -crf 30 -b:v 0 -cpu-used 4 -tiles -an output.avif Claude 3.5 thinks .opus is the best format to compress audio. It used ffmpeg -i audio.wav -c:a libopus -b:a 16k -application voip -vbr on -compression_level 10 audio.opus API coding best practices Source via Simon Willison: Always add screenshots to the Readme. They never break. Always add every example. Human think in examples. Avoid defaults and be explicit unless 99% of the usage is with the default. Make the feedback loops incredibly fast. Make deprecations easy for users to deal with. Keep objects immutable. PyMuPDF4LLM can convert PDFs to Markdown. It handles tables, too. 04 Oct 2024. PDF-Extract-Kit does PDF layout, formula, table, and OCR extraction using various models. 04 Oct 2024. llmsherpa extracts PDF layout, tables, not OCR When evaluating feasibility of technology with LLMs always ask for multiple options and pick from those. Simon Willison Gemini supports audio natively Google Vertex AI has an OpenAI compatible API but it works only for some models. Anthropic and Gemini are not compatible. When you paste HTML into Excel, it automatically changes the font of the cell to match the content in the HTML! Aptos is the new default font in Office - replacing Calibri. Anthropic’s Introducing Contextual Retrieval says: Use BM25 in addition to embeddings to match rare terms (e.g. identifiers) Add a context to each chunk’s metadata (generate it with a cheap LLM) and pass it to the summarizing LLM Reranking helps with cost AND accuracy. Use Cohere or Voyage Sentient lets you control the browser via Python in natural language