I explored ffmpeg on WASM, geocoding systems like Uber’s H3, and embeddable cloud databases like MotherDuck and Turso. I also examined Software 2.0 paradigms, OpenAI’s Realtime API, and the evolving dynamics of AI-assisted coding for different skill levels.

This week, I learned: ffmpeg on WASM works but is unstable and hard to use. You can't use it in a CDN without CORS issues, since it loads ffmpeg-core via a worker. It often runs into buffer allocation issues. Exotel and Plivo provide voice & SMS services in India (like Twilio). Plivo is more customer friendly. Uber's H3, Google's S2, and GeoHash are geocoding systems. H3 offers uniform cell sizes and better distance measurement S2 offers higher precision (factoring in Earth's curvature) for exact location matches GeoHash is the simplest There's a movement towards embeddable databases on the cloud. MotherDuck is hosted DuckDB. Turso is hosted SQLite (with local sync, multi-tenant) StarBase DB is SQLite with an API on top of Cloudflare Durable Objects. Software 2.0 by Andrej Karpathy. This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two: the 2.0 programmers (data labelers) edit and grow the datasets, while a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces. Adaptive UI ideas: Adaptive Fields: Show only required fields based on what the user field so far. Smart Inputs: Dropdowns and auto-complete based on user's context. Smart Themes: Change font size, contrast, theme guessing the user's age and preferences. Dynamic Menus: Show what they might need to do next. Like Nokia's right button, but using LLMs. Smart Tooltips: Check what the user's doing (delays, confusions, previous clicks, current actions) and show relevant tips. Personalized Layout: Show only the relevant sections of the app. E.g. based on what they're doing. Smart Charts: Create the right chart that solve the user's question. Adaptive Back-end Dynamic APIs: Create endpoints on the fly based on user needs Dynamic Indexing: Create & update indices on the fly based on user needs Dynamic Schema: Create & update schema on the fly based on user needs Dynamic Migration: Migrate to a new database or OS or language as required Dynamic Queries: Create SQL/NoSQL queries to solve the user problem Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required Dynamic Logging. Log what's required. Explain why it's logged and what's happening. Fix code that raised the error Dynamic Caching. Cache what's likely to be required. Evict what may not be required. Figure out cache keys. Aider LLM Leaderboards show which LLMs code better. As of now, o1-preview > claude-3.5 sonnet on code editing claude-3-opus > claude-3.5-sonnet on code refactoring deepseek-coder-v gpt-4o-mini sucks. Jaro-Winkler Distance is a string matching algorithm that weights the start of a string higher. Passing the feed of the following to NotebookLLM is a good way to get caught up with news and summaries. A blog / WhatsApp group (e.g. The Generative AI Group, Sithamalli, etc.) A Google Group / mailing list (e.g. genainews, datameet) YouTube channels (e.g. Vertiasium, GitHub) Hacker News top stories Research papers Emails (skipping marketing emails) OpenAI Evals and Distillation has a clever design. They just convert filtered history to .JSONL files that can be an input to either. Speak is a language learning app based on OpenAI's Realtime API. OpenAI's Realtime API can be used in a text-to-text chat mode without needing to send the entire context. If the pricing works out right, this can be far cheaper than sending the entire conversation context. Ref Matching addresses with just embeddings works well. Combine it with simple hard rules. Ref OpenAI's prompt caching works for images too -- both linked and embedded Quotes on Graph RAG from a Generative AI WhatsApp Group. "Damn so literally nobody uses Graph RAG yet. Good to know." Sumba "A big four consulting firm uses GraphRAG to retrieve related documents and excerpts from governance and compliance docs." Vinayak Hegde (Microsoft) "Graph RAG is expensive and unnecessary in most of the cases." Utkarsh Saxena ChatGPT's advanced mode includes: "...you can use various regional accents and dialects." Ref Source But the API can "laugh, whisper, and adhere to tone direction." Ref Hume API (INR 6/min) is far cheaper than OpenAI's real-time chat (6c/min input + 24c/min output) Devika is an open-source clone of Devin. DuckDB runs inside Pyodide Hungarian Jews have genetic diseases that increase their IQ. Gaucher’s disease, Torsion dystonia. People don't like hard stuff like maths or science, so richer societies have fewer scientists Ethan Mollick feels Claude 3.5 Sonnet is better at style and critiquing blog posts than OpenAI's o1 (which is better at reasoning.) News is going to be crazily disrupted again with voice mode. I can just listen to the topic I want In Singapore Airlines, You can't wear your seatbelt loose You have to keep the laptop in the pocket in front, not on your lap, during takeoff You can't charge during takeoff They verify if you ask for a veg meal and place a sticker on your seat Coders are more likely to edit LLM code. Non-coders don't have that bad habit. Vaishnavi and Ranjeet edited code Indal and Koustav didn't Coders are likely to get more out of an LLM because they know what it can do. But some non-coders will get more out of an LLM because they don't know what it can't do. E.g. Indal trying for a confetti animation, which is hard but do-able "You have to put in a lot of work to become productive at AI coding." Simon Willison

Things I Learned - 06 Oct 2024


I explored CSS nesting, AWS Lambda performance patterns, and how rain improved WiFi signals. I also learned about AI package hallucinations and summarized Paul Graham’s insights on using low standards and iteration to improve writing quality.

This week, I learned: CSS nesting is now available in browsers Cold starts in AWS Lambda: serverless functions stay alive for 5-7 min. All languages are fast but Docker is slow. More npm packages slow start dramatically. WiFi only works when it's raining because a tree was obstructing the signal but was weighed down when raining! Good reasons why finding a technical co-founder won't work. You want a unicorn to passionately trust YOUR idea after 2 meetings. Why should THEY risk money for YOUR idea? You're the money guy. RAISE the money for YOUR idea! How passionate are you about software? And you want to build one now? This is a subtle vulnerability. ChatGPT hallucinated pip install huggingface-cli. Sosomeone created the package and got 30,000 downloads! Video-Llava is a video LLM MusicCNN-embeddings provides embeddings for music genre classification How I write podcast. Paul Graham essays Write simply. It helps communicate. (Don't concise if communication worsens.). It forces you to make the idea better Do lame stuff. Else you won't start. Low standards drive creativity The more to delete, the better your writing. Read your piece. Highlight what feels poor. Fix it. Ask friends to highlight what's BORING? UNCONVINCING? Delete the first, brainstorm the second. Or ask, what's the 10% to cut and 10% to keep. Write about stuff you don't know above the. Writing GENERATES ideas Write about what's BUS. GENERAL and SURPRISING. (Laughter is a sign of comprehension.) Do HARD things to cultivate taste. Spend more time with people who generate ideas in you. Ravi chithappa. Ram. Ankor. Ganes. Books! Build taste. I have a taste for picking technologies. Data visualization. Retrospect. Write down what you like and dislike. Copy what you REALLY like. Guilty pleasures. A benefit of lower standards is that it let's you pick the path less travelled. ITERATE. Discuss ideas. Iterate. Acknowledge. ITERATE.

Things I Learned - 07 Apr 2024


I explore the history of TCS Ignite, Raman Srinivasan’s insights on Indian space tech, and the leadership style of S. Ramadorai. I also cover GPT-4's math limitations, new AI education tools, and technologies like GitOps and LMQL.

This week, I learned: Raman Srinivasan: IITM Profs and MTechs are spinning off deep tech startup. Agnicool is an example. They 3D print rockets with ceramic composites from Germany Sriram Krishnan (Facebook), Balaji Krishnan invested in pre-Series A Govt is de-regulating space tech and geospatial. Talking of de-regulating nuclear. ISRO seems to be focusing on cutting edge while others are doing commercial stuff There are about 100 space tech startups in India You can build your own modular reactor Geospatial AI is a big opportunity Have released a lot of 10m resolution geospatial data almost for free success is about getting NO factor wrong. Failiure just requires one aspect to fail. Brand, business savviness, financial stability, tech superiority, deep pockets, managing Gvt, long-term mindset, etc. - all of these matter. That's what made TCS monopolize the exam business in India. For deepening AI, we need, Talent, Data pipelines, Hardware Next wave is LMMs, not LLMs What's not captured in LLMs is verbal knowledge and tacit knowledge (in people's fingertips). India is rich in this. The road to tacit knowledge has to go through India We can get a welder to train a simulator and pay the welder We can get a storyteller to tell a few stories and train oral LLMs Tacit knowledge will have to cover robotics. Train robots to bring coffee in just 50 demos! "Project delays are within the 'rulebook'. Buyt paying skilled welders for ship building or nuclear pressure boilers needs breaking 100s of rules. Once they get certified, they abscond to Iran or somewhere." TCS Ignite started in 2006 by Ramadorai. Before recession. "There is going to be a talent shortage. Recruit from next rung. Science not engineering graduates. Break HR monopoly and corruption - colleges became placement agencies. Fewer people per college. Across the country. Train them." Tried in 2000. HR refused. Business refused. When Chandra was identified, Ramadorai took it up himself as a challenge. Ramadorai had very precise attention. Sat 7 am calls. "What are you doing?" 2 min call. Enough to energize. Would exchange and ask for brief updates. He reads and responds. You get a decision in a few hours early in the morning. No decision bottleneck He wanted to know ALL the details. Very precise, small, frequent probes on what's happening. E.g. one 6 am, he called. "What are the lectures planned for today?" He expected I would know this. If not, next time I would be prepared. He would call another person and ask the same question. So I updated the others. I've never seen anyone with that bility to ground-truth. He wants 10 birds from 1 stone. Get BSc, but don't comprimize. Get the best 2 per college but a full batch size of 500. We became the biggest training program as a single batch -- with 500 people. He wanted to demonstrate scale. HR and CFO said, 'You recruit first. Then we'll give you money. We don't think it's possible." We had anchor colleges and brought people from other campuses. We did digitized exams. Took big servers to the campus. Fully digitized with full auditability. Plugged the laptops into the college LANs. Kids had never used a mouse. We had to teach them. We said, "Don't worry. These are logical questions, not questions. We'll pay a full salary." We learned that 1 out of 2 didn't even join. Many took up a Masters. They didn't want to join the workforce. Unless they're desperate economically. Even poor parents, if they can afford to support you at home, they do that. It's weird. Every weekend, we visited a few campuses. 71 locations across the country. Found the NSS college in Ottapalam (Kumbakonam of Kerala. Cultural centre.) College had a nice nice Math dept website. I said "Mr Ramadorai, this looks promising." One Sat morning, he called and said, "When are you going to Ottapalayam?" We landed in the college. There was an impromptu communist student strike. We made 38 offers out of 100 who took the exam. Never had such a high conversion. One girl, whose father was a coolie, jad communication issues. Had a colleague talk in Malayalam. She was an amazing success. My colleague Murali made a documentary about her. We started in July. By Dec, we had 500 joinees. No one is doing such a thing now. You have to get dozens of things just right. Compromising on even one kills it. Ramadorain loaded it with multiple objectives. Fresh talent. Low cast. Sustainable. He kept pushing for innovation. I pushed back. But he was persistent. Over time, I came around and we started innovating. We restructured training program around innovation. Like a YCombinator. That unleashed extraordinary energy. Several of the kids are running their own startups. Ramadorai was very supportive of that. The assessment product came out of that. First batch, everyone was very sceptical. We got a lot of pushback. They're dumb. Ethics issues. Communication issues. Lot of prejudice. So we got them to do internal recruitment till they were satisfied. An internal placement market. THEN reputation was set. I told them, always stick to the dress code. One weaver's sone wore a bright yellow polyester T-shirt. I asked him why he didn't stick to the dress code. "Sir, it's my first T-shirt." Ramadorai tracked how many became billable. We were unable to place 70. He said, give them 1 more month training. Then we placed 64 of the 70. He said "Do something about the 6. I want 100% placement." We absorbed them as a teaching assistant. One was a weaver's son. One was a PC's daughter. A mestri's son. A shopkeeper's daughter from North Madras. None could speak English. They learned to code and helped build the exam software, with Srikumar who was a brilliant Java coder. That gave us the confidence that these are good kids, just from the wrong part of town. With a good guide, they're very capable. We bought a bunch of Nintendo Wiis. Kids have to play. He asked for a welding simulator. "Velu the Welder". The kids built it using the Wii. We got the most accomplished welder spend an afternoon at Ignite. He ripped us apart. 4 hrs non-stop. He told us EVERY thing wrong with it. Blasted us. I told Murali, "Let's call it a toy. It's not a simulator. Let kids play." He said, "I want to show that it can be done!" Murali churned out rapid iterations in a frenzy. Ramadorai said, "Deploy it in the field." So we went to all kinds of remote places like Gondiya below Nagpur. Surprisingly cosmopolatan. Junction of EW and NS train lines. We set up welding institutes in each. It was on the cloud. We could track everything. KPK killed the skills. Hard core bureaucrat. His view is colonial. Ignite philosophy is about unleashing energy of people. Colonoial model is about controlling people by keeping them poor. KPK and Chidambaram had that mindset. Ramadorai brought him in as Secy of NSDC. he killed the policies Modi did the first cut by creating a ministry. KPK ensured that it never gew. Like Yes Minister. Made sure nothing moved Had Govt not changed, he would have been Secy Finance. He was seen as Chidambaram's blue eyed boy. People know he was associated with NSE scam. Ramadorai helped by bringing him into skills He is very smart. Knows the IAS machinery in and out. Lives and breathes that. H Ramadorai likes him though. Put him on board of Tata Consumers. NSE Scam. He's part of the cabal with Ajay Shah. Private trading firms could co-locate within NSE and could make a huge amount of money. KPK ran some of this by proxy to fund Congress. But he left no fingerprints. But everyone knows it is him. He was running Chitra Ramakrishnan by proxy. He was the Himalayan Yogi. Ignite continued with unwavering focus. Kept increasing the kind of focus. We had a 99.5% success rate in placements. Just a handful of failures. Ramadorai has written about Ignite in "The TCS Story". My Dad translated it in Tamil. It's not a typical business biography. Worth reading. Should be a mandatory course in MBA courses in India. So many lessons. You have to read it knowing how Mr Ramadorai speaks. What is NOT said is just as important. Ch 5 is the thinnest - on the IPO. It is packed with so much stuff. Unless you know, you won't understand. "Tatas got the Govt to change a tax law to make the IPO meaningful." Behind that, there's a lot. You have to be alert to catch the sentene. He won't brag, or talk about the significance of some of these. Book is packed with dense insights. Unless you ARE LOOKING FOR IT, you'll miss it. Worth reading SEVERAL times. You need a foot-noting. Currently reading Pasquenelli -- Social History of Artificial Intelligence. Eye of the Master. Worth reading. I'm not Marxist by belief but they get some things right. Surprised how vibrant the European left is. "If someone is doing manual work, there is tacit knowledge that automation captures." India doesn't need self-driving cars. But a farmer would like a gaming controller that ploughs his fields while he sits under a tree. Semi-intelligent machines that removes the burden of hard labour in the country. Once a year, for a few weeks, I do manual labour. People are under-nourished. People typically work 5 hours a day. Not enough muscle mass. So use them for what they're good at I've seen the power tools. When Chinese power tools became cheap, the power welding became much more efficient. Everyone has become a monkey with power tools. They charge per inch. They know how to leverage the tech for economic benefit. Just bring in the power tools and rapidly finish and make money. But there are sections that are still poor and haven't made the transition. How can we create pathways for them? How can AI help? Anand: Why not use a gimball. RS: Good idea. Role modern psychologist DW Winnicott on ChatGPT (like Socrates) E.g. You don't need a perfect mother. A good enough mother is better Similarly, why not a "good enough" Bharat mata than a perfect one? To persuade someone, align it with their identity. ChatGPT 5 technologies of interest according to Gartner's latest hype cycle: GitOps Internal Developer Platform Graph Data Science Open Source Program Office Value Stream Management Platforms Gemini is an alternative to the Web. Sort of like Gopher, but recent SALI - Standards Advancement for the Legal Industry - has standards and ontology/taxonomy for legal documents, including patent litigation. Walking new routes habitualizes fighting fear and preferring novelty ⭐ GPT-4 is bad at math. It gets 60-70% of answers wrong. LMQL provides a constraint-based query language for interacting with LLMs. It uses token masking, which is clever. Hollywood writers signed a deal that limits AI in script writing. It's primarily aimed at protecting script writer wages. Adobe Firefly offers a "generative fill" that lets you remove or paint new objects into an image. I'm awaiting text to vector images. Duet AI is Google's answer to Github Copilot. Teachers are using LLMs to plan lessons, write emails to parents, create tests, adjust reading level of materials, personalize content with tools like MagicSchool, Diffit, Eduaide. WizardLM creates datasets for instruction tuning by cleverly using LLMs to create new prompts. Deita is an approach to improve instruction tuning datasets. Dhyeya: Attack on Titan is as good at Death Note Jaidev: Long car drives are a good place to explore new song genres. Try in taxis Same radio channels may have different frequencies across cities. Vividh Bharati is 100.5 FM in Chennai and 106.4 in Delhi Things to explore: Radio for new songs Clubhouse Twitter Spaces Instagram reels YouTube reaction videos (e.g. atheist, Indian songs, etc.) Stand-up comedies (Ricky Gervais, Louis CK, Jordan Peterson) Porn artists are at risk because of Gen AI

Things I Learned - 07 Jan 2024


I learned how Predibase uses LoRAX to serve multiple fine-tuned model adapters on a single GPU. This method leverages LoRA adapters to optimize hardware usage and scale specialized models more efficiently than traditional deployments.

This week, I learned: Predibase uses LORAX to run multiple fine-tunings of a base model in a single GPU via adapters. Ref

Things I Learned - 07 Jul 2024


I investigated Amazon Nova model costs, surveyed why employees avoid internal LLMs, and evaluated JavaScript text splitters. I also learned about Unicode characters in ChatGPT citations and how to install Docker on Windows without admin rights.

This week, I learned: ChatGPT uses several unusual unicode characters for citations. Ref NumLock can be dangerous. An IT support team member took control of Radheya's screen while debugging and had turned on NumLock. Radheya's login failed after that. After 5 tries, he was locked out. With LLMs, most architectural decisions are no longer one-way doors. Steve Yegge The cost of intelligence is trending to zero. How do we plan for this? Logan Kilpatrick If you are not planning for the price of intelligence to go to zero, the next 3-5 years are going to incredibly disruptive to your business / life. The important but not stated caveat: consumer willingness to pay for AI is going to go up (a lot). It will be fascinating to watch consumer willingness, cost, and the amount of AI being used all move in different directions. Everyone building things with AI has an economic incentive to limit the amount of AI because of cost, which inherent limits the value prop. This will change as intelligence goes up and cost goes down. What this means is: Admin automation: Administrative tasks vanish into background AI. Booking meetings, managing finances, or even planning family activities will require less thought. Hyper-personalization: Individuals get tailor-made everything—from medical advice to product recommendations to daily schedules. Systems learn your quirks. AI co-brains: AI co-worker “assistants support you at any moment. Productivity soars in knowledge work. “I’ll have my AI follow up becomes a normal response. Humanity valued more: As AI handles rote tasks, humans move up the value chain, focusing on creativity, empathy, or the “last-mile decisions. New business models: AI experts as a service Embedded AI Solutions AI micro-services for smart-calls Distributed AI Arena Hard is a set of hard prompts to test LLMs. Here is the code and evaluation LLMs can detect clear outliers easily. PROMPT: Which is the outlier in this dataset: (1,7), (2,7), (3,6), (4,6), (5,5), (6,1), (7,5), (8,3), (9,1), (10,1) (ANS: (6,1)) 🟢 GPT-4o on ChatGPT gets this. GPT-4o Mini on the API gets it too. 🟢 Gemini Pro, Flash, Flash 8b gets this right straight away, without even thinking. 🟢 Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3.5 Haiku get it on LLM Foundry. 🔴 Claude.ai, where it visualizes it and gets it wrong. 🟢 Nova Micro, Lite, and Pro get it right. 🟢 Llama 3.1 70b gets it right. 🔴 Llama 3.2 8b gets it wrong. Llama 3.2 70b, Llama 3.1 8b enter repetition. To install Docker on Windows without admin privileges, use net localgroup docker-users "your-user-id" /ADD A non-administrator in a Google Groups domain can only add 200 emails to a group from the UI directly without invitation at a time. The only programmatic way to add users is for an administrator to add them. Even apps that use the Google Admin SDK need an admin to log in to access the relevant API. Take 100% of your work, including complex, multi step processes and put it into an LLM. It might fail at some but you will discover the limitations. I emailed Straive employees about their use of LLM Foundry - the internal LLM portal. I picked 500 non-users from teams that otherwise have high (30%+) usage. Reasons they didn't use it were: 40% had not heard of it. 40% were unclear of the benefits 20% didn't have time 45% feel they don't have enough information and training to use it Some feedback Sharing training videos will help Live training sessions that allows for Q&A will help Developers prefer detailed documentation The same prompt gives different results Possible solution: Email non-users introducing the tool and sharing a quick 15-minute tutorial and a 1-page quick start. My notes on the Amazon Nova models. More on Hacker News Nova Micro (3.75c/MTok) has the same cost as Gemini 1.5 Flash 8b but does not support images or documents. Nova Lite (6c/MTok) has about the same cost as Gemini 1.5 Flash 002 and supports images and documents (but not audio or video). It may be a good alternative. But GPT-4o mini, which is 2.5X costlier, is much better. (It partly passes the Gr brx vshdn Fdhvdu flskhu? test which Nova Lite fails.) Nova Pro (80c/MTok) is cheaper than Gemini 1.5 Pro and a lot cheaper than GPT 4o, but does not match their quality. LLMs are great at convincing you of wrong things. A danger and something to be wary of. Ethan Mollick Fish eye text summary is a great way to read text while summarizing context. Amelia Wattenberger DuckDB's JavaScript API is still under development. For example, JSON, ARRAY are not insertable. Plus, re-creating persistent HNSW indices crashes. What's a good text splitter library to use in JS? LangChain: If you use it, use it with a simple wrapper decoupled from the implementation (e.g. your own parameters) that you can replace later. Popular Fit-for-purpose. MarkdownTextSplitter which inherits from RecursiveCharacterTextSplitter is what's needed in most cases. Unstable Poorly maintained Python docs indicate version 0.0 but it is in 0.1 Under-maintained Last update was 3 months ago, 13 Sep 2024 LlamaIndex: Popular Not an ideal fit. MarkdownNodeParser does not support chunk size. SentenceWindowNodeParser does not capture Markdown headings.

Things I Learned - 08 Dec 2024


I benchmarked RAM and CPU usage across FastAPI, Node, and Deno, while exploring several video and audio generative AI tools. I also identified whisperX for diarization and tested Reflection 70b's internal reflection mechanism for improved model accuracy.

This week, I learned: When running a Hello world app: FastAPI takes 26K RAM, 3% CPU NodeJS + Express takes 62K RAM, 2% CPU Deno + Express takes 62K RAM, 1% CPU Deno + Fresh takes 54K RAM, 0.4% CPU I was testing out different video LLMs: Luma Labs lets you create videos from text Runwal ML lets you create video from an image + text Viggle lets you add images to a video or move a character in a certain way Veed.io is a video editor that offers AI video editing features Deepmotion generates 3D animations from video Wonder Dynamics may be similar to DeepMotion I tested out a few audio LLMs: Suno is fast, has a better UI, lots of examples Udio is slow, poor UI, creates richer music Reflection 70b is one of the top models now, and is open source!. It works by making the LLM reflect on its answer inside ... tags. The best diarization model today is whisperX. Run on Colab T4 GPU with: Scale's SEAL Leaderboards seem fairly good. coedit-xxl is Grammarly's fine-tuned google/flan-t5-xxl model run on CoEdit - text editing dataset. It's mainly for single-line editing, though, and far from a full-document or full-email zero-shot editor.

Things I Learned - 08 Sep 2024


I explored how httpretty mocks Python HTTP libraries and learned about Japanese ancestral worship practices. I also researched GPT-4o's use of CNNs and OCR for image embeddings and visualized sinusoidal series as geometric spirograms.

This week, I learned: httpretty can mock ALL Python HTTP libraries Japanese pray to dead parents instead of gods. The dead are preserved in plates by priests. Japanese are generally non religious Looks like GPT-4o is using CNNs to create vector embeddings of images, with images gridded into a 1x1, 2x2, etc. PLUS OCR. Ref The sum of a sinusoidal series is like a spirogram. Spinning circle linked to another and so on

Things I Learned - 09 Jun 2024


I reflected on why data licensing is the new business model for Reddit and StackOverflow, the impact of usability in developer tools, and the internal culture shift needed at Google following recent AI controversies.

This week, I learned: Mughals just replaced the top of most temples with Mosque domes as part of the conquer or die policy "Math is racist". Because people who can't solve it can't because of their underprivileged background! Winners: commodity businesses, companies that own lots of data like Reddit and Stackoverflow, profitable bootstrapped businesses Making a tool more usable, e.g. a video, can have a 10-100X impact. Yet every developer thinks it's redundant All in one podcast. Can Google save itself? the success of a developer platform is the number of people using it. But not everyone uses it equally. Some people create winning products which drives attention to the platform. Use llm proxy like that to measure weekly average users and cost saving through caching one week ago, if someone at Google stood up and said we have too many black people in our images, the responsible AI team would have shut them down calling them racist. They had too much power and it was a one-way conversation. With the backlash now, there is a lot more awareness and acceptance of the balance. Security is like that. It's too easy to empower and shut things down until there is a backlash the lawyer's job is to tell you what's not possible. But like Travis, your job is to decide whether it's worth the risk of running a taxi company without a license or not Americans pronounce Sundar pichai's name as Sun Daar! data licensing has become a business model. Reddit, Accel springer, stack overflow and many others are licensing their content to Google and open AI for several million dollars.

Things I Learned - 10 Mar 2024


I explored OpenFreeMap and Zapier Actions for GPTs, then researched LLM vision use cases in energy and manufacturing. I also looked into IBM's Docling for PDF conversion and Hamel Husain's framework for building effective LLM-as-a-judge systems.

This week, I learned: OpenFreeMap is a free embeddable OpenStreetMap tile server. You can use MapLibre GL (more features) or Leaflet (simpler) to render it. It offers styling and self-hosting. Zapier Actions are an easy way to set up custom actions like GMail / Google Calendar APIs for GPTs, since GPTs' callback URLs keep changing. But they fail often, and don't work on mobile. At least for me. LLM Vision Use Cases in manufacturing and earth sciences (via Shivku) Automated geoscience image descriptions Ref Interpret Wind Turbine photos and charts, construction monitoring, equipment maintenance & charts Ref Forecast weather based on cloud photos! Ref Analyze thermal image of solar panels, electroluminescence images for warranty claims, ROI estimates from Google Sunroof rooftop images Ref Corrosion detection in electricity towers, turbines, storage tanks, penstock. Interpret non-destructive test images Ref Google counts auto-completion when saying "25% of all the code is written by AI at Google". "It's a helpful productivity tool but it's not doing any engineering at all. It's probably about as good, maybe slightly worse, than Copilot." YCombinator Workflow for AI video creation: Use Meshcapade (meshcapade.com) to generate body movement of a 3D-rendered character. Pass that video to Runway's video-to-video model to generate any visual. Add music from Suno Ref Someone sorted the X and Y columns independently for regression. Ref Android keyboard learning only sends model changes back to server and not local keywords. Model changes are aggregated! Ref Here is a prompt for audio transcription using Gemini. Ref Transcription: Accurately transcribe the audio clip in the original language. Include all spoken words, fillers, slang, colloquialisms, and any code-switching instances. Pay attention to dialects and regional variations common among immigrant communities. Do your best to capture the speech accurately, and flag any unintelligible portions with [inaudible]. Translation: Translate the transcription into English. Preserve the original meaning, context, idiomatic expressions, and cultural references. Ensure that nuances and subtleties are accurately conveyed. Capture Vocal Nuances: Note vocal cues such as tone, pitch, pacing, emphasis, and emotional expressions that may influence the message. These cues are critical for understanding intent and potential impact. Here are some approaches to large-scale classification of medical codes. ChatGPT Fine-Tuning LLMs on Medical Data: Enhance LLMs by training them on medical datasets, such as clinical notes and discharge summaries, to improve their understanding of medical terminology and context. Multi-Agent Frameworks: Implement a multi-agent system that simulates real-world coding processes with distinct roles (e.g., patient, physician, coder, reviewer, adjuster). Each agent utilizes an LLM to perform specific functions, enhancing interpretability and reliability. ArXiv Retrieve-Rank Systems: Develop a two-stage system where the LLM first retrieves potential ICD-10 codes and then ranks them based on relevance, improving precision in code assignment. ArXiv Embedding-Based Approaches: Use LLMs to generate embeddings for ICD-10 codes and medical texts, facilitating the matching of texts to appropriate codes through similarity measures. GitHub Hierarchical Classification: Leverage the hierarchical structure of ICD-10 codes by first classifying texts into broader categories before assigning specific codes, reducing complexity and improving accuracy. ArXiv Two-Stage Verification Models: Combine LLMs with verification models, such as Long Short-Term Memory (LSTM) networks, to validate and refine the codes suggested by the LLM, balancing recall and precision. ArXiv Also, a mixture of models approach might work. Feed any existing NLP model / rules as a second opinion. GraphRAG is better if data is naturally graph-structured. Else, it's slow and fills up the context window with even vaguely related stuff. Vigneshbabu, AMAT. ChatGPT for Windows desktop supports real-time voice and a global shortcut (Alt Space). uithub converts GitHub repos to Markdown. Just replace "g" in "github.com/..." with "u". Example WebContainers are a thing and Bolt.new uses them! Docling by IBM converts PDF, DOCX, etc. to Markdown. Like PyMuPDF4LLM but better. Check out Loom and Cleanshot are the recommended tools for screen recording and screenshotting. But Loom is paid and Cleanshot is Mac only. The Rubik's cube has a Hamiltonian cycle through every one of its 43 quintillion states. Ref OmniParser is great at parsing screenshots and identifying bounding boxes. Recraft.ai is currently SOTA in text to image. It's fairly impressive and could be a good alternative to Figma. Zed.dev is an AI code editor by the creators of Atom. It's written in Rust and is blazing fast. It has native AI integration. Artificial Analysis has a bunch of new leaderboards and arenas. Open AI TTS leads the TTS Leaderboard. ElevenLabs is a bit behind. Recraft V3 > Flux 1.1 leads Text to Image Leaderboard Hertz-Dev is an open source realtime voice chat model. But it doesn't fit in Google Colab T4's RAM Chain of Thought reduces performance where thinking makes humans worse. Ref. Specifically: Artificial grammar learning Facial recognition Classifying data that has exceptions Creating a LLM-as-a-Judge That Drives Business Results by Hamel Husain. Get THE domain expert (or approver) as the tester. Create a dataset that is DIVERSE. Covers EACH combination of: Features Scenarios: e.g. multiple matches, no match, ambiguous request, invalid/incomplete input, unsupported feature, system error Persona: e.g. new user, expert user, non-native speaker, busy professional, technophobe, elderly user Generate data using existing data + synthetic data for each SPECIFIC combination of the above Evaluate based only on PASS/FAIL with a CRITIQUE detailed enough for a new employee. Include: Nuances: Something a failed response did well or a passed response didn't quite do well Improvements: Suggest how model can improve Build an SPA to make it easy for the domain expert to review LLMs can be made to unlearn (copyright material) better by identifying components related to the knowledge to unlearn and applying a larger learning rate to these while leaving other parts unchanged. As opposed to low learning rates for all components. Ref

Things I Learned - 10 Nov 2024


I explored Agentic RAG for complex retrieval, fine-tuning with LoRAX, and practical LLM strategies. Key takeaways include using N-shot prompting before scaling models, automating workflows via disposable apps, and leveraging context caching to significantly reduce inference costs.

This week, I learned: Embedding models can be fine-tuned. Example: #TODO Agentic RAG (Ravi Theja, LlamaIndex) RAG via top-k retrieval fails with summarization => need to read all chunks comparison: compare product X vs Y => need to split and re-combine structured analytics. e.g. most expensive employees => Text2SQL first multi-part questions. e.g. Tell me about speed of model X AND cost of model Y and recommend => need to split and re-combine RAG failures: It's single shot. No query planning. No tools. No correction. No memory. Agents that help in RAG Route to the right tool E.g. retrieve via vector top-k search or vector summary search or keyword search or combination? One-shot query planning E.g. Break query into multiple specific queries. RAG those. Then combine. #TRY - maybe in DocSearch Tool use E.g. Schema retrieval, Text2SQL, Calendar, Chat, APIs, Search, etc. Agent orchestration ReAct: An agent reasoning loop. Reason + Act. {Thought, Action, Action Input, Observation}\. Orchestrate tools with a prompt Multi-agent task solver: Llama agents Instead of a single agent loop, use different agents. Also allows parallelization Allow services to register. (MS TaskWeaver stores tool descriptions in YAML) LlamaHub Tools has ideas for agents Notes on LLM Fine-Tuning Rouge 2 and Bleu and such metrics are NOT good. Create you own benchmarks Non-PEFT fine tuning needs 6X GPU RAM. Optimizer states, Gradient, Activations are the overhead. PEFT is about tuning a subset of parameters. LORA adds additional weights without updating the model. It's a low rank matrix multiplication. You can change these adapters in runtime. Saves space. Fast to train Quantization: Stick to bitsandbytes or AWQ (may be a bit better) QLORA = Quantization + LORA Predibase has open-sourced Lora Adapters in "Lora Land". Existing adapters are pretty good. ghcr.io/predibase/lorax:main Docker image works on Docker compose to run locally. devices: on Docker Compose lets you specify NVIDIA GPU devices Locust is a HTTP load testing lib in Python Techniques for inference optimization Dynamic adapters: Loads right LORAX adapters WHEN a request comes in Multi-adapter batching: Process all inputs in parallel on the same GPU, but different users are post-processed using different adapters Notes from a 4-hour flight: What We’ve Learned From A Year of Building with LLMs Strategy IS IT TOO HARD/EXPENSIVE? Log it. LLMs are getting cheaper and better. WILL OPENAI BUILD IT? If so, wait for it instead of building. HAS A STARTUP BUILT IT? If so, use it instead. It's a generic use case there's no point re-inventing. FOCUSED USE CASES over generic. Build trust by starting small. Tools for LLM Ops (feedback): LangSmith, Log10, LangFuse, W&B Weave, HoneyHive #TRY Human in the Loop is about humans evaluating model outputs. That's different from AI in the loop, human in the center, where AI accelerates human output (like Github Copilot) Operations CHECK EMBEDDINGS DRIFT over time. Users might be input-ing different things than before. LOG AND REVIEW everything. Instructor coaxes structured output from LLM APIs. #TRY IMPLICIT FEEDBACK collection is easy. Just let users edit stuff. #TRY Tactical Try n-shot prompting (n=5-12) before bigger models. #TRY Always structure for output: Markdown, XML/HTML tags. Combine RAG with Keyword search. It reduces user frustration in edge cases. Prefer multiple small prompts to one big prompt. Do X. Then Y. Then Z. Jitter prompts for diversity beyond temperature. LLM-as-judge works better when comparing outputs (not rating 1 output). Keep length similar (LLMs prefer wordiness). Swap order and compare. Allow for ties. Ask for reason FIRST. Hermes: A Text-to-SQL solution at Swiggy "Hermes performed significantly better for charters with well-defined metadata and a relatively smaller number of tables." "We collect feedback on the accuracy of the returned query from stakeholders directly within the Slack bot." How I use AI and "Replacing my right hand with AI" EMBED in every app/workflow. E.g. Auto-fix spellings. Auto-review code. Auto-ask LLM on errors and apply patch! Auto-search for answer, assess, continue. PERSIST. Stick with the LLM to the end. Don't fix it yourself. It's faster. #TRY INTERVENE FAST. If an LLM can't solve it by itself in 2 tries, it needs in-depth help. APP-IFY one-off tasks. Disposable tools. "Write web-app to convert JSON to tab-delimited." "Extract fields as a table." "Diff JSON." #TRY BEST language/frameworks preferred. CUDA in Python. Rust. C. Raspberry Pi. Arduino. Bluetooth. Modern ESM/JS. #TRY TEACH examples. "Here's the LLM Foundry API." "Here's how to use gramex.data." DUMP entire code. Models can handle it. Refactoring to SQLAlchemy 2, Pandas 2. API Documentation. Test case generation. #TRY ASK for features & packages. Docker without root access. GPU access inside docker. Windows CLI-only C++ compiler. TEST CASE writing. #TRY SPEC IN DETAIL. Use these libraries. Write like this: code example. SPEC USAGE in detail. "I will just pipe it into sqlite", or "I will just run ffmpeg -i filename [YOUR OPTIONS]. Describe the UI, API input/output, data structure, and internal data structure. HELP on usage. "ffmpeg to get audio.mp3". My benchmark for large language models LLM(text) is a useful function to have in JS and Python too. Useful as a simple pip install llmfoundry Allow images, files in LLM() Current list of #IMPOSSIBLE (or hard) things for LLMs Translate technical documents to Dutch -- because they don't understand the technical terms well Translate large documents (JSON to XML, English to Chinese, Python to Rust, Wrong to right spelling) -- because the output tokens are limited micro-agent generates test cases first when asked to build an app. Then it iterates until the test cases pass. Alternative interfaces to YouTube: Piped.video, CloudTube, Invidious, NewPipe, FreeTube Deepseek Context Caching reduces price to 1.4 cents/MTok for portions of chat messages that are repeated. That's a 10X reduction for long conversations!

Things I Learned - 11 Aug 2024


I explored building minimal Docker images from scratch, fine-tuned Mistral using Axolotl and Deepspeed, and studied communication strategies for winning hearts. I also practiced D3.js data visualization techniques and integrated Bard with my Google Workspace tools.

This week, I learned: Dockerfile can have FROM scratch and you can add specific binaries rather than an entire OS. via Fine-tuning session by Dan. Notebook Example of fine-tuning Mistral. Consumed 28 computes ($2.8) Axlotl is what the top fine-tuned LLMs are trained on Deepspeed provides distributed training Flash attention lets data stay on GPU Sample packing packs samples of different lengths into equal length tensors Visualize the RANK of a token in a generated stream instead of logprob The Knowledge Project. Tomorrow Gayner What I'd like in my obituary: Anand was happiness. A guru. Generous. To get what we seek we must deserve this. Build, measure, learn If you did the same thing daily for 50 years, would it be a great thing? If yes, do it. If not, stop. Do this in daily retrospectives My new role should be productivity through technology innovation. That may mean a CTO role. But be specific otherwise no one will understand it Hidden brain podcast. Us 2.0. Win hearts, then minds When in an interaction, ask yourself. Can I learn and change myself? Can I win their hearts, then mines, so their behavior will change. That identity will change Notice when you get emotionally triggered. That's exactly when you should not get emotionally triggered Try model humility and moral Look for close to people's identities in our conversations. What are things they like? What does it mean for them? Simply ask. With that understanding of identity, it becomes easier to reframe things in a way they will understand Bard can talk to Gmail and Google Drive! PREDICTION As automation takes over these mainstream activities, people will take over the niches. Since expertise like knowledge is fractal, there will be many more segments of one in the future and it will be easier to automate clusters of similar abilities. Recommenders and brands will become even more important Stephen Osserman's Observables have some nice notes. Visualizing partial election results D3 Force Dilemmas: Data Distortion Sandra Becker's 30 day D3 course

Things I Learned - 11 Feb 2024


I explored leadership insights from the Radio Free XP podcast, learning to announce plans early for alignment, provide actionable information instead of direct orders, and acknowledging the challenges of shifting company culture without being the boss.

This week, I learned: Radio free Xp podcast. Nudge 61 always announce first before doing. Give people time to plan comment and react. That gets you alignment without sacrificing freedom. give information, not orders. When someone is parking a car, tell them how much space they have, don't tell them to start stop or how much to turn left it's almost impossible to change the culture if you're not the boss

Things I Learned - 12 May 2024


I discovered how LLM planning time rivals massive parameter increases, compared text-to-speech pricing, and tested DuckDB's function chaining. I also explored Deno 2's Node compatibility, Marimo notebooks, and efficient Python Docker builds using uv.

This week, I learned: DuckDB supports function chaining DuckDB lets you create functions = macros HTML for People is a nice introduction to HTML. FlightRadar24 lets you watch airplanes live. sq is like jq but for SQL. Deno 2 is fully backward compatible with Node! via O1 is good at solving problems where the solution is easy to verify and generating options helps get closer to the solution Reverb ASR does diarration as well as transcription. It seems the state of art right now. Gemini Flash and Gemini Flash 8b can be fine-tuned at zero cost. Inference is at the same price! Ref Flux 1.1 Pro is released. I tried my Calvin & Hobbes test on it. Not great. ImageGen3 is better, ChatGPT is the best. Ref Revisiting text to speech models. Nothing much has changed since July 2024. OpenAI TTS: $15/1M chars Ref Deepgram Aura: $15/1M chars Ref Azure AI Speech: $15/1M chars Ref Google TTS Neural2: $16/1M chars Ref AWS Polly Neural TTS: $16/1M chars Ref Cartesia Pro: $50/1M chars Ref Elevenlabs Scale: $300/1M chars Ref GitHub co-pilot workspaces let you code using your mobile with AI and deploy it at one shot If you need an Ubuntu Docker container with Python, install it via uv rather than compiling from source. via VTracer is an open source library (and tool) to convert raster images to SVGs. via If you want to create a console.llm() function, a browser extension is the best way, because some pages have Content-Security-Policy that block eval, form submission, fetch from other domains, and script execution. PyPi lets you publish from GitHub Actions without a token. Also from Gitlab.com CI/CD and Google Cloud. ActiveState which made ActivePython, ActivePerl, etc. made these products paid for commercial use around 2013 after a series of acquisitions. Marimo supports: Publishing any notebook to static.marimo.app as a static app Creating a SINGLE link that embeds the ENTIRE notebook in the URL! Runnable via uvx marimo edit Parables on the Power of Planning in AI: Giving models about 30 seconds of thinking time consistently improves results - as much as increasing parameter size by a factor of 1,000 to 100,000! This works particularly well for verifiable results (code, math, etc.) Technique: Ask an LLM hundreds of times at low temperature and pick the most common one. (Google's Minerva used this on the MATH dataset.) Better Technique: Ask an LLM hundreds of times. Pick the best solution based on an evaluation metric (reward model) Better Technique: Apply a reward model at EACH step of the process. OpenAI's "Let's Verify Step by Step" Late chunking is an interesting approach to adding context to embeddings. (I don't understand it, but it's cheap and effective.) DeepInfra offers embedding models as APIs at about 0.5 to 1 cent per MTok in an OpenAI compatible API. It also supports text-to-image models like flux.dev and speech recognition models like Whisper. Jake Heller: "One of the things we learned is (an LLM app) after it passes passes frankly even 100 tests, the odds that it will do, on any random distribution of user inputs, the next 100,000 100% accurately is very high." OpenAI's O1 is like Daniel Kahneman's System 2 thinking - as against other LLMs' System 1 thinking. Continue.dev is another AI coding editor. It supports OpenRouter. So now I have heard good things about: Github Copilot Cursor Cody Continue.dev (supports OpenRouter) Aider (supports OpenRouter) Maybe: Codeium Not: Amazon Q Developer

Things I Learned - 13 Oct 2024


I discovered new VS Code shortcuts, Copilot terminal integration, and the SQLime browser playground. I also explored Python's fsspec for filesystem protocols and learned why embracing inaction and subtraction can be a strategic advantage.

This week, I learned: Prashant Pandey: we need to prepare before every meeting. Something to teach VS Code Select any code and command Explain this to understand the code %something in command bar searches ACROSS files for a term. Exactly like Ctrl+Shift+F Copilot has an Inline Chat: Start in Terminal (that needed me to unbind Ctrl+I in bash to work) Ctrl+2 opens a second window on the side. Ctrl+1 goes back to the first window Terminal: Open Detected Link lets you scroll through detected (file) links in terminal Terminal sticky scroll is transparent. (But Terminal stick scroll isn't working for me.) Copilot uses last 10 commit messages, Jupyter notebook kernel state (variables) as additional context 1.88: supports locked scrolling to sync scrolling of side-by-side windows fsspec is used by csvbase, Pandas, etc. to implement file system protocols like s3fs, gcfs, etc. SQLime is a SQLite client / playground on the browser! Do nothing. Then do less Humans have a bias against inaction. Hence a strategic advantage. What can you cancel today? Humans have a bias against subtraction or removal. That too is a strategic advantage. What can you remove today? Humans have a bias against constraints. That's a strategic advantage. What constraint can you embrace? No Yay! When declining something, add it your calendar so that when the time comes you can say yeah I got this time back

Things I Learned - 14 Apr 2024


I investigated Marimo for reactive Python, Python 3.13's JIT, and why 4-bit quantization beats smaller models. I also discovered an npm unpublishing flaw and discussed how AI shifts education toward troubleshooting and systems thinking.

This week, I learned: Transparent LED screens will be useful in windshieds to display maps as we drive. Marimo is a reactive alternative to Jupyter notebooks that saves files as pure Python. To run an org-specific chatbot on your own LLM: (via awesome-chatgpt) opengpts - but it doesn't support auth chatbot-ui - but Supabase is hard to install anse - but it doesn't support auth ChatGPT-Next-Web - but it doesn't support auth Python 3.13 gets a store and copy JIT If an npm package adds another package as a dependency with version "\", target package cannot unpublish ANY version! So this is a way of freezing EVERY repo and preventing unpublishing of EVERY version -- an unintentional flaw in the npm design. via Quantization is better than fewer parameters. So prefer high parameters (e.g. 70b) and quantize to 4-bit. In-browser playgrounds has compiled WASM versions of Python, PHP, SQLite. Happiness Lab podcast. Happiness lessons of the ancients Talking to strangers makes us happy Giving money makes us happy Free time makes us happier than working hard Tangi Domain-specific models being beaten by general purpose models is a phase. It will reverse towards domain. AI will potentially help build and understand domain-specific models Models are evolving so rapidly that humans cannot interpret models. We need a process to interpret models! xAI, Responsible AI, Physics-guided or Knowledge-guided models (called grey box models) are therefore a trend CS papers Don't review other papers, certainly not other fields. Disregard measurement errors. When CS papers get applied to climate, manufacturing or biology, we'll worry about Interpretability Domain-specific mechanics. (Introduce that into the training as a constraint.) Many domain experts are using AI to UNDERSTAND their process. Need to explore Uncertainty IB adds context to make learning applicable. But that distracts from the core learning, and if there's a gap it widens Most data science courses teach "Python science", not data science. They teach a bunch of models. They don't teach how even one kind of model e.g. LSTM works. Most coaching programs today teach FAMILIARITY with problems, not critical thinking Most of current education will become redundant thanks to LLMs. For students AND teachers Coding will become irrelevant Cognitive thinking, reasoning, human relations, systems thinking will become more relevant Troubleshooting will become more important. AI is not self-diagnosing. I would hire someone who can figure out something is going wrong, diagnose what's going wrong, and fix it TODO Hire for troubleshooting ability. Give a Q, an A, and ask them to figure out if it's wrong, why, and fix it All my exams and quizzes are open book, open ChatGPT. Onus is on me to give a problem that forces you to think. TODO Write a question paper that is ChatGPT proof. Exploring AI could be a ToK subject. "How to interact with an AI?" We need a manual on how to use AI. Like Simon Willison says Content doesn't suffice. You need pedegogy. What to serve you at what time, how, how to assess. Lots of businesses are filling this gap Students get great confidence when a teacher points to online content and says, I"ll tell you WHAT to see" and COMPLEMENTS that in their class "The map is not the territory." Most people confuse sample mean for the actual. ASK Parameter estimation -> Signal estimation -> State estimation Stats vs DL differ in that There is no notion of a defined "truth". Hence reliability is not measurable Parameters have no value. Hence interpretability is ignored. TODO Read 2020 National Education Policy. It's quite modern. We need a manual on self-learning too Listening is not learning. You know only if you implement. Levels for students: I can solve it. I can explain why it works. I can find alternatives. I can apply it to a new area, reformulating (requires imagination.) For teachers, you also need: Responsible learning (extra careful about what to teach and how to teach, to exceite them, to teach at THEIR level). Show the universality and connecting to other concepts. E.g. noise reduction with FT is like using water to remove dirt. Transform to water domain, remove dirt, transform back to air domain. It's better than dusting clothes to remove clothes. Washing machine programs are just different models of removing noise in the water domain. Teach people who WANT to learn AND who will APPLY it long-term. That's what maximizes impact Grad students are more satisfying that way. Else, it is WASTED effort. (Not that it's a bad thing for the student, but the effort IS wasted for the teacher) Therefore, I believe students should have general engineering first, and let students pick specialization later. Some universitie are doing that. THINK Students remember my philosophy more than my content. We impart character, not just knowledge. Astrology and horoscopes serve a different function. They provide explainability, not predictive ability. As the world becomes less explainable, the need for astrology will grow. Explainability is about creating STORIES that fits data plausibly. It has nothing to do with data or truth. Explainability and predictive ability and reproducibility are all different. Maybe, Science is about the latter two, less about explainability. Astrology is a model. The map is not the territory. It's an explanatory, not a predictive model. THINK Therefore, my lessons are just explanations. Stats about experiments are STILL explanations. They are NOT reproducible or predictive. Hence not yet science The meaning of our life is the transformation we undergo in our lives TODO Read "The Journey of Souls" by Michael Newton. A hypnotherapist TODO Try regression therapy / hypnosis. Record it and listen to it. Just for fun! Rohini Deshpande Slam book was the Facebook of the 1900s Prepared mind is an extremely powerful tool for learning. Practice prepared mind When women drop out of education or career, that is also a waste from the teacher and system perspective The time for career growth is the same as child bearing time for women. That's not true for men. But child rearing can be done by either. That's not recognised. It's 0K for a man to raise the child and make the home and 0K to treat that as the default Since men are more senior, it's usually logical for them to stay in their jobs. That's a systematic bias. When seniors advise women to step back. they respect it. That widens the barrier. Why not eliminate that situation? Be proud of the working women in the family Stats are just a symptom. They don't explain the cause. (Map is not the territory.) Explanations are what really helps us fix the cause. Hence stories are important. Read Tinker Tailor Soldier Spy RV Athimber health tips: Eat foods with low glycemic index Eliminate free salt completely Voyage AI Embeddings have a higher quality, similar price compared to OpenAI embeddings. There's a clear benefit to replacing text-embedding-3-large with voyage-3-lite. There's a 200 MTok free tier currently. mixtral-offloading cleverly loads only the model layer required at any point, letting you run Mixtral 8x7b on Colab Free and on 16GB GPUs. This notebook runs on Colab Free too. CodeGPT is an alternative to Github Copilot that can use any LLM.

Things I Learned - 14 Jan 2024


I explored VS Code venv setups, the benefits of writing for oneself, and why overthinking backfires. I also compared text-to-speech pricing across OpenAI, Deepgram, and cloud providers, noting Elevenlabs' significantly higher costs.

This week, I learned: Carlton's TDS session Always create a new venv via VS Code when starting a training session. Helps reproduce issues (though I could use Colab instead) Create an empty .ipynb notebook and double-click it. That's another way (though slower) to open a Jupyter notebook Share Parrish Knowledge Project podcast. Three generations of wealth There is a big difference between liking animals and being a vet. Between liking education and being a teacher. Even if no one reads your writing, you benefit from the writing. Emotional.crises like 9/11 or Covid are far easier for markets to recover from Hidden brain podcast. White trying to hard can back fire on you Sometimes conscious thinking makes our automated responses of sports music, dance are great examples Instead, SURRENDER to something outside of you. Like playing with kids. Exercise also sends blood away from brain. Drugs. ChatGPT. It's called Ue in Chinese philosophy A quick check on the pricing of text to speech models OpenAI TTS: $15/1M chars Ref Deepgram Aura: $15/1M chars Ref Elevenlabs Scale: $165/1M chars Ref Google TTS Neural2: $16/1M chars Ref Azure AI Speech: $15/1M chars Ref AWS Polly Neural TTS: $16/1M chars Ref

Things I Learned - 14 Jul 2024


I explored Windows 11 updates like native sudo and Paint Cocreator, learned about "Roaming RAG" for structured documents, and benchmarked SentenceTransformer embeddings, finding that smaller batches significantly improve encoding performance on Colab T4 GPUs.

This week, I learned: /.md can search for all Markdown files. Julia Evans Windows 11 2024 Update features: Ref Live captions (via the tray) can transcribe audio and microphone. Cocreator in Paint lets you draw crudely and enhances it with AI. The neat UI is a slider that lets you control how close it should be to your drawing. Voice Clarity automatically cancels echo, reduces background noise, and minimizes reverb. Studio Effects (via the tray) lets you apply camera effects on all apps. Eye contact feature is CLEVER! sudo lets you run commands with admin privileges from the command line. source Roaming RAG is an alternative to RAG without the vector database. Applicable to well structured documents, e.g. technical books, manuals, etc. Create a hierarchical outline of the document. Code Keep the top-level headings. Preserve the first 100 characters of opening text from each section. Present the second-level headings, but without any subsidiary content. Provide each section a unique 8 digit hex identifier. Each section heading is followed by a guiding comment for the model: Section collapsed - expand with expandsection("{identifier}"). Then read the relevant sections as context to answer the question. Code Traffic to StackOverflow has fallen considerably. Especially from young and Indian developers. StackOverflow revenue is down. Via Prashanth. They're exploring: Licensing their content. (Meta says high quality content improves LLM performance by 30% on HumanEval) Enterprise StackOverflow for system integration Fine-tuned versions of Enterprise Stackoverflow for enterprises Integrate StackOverflow within your IDE. Ask questions, post directly I surveyed the Gramener QA team on how they were using LLMs. 7 used it for code generation (e.g. date extraction, regex generation) 4 used it for learning (e.g. Robot Framework, how to define test cases, API usage) 3 used it for formula generation (e.g. Excel) 2 used it for test scenario identification 2 used it for test data generation 2 used it for comparing expected vs actual datasets 1 used it for data type identification (e.g. given sample values, identify the data type). 1 used it for evaluating resulting (LLM as a judge) I asked the Straive Digitalized Operations team what management techniques they would apply to manage LLMs. Here are the responses: Ask better questions. (Prompt engineering.) Create templates or step-by-step instructions. (Chain of Thought.) Ask for multiple options and pick from the best options. (Agentic approach?) Training. (Fine tuning.) Price weaker responses lower. (Stratified model pricing?) "LLM hallucinations are a good thing. They are a sign of diversity, allowing us to improve the answer by exploring multiple paths." -- A colleague from Straive. Hyperbrowser is a cloud based puppeteer service. Bedrock Llama models can't be directly called with their model names. You need to use their inference profile names, e.g. us.meta.llama3-2-11b-instruct-v1:0 if the model is in a US region. Hacker News RSS is a good way to get RSS feeds from Hacker News. It's also a good way to understand how to convert a news source into RSS feeds. BlueSky has RSS feeds too When embedding using a SentenceTransformer.encode(docs) it's best if we embed with smaller docs and call it multiple times (rather than embedding more at once). On Colab T4, for gte-base-en-v1.5, when embedding 1,000 docs of up to 8K chars each, here is the TOTAL time it took, based on batch sizes (lower is better) 1 doc per call: 10s 2 docs per call: 13s 4 docs per call: 19s 8 docs per call: 23s 16 docs per call: 32s 32 docs per call: 40s Running embeddings without a GPU is extremely slow. It takes 2.4 seconds per string.

Things I Learned - 15 Dec 2024


I explored how LLMs like Pixtral and Hume transform coding workflows while potentially hindering student retention. I also looked into agent-driven automation for daily tasks and why it's best to delegate OAuth implementation to external providers.

This week, I learned: Hume provides a voice-to-voice model (EVI 2) that handles emotions at 7 cents/minute. OpenArt workflows has image generation workflows Pixtral seems quite good at OCR LLM coding Makes you more ambitious Lets you code without stress. (Just pass it the error and have it fix it. Or find another approach) Is unlimited. You can run dozens of agents in parallel Simon Willison's crowdsourced list of prompt engineering hacks "Invest in things that don't change." Jeff Bezos. Like faster delivery, SQL, web platform. Medical cost in Singapore (for insurance coverage) - via Kumar Root canal at clinic: $1,300 Crown replacement at clinic: $1,300 Periodontist (gums) at hospital: $2,500 OAuth from First Principles is a SIMPLE explanation of OAuth. Conclusion: "You probably shouldn't implement your own OAuth client." Alphaxiv is Arxiv.org but with author comments and chat The Impact of AI on Computer Science Education: Eric Klopfer divided his undergrad CS class into three groups and gave them a Fortran task. One used ChatGPT. Another, Meta's Code Llama LLM. Third, only use Google. ChatGPT group was faster than Code Llama was faster than Google When tested on the approach, the ChatGPT remembered nothing. Half the Code Llama group passed. The Google group passed fully Server-side implementation of an OAuth2 client is too complex. Best to delegate this to Auth0 Via Pratap Vardhan: At Khan Academy, every developer working on Khanmigo has cursor. Everyone who's contributed to a Khan Academy GitHub repo has GitHub Copilot. I stopped using Google + StackOverflow 2 years ago. I use ChatGPT, Copilot, etc. For humans, I ask Reddit. Excited by async agents. Things that do my job while I sleep. Zapier notifications. Monitor what happens. Put it into a flow diagram and alert me. Every month, did my broker trade? Did my bank transaction fail? Did I pay my electricity bill? Every time you delegate, use an agent instead. Read my RSS feeds. Read my browser history and suggest interests. Plan a session in Bain, BCG, etc. on Artifacts. Explore sparse embeddings. More effective. ColiPali, ColBERT

Things I Learned - 15 Sep 2024


I found DuckDB significantly faster than Pandas and analyzed Claude 3's cost-effectiveness. I also learned about the Tavily search API, Vertex AI’s Model Garden, and how the isTrusted property prevents scraping on sites like Oracle Service Cloud.

This week, I learned: DuckDB is 2-10 times faster than Pandas. ClickHouse is supposedly faster but doesn't run on Windows. Claude 3 Haiku input costs is $0.25/MTok. That's half the GPT-3.5 cost. If it's of comparable quality, it's worth switching. But Claude 3 Opus is comparable to GPT-4 and twice the cost, so not worth it. Tavily is a search API for LLMs Interesting model garden models There are sites you TRULY cannot scrape even in the browser because of the isTrusted read-only property of events that you can never set to true. Oracle Service Cloud checks for isTrusted in mouse actions.

Things I Learned - 17 Mar 2024


I discovered Anthropic’s condensed documentation, Gemini’s OpenAI-compatible API, and Alphafold 3's open-sourcing. I also learned about LLM attention sinks, compared Whisper and Gemini for transcription, and explored using Prefect as a lightweight alternative to Airflow.

This week, I learned: Anthropic has single-plage docs for LLMs. Condensed version and Full version Malcolm Gladwell on the importance of self-correction Belonging to multiple social worlds is a good way to defend against no longer being good at what you used to be. Diverse values and social groups help. Self handicapping explains a lot about the world. You study late for a maths test - so you can fail for lack of trying, not aptitude. Ecosystems (e.g. sports teams) mitigate self-handicapping. You don't have to be good in athletics to get the benefits. A slow runner gets the same discipline, pumping up, etc that a fast runner does Mono cultures are good to accomplish a known mission. Diversity is good to pivot during uncertainty. So, localize mono cultures Diversity helps only if there are sufficient numbers, or if they have enough power to change the organization's thinking. Use a standardized password strategy, e.g. use the month like GramNov2024 (via Namit) Gemini has an OpenAI compatible API. Gemini Docs Ethan Mollick says Claude is solving MBA case studies well. x.com LLMs pay a lot of attention to the first 6 tokens. Ref This is an interesting article on "UI in the age of Gen AI". Ref Google Open sourced Alphafold 3. Repo Cloudflare R2 has the same API as S3 but is cheaper Prefect.io is a good alternative to Airflow / cron. Can use for synchronisation tasks, e.g. Drive to server. But no Auth, UI params or config. Gemini transcription does not give accurate timestamps. Whisper does. But the quality of transcription is similar. Pass a complex data structure to Claude.ai and have it create an app to visualize it. It does well. Simin Willison Tech Council Ventures and Sunicon VC invest in early stage startups, and aloso provide them technology support (via Naveen)

Things I Learned - 17 Nov 2024


I researched code agent frameworks including Factory and Cognition, tested multi-modal generators like Flux and Suno, discovered DocxTemplater for document automation, and experimented with the browser's File System API using showDirectoryPicker.

This week, I learned: Code agent frameworks to explore: Cognition Factory Codegen Some interesting multi-modal generation models / tools to explore: Flux for open-weights image generation Runway Gen 3 for video generation Suno for music generation DocxTemplater is SlideSense but open-core and handles DOCX as well! handle = await window.showDirectoryPicker() lets you access the browser File system API.

Things I Learned - 18 Aug 2024


I explored Docker memory footprints on WSL2, the economics of LLM fine-tuning, and techniques like two-pass generation for structured data. I also learned about CCTV-integrated alerts, graph databases, and why ad networks are ditching Vickrey auctions.

This week, I learned: Fine tuning makes economic sense only if the input tokens SAVED is twice the output token size on each call. Docker container memory usage on WSL2 docker stats frolvlad/alpine-glibc:alpine-3.17: 540KB ubuntu: 1MB (python3: +5MB) nikolaik/python-nodejs:python3.10-nodejs18-bullseye: 1.4MB (python3: +5MB) python:3-alpine: 612KB (python3: +7.5MB) python:3: 500KB (python3: +11.2MB) continuumio/miniconda3: 7.6MB (+6.5MB) Discussion with Vinu Yamunan Databuck by FirstEigen. Autolysis plus monitoring Quality council has the data steward (maintainer of each dataset) coming together with the uses on a weekly basis to understand what quality problems to users are facing. Data owners jaundice at a lower frequency to get an understanding TODO Automate rules for data quality in our projects and intranet Convert a config rule into business language. Explain SQL. These are good use cases for llm's Graph DBs are powerful for flexible data structures, but query generation needs AI or expertise. Check the Neo4J language cypher Explore storing SAME data in relational DBs AND in graph DBs / document DBs for different use cases Dallas rocketry challenge. Build a rocket that can take an egg to 800 feet exactly and land without breaking it Discussion with Karthik A TODO Ask IIT students to do internship tasks. Use advent of code is a qualifying criterion Tata motors unionized DB admins for longevity. No one can take their jobs. Hires people who LIKE their jobs Rust gives me typing. It's very efficient. Pola.rs is interesting but Pandas as good enough. Explore alerts from CCTV feeds. Karthik sends email alerts with pictures for: "Is the machine on or off"? for productivity "Are people not wearing helmets?" for safety at Cummins TODO Integrate with WhatsApp. Use LLMs with function calling for responses Use expiring links (to pictures or content). It increases engagement Check Deno licensing. Is there a commercial clause? #ANS No - it's MIT license Centre or excellence for zero emission tech at IIT. Karthik is part of it Explore auth0. 7000 users are free toml is part of the Python 3.11 standard library! If copilot writes code we don't understand we are screwed. Hence expertise matters Discussion with Vikas Kedia TODO Plan an AMA The mind becomes lazy with financial success. Vikas is treating his podcast as a startup Hire a professional videographer for your content Financial RoI in financial markets is the highest. Programming is high too but FS is even better "Performative power" -- when you're forced to perform, you get better ideas Observable 2.0 is an open source static site generator for data Python dataclasses SORA is OpenAI's video generation model, and is stunning! If Appa comes to Singapore even for a week, he will feel better and can boast to his friends. At over 90, it may be better to move Appa to where I am since many of his friends would be no more and shops, doctors, etc can be managed and getting an independent house nearby is not hard. There is an SEZ in Gujarat where Indians can invest like in Mauritius without forex restraint Shubha: Media sites are moving away from Vickrey auctions to first-price auctions for ads. That's because they send the auction price forward to a search engine and the winning second-price value can lose even though the owner is willing to pay more. Second-price auctions don't work unless ALL bidders are in the SAME auction. Ad networks are a hierarchy of auctions! Gemini 1.5 launched. Fly.io offers GPU hosting and auto stop when they have nothing to do. Embeddings in random forest are very effective at classification -- much better than dot product. To deploy apps with OAuth + templating support in a small Docker container, use Caddy Deno has native TypeScript, browser APIs, and compiles to multiple OSs Ruff is a MUCH faster flake8 Two pass generation is a clever technique to get multiple SEQUENTIAL answers in a single API request. For example the schema {'code', 'optimizedcode'} will generate code and then optimize it. Unions in function calling allows flexible multi-step prompts in a single API.

Things I Learned - 18 Feb 2024


I learned about the post-WW2 history of the letter Å in Scandinavian alphabets, the construction of Zalgo text through Unicode characters, and Artificial Analysis for benchmarking LLM API performance metrics like speed and cost.

This week, I learned: In Scandinavia, Århus comes after Zürich because Å is a different letter. It was added by the Dutch after WW2 to distance themselves from the Germans. via Zalgo text is where we combine multiple Unicode combining characters Artificial Analysis benchmarks LLM APIs on speed, cost, and quality.

Things I Learned - 19 May 2024


I learned SQLite optimizations like WAL mode and synchronous settings for better concurrency. I also explored how AI impacts apprenticeship and education, alongside technical demos involving the OpenAI Realtime API and high-performance video rendering in Minecraft.

This week, I learned: SQL optimizations for multi-threaded web applications. Ref PRAGMA journalmode = WAL. Improves performance for frequent writes. It allows concurrent reads and writes. PRAGMA synchronous = NORMAL. Improves performance. We might lose a few transactions but won't corrupt the database. PRAGMA mmapsize = 128000000. Set global memory map for processes to share data PRAGMA journalsizelimit = 64000000. Limit WAL file to prevent unlimited growth BEGIN IMMEDIATE instead of BEGIN. Prevents writes to the journal file until the transaction is complete. Improves concurrency. AI seems to be slowing down apprenticeship since experts would rather use an AI than train an apprentice. Example: Robotic surgery. Ref How AI can improve education performance and engagement. Ref Student: Create study plan based on course and schedule Student: Focus on what you need to learn more Student: Align with your study style and pace Teacher: Grading MCQs Teacher: Writing conceptual guides "New collar workers" was coined by Ginny Rometty Embed tutor or document in video and ask for clarification! This is a new embedded interface. #TODO Playing Bad Apple in Minecraft. Ultra cool! OpenAI has a prompt generator. Currently it uses a meta-prompt but may later move to DSPy or Gradient Descent. Ref Great demo of using the Realtime API to read the latest Hacker News. Ref LLMs have reached the point where they can show a world, like CounterStrike, in near real time. Ref

Things I Learned - 20 Oct 2024


I gathered insights on GPT-4 prompting, structured LLM outputs via Outlines’ logit bias trick, and the browser's Shape Detection API. I also found that personal invitations and mandatory in-person sessions are crucial for driving engagement in online learning.

This week, I learned: Effort engine introduces "effort" as a parametrizable way to speed up LLMs with a quality trade-off. Works on Mistral for now. Many arts demand devotion. Devoting unrestricted time is part of that. 16 hours of practice a day is not uncommon. Sessions don't start and end on time. Instruments take a lot longer to learn than vocal music. The instrument needs to become an extension of you. Tests and homework have a purpose. It helps people figure out whether they've learnt. So: Write tests that make people think! Like DuckDB workshop Share a list of exercises that people can explore People need to explicitly be INVITED, and potentially IN PERSON, before they will engage with something new. For example, no one posted to until the VIA Talks session where we got them to post. For example, having one day at IITM mandatory (especially early in the course) gets online students familiar with TAs. They understand that TAs actually help, at high quality. That they can use Discord. What makes Delhi students more assertive? How can we inculcate that in others? jsr-io/migrations is a great example of database migrations. Shape Detection API in the browser detects QR codes, face bounding boxes, Browsers also natively support blurring and face tracking. via Lessons after half a billion GPT tokens for GPT-4: Vague instructions are better than over-specifying Avoid libraries like Langchain. APIs are stabler 1 token = 3 characters is good enough GPT4 doesn't hallucinate much, except it does a poor job of saying "I don't know" or "There's no such data" (the null hypothesis) Keep the output down to 10 items or so if you're listing. For longer lists, have it explicitly enumerate Don't worry about niches. Just wait for GPT5 WRITE GPT clearly prefers 42 as a random number. WRITE fal.ai "animates" pictures, creating videos. It made one from my talk. I morphed into various somewhat similar people rapidly in a 2-second span. Very promising, and far from good. llmsherpa extracts PDFs using LLMs. It has errors but it preserves hierarchy, extracts tables well, and retains image coordinates. Via +91 90031 35354 Vetrivel PS www.web.sp.am is a content farm that's getting hit by OpenAI. Highlights how easy it is to create content farms, and therefore "easy" it can be to introduce bias into LLMs. OpenAI supports batching requests. Didn't know that. Marvin provides Python decorators to create AI functions. Pretty intuitive! Outlines generates structured test with LLMs. It uses the ⭐ logitbias trick to limit choices in output. See getchoice() Lemur from Assembly.ai does real time call transcription and summary W3C is exploring ways to allow web pages to train LLMs, to flag content as AI generated, etc. Data Provenance Explorer lists open datasets used to train LLMs. Summarize.tech summarizes YouTube videos. WRITE Stable Audio 2.0 generates 3 min of music from a prompt. I tried Bollywood Tamil film background music. Dark, soulful and Horror movie background. Drums starts darkly. Build up to a crescendo of intense chaos.. Great that it managed, but not great music. Somewhat stereotyped. I need to learn how to prompt better. BTW, Udio is another such. Harpa.ai is a well designed Chrome extension / plugin that can chat with or automate any page. Due to in-context learning, giving 100s of examples in the prompt can teach LLMs to jailbreak. Ref With RAG on search becoming big, search APIs are growing. serper.dev, you.com, searxng being examples.

Things I Learned - 21 Apr 2024


I compared quantized Mistral outputs, tuned ElevenLabs voice cloning, and explored Tim Ferriss’s writing constraints. I also learned about Lilac for data curation and why lungs have a high Hausdorff dimension of 2.97.

This week, I learned: When comparing Mistral with 4b quantization vs unquantized: 2 responses were significantly shorter and fairly different 1 was identical 1 was almost identical but shorter by a few words 1 was slightly longer and fairly different PREDICTION As humans have more conversations with LLMs, they will replace video watching and interactive gaming with conversation based role play. New game genres will evolve Lilac is an LLM-based data curation tool. Use it to search by concept (e.g. PII, duplicates, etc.) and then drop/update the results. Lungs have a Hausdorff dimension of 2.97 -- giving them one of the highest surface area to volume ratio. Brains are 2.8. Sierpinski Pyramid is exactly 2 -- which is weird. To solid-paint twice the size, you need 4 times as much paint. How I write podcast. Tim Ferriss High bars are constraints. I set the strongest constraints against the scarcest resources. Like reputation Being a category of one is more defensible than a competitive advantage Content always beats presentation. When in doubt, push for more interesting content Regular publishing improves thinking To build a habit, do less than you think you can do. That makes it easier to build momentum on the habit and sustain during crunch times There is a lot of mediocrity in the world. If you're doing something (in a winner take all ecosystem), be the best. Top lawyers are exceptional proofreaders. They are able to see what is unclair, and what is redundant, and what has loop holes very quickly. Forcing yourself to cut down from a thousand words to 200 to a paragraph to a sentence takes you through a phase transition where you discover something unexpected The more outrageous the question, the more likely it is to be useful in generating a new perspective Eleven-labs speech synthesis with voice cloning is at the uncanny valley. With two 5-minute samples, my voice sounds a fair bit like my voice but is very clearly not my voice. I find stability 30%, similarity 80% and style 50% gives a reasonable outcome. But the default voices (e.g. Joseph, George, Charlie) are excellent. Practical AI podcast: AI predictions for AI by API is the norm today and will grow Just having AI is no longer a differentiator AI is part of life, not just work TODO Explore quickdrop from Stability for Maruti TODO Explore Codium VS Code plugin and Continue.dev Hybrid systems that combine stats, ML, DL and AI models will grow AGI and AutoGPT resurgence RAG will continue to be a focus GPT4 will be beaten by open source models. Special purpose models beat it already Self hosted and cloud hosted models will grow for security Small language models will grow Productivity will be enhanced rather than replaced Multi modal models will grow Cost efficiency will grow in focus GPT Builder help explains how the GPT Builder updates GPTs - including some very interesting prompts

Things I Learned - 21 Jan 2024


I explored GPT functions for spreadsheets, Xata's free PostgreSQL API, and Nginx's least_conn load balancing. I also looked into GitHub Copilot's prompt construction and the importance of tracking evolving LLM capabilities and hardware-specific package managers.

This week, I learned: GPT For Work has a set of useful spreadsheet LLM functions Xata offers a free PostgreSQL tier with REST API Mamba now uses mambaforge as the default installation, i.e. conda-forge is the default and only channel! Update: 6 Jun 2025. Mambaforge is sunset as of 29 Jul 2024. Conda-forge now uses Miniforge as the standard installer Ref conda-forge.org. Users should switch to Miniforge instead. nginx supports a load-balancing method leastconn which is far better than the default round-robin. IMPOSSIBLE LLMs cannot provide a bounding box of objects in images. (Maybe Florence 2 can). Update: Mar 2025. Gemini has good timestamps and bounding boxes Models gently grow in capability. It helps to maintain an impossibility list that steadily gets invalidated. Ref Github Copilot internals walks through how Copilot constructs its prompts

Things I Learned - 21 Jul 2024


I categorized hosting platforms, broke down Anthropic’s agentic patterns like routing and prompt chaining, and found a useful yt-dlp snippet for subtitles. I also analyzed OpenAI's GPT-4o mini Realtime pricing and Python's markdown2 library.

This week, I learned: What to use for hosting: ChatGPT GitHub Pages: Static websites, medium files Cloudflare Pages: Static websites, global delivery Vercel: Frontend frameworks (e.g. Next.js) with high DX and ISR, small files Netlify: JAMstack projects, minimal back-end, moderate files Glitch: Small static projects Render: Full-stack apps requiring databases and server-side compute Firebase Hosting: Small sites, limited large files Archive.org: Public archival, large files Google Drive: File sharing, large files Dropbox: File sharing, moderate files Cloudflare R2: Static assets, large file delivery Anthropic defines agents. Building effective agents + Cookbook Augmented LLMs are LLMs enhanced with augmentations such as retrieval, tools, and memory. Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Prompt chaining: Pipe each LLM output to the next LLM. A->B->C->Z. E.g. Write report, then translate. Extract results, then verify them. Successively ask follow-up questions. Routing: One LLMs decides which other LLM to call next. A->B|C|D->Z. E.g. Evaluate complexity, then pick the right model. Classify request time, then pick the right prompt. Parallelize: Sectioning (and Orchestrator-workers): Break tasks into independent subtasks, then aggregate. A->B+C+D->Z. E.g. Evaluate contracts against different clauses in parallel. Parallelize: Voting: Run same task multiple times, then vote. A->B+B+B->Z. E.g. Review code for prompt injection using different prompts. Evaluate content safety with different thresholds. Evaluator-optimizer: One model checks another in a loop. A->B->A->B->...->Z. E.g. Literary translation. Self-healing code. Policy violation checks. Human-in-the-loop Checkpoints: The workflow explicitly requests human review at certain stages. A->B->(Human)->C->Z. E.g. Sensitive content review. High-stakes decision making. Ambiguous tasks. Agents are LLMs that dynamically direct their own processes and tool usage, consulting tools or the user as needed. To download YouTube subtitles, use: yt-dlp -q --skip-download --convert-subs srt --write-sub --sub-langs "en" --write-auto-sub --print "requestedsubtitles.en.url" "$url" Simon Willison o1-preview diagnoses better than doctors. Harvard OpenAI's release of ephemeral tokens via sessions (valid for 1 minute) are a useful way of exposing apps for public demos. Currently it works only for the Realtime API, though. SpreadsheetLLM is a way of encoding spreadsheets in an LLM friendly format. It's good for 1K+ rows. For lower, Markdown > XML > HTML. However, Table Meets LLM suggests that HTML > XML > Markdown, so this is unclear. HARD prompt. Ask video generators like SORA to generate text in videos. It is of average quality. GPT 4o Mini Realtime was released. A realtime conversation will cost 50c/hr. About 36c for input, 72c for output. (I extrapolated from the 6c/min audio input cost for GPT 4o Realtime when it was $100/MTok. GPT 4o Mini Realtime is $10/MTok input and $20/MTok output.) This is an interesting way to understand software. Generate a Mermaid sequence diagram showing interactions based on this code. Ref The King James Bible and all Harry Potters, each, are about $1M tokens (rounded off). markdown2 is the new de facto Markdown library for Python. Claude 3.5 Sonnet is way ahead of competition on the LMSYS Webdev Arena Raspberry Pi 5 has a faster CPU, more RAM and GPU, 4K support, multiple USB 3 ports Government websites like the official press releases cannot be crawled from outside India. Hence the need for server farms in India!

Things I Learned - 22 Dec 2024


I discovered E2E for cheap Indian GPU hosting, explored XML tags for better LLM prompting, and tested tools like Jupyter Lite and VoidEditor. I also learned about Ollama's concurrency and animating faces with Segmind's Hallo.

This week, I learned: E2E is a cheap GPU hosting provider for India. About Rs 100/hr for a V100 16GB Jetson NVIDIa is like Raspberry Pi with a GPU! But it's expensive. Sarvam.ai offers Indic text to speech Jupyter Lite lets you run Jupyter notebooks in the browser Piston lets you run Python code via a REST API lets you load and compile modules early! Ollama 0.2 can handle concurrent requests with only a little additional memory. (So can vLLM and DeepSpeed.) Prompt engineering for code generators: Claude Artifacts Prompt Val.Townie system prompt. Good example of how to create Cursor editing Cursor debugging Cursor conversation XML tags seem best to structure prompts across LLMs. Claude OpenAI Gemini Instructor prompts by Ethan Mollick help teach better Non-Negative matrix factorization apparantly aligns to intuition more than K-Means and hence would be a great fit for most cosine-similarity matrices (via Jaidev). Segmind's Hallo lets you animate a face to an audio clip VoidEditor aims to be an open source Cursor alternative Video of ChatGPT o1 + mini reproducing the methodology of a paper by writing the code - in 6 iterations. Here's the repo. Prompts: 1. You are a Python and Astrophysics expert who is tasked with helping me on my research project. Please read the following methods section of this research paper and re-create the Python code described. 2. Thank you, this code looks really nice. I don't have any actual data or noise cube ready at the moment, but could you please generate some test data that can be used in the code you just wrote: {CODE} 3. Hi. thank you for writing the code! Unfortunately, it seems that I get an error when I try to run it. I've attached the error message below, can you please refine the code so that the error is resolved? {ERROR} 4. Thank you, but when attempting to run the code that you provided, I received the following error: {ERROR} 5. Hello, thank you for the code. but now I get the following error pasted below: {ERROR} 6. Thank you, I think we are getting close to a final solutiom I still get an error, which I've pasted below: {ERROR} Groq, SembaNova and Cerebras are fast inference models. All appear to be free The skills required to vet the AI's response is the same skillset used to vet a Pull Request. It's a good way to teach code review. Source: My personal guide for developing software with AI Prompt engineering tip: Tell LLMs another AI wrote code. Else they will agree with you!

Things I Learned - 22 Sep 2024


I discovered Luma Labs Dream Machine for AI video generation and the LLM DataHub for training datasets. I also learned to prioritize duration over returns in compounding and received tips for running distraction-free workshops.

This week, I learned: Luma Labs Dream Machine generated videos. It's free and is of reasonable quality. Update: 6 Jun 2025. Costs $10/month LLM DataHub has LLM training datasets, regularly updated From Dan Becker on running a workshop Answer questions at the end, not in parallel in a chat, to avoid distraction Have fewer words in slides when presenting. It's less distracting Morgan Housel Shane Parrish podcast Risk is what stops you from achieving YOUR goals. What's risky for me may not be risky for you The lesson from compounding is that you want to optimize for duration, not return. That's what does the heavy lifting. Survival, consistency, long term - these matter. The performance does NOT matter.

Things I Learned - 23 Jun 2024


I explored expanding mental models through inversion techniques like pre-mortems and red teaming. I also discovered Planka for project management, the Instructor library for structured JSON, and how real-time control vectors influence model responses.

This week, I learned: Ways to expand mental models DISCOVER mental models. Review beliefs diary. DIVERSIFY. Find INFLUENTIAL (not dull) people with different backgrounds. Experiment! New environment, approach, perspective Be open. Change your mind. APPLY. Practice regularly Ways to use inversion "Pre-mortem" is an analysis at the beginning of how a project failed. Then avoid that "Red team" or "Black hat" are designated to contradict. Having a PoV IS a hypothesis. Always having a PoV allows us to detect anomalies and learn. Control vectors in real-time lets you control response in real-time OIDC is Open ID Connect. It's like OAuth2 but more. Azure and Google support it. Planka is an open-source Trello There is a that serves as a starting point. Might be helpful Instructor lets you create structured JSON output.

Things I Learned - 24 Mar 2024