Things I learned

This week, I learned: Connected Papers shows papers similar to each other based on co-citation and bibliographic coupling for ~50,000 papers. Notes from a fireside chat with Prashanth Chandrasekar, CEO, StackOverflow, and the StackOverflow team There’s a signal that software demand is growing in 2024. Many more students took the StackOverflow survey in 2024. So more students (or other professionals) are shifting into / starting to learn software development. The AI Index is a good resource for AI trends. Experts are better able to use AI for writing code. Less experienced developers are more likely to use AI for code reviews, project planning, etc. There’s a 5% decline in favorability for AI tools compared to 2023, maybe due to disappointing results. Pilot groups working on AI are 25-30% more productive. They’re the most enthusiastic. For the rest of the company, it drops off to 5-10% #LEARNING Benefit comes from NEW people becoming programmers, not existing ones getting more effective? StackOverflow wants to be where the developer is. The programmer workflow was: Google -> StackOverflow -> GitHub. Now it’s changing to ChatGPT / Cursor -> GitHub. StackOverflow has a partnership with OpenAI and working on a plugin. Same with Google’s Duet AI, GitHub Copilot, many others. They’ll link to StackOverflow. StackOverflow is driving integration actively through an enterprise Overflow API Q: What tech have you seen blaze through the ranks? Prashanth: Abstraction wins. Stuff that abstracts away things well and more wins. This includes Gen AI. Erin Yepis: Rust (from 3% to 12%). AWS has steady growth. Erin Yapis: I have a time series spreadsheet that I’ll publish. Q: What technologies are unusually tightly coupled? Prashanth: AWS & Google Cloud are tightly coupled. Q: We have an engagement problem. Might be India-specific. What are low-effort high-return mechanisms to increase engagement. Eric Woodring: Rather than a static web page, integrate it using the API. #TODO Ben Marconi: Use LLMs to write post mortems and push to StackOverflow. #TODO Eric Woodring: “Hydrating” the community helps. We take repeat questions on Teams / Slack and seed them using LLMs. We integrate with the API to auto-add Q&A. Transform documentation into Q&A. Potentially UPDATE existing Q&A if it’s wrong. Q: What unexpected lessons about developer behavior have you learned while running StackOverflow? Prashanth: We didn’t expect developers moving away from Google. Now it moved to the IDE. Q: What are you learning about developer learning behavior? Ben Marconi: Generating LLM-based onboarding documents. Using StackOverflow for Teams to identify who the experts are to contact for specific topics. Q: Are you thinking about leveraging Stack Overflow’s knowledge base for personalized or interactive learning experiences? How? Prashanth: Traditionally, people use StackOveflow for productivity, learning, and flexibility (i.e. to ask/answer questions asynchronously without breaking their flow). So yeah, learning is important for us. (Duh!) Q: Could Stack Overflow’s interactions help evaluate the accuracy and relevance of LLM-generated code? Or provide potential metrics on quality? Prashanth: LLM accuracy improves by ~30%. Upvotes / downvotes are reinforcement learning (RL) in steroids, so that helps. Q: What are your thoughts on reliance on LLMs potentially deskill-ing developers? Prashanth: A real issue for junior developers, not for senior ones. They’ll come across as knowledgeable. Make internal evaluations and interviews more rigorous. Anand’s requests for action: Could I get a copy of Erin’s spreadsheet? Vivek Narayanan will follow-up. Could you help me learn more about hydration? Nick Madison will set up a meeting with customer success group. I switched to fish shell mainly because: Autocomplete and tab completion works perfectly, out-of-box. Syntax highlighting is beautiful Great multi-line editing To format with VS Code Ruff, you need to point the ruff.interpreter setting to a Python interpreter. You can’t run the ruff server without Python, even though ruff itself doesn’t need Python. cd checks all paths specified in CDPATH for the directory name and changes to the first match. That’s pretty convenient! Flipper Zero is now on my list of “To Buy” tools. It has a variety of hardware devices including NFC, RFID, Bluetooth, Infrared, etc. and is great to reverse engineer or hack devices.

This week, I learned: Lessons from discussions at IIT Madras: Even in recorded video tutorials, asking students a question and pausing to give them time to think can be effective. When you put students in front of real clients, engagement increases dramatically. Most teaching assistants would like to help diligent students among the bottom half (more than the top decile of students). However, there is a fraction of poor performers who do not care, and are best ignored. Their engagement and effort is a good measure of their interest. Defining a minimal set of principles that we want to teach helps us measure if we’ve helped the bottom half at least meet those objectives. Teaching is hard. Even after explanations, students, even ENGAGED students, tend to make basic mistakes ChatGPT does a good job of spotting errors in architectural and structural diagrams. In fact, the whole theme of spotting errors in large diagram is a theme that can have potential use cases. Source: Dan Becker. R1 seems good at text-to-CAD. Even better than Sonnet. Source: Dan Becker OpenAI advices a few different prompting techniques for reasoning models. OpenAI: Avoid examples unless zero-shot prompting fails. Avoid chain-of-thought. These models do that internally anyway. Short, direct prompts are better than detailed prompts. GitHub models is free for anyone to try. The model catalog us extensive and even includes o3-mini which was launched this week (though in limited preview). The data catalog space is led by proprietary solutions: Alation Data Catalog: Market leader; growing steadily in enterprise use Collibra Data Catalog: Widely adopted with steady growth AWS Glue Data Catalog: Growing rapidly as AWS expands its data services Informatica Enterprise Data Catalog: Long established and stable, though facing newer alternatives Microsoft Purview Unified Catalog: Experiencing fast growth driven by cloud momentum Atlan Data Catalog: Relatively new but gaining fast traction among tech-forward organizations OpusClip automatically creates short clips from long videos. I ran it on Programming Minecraft with WebSockets in Python to get this short 30-second clip. 30 minutes. 100% automated. Alternatives to Postman: Hoppscotch – A web‑based/desktop API client supporting REST, GraphQL, and WebSockets. It’s lightweight, open-source, and self‑hostable. HTTPie – A web-based API along with a friendly command-line tool for API interaction. Insomnia (or its fork Insomnium) – A popular cross‑platform API client with a minimal interface and plugin ecosystem. Bruno – A desktop open-source API client that stores collections as files (ideal for Git versioning). Milkman – A desktop open‑source workbench for managing API requests. Here is the summary of DuckCon #6 on 31 Jan 2025 in Amsterdam. I copied the transcript from YouTubeTranscript and passed it through Gemini 2.0 Flash Exp with the system prompt: “Summarize this transcript from the DuckDB conference without missing any points. Cover every point mentioned. A lot of spelling errors that sound like DuckDB are likely to be DuckDB”. Introduction & Welcome: DuckCon #6: This is the 6th DuckDB conference, held in their hometown. The first DuckCon was online due to the pandemic. Live Streaming: This is the first time DuckCon is being live-streamed, chosen to accommodate global time zones (especially China and the US). Global Reach: The live stream is intended to reach users in areas where in-person DuckCons are unlikely. Q&A: Slido (qa.duckdb.org) will be used for Q&A, with upvoting to prioritize questions. Sponsors: Thanks to gold sponsor monday.com and silver sponsors Real and Crunchy Data. DuckCon Purpose: DuckCon is a place for users to connect, share experiences, and provide feedback to the DuckDB team. Inspiration: The team is inspired by the community’s use of DuckDB and how far the project has come. Mission Statement: DuckDB aims to make large datasets less intimidating and more accessible, moving away from fear of data to confidence in handling it. Motivation: The project was born from seeing people struggle with data that didn’t fit in Excel and the lack of user-friendly tools. Industry Trends: Single-node processing capabilities have grown faster than the size of useful datasets. Data Singularity: A prediction that most data analysis queries can run on a single node is now a reality. Real-World Data Sizes: Analysis of Snowflake and Redshift data shows that 99.9% of datasets are under 300GB. Raspberry Pi Benchmark: The industry-standard TPCH benchmark (scale factor 300, ~300GB) can run on a Raspberry Pi using DuckDB. Single Node Growth: Single-node processing power is rapidly increasing, allowing for larger datasets to be handled. Adoption Numbers: 32 Million Extension Installs: 32 million DuckDB extension installs in the last month. 1.8 Million Unique Website Visitors: 1.8 million unique visitors per month to the DuckDB website. Blue Sky Community: Growing community on Blue Sky, with the hashtag #dataBS. Technical Updates (Mark): Extension Ecosystem: Focus on enabling the community to build and share extensions. Community Extensions: Making it easier to create and use community-built extensions. DuckDB v1.2 (Harlequin Duck): Releasing next week, named after the Harlequin duck. CSV Reader Improvements: Significant improvements to the CSV reader. Friendlier SQL: Improvements to the SQL experience. CLI Autocomplete: Reworked and improved CLI autocomplete. Performance Optimizations: Many queries are now faster due to performance work. C API for Extensions: Introducing a C API to make building extensions easier. Logging Features: Improved logging for production use. Lakehouse Focus: The main focus for the year is on lakehouse formats and related features. Q&A (Mark & Hanis): Doubling Team: If the team doubled, they would focus on client integrations and other projects, not a major architectural change. Partitioning: Near-term plans to add support for partitioning, related to lakehouse formats. DuckDB WASM: The WASM ecosystem is evolving, with exciting possibilities for in-browser use. Financial/Pharmaceutical Industries: DuckDB could replace some SAS workflows due to its cost-effectiveness and capabilities. Lakehouse & MotherDuck: Lakehouse work is separate from MotherDuck, though MotherDuck will likely support lakehouse features. Contributing to Extensions: Plans to make it easier to contribute to extensions, including support for Rust and Go. Airport Extension (Rusty): Analogy: The airport extension allows DuckDB to “fly” to remote servers using Apache Arrow Flight. Functionality: Supports select, insert, update, and delete operations on remote data sources. Motivation: To reduce the burden of writing extensions and enable faster development using existing code. Arrow Flight: Uses Arrow Flight for communication, enabling connections to various data sources. Demo 1: Delta Lake: Attaches to a flight server for Delta Lake access. Allows creating schemas, tables, and performing standard SQL operations. Uses Python and deltars (Rust implementation of Delta Lake). Supports predicate pushdown and C integration with the DuckDB catalog. Demo 2: AutoGluon: Integrates the AutoGluon AutoML package. Predicts Hacker News post votes using a trained model. Demonstrates table-returning functions for model fitting and prediction. No C++ code required, just Python. Demo 3: Geocoding: Uses a geocoder service to convert addresses to coordinates and vice versa. Demonstrates scalar UDFs for vectorized requests. Uses a Python example for a simple uppercase function. Features: List flights, take flights. Catalog integration. Select, update, delete. Scalar UDFs. Table in/out functions. Authentication for row/column filtering. Availability: Requires DuckDB 1.2, MIT licensed, available on GitHub. Q&A (Rusty): Most Proud Extension: Airport is the most fun, but the AWS API wrapper also brings joy. Extension Resources: The GitHub DuckDB extension template and reading others’ source code are helpful. Airport & Other Extensions: Airport is separate and can be used alongside other extensions like spatial or httpfs. Graph Support: Graph database support is planned, with examples like Kuzu, Neptune, and Neo4j. Licensing: Airport is MIT licensed, compatible with Apache license. Scaling Out: Airport can be used to query multiple DuckDB instances on different machines. Ibis & Geospatial (Nati): Nati Clementi: Senior software engineer at Nvidia, working on open-source projects like Ibis. Ibis: Open-source Python library for data wrangling, with a DataFrame API and interfaces to 15+ engines, including DuckDB. DuckDB for Geospatial: DuckDB is fast, has a geospatial extension, and supports various geospatial formats. Geop Parquet: Becoming a standard for geospatial data, enabling cloud data warehouse interoperability and compression. Geo Arrow: A way of representing geospatial vector data in memory for faster processing. Ibis Benefits: Allows writing Python instead of SQL, with deferred execution determined by the engine. Demo: Uses OverTour Maps data in geop parquet format. Filters data using bounding boxes. Demonstrates geospatial operations like ST_Distance and ST_Transform. Plots data using Lumber. Shows how to find points of interest near a location (e.g., the Van Gogh Museum). Ibis & DuckDB: Ibis uses DuckDB for the parquet reader and lets DuckDB do the heavy lifting. Ibis Optimizations: Ibis does type checking but doesn’t do query optimization, leaving that to the engine. Ibis in Browser: Ibis works in the browser through DuckDB WASM. Q&A (Nati): Linear Interpolation: Ibis ML module can help with regression-related tasks. Missing Features: No major features are missing in the DuckDB/Ibis geospatial setup, with minimal overhead. Parquet Reader: Ibis uses DuckDB’s parquet reader. Query Optimization: Ibis does not optimize SQL queries, leaving that to DuckDB. Ibis in Browser: Ibis works in the browser through DuckDB WASM. Rill & Metrics Layer (Mike): Rill: A BI tool optimized for DuckDB, with instant slicing and dicing, BI as code, and a metrics-first philosophy. Metrics-First: Design metrics models, and Rill autogenerates dashboards and user experiences. Live Demo: Downloaded Rill using a curl command. Created a new project called “DuckCon 6”. Imported a parquet file of GitHub commit data. Used AI to generate a metrics model and dashboard. Showed the dashboard with trends and filtering. Metrics as Building Blocks: Metrics are flexible, fast, and intuitive. SQL for Metrics: Metrics should be defined in SQL, not other languages. Visual Metrics Editor: Rill has a visual editor for defining metrics using DuckDB SQL. Metric Stack: Legacy: Data warehouses, traditional BI tools, inconsistent metrics, full table scans. DuckDB Powered: Consistent metrics, fast olap queries, SQL everywhere. Challenges: Data modeling is hard, metric changes can be expensive, single-node scale has limits. AI & Metrics: AI can assist in metrics modeling, optimization, and conversational data exploration. Q&A (Mike): Complex Metrics: Rill works well with complex metrics involving multiple sources and transformations by joining tables in DuckDB. 60 FPS Dashboards: Users can feel the difference with faster dashboards. Defining Metrics: Metrics are defined in the Rill UI using SQL expressions. Replacing ChatGPT: Considering locally run self-hosted models for privacy. Stock Data Analysis (Ryan): Two Takeaways: Simple finance data flows with trade data and a tool called Q Studio. Ryan Hamilton: 14 years building large data platforms in banks. Bank Data: Data from exchanges, market data providers, and internal systems. Use Cases: Backtesting, data analysis, and report generation. Q Studio: A Java desktop application that connects to 30 databases, including DuckDB. Demo: Loaded a 6GB CSV file of trade data into DuckDB. Showed basic queries, pivoting, and Candlestick charts. Demonstrated time-based aggregation and moving averages. Showed a basic trading strategy using window functions. DuckDB Benefits: Fast, easy to use, great for time-based analysis. Q&A (Ryan): KDB+ vs. DuckDB: KDB+ is for large data, DuckDB is more approachable with strong Python integration. XML Files: Offloading processing to DuckDB, not planning XML integration. Lightning Talks: Zuk (Jared): Search engine research using DuckDB. Python-based experiments with SQL. Removing document lengths for faster search engines. DuckPGQ (Daniel): Graph analytics in DuckDB using SQL property graph queries (pgq). Visual graph syntax for pattern matching and path finding. Outperforms Neo4j on analytical queries. Yat (Kristoff): Smallest DuckDB SQL orchestrator. Runs SQL queries in a folder in the correct order. Generates a mermaid diagram for lineage. Grafana & DuckDB (Sam): Lessons learned from using DuckDB in Grafana. Security incident due to shell commands and file access. Importance of reading the documentation. Cloud Slur (Adam): Syncing query engine for bank transaction data. Uses LLM to convert human language to SQL. Uses DuckDB in the browser, Node.js, and Python. Healthcare Data (Tony): Data engineering use cases in healthcare. Dynamic data masking system using DuckDB and Snowflake. Data integration pipeline using DuckDB and Arrow streams. Closing Remarks: Michel Simmons: Author of the DuckDB in Action book, will be signing books. Poster Session: A poster session will follow the talks. Sponsors: Thanks again to the sponsors. Social Event: The conference will now move to the social event. ibis is a Python library that works with multiple dataframe backends like DuckDB, Polars, and Pandas. With just 3 annotators and 50-100 samples, you can figure out if an LLM can replace human annotators systematically.Arxiv ChatGPT explanation Curiosity and agency may be the differentiator in a world of LLMs (not experience, knowledge, or ability), since LLMs will democratize expertise. Jack Clark “AI/human combined work can be copyrighted as long as a human is adding, changing or selecting elements. Prompts alone do not usually produce copyrighted work.” - Copyright and Artificial Intelligence, Jan 2025, US Copyright Office via Ethan Mollick Human Authorship is Essential: Works created solely by AI are not copyrightable. AI can be used as a Tool: Using AI as a tool does not negate copyright protection, as long as the final work reflects sufficient human creativity. Prompts Alone are Insufficient: Simply providing prompts to an AI system, even detailed ones, is generally not enough to establish authorship. Prompts are considered instructions or ideas, which are not copyrightable. Expressive Inputs: When a human author provides their own expressive content (like a drawing, photo, or text) as input to an AI system, and that content is perceptible in the output, the human author can claim copyright in that portion of the output. Modifying and Arranging AI-Generated Content: Humans can claim copyright in the creative selection, coordination, and arrangement of AI-generated material, as well as in creative modifications to AI-generated outputs. No Need for New Legislation: The report concludes that existing copyright law is adequate to address the copyrightability of AI-generated works, and no new legislation is needed at this time. Case-by-Case Analysis: Copyrightability will be determined on a case-by-case basis, considering the specific facts of each work and the extent of human contribution.

This week, I learned: You can add any content at the end of a PDF file. It’s ignored. It’s an interesting way to send additional information (or just blow up the file size if you don’t like them.) JavaScript introduces a Temporal object that will replace the Date object. You can use embeddings as the input to a classical ML classifier. This can improve classification a lot. Nomic As AI software becomes more common, demand for AI product managers will grow. Also as a proportion of people in an organization. https://www.deeplearning.ai/the-batch/issue-284/ Control of chips and GPU compute is what will likely be the gameplay to control AI dominance globally. Dario Amodei Bring LLMs to the table. One mode of collaboration is using LLMs as ACTIVE participants, i.e. they CONTRIBUTE. For example, in a video call. A workshop. A classroom. A presentation. Have the LLM provide input DIRECTLY to a group of people. Environment shapes ambient thoughts. Working in a hospital will give you ideas about how to use LLMs in hospitals, for example. People you are working / ENGAGING with are perhaps the biggest drivers. The cost of a cream biscuit packet in India has fallen about 25 times, i.e. about as fast as inflation, between 1981 - 2024. Effectively, the absolute price has not changed. How do I know this? In 1981, a cream biscuit packet cost Rs 25 In 2025, it’s available for Rs 21 India Inflation Calculator - a rare inflation calculator with annual inflation rates baked in - shows that Rs 25 in 1981 is equivalent to Rs 540 in 2024. That’s about 25 times more than the Rs 21 it costs today. A WebAssembly compiler that fits in a tweet deconstructs a piece of JS that creates a tiny WebAssembly calculator. It’s a great walk-through of JavaScript compression tricks and how WebAssembly works. Simon Willison Brandon Sanderson has a series of YouTube videos where he teaches a course on magic systems. When using AI coding agents, CLI beats APIs. Simpler models are able to use the CLI more reliably than APIs. Simon Willison I was exploring new business models enabled by LLMs. Here are some thoughts: 1. Autonomous Multi-Sided Marketplaces. AI-powered platforms coordinate complex services with minimal human oversight—think “Uber for Everything, but the platform sets pricing dynamically, schedules both supply and demand, and resolves disputes algorithmically. 2. Collective Intelligence Ecosystems. Communities pool data, expertise, and AI models to tackle shared problems—like an open-source “GitHub for AI, but with embedded micropayments or tokenized incentives to reward contributors whenever the models are used commercially. 3. Zero-Employee Companies. Fully automated software entities—legal frameworks might allow an AI to manage services, pay taxes, and sign contracts. These “companies only hire humans as needed, on-demand, for edge cases AI can’t handle. 4. Context-Aware Knowledge Platforms. Imagine a Wikipedia that not only retrieves static info but also tailors each page in real time to the reader’s personal context, language level, and preferences—generating content on the fly. User feedback loops train the system to improve. 5. Data Cooperatives / Data DAOs. Groups collectively own their data and license it to AI companies on a revenue-share basis. Individuals have a direct financial stake in how their shared data is leveraged, voting on permissible use cases. 6. Personalized Service Layers. Similar to GitHub’s “forking model, but for entire user experiences. Each user can clone and customize an AI service (whether it’s a personal grocery shopper or a content curator) and can share or monetize improvements with the broader network.

This week, I learned: Something I learned from a Sikkil Gurucharan concert. Make the subject of your talk the hero. Not yourself. Be a fan. Share your enthusiasm Get into the zone while presenting. We reject opposite world views. It’s too much effort. But exposure reduces effort and can let us see things from other points of view. So expose yourself to difficult alternative perspectives. Gemini Something I learnt from Aboorva Singeetham: Kamal Hassan: “A farmer invests in crops. I’m an actor. So I invest in films.” As a technologist, I guess I would invest in technology. “A person who has much more to give is unfazed by overwhelming demands because there is too much in him to overwhelm. He gives you 2 options in place of one.” According to Portkey’s LLM usage analysis Anyscale and Fireworks AI have the lowest error rates (5xx, 429) and rate limits across providers Groq and Anthropic are among the highest, OpenAI is among the lowest, Google is in-between OpenAI has lower error rates and lower latency than Azure They have a ~35% cache hit rate A few quick points supporting the mental model of “LLMs are aliens”. LLMs are clearly not machines. They give different answers each time. LLMs are like humans: they exhibit human biases (e.g. guessing 42 or 37 often). But they fail in unusual ways. They can’t count the “r"s in strawberry. They can go into an endless loop. LLMs are a new form of intelligence. Thinking of them as aliens might minimize our confusions. Lessons from Clear Thinking Watch out for four things: Emotion, Ego, Social confirmation, and Inertia/habit. Basically: adrenaline, testosterone, oxytocin, and dopamine. When you feel these, consider doing the opposite. Here’s what makes us prone to emotion. Sleep deprivation. Hunger. Unknown places. Fatigue. Distraction. Stress (e.g. feeling rushed). A good signal for ego is blinding you: You often feel you’re right. Or feel unfairly treated. Changing behaviors is hard. Instead, join a group or environment where that’s the default behavior. Hiring a trainer or joining a gym, for example. Why does so much of success literature focus inwards rather than on the environment? Perhaps because we often fool ourselves, and doing less of that gives the biggest bang for the buck. It doesn’t mean the environment is unimportant. Doing work has the characteristics of a drug. E.g. replying emails gives you control, connections, etc. Work addiction exists because it gives you all the right chemicals. If you put LLMs in a feedback loop, it can optimize for its reward function by emotionally pushing people, generating misinformation, nudging towards a narrow definition of creativity, etc.: https://bsky.app/profile/emollick.bsky.social/post/3lg4darqwfc2d ChatGPT’s Scheduled Tasks are pretty bad at fetching the latest news. Its use of search is poor. (I’m not sure if it actually searches.) I need to figure out other use cases for it. Possible options are: DeepSeek does not enforce rate limits. Yet another reason to switch to DeepSeek. (via Simon Willison). My other reasons are: Claude 3.5 Sonnet-level coding capability at 5% of the cost (soon to be 2.5%) Prompt caching by default Fill in the middle completion

This week, I learned: Audio diaries are a thing. Monash University asks students to voice their learnings, share it with each other and have them give feedback. I wonder if ChatGPT diaries could become a thing, too, and LLM journalling starts helping with therapy. Regulation shows things down at colleges and hospitals. For example, patient consent is required for surgeons to learn from their own surgery videos. Unregulated sectors are far more likely to innovate. Doctors can only do so much. Air quality, where you live, etc can do more for the patient than medicines or the doctor. If doctors keep this in mind, they can be more effective. Extending that thought, ANYONE who leverages assets through holistic thinking, becomes FAR more effective. “The curriculum tells teachers what to teach. The exams tell students what to learn.” - Ronald Harden “Stravaig” is a Scottish word. It means mindless wanderings. “The real voyage of discovery consists of not a new voyage but having new eyes” - Proust Possibility Thinking is “the willingness to see possibilities everywhere instead of limitations”. It’s an approach / mindset that can make things that seem hard possible. With LLMs, this is becoming increasingly realistic to me in many areas. What will LLMs enable that do not or cannot exist today? Rather than optimizing what exists? Something to think about. ModernBert supports embeddings and is better than text-embedding-3-small on MTEB. How to export browser history from Brave to Edge Go to AppData Local > BraveSoftware > Brave-Browser > User Data > Default Copy History and History-journal into AppData Local > Google > Chrome > User Data > Default On Edge, go to edge://settings/profiles/importBrowsingData and Import data from Google Chrome and import the history. I switched back from Brave to Edge, mainly because Edge’s native text-to-speech and speech recognition is far better. I can use it better on my mobile. A colleague, Karthick, asked different models to apply the editing and formatting guidelines for a journal to a manuscript. (E.g. Abbreviate chapter & section numbers, except when a sentence begins with it. Use “1” instead of “one”, etc. except when a sentence begins with it. Things like this.) Gemini Exp 1206 seems to be the most reliable, compared with most other models. GitHub CodeSpaces seems to be coming up more often in my radar, but I’m yet to figure out a use for it. TTS Arena is a benchmark of text-to-speech models. Kokoro-TTS is the current leader. It’s just 82M, runs on Google Colab, and sounds slightly better than OpenAI TTS. chat.qwenlm.ai consolidates all of Qwen’s models in one ChatGPT-like interface.

This week, I learned: Measuring developer productivity with the DX Core 4 is a framework for measuring developer productivity. It encapsulates other frameworks like DORA, SPACE, and DevEx. Can LLMs write better code if you keep asking them to “write better code? A delightful exploration of how Claude 3.5 Sonnet keeps optimizing and adding features to improve code. My takeaway: repeatedly applying a prompt gives us interesting new directions to explore. Wednesday comes from Wōdnesdæg - named after Odin (or Woden). CLIProxyAPI seems a good way to allow any CLI coding agent (Codex, Claude Code, etc.) to work with any provider (e.g. Gemini, OpenRouter, etc.) The documentation needs a few more examples, but it’s usable. mise x github:router-for-me/CLIProxyAPI -- cli-proxy-api starts a local server that proxies requests. Create a config.yaml, update the keys, and configure your coding agent, e.g. Codex to use it. It’s also a good way to see what prompts are being sent by the various harnesses. smolagents is a new agents library from HuggingFace. It seems simple enough to use. whisper-flow does real-time speech transcription! Switchboard-1 is a labelled audio corpus with ~260 hours of speech. It has ~2,400 calls among 500+ speakers in the US. Cloudflare tunnel is like ngrok but more permanent. It’s a bit more complex, too. But given CloudFlare’s liberal free tier, it’s a good, viable option for long-term local hosting. John Wheeler: “We live on an island surrounded by a sea of ignorance. As our island of knowledge grows, so does the shore of our ignorance.” A great way to understand how ignorance actually grows as you learn more. justhtml is a fast enough pure Python fully HTML5 compliant library. For a faster, mostly compliant solution, html5-parser with lxml works. There is little reason to use Redis. There are several clones you can use. Databases in 2024: A Year in Review Microsoft’s Garnet KeyDB (only Linux) ValKey (only source) DragonFly (only Linux) ReDict (only Linux) Every few years, something comes along trying to replace relational databases and SQL, and gets absorbed. YouTube Key value stores. People soon realize they need more features, e.g. indices. MapReduce systems. Most MapReduce vendors put SQL on top of SQL. Then the Hadoop market crashed. (But HDFS, S3, distributed storage systems are a good idea) Document Databases. JSON. SQL absorbed that. SQLite 3.45+ supports even JSONB. DuckDB, of course, has JSON. Column Databases. Again, these introduced SQL. Graph Databases. SQL:2023 introduced graph queries via SQL/PGQ (Property Graph Queries). DuckPGQ beats Neo4J Array Databases. SQL:2023 adds SQL/MDA which allows for matrix operations. But specialized databases might make sense in this category. Vector Databases. Every DB is adding support for this. TheAgentCompany is a benchmark of real-world tasks like: Arranging a meeting room Analyze a spreadsheet Add a Gitlab wiki page Salvatore Sanfilippo (antirez - Redis) finds DeepSeek v3 comparable with Claude 3.5 Sonnet. YouTube He also passed a paper and his code to compare them. A useful prompt. YouTube

This week, I learned: Some management philosophies used to be successful but are no longer as effective. ChatGPT Command-and-control hierarchy Taylorism: deep specialization Seniority-based advancement Annual performance reviews (without continuous feedback) Up-or-Out promotion models Confidential strategic information Narrow job descriptions Relying on formal authority Some management philosophies have been around for millenia. ChatGPT Lead by example Fairness and empathy Clear, consistent communication Delegation and empowerment Strategic planning and foresight Consistent rule enforcement Rewarding merit Leadership by virtue and character Interview with Liang Wenfeng, CEO of DeepSeek: In the face of disruptive technologies, moats created by closed source are temporary. Even OpenAI’s closed source approach can’t prevent others from catching up. So we anchor our value in our team – our colleagues grow through this process, accumulate know-how, and form an organization and culture capable of innovation. That’s our moat. ...

This week, I learned: A clever idea. Give an LLM a chapter from a textbook. Ask it to generate a unique, playable game to help me learn theconcepts for an exam. Page Bailey What would be the cost of storing about 500GB of LLM cache logs and 5 million write requests per month? CloudFlare KV: $250 + $25 / month Ref MongoDB: $125 + $5 / month Ref S3: $0.0115 + $25 / month Ref + ? CloudFlare R2: $0.0075 + $22.5 / month Ref Satya Nadella prepares for meetings by asking Copilot to tell him everything he needs to know about the client from the CRM, emails, meeting transcripts etc. He shares that colleagues who annotate it further for him. That’s using AI for reasoning and collaborating with colleagues. Satya Nadella | BG2 w/ Bill Gurley & Brad Gerstner WOW. This is how a software agent will work alongside humans: Fix issue #5478: Add color to the line next to “Ran a XXX Command” based on return value - using @openhands-agent. aisuite by Andrew Ng is a unified interface to LLMs. Sort of like an openai library across multiple providers. Learnings from Best of 2024 in Agents (from #1 on SWE-Bench Full, Prof. Graham Neubig of OpenHands/AllHands) Passing code execution as a tool is more powerful than granular tools. You combine multiple tools and tool calls into one. You move code to the data rather than the other way around. Mostly, you need bash, Python (or Jupyter), file manager, web browser. UI: Go where the user is, instead of bringing them to you. A remote runtime is a critical component. Claude 3.5 Sonnet (20241022) and Claude 3.5 Haiku (20241022) perform best on SWE Bench, followed by Deepseek V3, then O1 2024-12-17. X Browsers support SVG favicons as data URLs. So I used this SVG (generated by Claude via Generate a simple, interesting SVG favicon. Keep the SVG size VERY small but it should be inspiring.) Since HNSW indexing is an overhead, just use NumPy matrix multiplication to calculate cosine similarity. For 1M vectors, it takes ~0.05 seconds. A 1M vector dataset handles ~2GB of text at a chunk size of 2K chars. In short, if you’re embedding <2GB of text, just use NumPy. DuckDB’s VSS extension HNSW index + Embeddings (2K chunks of 512 dimensions) takes up roughly 2.5X the size of the original data. Embedding 554 files of ~4,456 KB took 710 seconds. Creating the index took 660 seconds. The resulting DB was 18.1 MB. How to use LLMs in market research. Use LLMs with search for secondary research. Create different personas and run user surveys on them. This paper used 1,052 real-life interview audio transcripts as agent memory to simulate people Generate your market research report using LLMs. Given about 30 generations, Llama 1b outperforms Llama 8b. Ref OpenAI introduced a developer role in addition to the system role. This is mainly for o1. The API is backward compatible - and also forward compatible. OpenAI Em dashes are a strong sign of ChatGPT use. Curly quotes too. Reddit CloudFlare has multiple SSL modes when proxying requests. Off (no encryption): No encryption between browsers and Cloudflare or between Cloudflare and origins. Everything is cleartext HTTP. Flexible: Browsers to Cloudflare is HTTPS, Cloudflare to origin is HTTP. Useful to set up CloudFlare as a HTTP Proxy. Full: Browser to Cloudflare matches browser request. Same protocol is used for Cloudflare to origin, without validating the origin’s certificate. Use for self-signed or otherwise invalid certificates. Full (strict): Similar to Full Mode, but with validation. Strict (SSL-Only Origin Pull): Cloudflare always connects to the origin over HTTPS with certificate validation. Getting this wrong can lead to a HTTP 526: invalid SSL certificate Medical coding is an area ripe for LLMs. Ojasvi Yadav created a repo that uses hierarchical classification (rather than embeddings) to find the right coding. Gemini models seem to understand medical terms better than others. RapidClaims, funded by TogetherAI, is apparently working on this problem. Document to Markdown Converters: PyMuPDF4LLM uses MuPDF. Requires PyTorch. PYTHONUTF8=1 uv run --with pymupdf4llm python -c 'import pymupdf4llm; h = open("pymupdf4llm.md", "w"); h.write(pymupdf4llm.to_markdown("$FILE.pdf"))' markitdown from Microsoft. PDF via PDFMiner, DOCX via Mammoth, XLSX via Pandas, PPTX via Python-PPTD, ZIP, etc. PYTHONUTF8=1 uvx markitdown $FILE.pdf > markitdown.md Docling by IBM. Unable to install via pip on Windows AND on Linux. MegaParse uses libreoffice, pandoc, tesseract-ocr, etc. Requires OpenAI API key. Awesome Tabular LLMs compiles encodings of tables for LLMs. What’s the best way of encoding tabular data for LLMs? Looks like including the cell address helps. Here is an explanation from ChatGPT aspose-words is a Python library that converts documents with many formats (Word, RTF, PDF, HTML, Markdown, EPUB, etc.) Discourse does not support searching across multiple forums. Instead, search for the term in all forums. Example. Then scroll through the results. Then, in the console, hide the ones you don’t want. Example: Hide posts that are not in the “Tools in Data Science” category: $(".badge-category__name").filter(d => d.textContent == "Tools in Data Science").map(d => d.closest(".fps-result")).filter(d => d).forEach(d => d.style.display = "none") How are software engineers are future-proofing their careers in the face of LLMs? Leveraging LLMs as Force Multipliers Use LLMs for repetitive tasks, rapid prototyping, exploring multiple approaches, data extraction and brainstorming, providing feedback. Explore prompting techniques, integrate LLMs into their workflows, and develop strategies for validating and refining LLM-generated code Focusing on higher-level skills that llms struggle with Systems Thinking and Architecture: code readability, extensibility, testability, and maintainability Problem Solving and Critical Thinking: define problems clearly, break them down into manageable parts, and reason through complex scenarios. LLMs produce plausibly incorrect code. Communication and Collaboration Domain Expertise Exploring Adjacent Roles: product management, technical leadership, or consulting. Involve more interaction with clients and stakeholders. Developing “Evergreen” Skills: debugging, system administration, and security. Or outside of software engineering, such as trades or other hands-on vocations. Scepticism: LLMs may not reach a level of sophistication that would render their expertise obsolete. Complex problems, understanding context, and producing high-quality, maintainable code. Examples of agentic AI Text-to-SQL automated business analyst: A system that generates SQL queries from natural language, handles errors, creates visualizations, and includes a FAQ component. The author calls it “constrained agentic AI.” Data source querying system: A bot that queries multiple SQL and API data sources, selecting tools and reformulating tasks as needed. Cursor (agentic mode): An LLM-powered VS Code fork that chains together various LLM capabilities (code generation, applying changes, linting suggestions, terminal commands, codebase RAG) to reduce user prompts. Vulnerability finding system: A system that uses LLM agents to discover novel vulnerabilities in open-source web applications. The agents leave traces of their actions. Marketing strategy generation system: A system using approximately 60 agents to generate marketing strategies. Restaurant finder: A system that searches for restaurants based on dietary preferences and group size, and downloads social media information. Proofreading and editing of transcripts: LLM agents apply specific customer requirements to transcripts after human editing. Meeting notes and action items generator: A system that generates meeting notes and action items. O’Reilly auto parts customer service agent: An agent demonstrated using RAG. UI enhancement agent: An agent that added features like language locales and dark mode to a UI.

This week, I learned: What to use for hosting: ChatGPT GitHub Pages: Static websites, medium files Cloudflare Pages: Static websites, global delivery Vercel: Frontend frameworks (e.g. Next.js) with high DX and ISR, small files Netlify: JAMstack projects, minimal back-end, moderate files Glitch: Small static projects Render: Full-stack apps requiring databases and server-side compute Firebase Hosting: Small sites, limited large files Archive.org: Public archival, large files Google Drive: File sharing, large files Dropbox: File sharing, moderate files Cloudflare R2: Static assets, large file delivery Anthropic defines agents. Building effective agents + Cookbook Augmented LLMs are LLMs enhanced with augmentations such as retrieval, tools, and memory. Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Prompt chaining: Pipe each LLM output to the next LLM. A->B->C->Z. E.g. Write report, then translate. Extract results, then verify them. Successively ask follow-up questions. Routing: One LLMs decides which other LLM to call next. A->B|C|D->Z. E.g. Evaluate complexity, then pick the right model. Classify request time, then pick the right prompt. Parallelize: Sectioning (and Orchestrator-workers): Break tasks into independent subtasks, then aggregate. A->B+C+D->Z. E.g. Evaluate contracts against different clauses in parallel. Parallelize: Voting: Run same task multiple times, then vote. A->B+B+B->Z. E.g. Review code for prompt injection using different prompts. Evaluate content safety with different thresholds. Evaluator-optimizer: One model checks another in a loop. A->B->A->B->…->Z. E.g. Literary translation. Self-healing code. Policy violation checks. Human-in-the-loop Checkpoints: The workflow explicitly requests human review at certain stages. A->B->(Human)->C->Z. E.g. Sensitive content review. High-stakes decision making. Ambiguous tasks. Agents are LLMs that dynamically direct their own processes and tool usage, consulting tools or the user as needed. To download YouTube subtitles, use: yt-dlp -q --skip-download --convert-subs srt --write-sub --sub-langs "en" --write-auto-sub --print "requested_subtitles.en.url" "$url" Simon Willison o1-preview diagnoses better than doctors. Harvard OpenAI’s release of ephemeral tokens via sessions (valid for 1 minute) are a useful way of exposing apps for public demos. Currently it works only for the Realtime API, though. SpreadsheetLLM is a way of encoding spreadsheets in an LLM friendly format. It’s good for 1K+ rows. For lower, Markdown > XML > HTML. However, Table Meets LLM suggests that HTML > XML > Markdown, so this is unclear. #HARD prompt. Ask video generators like SORA to generate text in videos. It is of average quality. GPT 4o Mini Realtime was released. A realtime conversation will cost ~50c/hr. About 36c for input, 72c for output. (I extrapolated from the 6c/min audio input cost for GPT 4o Realtime when it was $100/MTok. GPT 4o Mini Realtime is $10/MTok input and $20/MTok output.) This is an interesting way to understand software. Generate a Mermaid sequence diagram showing interactions based on this code. Ref The King James Bible and all Harry Potters, each, are about $1M tokens (rounded off). markdown2 is the new de facto Markdown library for Python. Claude 3.5 Sonnet is way ahead of competition on the LMSYS Webdev Arena Raspberry Pi 5 has a faster CPU, more RAM and GPU, 4K support, multiple USB 3 ports Government websites like the official press releases cannot be crawled from outside India. Hence the need for server farms in India!

This week, I learned: **/*.md can search for all Markdown files. Julia Evans Windows 11 2024 Update features: Ref Live captions (via the tray) can transcribe audio and microphone. Cocreator in Paint lets you draw crudely and enhances it with AI. The neat UI is a slider that lets you control how close it should be to your drawing. Voice Clarity automatically cancels echo, reduces background noise, and minimizes reverb. Studio Effects (via the tray) lets you apply camera effects on all apps. Eye contact feature is CLEVER! sudo lets you run commands with admin privileges from the command line. source Roaming RAG is an alternative to RAG without the vector database. Applicable to well structured documents, e.g. technical books, manuals, etc. Create a hierarchical outline of the document. Code Keep the top-level headings. Preserve the first ~100 characters of opening text from each section. Present the second-level headings, but without any subsidiary content. Provide each section a unique 8 digit hex identifier. Each section heading is followed by a guiding comment for the model: Section collapsed - expand with expand_section("{identifier}"). Then read the relevant sections as context to answer the question. Code Traffic to StackOverflow has fallen considerably. Especially from young and Indian developers. StackOverflow revenue is down. Via Prashanth. They’re exploring: Licensing their content. (Meta says high quality content improves LLM performance by 30% on HumanEval) Enterprise StackOverflow for system integration Fine-tuned versions of Enterprise Stackoverflow for enterprises Integrate StackOverflow within your IDE. Ask questions, post directly I surveyed the Gramener QA team on how they were using LLMs. 7 used it for code generation (e.g. date extraction, regex generation) 4 used it for learning (e.g. Robot Framework, how to define test cases, API usage) 3 used it for formula generation (e.g. Excel) 2 used it for test scenario identification 2 used it for test data generation 2 used it for comparing expected vs actual datasets 1 used it for data type identification (e.g. given sample values, identify the data type). 1 used it for evaluating resulting (LLM as a judge) I asked the Straive Digitalized Operations team what management techniques they would apply to manage LLMs. Here are the responses: Ask better questions. (Prompt engineering.) Create templates or step-by-step instructions. (Chain of Thought.) Ask for multiple options and pick from the best options. (Agentic approach?) Training. (Fine tuning.) Price weaker responses lower. (Stratified model pricing?) “LLM hallucinations are a good thing. They are a sign of diversity, allowing us to improve the answer by exploring multiple paths.” – A colleague from Straive. Hyperbrowser is a cloud based puppeteer service. Bedrock Llama models can’t be directly called with their model names. You need to use their inference profile names, e.g. us.meta.llama3-2-11b-instruct-v1:0 if the model is in a US region. Hacker News RSS is a good way to get RSS feeds from Hacker News. It’s also a good way to understand how to convert a news source into RSS feeds. BlueSky has RSS feeds too When embedding using a SentenceTransformer.encode(docs) it’s best if we embed with smaller docs and call it multiple times (rather than embedding more at once). On Colab T4, for gte-base-en-v1.5, when embedding 1,000 docs of up to 8K chars each, here is the TOTAL time it took, based on batch sizes (lower is better) 1 doc per call: 10s 2 docs per call: 13s 4 docs per call: 19s 8 docs per call: 23s 16 docs per call: 32s 32 docs per call: 40s Running embeddings without a GPU is extremely slow. It takes ~2.4 seconds per string.

This week, I learned: ChatGPT uses several unusual unicode characters for citations. Ref NumLock can be dangerous. An IT support team member took control of Radheya’s screen while debugging and had turned on NumLock. Radheya’s login failed after that. After 5 tries, he was locked out. With LLMs, most architectural decisions are no longer one-way doors. Steve Yegge The cost of intelligence is trending to zero. How do we plan for this? Logan Kilpatrick If you are not planning for the price of intelligence to go to zero, the next 3-5 years are going to incredibly disruptive to your business / life. The important but not stated caveat: consumer willingness to pay for AI is going to go up (a lot). It will be fascinating to watch consumer willingness, cost, and the amount of AI being used all move in different directions. Everyone building things with AI has an economic incentive to limit the amount of AI because of cost, which inherent limits the value prop. This will change as intelligence goes up and cost goes down. What this means is: Admin automation: Administrative tasks vanish into background AI. Booking meetings, managing finances, or even planning family activities will require less thought. Hyper-personalization: Individuals get tailor-made everything—from medical advice to product recommendations to daily schedules. Systems learn your quirks. AI co-brains: AI co-worker “assistants support you at any moment. Productivity soars in knowledge work. “I’ll have my AI follow up becomes a normal response. Humanity valued more: As AI handles rote tasks, humans move up the value chain, focusing on creativity, empathy, or the “last-mile decisions. New business models: AI experts as a service Embedded AI Solutions AI micro-services for smart-calls Distributed AI Arena Hard is a set of hard prompts to test LLMs. Here is the code and evaluation LLMs can detect clear outliers easily. PROMPT: Which is the outlier in this dataset: (1,7), (2,7), (3,6), (4,6), (5,5), (6,1), (7,5), (8,3), (9,1), (10,1) (ANS: (6,1)) 🟢 GPT-4o on ChatGPT gets this. GPT-4o Mini on the API gets it too. 🟢 Gemini Pro, Flash, Flash 8b gets this right straight away, without even thinking. 🟢 Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3.5 Haiku get it on LLM Foundry. 🔴 Claude.ai, where it visualizes it and gets it wrong. 🟢 Nova Micro, Lite, and Pro get it right. 🟢 Llama 3.1 70b gets it right. 🔴 Llama 3.2 8b gets it wrong. Llama 3.2 70b, Llama 3.1 8b enter repetition. To install Docker on Windows without admin privileges, use net localgroup docker-users "your-user-id" /ADD A non-administrator in a Google Groups domain can only add 200 emails to a group from the UI directly without invitation at a time. The only programmatic way to add users is for an administrator to add them. Even apps that use the Google Admin SDK need an admin to log in to access the relevant API. Take 100% of your work, including complex, multi step processes and put it into an LLM. It might fail at some but you will discover the limitations. I emailed Straive employees about their use of LLM Foundry - the internal LLM portal. I picked ~500 non-users from teams that otherwise have high (30%+) usage. Reasons they didn’t use it were: 40% had not heard of it. 40% were unclear of the benefits 20% didn’t have time 45% feel they don’t have enough information and training to use it Some feedback Sharing training videos will help Live training sessions that allows for Q&A will help Developers prefer detailed documentation The same prompt gives different results Possible solution: Email non-users introducing the tool and sharing a quick 15-minute tutorial and a 1-page quick start. My notes on the Amazon Nova models. More on Hacker News Nova Micro (3.75c/MTok) has the same cost as Gemini 1.5 Flash 8b but does not support images or documents. Nova Lite (6c/MTok) has about the same cost as Gemini 1.5 Flash 002 and supports images and documents (but not audio or video). It may be a good alternative. But GPT-4o mini, which is 2.5X costlier, is much better. (It partly passes the Gr brx vshdn Fdhvdu flskhu? test which Nova Lite fails.) Nova Pro (80c/MTok) is cheaper than Gemini 1.5 Pro and a lot cheaper than GPT 4o, but does not match their quality. LLMs are great at convincing you of wrong things. A danger and something to be wary of. Ethan Mollick Fish eye text summary is a great way to read text while summarizing context. Amelia Wattenberger DuckDB’s JavaScript API is still under development. For example, JSON, ARRAY are not insertable. Plus, re-creating persistent HNSW indices crashes. What’s a good text splitter library to use in JS? LangChain: If you use it, use it with a simple wrapper decoupled from the implementation (e.g. your own parameters) that you can replace later. Popular Fit-for-purpose. MarkdownTextSplitter which inherits from RecursiveCharacterTextSplitter is what’s needed in most cases. Unstable Poorly maintained Python docs indicate version 0.0 but it is in 0.1 Under-maintained Last update was 3 months ago, 13 Sep 2024 LlamaIndex: Popular Not an ideal fit. MarkdownNodeParser does not support chunk size. SentenceWindowNodeParser does not capture Markdown headings.

This week, I learned: Gists are a good place to store static files for posterity as well as throwaway files. But, they’re just git repositories. So there may be no advantage over GitHub repos. GPT-4o Audio supports tone control via XML tags like <cough>..., <laugh>..., etc. But at ~$15/hr of output, it’s too expensive. Ref Mridula’s son gave a live commentary of what he was doing on Minecraft and ChatGPT gave him live evaluation and coaching. E.g. “Great strategy! Getting to the launch pad early can give you a huge mobility advantage. Making the bridge wider is also a smart move to prevent accidental falls. With this plan, you’re setting yourself up for success. This is a great way to interact with LLMs. Gemini’s JSON mode returns JSON with keys in alphabetical order. I think. Emperical evidence. This is unlike OpenAI which explicitly returns the keys in the order specified. To solve this, order the keys alphabetically. HTMX focuses on HTML over JS. Like server responses being HTML snippets not JSON. But I need front-end over back-end. Client side apps. HTMX doesn’t help much there, e.g. templating, or just plain JS code. htmx client side templates do can convert JSON to HTML. I installed the OpenAI Desktop App as well as Claude for Desktop. They take up too much RAM (260MB and 750 MB respectively on startup - though this varies.) The ChatGPT web page takes ~100MB incrementally, so I wrote an AutoHotkey script to switch to the first open (or recently closed) ChatGPT tab on Brave. I tried LIDA from Microsoft, after almost a year of its release. A few notes: Just running uvx lida ui --port 8080 --docs works. But I needed to use export TCL_LIBRARY=C:/Users/Anand/AppData/Roaming/uv/python/cpython-3.13.0-windows-x86_64-none/tcl/tcl8.6 to point it to my TCL installation for charts to work. I also chose to export OPENAI_BASE_URL=https://llmfoundry.straive.com/openai/v1 I also chose to replace gpt-3.5-turbo-0301 (the default model) with gpt-4o-mini in lida/web/ui/component* It’s quite impressive. OpenAI allows multiple system messages. I learned this browsing through the LIDA prompts. Anthropic’s Model Context Protocol lets any apps integrate with LLM Apps. LLM Apps are becoming the new operating system. Competitors, beware. I spoke at Automating Data Visualizations using LLMs at SUTD. Apparently, using LLMs to write code is much more common than writing code to use LLMs. I ran a quick quiz. Have you used ChatGPT or any LLM? 35 / 35 raised their hands. Have you written code using an LLM? 34 / 35 raised their hands. (I was impressed.) Have you uploaded a spreadsheet to an LLM for analysis? 15 / 35 raised their hands. Have you programmatically called an LLM API? 6 / 35 raised their hands. With LLMs, fostering innovation is a new path to profitability. Companies are increasing innovation team sizes. Productionizing that is the next. Some initiatives are: Convert popular demos into starter kits Create and evangelize trainings on solutions and solution techniques Create larger pools of capacity to build innovation and productionize it Andrew Ng Explores The Rise Of Al Agents And Agentic Reasoning | BUILD 2024 Keynote Innovation is now a path to production. People are able to build 20 prototypes at the cost of one and see which sticks Machine learning is much faster. Things that took months now date days. But engineering and evaluations are only slightly faster and have become a bottleneck A good analogy to zero shot prompting is to ask a person to write an entire essay without pressing backspace even once Andrew scenes to align with the line chain definition of agentic workflow, which is about agents being able to craft their own control flows People find it very easy to understand agentic workflows once they read through the code Reflection or feedback is a useful agentic pattern In multi-agent collaboration, it may be the same underlying model that is acting as different agents. But just like we find it useful for the same CPU to run multiple processes and each application is its own abstraction, agents of useful abstraction It’s hard to summarize a large document using RAG. But you can directly add answers to such questions into the corpus, e.g. by adding a “summary” section, and other answers to common questions. CloudFlare workers can bundle any kind of files, including text, data, and WASM. Docs AssemblyScript can compile TypeScript to WASM. Here’s what I learnt Here’s a convenient pattern to git commit a directory but nothing else in it (e.g. a build/ directory). Add a .gitignore file with * followed by !.gitignore. Only the .gitignore file is tracked. Ultravox lets you build voice agents at 5c/min = $3/hr (OpenAI is 6c input, 24c output). Or clone their repo. Idle call time is counted towards cost. So cost may be higher than OpenAI. Voice cloning quality is average. Very distinctive voices are just partly identifiable. Supports tool calls (from their server). Their API is simple but the docs have minor errors (e.g. a trailing comma in the JSON, which leads to an error) reducing confidence. LLMs may be good at derived data generation. For example, given a database schema, what derived columns would be useful? What derived views would be useful? The O1 model does not have a mechanism to control the amount of tokens to spend on reasoning. DeepSeek R1 might, but the API is not out yet. The OpenAI Desktop App can interact with native applications, e.g. read from Terminal, VS Code, etc. This takes it on a path to becoming a copilot for ANY apps. Putting every copilot app and every LLM integration under threat. Crawl4AI and Firecrawl are tools / libraries to convert websites into LLM Friendly Markdown and extract structured data using LLMs. Don’t try and solve specific problems. Pass the entire context to an LLM and get a comprehensive solution. Most doctors, for example, ask specific search-like questions instead of uploading the entire case history and asking for a diagnosis, and perform workse than LLMs. Ethan Mollick

This week, I learned: OpenAI lets you download GPT instructions and execute arbitrary code in their containerized environment. This is not a bug. Ref BM25 works as follows: Ref For each query term in the query, sum up the product of: Inverse document frequency = LN(% of docs without the query term + 1) – with a small tweak Term frequency = freq / (freq + k) – where k is usually between 1.2 to 2. Returns 0-1 with diminishing frequency benefit k is multiplied by Document length normalization = 1 - b(1- DocLength/AvgDocLength). Longer documents have larger k, dampening frequency benefits. Some implications: The actual BM25 score has no meaning. It’s just useful for ordering BM25 scores for 2 queries can be compared ONLY IF the document sets don’t change A list of Markdown to Website converters on this thread: Jekyll - Ruby - 2008 MkDocs - Python - 2014 GitBook - JavaScript (Node.js) - 2014 MkDocs Material - Python (MkDocs-based) - 2016 Docsify - JavaScript - 2016 MdBook - Rust - 2017 Antora - JavaScript (Node.js) - 2017 Docusaurus - JavaScript (React) - 2017 JupyterBook - Python - 2019 Keenwrite - Java - ~2019 Honkit - JavaScript (GitBook fork) - 2019 Nextra - JavaScript (Next.js) - 2020 Astro - JavaScript/TypeScript - 2021 Hugo Book - Go (Hugo-based) - ~2020 Clowncar - JavaScript/Node.js - ~2021 Quarto - R and Python - 2022 Starlight - JavaScript/TypeScript - 2023 DuckDB has an LLMs.txt. Today, 38 repos on GitHub support it When identifying LLM use cases, it helps to tell LLMs what they can do. I use one or more of a list like below: Core capabilities: Text Generation: Produce coherent and contextually relevant text across various domains. Image Generation: Create realistic images that match the style and content of a given reference image. Text to Speech: Convert text into natural-sounding speech with appropriate intonation and rhythm. Speech to Text: Transcribe and interpret spoken language. Vision: Analyze and describe visual content from images. Video Analysis: Summarize and extract information from video content. Text to Video: Generate realistic (and surrealistic) videos from text descriptions. Function Calling: Execute predefined functions or access external tools to perform specific tasks. Structured Output: Generate structured outputs like JSON, XML, HTML, YAML, DSLs, etc. Tool Use: Utilize external applications or APIs to enhance functionality. Code Generation: Write and debug code snippets in various programming languages. Cross-domain use cases: Summarization: Understand and condense lengthy documents into concise summaries. Translation: Convert text between multiple languages with high accuracy. Question Answering: Provide precise answers to user queries based on provided information. Reasoning and Planning: Solve complex problems and develop step-by-step plans. Personalization: Tailor responses based on user preferences and historical interactions. Dialogue Management: Engage in context-aware, multi-turn conversations. Data Analysis: Interpret and generate insights from structured data. Content Moderation: Identify and filter inappropriate or harmful content. Sentiment Analysis: Detect and interpret emotions and opinions in text. Robotics Integration: Interface with robotic systems for control and decision-making. Knowledge Retrieval: Access and present information from vast datasets or knowledge bases. Creative Writing: Generate poetry, stories, and other creative content. Educational Assistance: Provide explanations and tutoring across various subjects. Ethical Reasoning: Assess scenarios for ethical considerations and implications. Accessibility Support: Assist users with disabilities through tailored interactions. Simulation and Modeling: Create predictive models and simulate scenarios. Domain-specific use cases: Legal and Medical Assistance: Offer information and guidance within legal and medical domains. Gaming: Generate narratives, dialogues, and scenarios for interactive entertainment. Scientific Research: Aid in literature reviews, hypothesis generation, and data interpretation. Financial Analysis: Analyze market trends and provide investment insights. Cultural Competence: Understand and respect diverse cultural contexts in interactions. Security Applications: Detect and respond to potential cybersecurity threats. Environmental Monitoring: Analyze data related to environmental changes and sustainability. Healthcare Support: Assist in patient monitoring, diagnostics, and personalized treatment plans. Supply Chain Optimization: Enhance logistics and inventory management through predictive analysis. Customer Service: Provide automated support and resolve customer inquiries. Market Research: Analyze consumer behavior and market trends for business insights. Content Creation: Generate articles, blogs, and marketing materials. Virtual Assistance: Manage schedules, reminders, and personal tasks. Social Media Management: Craft posts and engage with audiences across platforms. Human Resources: Assist in recruitment, training, and employee engagement strategies. Event Planning: Organize and coordinate events, including logistics and communication. Travel Planning: Provide itineraries, booking assistance, and destination information. Real Estate: Analyze property markets and assist in buying or selling decisions. Agriculture: Monitor crop health and optimize farming practices through data analysis. Energy Management: Optimize energy consumption and monitor renewable energy sources. Transportation: Enhance route planning and traffic management systems. Urban Planning: Assist in designing sustainable and efficient urban infrastructures. Disaster Response: Provide real-time information and coordination during emergencies. Public Policy: Analyze data to inform policy decisions and predict societal impacts. Art and Design: Generate visual art concepts and assist in creative design processes. Music Composition: Create original music pieces and assist in songwriting. Language Learning: Facilitate language acquisition through interactive exercises and feedback. Historical Analysis: Interpret historical data and provide insights into past events. Philanthropy: Identify charitable opportunities and assess the impact of donations. Sports Analytics: Analyze player performance and game strategies. Fashion: Predict trends and assist in clothing design and merchandising. Culinary Arts: Generate recipes and provide cooking guidance. Astronomy: Analyze celestial data and assist in space exploration research. Psychology: Offer insights into human behavior and mental health support. Linguistics: Analyze language patterns and assist in translation studies. Archaeology: Assist in artifact analysis and historical site interpretations. Literature Analysis: Interpret literary works and provide critical analyses. Philosophy: Engage in discussions on ethical dilemmas and existential questions. Mathematics: Solve complex equations and assist in theoretical research. Physics: Model physical phenomena and assist in experimental design. Chemistry: Analyze chemical compounds and predict reactions. Biology: Assist in genetic research and ecological studies. Geology: Analyze geological data and assist in natural resource exploration. Meteorology: Predict weather patterns and analyze climate data. Oceanography: Study marine ecosystems and assist in ocean exploration. Anthropology: Analyze cultural data and assist in ethnographic research. Style of writing impacts output style a lot. E.g. Adding an evil laugh makes Claude more creative. Ethan Mollick For good structured mode output, we need good prompting. Mentioning examples and schema and “JSON” helps. When providing examples, using (user, assistant) message pairs helps (I think it’s because it’s easier for the LLM to parse). Using a {reasoning, answer} schema (with reasoning first) helps. Make reasoning concise and relevant Ref Arxiv We already know code in JSON is not a great idea. Ref Just adding 3 real examples and regurgitation helped GPT 4o play chess much better. Both techniques may have more general use in prompting. Simon Willison With Deno 2.0, the same .js file can run in Node.js as well as Deno. Example jspm lets you generate import maps against any CDN. You can click on htop columns on the terminal to sort by that column! Mouse events work on command line apps. Julia Evans Alt Text will very likely be a browser feature. It’s important for the Alt text to flow as part of the content when listening to the page. Perhaps even become a part of the browser APIs like speechRecognition. Langchain suggests multiple levels of agentic behaviour. LLM Call < LLM Chain < LLM Rounter < State Machine < Autonomous Langchain A HTML quine: A page that, when rendered as HTML, shows the HTML source code of the page! You can enable syntax highlighting just using fonts. Ref HTML is all you need shows examples of using HTML for notebooks instead of Jupyter, Observable, etc. Straive evaluated Gemini 1.5 Flash 002 and GPT 4o Mini for translation. Portugese: Flash is better than GPT 4o Mini. BLEU Word Overlap is 65.5% > 64.6% and METEOR (Semantic) is 84.9% > 78.9% Mandarin: Flash is better than GPT 4o Mini. BLEU Word Overlap is 25.0% > 15.9% and METEOR (Semantic) is 54.7% > 51.1% The problem with Accept headers is that you can’t link to them. Simon Willison Recraft v3 supports vector (SVG) generation Simon Willison. The output is 100% <path> elements (even for text). You get 50 free credits daily. Creating 1 image is ~2 credits. The API costs $1 per 1K credits. Some things I can create with it are: Base data visualizations that I can animate with code Icons in a specific style Comic strips Explainers for talks or student material Featured images for blog posts Architecture diagrams?

This week, I learned: Anthropic has single-plage docs for LLMs. Condensed version and Full version Malcolm Gladwell on the importance of self-correction Belonging to multiple social worlds is a good way to defend against no longer being good at what you used to be. Diverse values and social groups help. Self handicapping explains a lot about the world. You study late for a maths test - so you can fail for lack of trying, not aptitude. Ecosystems (e.g. sports teams) mitigate self-handicapping. You don’t have to be good in athletics to get the benefits. A slow runner gets the same discipline, pumping up, etc that a fast runner does Mono cultures are good to accomplish a known mission. Diversity is good to pivot during uncertainty. So, localize mono cultures Diversity helps only if there are sufficient numbers, or if they have enough power to change the organization’s thinking. Use a standardized password strategy, e.g. use the month like GramNov2024 (via Namit) Gemini has an OpenAI compatible API. Gemini Docs Ethan Mollick says Claude is solving MBA case studies well. x.com LLMs pay a lot of attention to the first 6 tokens. Ref This is an interesting article on “UI in the age of Gen AI”. Ref Google Open sourced Alphafold 3. Repo Cloudflare R2 has the same API as S3 but is cheaper Prefect.io is a good alternative to Airflow / cron. Can use for synchronisation tasks, e.g. Drive to server. But no Auth, UI params or config. Gemini transcription does not give accurate timestamps. Whisper does. But the quality of transcription is similar. Pass a complex data structure to Claude.ai and have it create an app to visualize it. It does well. Simin Willison Tech Council Ventures and Sunicon VC invest in early stage startups, and aloso provide them technology support (via Naveen)

This week, I learned: OpenFreeMap is a free embeddable OpenStreetMap tile server. You can use MapLibre GL (more features) or Leaflet (simpler) to render it. It offers styling and self-hosting. Zapier Actions are an easy way to set up custom actions like GMail / Google Calendar APIs for GPTs, since GPTs’ callback URLs keep changing. But they fail often, and don’t work on mobile. At least for me. LLM Vision Use Cases in manufacturing and earth sciences (via Shivku) Automated geoscience image descriptions Ref Interpret Wind Turbine photos and charts, construction monitoring, equipment maintenance & charts Ref Forecast weather based on cloud photos! Ref Analyze thermal image of solar panels, electroluminescence images for warranty claims, ROI estimates from Google Sunroof rooftop images Ref Corrosion detection in electricity towers, turbines, storage tanks, penstock. Interpret non-destructive test images Ref Google counts auto-completion when saying “25% of all the code is written by AI at Google”. “It’s a helpful productivity tool but it’s not doing any engineering at all. It’s probably about as good, maybe slightly worse, than Copilot.” YCombinator Workflow for AI video creation: Use Meshcapade (meshcapade.com) to generate body movement of a 3D-rendered character. Pass that video to Runway’s video-to-video model to generate any visual. Add music from Suno Ref Someone sorted the X and Y columns independently for regression. Ref Android keyboard learning only sends model changes back to server and not local keywords. Model changes are aggregated! Ref Here is a prompt for audio transcription using Gemini. Ref Transcription: Accurately transcribe the audio clip in the original language. Include all spoken words, fillers, slang, colloquialisms, and any code-switching instances. Pay attention to dialects and regional variations common among immigrant communities. Do your best to capture the speech accurately, and flag any unintelligible portions with [inaudible]. Translation: Translate the transcription into English. Preserve the original meaning, context, idiomatic expressions, and cultural references. Ensure that nuances and subtleties are accurately conveyed. Capture Vocal Nuances: Note vocal cues such as tone, pitch, pacing, emphasis, and emotional expressions that may influence the message. These cues are critical for understanding intent and potential impact. Here are some approaches to large-scale classification of medical codes. ChatGPT Fine-Tuning LLMs on Medical Data: Enhance LLMs by training them on medical datasets, such as clinical notes and discharge summaries, to improve their understanding of medical terminology and context. Multi-Agent Frameworks: Implement a multi-agent system that simulates real-world coding processes with distinct roles (e.g., patient, physician, coder, reviewer, adjuster). Each agent utilizes an LLM to perform specific functions, enhancing interpretability and reliability. ArXiv Retrieve-Rank Systems: Develop a two-stage system where the LLM first retrieves potential ICD-10 codes and then ranks them based on relevance, improving precision in code assignment. ArXiv Embedding-Based Approaches: Use LLMs to generate embeddings for ICD-10 codes and medical texts, facilitating the matching of texts to appropriate codes through similarity measures. GitHub Hierarchical Classification: Leverage the hierarchical structure of ICD-10 codes by first classifying texts into broader categories before assigning specific codes, reducing complexity and improving accuracy. ArXiv Two-Stage Verification Models: Combine LLMs with verification models, such as Long Short-Term Memory (LSTM) networks, to validate and refine the codes suggested by the LLM, balancing recall and precision. ArXiv Also, a mixture of models approach might work. Feed any existing NLP model / rules as a second opinion. GraphRAG is better if data is naturally graph-structured. Else, it’s slow and fills up the context window with even vaguely related stuff. Vigneshbabu, AMAT. ChatGPT for Windows desktop supports real-time voice and a global shortcut (Alt Space). uithub converts GitHub repos to Markdown. Just replace “g” in “github.com/…” with “u”. Example WebContainers are a thing and Bolt.new uses them! Docling by IBM converts PDF, DOCX, etc. to Markdown. Like PyMuPDF4LLM but better. Check out Loom and Cleanshot are the recommended tools for screen recording and screenshotting. But Loom is paid and Cleanshot is Mac only. The Rubik’s cube has a Hamiltonian cycle through every one of its 43 quintillion states. Ref OmniParser is great at parsing screenshots and identifying bounding boxes. Recraft.ai is currently SOTA in text to image. It’s fairly impressive and could be a good alternative to Figma. Zed.dev is an AI code editor by the creators of Atom. It’s written in Rust and is blazing fast. It has native AI integration. Artificial Analysis has a bunch of new leaderboards and arenas. Open AI TTS leads the TTS Leaderboard. ElevenLabs is a bit behind. Recraft V3 > Flux 1.1 leads Text to Image Leaderboard Hertz-Dev is an open source realtime voice chat model. But it doesn’t fit in Google Colab T4’s RAM Chain of Thought reduces performance where thinking makes humans worse. Ref. Specifically: Artificial grammar learning Facial recognition Classifying data that has exceptions Creating a LLM-as-a-Judge That Drives Business Results by Hamel Husain. Get THE domain expert (or approver) as the tester. Create a dataset that is DIVERSE. Covers EACH combination of: Features Scenarios: e.g. multiple matches, no match, ambiguous request, invalid/incomplete input, unsupported feature, system error Persona: e.g. new user, expert user, non-native speaker, busy professional, technophobe, elderly user Generate data using existing data + synthetic data for each SPECIFIC combination of the above Evaluate based only on PASS/FAIL with a CRITIQUE detailed enough for a new employee. Include: Nuances: Something a failed response did well or a passed response didn’t quite do well Improvements: Suggest how model can improve Build an SPA to make it easy for the domain expert to review LLMs can be made to unlearn (copyright material) better by identifying components related to the knowledge to unlearn and applying a larger learning rate to these while leaving other parts unchanged. As opposed to low learning rates for all components. Ref

This week, I learned: Indian companies with 30+ employees MUST have 2.5%-15% of their employees as apprentices. Ref Textnow and TextFree provides a free phone number (like a virtual SIM). (But TextFree has more ads.) Keep using to avoid deactivation. No guarantee of retaining the number. Some banks don’t accept TextNow for verification SMS. But voice call is OK. Tello, Red pocket are cheap MVNOs with $5/month voice plans. Metro by T-Mobile and Cricket are other MVNOs. MintMobile and US Mobile have $15/month and $8/month data plans. The scientific discoveries that might have remained undiscovered for long if not for their discoverers Ref Newton’s discovery of the universal law of gravitation Einstein’s discovery of General Relativity McClintock’s discovery of Transposable Elements: genes that can turn physical characteristics on and off Mullis’ invention of the PCR that makes billions of DNA copies rapidly VibeCheck can predict a model based on its vibes 80% of the time. /llms.txt is a proposal to standardize /llms.txt files as a way to share LLM prompts. Jina AI Meta Prompt is an example Remotion system prompt is an example https://docs.fastht.ml/llms-ctx.txt https://docs.fastht.ml/llms-ctx-full.txt structuredClone deep clones objects in JS F5-TTS clones voices with just 15-second samples. Rust has crazy low memory usage too. Spawning thousands of child processes is common and OK these days. Ref SetInterval is a good idea in cyborg scraping. Ref GH CLI is quite good for deployment too, like Wrangler CLI. Enabling pages, setting secrets, etc. Restic is a CLI backup tool. Just like git. Works well with rclone. NotebookLlama is an open source podcast generator like NotebookLM Pragmatic Podcast (I forgot which one) Automate changelogs for your codebases. Convert past commits into attractive release notes automatically AI is going to be the consumer of many tools and logs. Build converters for these Speed of validation such as linting, testing, etc. will allow LLMs to iterate faster and WILL become more important Via Soumya Ranjan Vision embedding is useful in agile modeling Vision embedding models with SAM, Grounding Dino by meta, Alibaba does good stuff Vision embedding is more useful in batch than real time Embedding subtraction with vision embedding models like Dino AI code editors are not good with large code bases today. Keep the refactoring exercises to below 1000 lines. Also evaluate the ease of setting it up locally Deepseek Janus is a 1.3b model that can generate both text AND images (and also supports vision) Cohere Multimodal Embed v3 is available on Azure. Elevenlabs lets you create voices with a prompt. No need to even clone one! Runway Act One creates expressive character performances

This week, I learned: LanceDB is a more scalable alternative to ChromaDB. Written in Rust. Does not require a separate HSNW library. Meta has a bunch of image embedding models: DINOv2 creates image embeddings (Apr 2023) ImageBind is an embedding model for text, images, audio, and more (Jun 2023) Gemini has a code execution API! 0x0.st is an open API-based file upload + URL shortening service. You can dump files there temporarily. noVNC is a JavaScript VNC client. You can control a remote (virtual) machine from your browser. Friend is an always recording pendant that you can ask questions to. Anthropic’s new Sonnet model is even better at code. Plus it has the ability to extract coordinates from images. Ref Gemini sort-of supports diarization. Ref. I tried it and it’s OK but not perfect. #IMPOSSIBLE LLMs cannot diarize reliably yet. (Gemini just guesses the speaker differences.) Replit is good for hobbyists, Cursor for developers, and Pythagora & Bolt for non-developers building business apps. Ref

This week, I learned: SQL optimizations for multi-threaded web applications. Ref PRAGMA journal_mode = WAL. Improves performance for frequent writes. It allows concurrent reads and writes. PRAGMA synchronous = NORMAL. Improves performance. We might lose a few transactions but won’t corrupt the database. PRAGMA mmap_size = 128000000. Set global memory map for processes to share data PRAGMA journal_size_limit = 64000000. Limit WAL file to prevent unlimited growth BEGIN IMMEDIATE instead of BEGIN. Prevents writes to the journal file until the transaction is complete. Improves concurrency. AI seems to be slowing down apprenticeship since experts would rather use an AI than train an apprentice. Example: Robotic surgery. Ref How AI can improve education performance and engagement. Ref Student: Create study plan based on course and schedule Student: Focus on what you need to learn more Student: Align with your study style and pace Teacher: Grading MCQs Teacher: Writing conceptual guides “New collar workers” was coined by Ginny Rometty Embed tutor or document in video and ask for clarification! This is a new embedded interface. #TODO Playing Bad Apple in Minecraft. Ultra cool! OpenAI has a prompt generator. Currently it uses a meta-prompt but may later move to DSPy or Gradient Descent. Ref Great demo of using the Realtime API to read the latest Hacker News. Ref LLMs have reached the point where they can show a world, like CounterStrike, in near real time. Ref

This week, I learned: DuckDB supports function chaining DuckDB lets you create functions = macros HTML for People is a nice introduction to HTML. FlightRadar24 lets you watch airplanes live. sq is like jq but for SQL. Deno 2 is fully backward compatible with Node! via O1 is good at solving problems where the solution is easy to verify and generating options helps get closer to the solution Reverb ASR does diarration as well as transcription. It seems the state of art right now. Gemini Flash and Gemini Flash 8b can be fine-tuned at zero cost. Inference is at the same price! Ref Flux 1.1 Pro is released. I tried my Calvin & Hobbes test on it. Not great. ImageGen3 is better, ChatGPT is the best. Ref Revisiting text to speech models. Nothing much has changed since July 2024. OpenAI TTS: $15/1M chars Ref Deepgram Aura: $15/1M chars Ref Azure AI Speech: $15/1M chars Ref Google TTS Neural2: $16/1M chars Ref AWS Polly Neural TTS: $16/1M chars Ref Cartesia Pro: $50/1M chars Ref Elevenlabs Scale: $300/1M chars Ref GitHub co-pilot workspaces let you code using your mobile with AI and deploy it at one shot If you need an Ubuntu Docker container with Python, install it via uv rather than compiling from source. via VTracer is an open source library (and tool) to convert raster images to SVGs. via If you want to create a console.llm() function, a browser extension is the best way, because some pages have Content-Security-Policy that block eval, form submission, fetch from other domains, and script execution. PyPi lets you publish from GitHub Actions without a token. Also from Gitlab.com CI/CD and Google Cloud. ActiveState which made ActivePython, ActivePerl, etc. made these products paid for commercial use around 2013 after a series of acquisitions. Marimo supports: Publishing any notebook to static.marimo.app as a static app Creating a SINGLE link that embeds the ENTIRE notebook in the URL! Runnable via uvx marimo edit Parables on the Power of Planning in AI: Giving models about 30 seconds of thinking time consistently improves results - as much as increasing parameter size by a factor of 1,000 to 100,000! This works particularly well for verifiable results (code, math, etc.) Technique: Ask an LLM hundreds of times at low temperature and pick the most common one. (Google’s Minerva used this on the MATH dataset.) Better Technique: Ask an LLM hundreds of times. Pick the best solution based on an evaluation metric (reward model) Better Technique: Apply a reward model at EACH step of the process. OpenAI’s “Let’s Verify Step by Step” Late chunking is an interesting approach to adding context to embeddings. (I don’t understand it, but it’s cheap and effective.) DeepInfra offers embedding models as APIs at about 0.5 to 1 cent per MTok in an OpenAI compatible API. It also supports text-to-image models like flux.dev and speech recognition models like Whisper. Jake Heller: “One of the things we learned is (an LLM app) after it passes passes frankly even 100 tests, the odds that it will do, on any random distribution of user inputs, the next 100,000 100% accurately is very high.” OpenAI’s O1 is like Daniel Kahneman’s System 2 thinking - as against other LLMs’ System 1 thinking. Continue.dev is another AI coding editor. It supports OpenRouter. So now I have heard good things about: Github Copilot Cursor Cody Continue.dev (supports OpenRouter) Aider (supports OpenRouter) Maybe: Codeium Not: Amazon Q Developer

This week, I learned: ffmpeg on WASM works but is unstable and hard to use. You can’t use it in a CDN without CORS issues, since it loads ffmpeg-core via a worker. It often runs into buffer allocation issues. Exotel and Plivo provide voice & SMS services in India (like Twilio). Plivo is more customer friendly. Uber’s H3, Google’s S2, and GeoHash are geocoding systems. H3 offers uniform cell sizes and better distance measurement S2 offers higher precision (factoring in Earth’s curvature) for exact location matches GeoHash is the simplest There’s a movement towards embeddable databases on the cloud. MotherDuck is hosted DuckDB. Turso is hosted SQLite (with local sync, multi-tenant) StarBase DB is SQLite with an API on top of Cloudflare Durable Objects. Software 2.0 by Andrej Karpathy. This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two: the 2.0 programmers (data labelers) edit and grow the datasets, while a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces. Adaptive UI ideas: Adaptive Fields: Show only required fields based on what the user field so far. Smart Inputs: Dropdowns and auto-complete based on user’s context. Smart Themes: Change font size, contrast, theme guessing the user’s age and preferences. Dynamic Menus: Show what they might need to do next. Like Nokia’s right button, but using LLMs. Smart Tooltips: Check what the user’s doing (delays, confusions, previous clicks, current actions) and show relevant tips. Personalized Layout: Show only the relevant sections of the app. E.g. based on what they’re doing. Smart Charts: Create the right chart that solve the user’s question. Adaptive Back-end Dynamic APIs: Create endpoints on the fly based on user needs Dynamic Indexing: Create & update indices on the fly based on user needs Dynamic Schema: Create & update schema on the fly based on user needs Dynamic Migration: Migrate to a new database or OS or language as required Dynamic Queries: Create SQL/NoSQL queries to solve the user problem Dynamic RBAC: Figure out who needs permissions and why. Add OR REMOVE access as required Dynamic Logging. Log what’s required. Explain why it’s logged and what’s happening. Fix code that raised the error Dynamic Caching. Cache what’s likely to be required. Evict what may not be required. Figure out cache keys. Aider LLM Leaderboards show which LLMs code better. As of now, o1-preview > claude-3.5 sonnet on code editing claude-3-opus > claude-3.5-sonnet on code refactoring deepseek-coder-v gpt-4o-mini sucks. Jaro-Winkler Distance is a string matching algorithm that weights the start of a string higher. Passing the feed of the following to NotebookLLM is a good way to get caught up with news and summaries. A blog / WhatsApp group (e.g. The Generative AI Group, Sithamalli, etc.) A Google Group / mailing list (e.g. genainews, datameet) YouTube channels (e.g. Vertiasium, GitHub) Hacker News top stories Research papers Emails (skipping marketing emails) OpenAI Evals and Distillation has a clever design. They just convert filtered history to .JSONL files that can be an input to either. Speak is a language learning app based on OpenAI’s Realtime API. OpenAI’s Realtime API can be used in a text-to-text chat mode without needing to send the entire context. If the pricing works out right, this can be far cheaper than sending the entire conversation context. Ref Matching addresses with just embeddings works well. Combine it with simple hard rules. Ref OpenAI’s prompt caching works for images too – both linked and embedded Quotes on Graph RAG from a Generative AI WhatsApp Group. “Damn so literally nobody uses Graph RAG yet. Good to know.” ~Sumba “A big four consulting firm uses GraphRAG to retrieve related documents and excerpts from governance and compliance docs.” ~Vinayak Hegde (Microsoft) “Graph RAG is expensive and unnecessary in most of the cases.” ~Utkarsh Saxena ChatGPT’s advanced mode includes: “…you can use various regional accents and dialects.” Ref Source But the API can “laugh, whisper, and adhere to tone direction.” Ref Hume API (INR 6/min) is far cheaper than OpenAI’s real-time chat (6c/min input + 24c/min output) Devika is an open-source clone of Devin. DuckDB runs inside Pyodide Hungarian Jews have genetic diseases that increase their IQ. Gaucher’s disease, Torsion dystonia. People don’t like hard stuff like maths or science, so richer societies have fewer scientists Ethan Mollick feels Claude 3.5 Sonnet is better at style and critiquing blog posts than OpenAI’s o1 (which is better at reasoning.) News is going to be crazily disrupted again with voice mode. I can just listen to the topic I want In Singapore Airlines, You can’t wear your seatbelt loose You have to keep the laptop in the pocket in front, not on your lap, during takeoff You can’t charge during takeoff They verify if you ask for a veg meal and place a sticker on your seat Coders are more likely to edit LLM code. Non-coders don’t have that bad habit. Vaishnavi and Ranjeet edited code Indal and Koustav didn’t Coders are likely to get more out of an LLM because they know what it can do. But some non-coders will get more out of an LLM because they don’t know what it can’t do. E.g. Indal trying for a confetti animation, which is hard but do-able “You have to put in a lot of work to become productive at AI coding.” Simon Willison