An OpenAI model solved the Erdos unit distance problem. Erdos roughly said, “The number of edges of the same distance between N points can’t compound faster than close to 0%.” The model found a method of placing points so that it compounds at about 1.4%. This visualization is a crude way of visualizing how that works.
OpenAI launched Prism - an AI LaTeX IDE. It’s a boon for anyone writing LaTeX documents. All the nitty-gritty of formatting, syntax, etc. is handled by AI. You can collaborate, too. It brings the power of AI code editors to scientific document editing. It still has some way to go, though. I asked it to convert a portion of this paper into LaTeX. Here’s the image I passed: … and here’s the LaTeX output it generated: ...
The OpenAI text-to-speech cost documentation is confusing. As of 2 Nov 2025: GPT-4o mini TTS costs $0.60 / MTok input and $12.00 / MTok audio output according to the model page and the pricing page. They also estimate this to be ~1.5c per minute - both for input and output. It supports up to 2,000 tokens input. TTS-1 costs $15 / MTok speech generated according to the model page but the pricing page says it's $15 / MChars. No estimate per minute is provided. Is supports up to 4,096 characters input. TTS-1 HD is twice as expensive as TTS-1 I wanted to find the approximate total cost for a typical text input measured per character and token. ...
I charted the OpenAI GDPVal paper with industry compensation as the size and AI augmentation as color. Big green areas are we’re paying people where AI does better. Click here to see the interactive visualization. Clicking to see some actual tasks compared. I use this to check whom to ask advice: AI or professional. AI beats Personal Financial Advisors ~64% of the time. So I invested half my money using ChatGPT’s recommendation. (UTI Nifty 50, if you’re curious.) ...
This week, I learned: OpenAI supports a tool "type": "custom" that lets it write code as an argument to a tool call. Great for code / SQL generation. Even more powerfully, you can generate output following specific grammars, e.g. STL files, PostgreSQL dialect, Mermaid/PlantUML diagrams, OpenAPI specs, Vega-Lite JSONs, Cron expressions, GraphQL SDLs, Dockerfiles, Terraform HCLs, or any DSL! # #ai-coding The OpenAI playground has a GPT-5 Prompt Optimizer that can migrate prompts to GPT-5. Docsify 4.13.1 is 2 years old and uses [email protected] which is 5 years old. Newer plugins like marked-directive don’t work with it. Though docsify v5.0.0-rc1 is in development, it may be the better option for modern Markdown plugins. Here’s sample code. CommonMark has a powerful directive syntax proposal that lets you add classes, attributes, and arbitrary plugins to Markdown. For example, :abbr[MD]{#id .class title="Markdown"} for inline directives. Plugins exist for marked, markdown-it and remark. biomejs and dprint are gaining traction as prettier alternatives. I’m yet to try them but keen to explore. Skip biomejs for now. It uses tabs (not spaces) and does not respect .gitignore by default. Handling these is too much work. ⭐ Code generation is more flexible than tool calling. LLMs can’t write a tool-call loop, for example, but they can write code to run an API in a loop. So, I like telling the LLM to “write code using these APIs” than giving it APIs to tool-call. #ai-coding npx -y ccusage is an easy way of summarizing your Claude Code usage and cost. My cost so far (since 21 July) is about $10. The median session cost is ~50 cents. Most of it ($7) was from a single temporary coding chat that I kept continuing for way too long, building up the context window. # defuddle can be used in the browser to get the main content from web pages. A replacement for Mozilla Readability. # Modern Node.js Patterns for 2025 include these 5 features I’m excited by: Single-executable bundling. node --experimental-sea-config sea-config.json builds standalone binaries. ES Modules. Use node: prefix for built-in imports. import { createServer } from 'node:http'; Watch mode. Use node --watch file.js auto-reloads when file.js or dependencies change. Env file. Use node --env-file=.env loads .env as environment variables. node:test is a full-featured test framework with --watch and coverage. Concise explanations speed up decisions because they’re faster to read and understand (obvious). They’re also easier to combine with other ideas (less obvious). # I’ve been uncertain about htmx for some time now. This tutorial, HTMX is hard, so let’s get it right, convinced me that it’s too far from my mental model, so I’m unlikely to ever use it. ⭐ Slow, effortful practice (spaced recall, interleaving topics, self-testing) builds lasting knowledge but looks inefficient and doesn’t help with exams. # #beliefs GitDoc VS Code extension auto-commits and syncs notes. I dropped gitwatch in favor of this. It’s interesting that Gemini Deep Research cannot access Google Drive while Gemini can. On the other hand, ChatGPT Deep Research can access Google Drive but ChatGPT cannot. A trend that AI coding will only accelerate: “It is now possible for tiny teams to make principled software that millions of people use, unburdened by investors. … you need far less money and far fewer employees to reach far more customers. That wave is only just beginning.” # #ai-coding Typed languages are better suited for vibe coding. This will likely lead to the growth of typed languages (TypeScript, Rust, Go) but also of typing in untyped languages (e.g. Python) # #ai-coding Instead of Celery, Redis, Kafka, etc. as task queues, we could the file system as a message queue. For example, pending/task-01.json moves to wip/task-01.json to done/task-01.json. Folders for state/tags, files for task details. Foam is a note-taking VS Code extension. The WikiLinks, tags and backlinking features align naturally with Markdown note-taking. Via Steph Ango who uses Obsidian which nudged me to search for WikiLink-ing features in VS Code. I’m an open data hawk. But here are things I should remind myself of. # Privacy incubates creativity. People self-censor when watched. Privacy shields fragile ideas. Power assymetry. Big players can leverage openness more, e.g. Cambridge Analytics + Facebook data. Context matters. What’s harmless in one setting can be toxic in another. One-way door. Data can’t be unshared. Don’t scrap brakes dreaming of perfect roads. Anticipate tyrannical regimes / cultures. Not your call. You don’t share your neighbour’s medical records. One Punch Man is available as manga. I watched the anime first and assumed that came first. Apparently not. ⭐ In “kind” environments (stable rules, rapid and accurate feedback), specialize. In “wicked” environments (rules shift, feedback is noisy/late), generalize. ChatGPT Models’ ability to orchestrate longer workflows will improve. Factor that into your application design. Claude Code can already handle over 70 tasks in a workflow What happens when LLMs play Chinese Whispers / the Telephone Game? Here are learnings. ChatGPT Drift increases faster than linear with hops. Bigger models do better, but constrained prompts (“Copy the text exactly; change nothing.”) have a bigger impact. Low temperature improves copying fidelity. But even after “forgetting”, LLMs reproduce rare content if they’re trained on it. “In fact, React Native looks set to become the most engine-agnostic JavaScript runtime around”. The Many, Many, Many, JavaScript Runtimes of the Last Decade OMDb (simple) and TMDb (comprehensive) are API-friendly alternatives to the IMDb. copyparty seems one of the most feature-rich file servers out there. Single Python file, runs on any OS, works with any client, and optimized for speed. Video Quotes I enjoyed from Linus Torvalds’ TED interview I want to not have external stimulation. You can kind of see, on the walls are this light green. I’m told that at mental institutions they use that on the walls. It’s like a calming color. … the main thing I worry about in my computer is – it really has to be completely silent. If the cat comes up, it sits in my lap. And I want to hear the cat purring. I did not start Linux as a collaborative project. I started it as one in a series of many projects I had done at the time for myself, partly because I needed the end result, but even more because I just enjoyed programming. I’m actually not a people person. But I do love other people who comment and get involved in my project. The big point for me was not being alone and having 10, maybe 100 people being involved. Going from 100 people to a million people is not a big deal – to me. Well, I mean, maybe it is if you want to sell your result then it’s a huge deal. But if you’re interested in the technology and you’re interested in the project, the big part was getting the community. So Git is my second big project, which was only created for me to maintain my first big project. And this is literally how I work. Well, I do code for fun – but I want to code for something meaningful so every single project I’ve ever done has been something I needed. Apparently, my sister said that my biggest exceptional quality was that I would not let go. I can’t do UI to save my life. Good taste is about really seeing the big patterns and kind of instinctively knowing what’s the right way to do things. Companies like Google and many others have made, arguably, like, billions of dollars out of your software. Does that piss you off? No. No, it doesn’t piss me off for several reasons. And one of them is, I’m doing fine. But the other reason is – I mean, without doing the whole open source and really letting go thing, Linux would never have been what it is. I think one reason open source works so well in code (is that …) Code either works or it doesn’t. The Uses This site has interviewed professionals for decades. From their repo I scraped the top developer apps post 2020: CloudFlare has an Iceberg data catalog in R2 Data Catalog. Iceberg is like Parquet but supports metadata, time-travel, and schema edits. But I’m yet to find a single publicly accessible Iceberg catalog. Its open-data adoption is not as high as Parquet’s. Apache Iceberg vs Parquet Observable Notebook 2 is the new notebook format from Mike Bostock. It is vanilla JS and embeddable into other pages. THis would have been a big deal 2 years ago, but with the LLM ecosystem today, I’m not sure if it matters as much. To add CORS support to CloudFlare pages protected by Zero Trust, add a _headers file to your repo. (This is different from the Zero Trust CORS which allows automated logins.) Sample _headers that lets logged-in users fetch pages via fetch("...", { credentials: "include" }): /* Access-Control-Allow-Credentials: true Access-Control-Allow-Origin: https://your-site.example.com Access-Control-Allow-Methods: GET, HEAD Access-Control-Allow-Methods: * As corporates restrict the use of LLMs, I see employees purchasing personal laptops to use LLMs on. An interesting trend! openai-python has a CLI. You can run uvx openai api chat.completions.create --stream -m gpt-4.1-nano -g developer 'Translate to Chinese' -g user "Hello" for example Anthropic has an OpenAI compatible API at https://api.anthropic.com/v1/. Claude Code tips from Things that didn’t work by Armin Rocher #ai-coding Speech-to-text. Cannot stress this enough but talking to the machine means you’re more likely to share more about what you want it to do. I maintain some basic prompts and context for copy-pasting at the end or the beginning of what I entered. I ended up preloading executables on the PATH that override the default ones, steering Claude toward the right tools, e.g. running python asks it to use uv. I use the task tool frequently for basic parallelization and context isolation. Simply taking time to talk to the machine and give clear instructions outperforms elaborate pre-written prompts. Forcing myself to evaluate the automation has another benefit: I’m less likely to just blindly assume it helps me. Research indicates that we don’t know in advance which prompts will help. Evals beat prompt engineering. Ethan Mollick
I asked 50 LLMs to multiply 2 numbers: 12 x 12 123 x 456 1,234 x 5,678 12,345 x 6,789 123,456 x 789,012 1,234,567 x 8,901,234 987,654,321 x 123,456,789 LLMs aren’t good tools for math and this is just an informal check. But the results are interesting: Model %Win Q1 Q2 Q3 Q4 Q4 Q6 Q7 openai:o3 86% ✅ ✅ ✅ ✅ ✅ ✅ ❌ openrouter:openai/o1-mini 86% ✅ ✅ ✅ ✅ ✅ ✅ ❌ openrouter:openai/o3-mini-high 86% ✅ ✅ ✅ ✅ ✅ ✅ ❌ openrouter:openai/o4-mini 86% ✅ ✅ ✅ ✅ ✅ ✅ ❌ openrouter:openai/o4-mini-high 86% ✅ ✅ ✅ ✅ ✅ ✅ ❌ deepseek/deepseek-chat-v3-0324 71% ✅ ✅ ✅ ✅ ✅ ❌ ❌ openai/gpt-4.1-mini 71% ✅ ✅ ✅ ✅ ✅ ❌ ❌ openai/gpt-4.5-preview 71% ✅ ✅ ✅ ✅ ✅ ❌ ❌ openai/gpt-4o 71% ✅ ✅ ✅ ✅ ✅ ❌ ❌ openrouter:openai/o3-mini 71% ✅ ✅ ✅ ✅ ✅ ❌ ❌ anthropic/claude-3-opus 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ anthropic/claude-3.5-haiku 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ anthropic/claude-3.7-sonnet:thinking 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemini-2.0-flash-001 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemini-2.0-flash-lite-001 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemini-2.5-flash-preview 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemini-2.5-flash-preview:thinking 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemini-2.5-pro-preview-03-25 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemini-flash-1.5 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemini-pro-1.5 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemma-3-12b-it 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ google/gemma-3-27b-it 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ meta-llama/llama-4-maverick 57% ✅ ✅ ✅ ❌ ✅ ❌ ❌ meta-llama/llama-4-scout 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ openai/gpt-4-turbo 57% ✅ ✅ ✅ ✅ ❌ ❌ ❌ openai/gpt-4.1 57% ✅ ✅ ✅ ❌ ✅ ❌ ❌ amazon/nova-lite-v1 43% ✅ ✅ ✅ ❌ ❌ ❌ ❌ amazon/nova-pro-v1 43% ✅ ✅ ✅ ❌ ❌ ❌ ❌ anthropic/claude-3-haiku 43% ✅ ✅ ✅ ❌ ❌ ❌ ❌ anthropic/claude-3.5-sonnet 43% ✅ ✅ ✅ ❌ ❌ ❌ ❌ meta-llama/llama-3.1-405b-instruct 43% ✅ ✅ ❌ ✅ ❌ ❌ ❌ meta-llama/llama-3.1-70b-instruct 43% ✅ ✅ ❌ ✅ ❌ ❌ ❌ meta-llama/llama-3.2-3b-instruct 43% ✅ ✅ ❌ ✅ ❌ ❌ ❌ meta-llama/llama-3.3-70b-instruct 43% ✅ ✅ ❌ ✅ ❌ ❌ ❌ openai/gpt-4.1-nano 43% ✅ ✅ ✅ ❌ ❌ ❌ ❌ openai/gpt-4o-mini 43% ✅ ✅ ✅ ❌ ❌ ❌ ❌ qwen/qwen-2-72b-instruct 43% ✅ ✅ ✅ ❌ ❌ ❌ ❌ anthropic/claude-3-sonnet 29% ✅ ✅ ❌ ❌ ❌ ❌ ❌ deepseek/deepseek-r1 29% ✅ ✅ ❌ ❌ ❌ ❌ ❌ google/gemini-flash-1.5-8b 29% ✅ ✅ ❌ ❌ ❌ ❌ ❌ google/gemma-3-4b-it 29% ✅ ✅ ❌ ❌ ❌ ❌ ❌ meta-llama/llama-3-8b-instruct 29% ✅ ✅ ❌ ❌ ❌ ❌ ❌ meta-llama/llama-3.1-8b-instruct 29% ✅ ❌ ❌ ✅ ❌ ❌ ❌ openai/gpt-3.5-turbo 29% ✅ ✅ ❌ ❌ ❌ ❌ ❌ amazon/nova-micro-v1 14% ✅ ❌ ❌ ❌ ❌ ❌ ❌ meta-llama/llama-2-13b-chat 14% ✅ ❌ ❌ ❌ ❌ ❌ ❌ meta-llama/llama-3-70b-instruct 14% ✅ ❌ ❌ ❌ ❌ ❌ ❌ meta-llama/llama-3.2-1b-instruct 14% ✅ ❌ ❌ ❌ ❌ ❌ ❌ google/gemma-3-1b-it:free 0% ❌ ❌ ❌ ❌ ❌ ❌ ❌ meta-llama/llama-2-70b-chat 0% ❌ ❌ - - ❌ ❌ ❌ Average 96% 86% 66% 58% 24% 10% 0% OpenAI’s reasoning models cracked it, scoring 6/7, stumbling only on the 9-digit multiplication. ...
This week, I learned: OpenAI’s reasoning models are much ahead of other models when multiplying two numbers in their heads. Ref ⭐ Promptfoo may be the most mature open source LLM evals tool. Simon Willison Dyson Sphere. LemonSlice showcases real-time audio-video models (avatars) that are close enough to real. Notes from Latent Space ICLR 2025, Singapore Daniel: Menlo’s ReZero. A model that keeps searching till it finds the answer. There are multiple search techniques: Multi-step retreival, Iterative retrieval, Query rewriting. Also, reasoning. The LLM token generation sequence is normally: <think>, <search>, <answer>. Insight: “If we explicitly reward LLMs for retrying after a failed search, they out-perform one-attempt systems.” So <think>, <search>, <think>, <search>, <think>, <search>, <answer>. ⭐ Prompt reasoning models, e.g. “Keep searching till you find the best answer.” Roger, Nous Research Supervised learning is limited because accuracy is piece-wise linear, i.e. it’s broken up. Continuous optimization is meaningless. Reinforcement learning works better because rewards can be discrete. (But it converts things back into differentiable loss functions behind the scenes.) Rewards can be good/bad. Single or multi-step. Whatever. We’re in the “Era of experience”, i.e. models gain experience from the environment themselves. ⭐ So, we need environments models can learn in. This is the next thing after training data. That needs a standard for environments. We’d need a model, a trainer, and the environment. The environments whatever capabilities. Run code. Browser. A game. … With an exposed interface Eugene Cheah (Featherless.ai) Transformer architectures need n-square GPUs as # of tokens grow. Featherless is exploring an RWKV architecture that scales linearly. THere are other such architectures. Performer, Linformer, Reformer, Hyena. Mistral-Nemo-12b-ic is one of the most popular fine-tuned model. It’s small enough to run on a server. Justus Mattern (Prime Intellect) Intellect-2 is a continously learning (RL) model that uses decentralized training on peer-to-peer GPUs. Solving problems on bandwidth, verifiable contributions, etc. ChatGPT Deep Research now also has an O4-Mini version to serve smaller reports. Free users get 0 original + 5 lightweight 5 tasks / month. $20 version gets 10 + 15. $200 version gets 100 + 150. The month begins on first use of Deep Research and runs on a 30 day “window”. Ref O4-Mini-High is great at going through an under-documented repo and finding things. For example, here’s how I configured cmdg. ChatGPT is my new Jupyter Notebook :-) Google announced new AI capabilities at Google Next APAC 2025. Blog. Interesting ones are: @Gemini in chat Google Meet support for “Catch me up” Google Vids: Create short video clips Google Sheets: does better analysis Google Slides: image generation Google Docs: Create Audio Clips (like NotebookLM in Google Docs) Google Docs: “Help me refine” is better than before Google Workspace Flows gcalcli is a convenient way to export Google Calendar. Example: uvx gcalcli agenda --tsv 2025-01-01 2025-01-05 cmdg is a command line GMail client that I’ve now switched to for quick email checks. 80% of my email is spam and this is good enough to scan and delete those. It also avoids running a 200-500 MB tab in the browser that constantly shows me how many unread emails I have. From Worklife with Adam Grant: Cancelling cancel culture with Loretta Ross “Lighten up! Fighting Nazis should be fun. It’s being a Nazi that sucks. If you’re not having fun fighting for hope and joy and human rights, maybe you’re doing the fight wrong. We are the ones who should be having fun.” “You can say what you mean. But you don’t have to say it mean.” There is always a way to put it across better. Refusing to say mean things is about to discover these approaches. “The true mark of a lifelong learner is knowing that you can learn something from every single person you meet.” If you remember that, you can’t be a know it all. semantic-text-splitter could be the go-to text splitter. It’s Rust-based, supports MarkdownSplitter, and multiple tokenizers. Alternatives like semchunk, advanced-chunker, chonkie, etc. seem clunkier. ULID is like UUID but time-sortable. That’s an improvement over timestamp IDs (definitely) and potentially even UUIDs. They can be generated by clients as a globally unique ID. Try pip install python-ulid and npm install ulid. The Consumer Product Safety Commission Data has thousands of reports of product safety over time You can run xclip -sel clip -o | pandoc -f markdown -t html --no-highlight | xclip -sel clip -t text/html -i to convert Markdown in the clipboard to rich text. But xclip doesn’t support multiple selections, so the text is lost. ChatGPT DuckDB UI & Notebooks will potentially be a good alternative to Datasette, DBeaver, etc. But for now, there are still glitches. It crashes with a SIGSEGV (Address boundary error) when connecting to SQLite databases. Ollama limits MAX_TOKENS to 2K by default. AI assisted search helps wherever I would have used Google, e.g. Debugging. “Fix CUDA initialization: CUDA unknown error” Tool search. “Find an online word counter tool.” Library search. “Find a JS micro library to render Markdown.” OpenAI API capabilites lag ChatGPT features. For example: o4-mini via the API does not search the web natively as part of its reasoning. o4-mini, o3, o3-mini, o1, gpt-4.1-nano don’t yet support the web_search_preview tool. Only gpt-4.1 and gpt-4.1-mini do. Limitations Search results are NOT visible via the API. They’re fed directly to the model. The number of searches or results is unknown. Each search costs 0.25-0.5 cents. Pricing For reasoning traces (e.g. .reasoning.summary: "medium") you need to verify your organization via withpersona.com which failed with my Indian passport AND Singapore work permit. The ChatGPT Plus plan ($20) gives you 50 O4 mini messages a day, which I exceeded! It’s supposed to reset at midnight UTC Ref but might operate on a rolling window ChatGPT. “Currently, there is no way to check how many messages you have used in your usage budget.” OpenAI SignalBloom reads SEC filings and writes analyst reports on it using LLMs “Evaluation in the loop” or “Evals-in-the-loop” is a new term I learnt. SignalBloom’s Hallucination Bechmark If AI interacts with the world and generates data from its own experience and learns from that, we have a new scaling mechanism. DeepMind podcast OpenAI’s search API is fairly expensive at $30+/1K calls. Typically, to read interesting HN articles, I will make 30 calls which is about 75c. Instead I should use the app and summarise HM news across different days manually based on my interests! Finally! t-strings land in Python. They’re like JavaScript template literals. DuckDB’s CSV parser might be one of the most forgiving parsers. Even better than Pandas or SQLite3. Ref Good managers will probably make good AI managers. AI agents can probably substitute humans in business experiments. Ethan Mollick If Windsurf stops working, reload the extension. GitHub TLS certificates will start expiring in 47 days from 15 Mar 2029, forcing automated domain renewals. Digicert Nix flakes are a reliable alternative to DevContainers that don’t need Docker - but don’t work on Windows. Ink is like React for the CLI. The Unsure Calculator is a great tool to calculate formulas with multiple uncertainties, like: My office is 9-11 km away and it takes me 45-55 min to reach. So I cycle at 9~11 / 45~55 * 60 ~ 10-14 kmph (12 most likely). I spend $6-15 on lunch and eat out 80-120 days a year. So I spend 6~15 * 80~120 ~ $600~1550 ($1000 most likely) eating out yearly. I take 30-120 min to prepare a quiz question. Each exam has 6-12 questions. So I need 30~120 * 6~12 / 60 = 4~20 hours (11 most likely) Using Kiran’s macOS setup for dev I enabled colorized less and mouse options for tmux. time fish -i -c exit prints the time taken for fish startup. fish --profile-startup ~/fish.profile -i -c exit prints the time taken by each command on fish startup to ~/fish.profile. I used this to speed up my fish startup. The 8 top features of the OpenAI Responses API that are an improvement over the Completions API (IMHO) are: Link to previous response rather than sending history Uploading files directly Swappable system instructions while retaining the chat history Customisable reasoning effort AND reasoning summary detail Truncation in the middle option Web search context size option File search filters by file attributes Flex service tier for lower cost OpenAI doesn’t charge for file storage but does charge 10 cents / GB-day for vector storage beyond 1 GB. The first 1GB is free Augment Code is an AI code editor that’s growing popular on Reddit. #ai-coding The GPT 4.1 models have a 75% discounted prompt caching (instead of the usual 50%), making them particularly suited for repetitive tasks. OpenAI chatgpt.com shortcut keys are revealed via Ctrl + /. Here’s my ranking on usefulness: Ctrl + Shift + C: Copy last response as Markdown! Ctrl + Shift + ;: Copy last code block Ctrl + Shift + S: Sidebar toggle Ctrl + Shift + O: Open new chat Shift + Esc: Focus chat input Ctrl + Shift + I: Ccustom instructions Ctrl + Shift + X: Delete chat
What if you keep ask an LLM Improve the code - dramatically!? We used the new GPT 4.1 Nano, a fast, cheap, and capable model, to write code for simple tasks like “Draw a circle”. The we fed the output back and asked again, Improve the code - dramatically! Here are the results. Draw a circle rose from a fixed circle to a full tool: drag it around, tweak its size and hue, and hit “Reset” to start fresh. Animate shapes and patterns turned simple circles and squares into a swarm of colored polygons that spin, pulse, and link up by distance. Draw a fully functional analog clock grew from a bare face to one that builds all 60 tick marks in code—no manual copy‑paste needed. Create an interactive particle simulation went from plain white dots on black to hundreds of bright, color‑shifting balls that bounce, die, and come back to life. Generate a fractal changed from a single Mandelbrot image to an explorer you can zoom, drag, and reset with sliders and the mouse wheel. Generate a dashboard jumped from static charts to a live page with smooth card animations, modern fonts, and a real‑time stats box. A few observations. ...
This week, I learned: OpenAI lets you download GPT instructions and execute arbitrary code in their containerized environment. This is not a bug. Ref BM25 works as follows: Ref For each query term in the query, sum up the product of: Inverse document frequency = LN(% of docs without the query term + 1) – with a small tweak Term frequency = freq / (freq + k) – where k is usually between 1.2 to 2. Returns 0-1 with diminishing frequency benefit k is multiplied by Document length normalization = 1 - b(1- DocLength/AvgDocLength). Longer documents have larger k, dampening frequency benefits. Some implications: The actual BM25 score has no meaning. It’s just useful for ordering BM25 scores for 2 queries can be compared ONLY IF the document sets don’t change A list of Markdown to Website converters on this thread: Jekyll - Ruby - 2008 MkDocs - Python - 2014 GitBook - JavaScript (Node.js) - 2014 MkDocs Material - Python (MkDocs-based) - 2016 Docsify - JavaScript - 2016 MdBook - Rust - 2017 Antora - JavaScript (Node.js) - 2017 Docusaurus - JavaScript (React) - 2017 JupyterBook - Python - 2019 Keenwrite - Java - ~2019 Honkit - JavaScript (GitBook fork) - 2019 Nextra - JavaScript (Next.js) - 2020 Astro - JavaScript/TypeScript - 2021 Hugo Book - Go (Hugo-based) - ~2020 Clowncar - JavaScript/Node.js - ~2021 Quarto - R and Python - 2022 Starlight - JavaScript/TypeScript - 2023 DuckDB has an LLMs.txt. Today, 38 repos on GitHub support it When identifying LLM use cases, it helps to tell LLMs what they can do. I use one or more of a list like below: Core capabilities: Text Generation: Produce coherent and contextually relevant text across various domains. Image Generation: Create realistic images that match the style and content of a given reference image. Text to Speech: Convert text into natural-sounding speech with appropriate intonation and rhythm. Speech to Text: Transcribe and interpret spoken language. Vision: Analyze and describe visual content from images. Video Analysis: Summarize and extract information from video content. Text to Video: Generate realistic (and surrealistic) videos from text descriptions. Function Calling: Execute predefined functions or access external tools to perform specific tasks. Structured Output: Generate structured outputs like JSON, XML, HTML, YAML, DSLs, etc. Tool Use: Utilize external applications or APIs to enhance functionality. Code Generation: Write and debug code snippets in various programming languages. Cross-domain use cases: Summarization: Understand and condense lengthy documents into concise summaries. Translation: Convert text between multiple languages with high accuracy. Question Answering: Provide precise answers to user queries based on provided information. Reasoning and Planning: Solve complex problems and develop step-by-step plans. Personalization: Tailor responses based on user preferences and historical interactions. Dialogue Management: Engage in context-aware, multi-turn conversations. Data Analysis: Interpret and generate insights from structured data. Content Moderation: Identify and filter inappropriate or harmful content. Sentiment Analysis: Detect and interpret emotions and opinions in text. Robotics Integration: Interface with robotic systems for control and decision-making. Knowledge Retrieval: Access and present information from vast datasets or knowledge bases. Creative Writing: Generate poetry, stories, and other creative content. Educational Assistance: Provide explanations and tutoring across various subjects. Ethical Reasoning: Assess scenarios for ethical considerations and implications. Accessibility Support: Assist users with disabilities through tailored interactions. Simulation and Modeling: Create predictive models and simulate scenarios. Domain-specific use cases: Legal and Medical Assistance: Offer information and guidance within legal and medical domains. Gaming: Generate narratives, dialogues, and scenarios for interactive entertainment. Scientific Research: Aid in literature reviews, hypothesis generation, and data interpretation. Financial Analysis: Analyze market trends and provide investment insights. Cultural Competence: Understand and respect diverse cultural contexts in interactions. Security Applications: Detect and respond to potential cybersecurity threats. Environmental Monitoring: Analyze data related to environmental changes and sustainability. Healthcare Support: Assist in patient monitoring, diagnostics, and personalized treatment plans. Supply Chain Optimization: Enhance logistics and inventory management through predictive analysis. Customer Service: Provide automated support and resolve customer inquiries. Market Research: Analyze consumer behavior and market trends for business insights. Content Creation: Generate articles, blogs, and marketing materials. Virtual Assistance: Manage schedules, reminders, and personal tasks. Social Media Management: Craft posts and engage with audiences across platforms. Human Resources: Assist in recruitment, training, and employee engagement strategies. Event Planning: Organize and coordinate events, including logistics and communication. Travel Planning: Provide itineraries, booking assistance, and destination information. Real Estate: Analyze property markets and assist in buying or selling decisions. Agriculture: Monitor crop health and optimize farming practices through data analysis. Energy Management: Optimize energy consumption and monitor renewable energy sources. Transportation: Enhance route planning and traffic management systems. Urban Planning: Assist in designing sustainable and efficient urban infrastructures. Disaster Response: Provide real-time information and coordination during emergencies. Public Policy: Analyze data to inform policy decisions and predict societal impacts. Art and Design: Generate visual art concepts and assist in creative design processes. Music Composition: Create original music pieces and assist in songwriting. Language Learning: Facilitate language acquisition through interactive exercises and feedback. Historical Analysis: Interpret historical data and provide insights into past events. Philanthropy: Identify charitable opportunities and assess the impact of donations. Sports Analytics: Analyze player performance and game strategies. Fashion: Predict trends and assist in clothing design and merchandising. Culinary Arts: Generate recipes and provide cooking guidance. Astronomy: Analyze celestial data and assist in space exploration research. Psychology: Offer insights into human behavior and mental health support. Linguistics: Analyze language patterns and assist in translation studies. Archaeology: Assist in artifact analysis and historical site interpretations. Literature Analysis: Interpret literary works and provide critical analyses. Philosophy: Engage in discussions on ethical dilemmas and existential questions. Mathematics: Solve complex equations and assist in theoretical research. Physics: Model physical phenomena and assist in experimental design. Chemistry: Analyze chemical compounds and predict reactions. Biology: Assist in genetic research and ecological studies. Geology: Analyze geological data and assist in natural resource exploration. Meteorology: Predict weather patterns and analyze climate data. Oceanography: Study marine ecosystems and assist in ocean exploration. Anthropology: Analyze cultural data and assist in ethnographic research. Style of writing impacts output style a lot. E.g. Adding an evil laugh makes Claude more creative. Ethan Mollick For good structured mode output, we need good prompting. Mentioning examples and schema and “JSON” helps. When providing examples, using (user, assistant) message pairs helps (I think it’s because it’s easier for the LLM to parse). Using a {reasoning, answer} schema (with reasoning first) helps. Make reasoning concise and relevant Ref Arxiv We already know code in JSON is not a great idea. Ref Just adding 3 real examples and regurgitation helped GPT 4o play chess much better. Both techniques may have more general use in prompting. Simon Willison With Deno 2.0, the same .js file can run in Node.js as well as Deno. Example jspm lets you generate import maps against any CDN. You can click on htop columns on the terminal to sort by that column! Mouse events work on command line apps. Julia Evans Alt Text will very likely be a browser feature. It’s important for the Alt text to flow as part of the content when listening to the page. Perhaps even become a part of the browser APIs like speechRecognition. Langchain suggests multiple levels of agentic behaviour. LLM Call < LLM Chain < LLM Rounter < State Machine < Autonomous Langchain A HTML quine: A page that, when rendered as HTML, shows the HTML source code of the page! You can enable syntax highlighting just using fonts. Ref HTML is all you need shows examples of using HTML for notebooks instead of Jupyter, Observable, etc. Straive evaluated Gemini 1.5 Flash 002 and GPT 4o Mini for translation. Portugese: Flash is better than GPT 4o Mini. BLEU Word Overlap is 65.5% > 64.6% and METEOR (Semantic) is 84.9% > 78.9% Mandarin: Flash is better than GPT 4o Mini. BLEU Word Overlap is 25.0% > 15.9% and METEOR (Semantic) is 54.7% > 51.1% The problem with Accept headers is that you can’t link to them. Simon Willison Recraft v3 supports vector (SVG) generation Simon Willison. The output is 100% <path> elements (even for text). You get 50 free credits daily. Creating 1 image is ~2 credits. The API costs $1 per 1K credits. Some things I can create with it are: Base data visualizations that I can animate with code Icons in a specific style Comic strips Explainers for talks or student material Featured images for blog posts Architecture diagrams?
Looks like XML tags are the best way to structure prompts and separate sections for an #LLM. It’s the only format that all of Anthropic, Google, and OpenAI LLMs encourage. For example: … … … … Anthropic Docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/use-xml-tags OpenAI Docs: https://platform.openai.com/docs/guides/prompt-engineering/strategy-write-clear-instructions Google Docs: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/structure-prompts Alternatives are using JSON, Markdown, templating formats like Mustache/Jinja, etc. Even Llama’s system tokens seem a little XML-like. https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py#L61-L74 Personally, I’ve been using Markdown so far. But it’s time to switch over. (Only on the prompt side. On the generation side, Markdown still seems the best.) ...
At Straive, we use an LLM Router. Since ChatGPT, etc. are blocked for most people, this is the main way to access LLMs. One thing we measure is the speed of models, i.e. output tokens per second. Fast models deliver a much smoother experience for users. This is a different methodology than ArtificialAnalysis.ai. I’m not looking purely at the generation time but the total time (including making the connection and the initial wait time) for all successful requests. So, if the provider is having a slow day or is slowing down responses, these numbers will be different. ...
Fascinating to see the how LLM cost-quality frontier moves. Recent fights were mostly on cost. Yesterday, #OpenAI halved the GPT-4o cost. At $2.5/MTok (and with GPT-4o-min at 15 cents/MTok), the best and cheapest models are back with OpenAI, IMHO. Sigh, time to move all our stuff back from #Anthropic. For now… https://gramener.com/llmpricing/ LinkedIn
This week, I learned: Alzhara is one of the VFX companies that worked on Leo’s hyena scene. Their 3D modeling is incredible. Enterprise scenarios leaderboard. Mistral 7B leads. Veda Srinivasan. How does Google manage culture? AMA sessions Manager feedback. Entirely anonymous. Avoid taking feedback for teams less than 5 Workplace concerns team exists. Put managers on watch Books Mohammad Younus. Three zeroes book. Read about his social business theme Pluriverse. Anti fragile. Aurobindo Vedas. Barry Oshry. Seeing systems. Runs workshops but book is better Raghu Anantanarayana has written about Indian archetypes based on Mahabharatha India that is Bharath. Sai Deepak. Podcasts Listen to Nilesh Oak. Sugreeva’s Atlas. Pankaj Tripathi podcast on geography influences acting Areas of focus “I’m an Expert on synthesis and implementation” Intersectionality is another word for complex failures. Also for deep segmentation. Swiss cheese model. Dialogic self theory is about multiple voices in the head. How do we make meaning? Psychological rupture is when cognitive activity is maximum. At any point there are MULTIPLE voices in our heads that are sources of action. We don’t listen to them. Epistemology. Language determines thought. like the word productivity. How does appreciation of a rose become productive? Words from other languages may have incredible power. From other cultures. Paul Sloan. Lateral thinking podcasts from multiple sources Deliberately engage with topics randomly. Deliberately engage with random people Read a random book from the library Watch a random film in a different language Consciously where the six thinking hats or look hard for the silent voices in your head and express them Ask children. They tend to think of more creative and childlike solutions He converted a hiring process into a contest Constantly ask yourself. What if every assumption I’m making about this is wrong? Scenario planning is really about this. List a few scenarios. They’d have high impact or high probability. What happens in this scenario? Ideate You can @mention GPTs to ask a specific GPT a question in ChatGPT. This is really powerful. Hidden brain podcast. Making the most of your mistakes FIX every small mistake. You never know how they might line up in the future You also never know how small little things done well might line up to give you a boost in the future The Toyota cord does not actually stop the production line. It brings a team lead over who quickly diagnoses the problem with you. The responsiveness of the league is a critical factor and so is encouragement That isn’t always a single bottleneck to stop that is the case of a simple failure. There can be a series of holes that happen to align perfectly. These are events that lead to catastrophic failures or successes Do as little as possible, waste as little as possible, until you know that the outcome is worthwhile. Figure out what is the value of the outcome and the most important piece of information you need to discover that Do full research before you try and fail. The aim of failure is learning at the least possible cost How I write podcast. 2023 summary Ask for feedback from friends in a specific way. What 20% should I retain no matter what? What 20% should I cut? This allows them to compliment while providing genuine feedback Hire lawyer interns to proofread. They are the ones that find fault the best Be in a segment of one. Where there is zero competition. Something only you can do Don’t try to do stuff faster. Try to do stuff you don’t want to stop doing Read books older than 50 years Read Michael Collins book on things that sustain Temp service make sure he has some energy to spare. Cuz Riley does the opposite. She waits till she can’t stand it anymore and then writes like crazy until she drops dead. The former leads to thoughtful writing. The latter is emotionally powerful. Be able to do that Vanna is a SQL generation LLM. An alternative to SQLCoder. This thread has a detailed discussion on SQL generation and BI Intel developer cloud has a liberal GPU in the free tier. OpenAI releases text-embedding-3-large which can be truncated. The embedding values have descending importance, so picking the first n is a good approximation. Also, gpt-3.5-turbo-0125 is 50% cheaper. AppAgent is an LLM that can navigate mobile / web apps Retrieval Centric Generation is an emerging alternative to RAG, where the LLM is explicitly built to leverage external knowledge. SimplyRetrieve is an early implementation. Big Code Models Leaderboard is a leaderboard for open source code models.
text-embedding-ada-002 used to give high cosine similarity between texts. I used to consider 85% a reasonable threshold for similarity. I almost never got a similarity less than 50%. text-embedding-3-small and text-embedding-3-large give much lower cosine similarities between texts. For example, take these 5 words: “apple”, “orange”, “Facebook”, “Jamaica”, “Australia”. Here is the similarity between every pair of words across the 3 models: For our words, new text-embedding-3-* models have an average similarity of ~43% while the older text-embedding-ada-002 model had ~85%. ...
This week, I learned: ⭐ OpenAI’s prompt engineering strategies are an excellent start for prompt engineering. A few lessons: Use detailed system prompts, often containing the entire instruction set, if it won’t change over the course of a conversation. “… summary of the prior conversation could be included as part of the system message” is an interesting history compression tactic. OpenAI summarizes books by recursively summarizing sections and maintaining a running commentary of the summary so far. Dan sends Google documents with essays instead of emails. This allows people to comment on it. But commenting is a culture and not many people do it. Adriano does it a lot and we’ll. Dan and Adriano actively converse on GitHub issues llm-guard is an LLM content validation tool.