openai on S Anand

Erdos Unit Distance Problem

Tue, 26 May 2026 22:36:06 +0800

An OpenAI model solved the Erdos unit distance problem.

Erdos roughly said, “The number of edges of the same distance between N points can’t compound faster than close to 0%.”

The model found a method of placing points so that it compounds at about 1.4%.

This visualization is a crude way of visualizing how that works.

OpenAI Prism for LaTeX

Thu, 29 Jan 2026 14:52:00 +0800

OpenAI launched Prism - an AI LaTeX IDE.

It’s a boon for anyone writing LaTeX documents. All the nitty-gritty of formatting, syntax, etc. is handled by AI. You can collaborate, too. It brings the power of AI code editors to scientific document editing.

It still has some way to go, though. I asked it to convert a portion of this paper into LaTeX. Here’s the image I passed:

… and here’s the LaTeX output it generated:

The number of errors it made are too many to list. So, it’s still some way from being picture-perfect. But for those experimenting, not publishing, it’s a useful accelerator.

UPDATE: I assumed that (because the chemical formulas looked so different) it had misread the image. But experts tell me that it actually got it right! So, Prism (and the underlying GPT models) may not be perfect but are certainly better than I thought.

OpenAI TTS cost

Sun, 02 Nov 2025 05:12:32 +0000

The OpenAI text-to-speech cost documentation is confusing.

As of 2 Nov 2025:

GPT-4o mini TTS costs $0.60 / MTok input and $12.00 / MTok audio output according to the model page and the pricing page. They also estimate this to be ~1.5c per minute - both for input and output. It supports up to 2,000 tokens input.
TTS-1 costs $15 / MTok speech generated according to the model page but the pricing page says it's $15 / MChars. No estimate per minute is provided. Is supports up to 4,096 characters input.
TTS-1 HD is twice as expensive as TTS-1

I wanted to find the approximate total cost for a typical text input measured per character and token.

I converted this podcast with 4,096 ASCII characters and 877 tokens on o200k_base using:

I ran:

curl https://api.openai.com/v1/audio/speech \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg text "$(cat podcast.txt)" '{
    model: "tts-1",
    voice: "coral",
    input: $text
  }')" \
  --output tts-1.mp3

This took 46 seconds to generate and produced a 5.1 MB MP3 file (256 seconds)

To measure the cost, I ran:

curl "https://api.openai.com/v1/organization/costs?start_time=$(date -d '1 day ago' +%s)&project_ids=$PROJECT_ID&group_by=line_item" \
  -H "Authorization: Bearer $OPENAI_ADMIN_KEY" \
  -H "Content-Type: application/json"

This cost: USD 0.061425. Then I ran:

curl https://api.openai.com/v1/audio/speech \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg text "$(cat podcast.txt)" '{
    model: "gpt-4o-mini-tts",
    voice: "coral",
    input: $text
  }')" \
  --output gpt-4o-mini-tts.mp3

This took 44 seconds to generate and produced a 4.3 MB MP3 file (268 seconds).

When I ran the admin API call again, the costs did not reflect for 5 minutes. So I ran it the GPT-4o mini TTS call again with the same input. This took 44 seconds to generate a 4.3 MB MP3 file. When I ran the admin API call again, the total cost for the 2 requests was: USD 0.12942 audio output and USD 0.0010524 input.

I also checked for TTS-1 HD. Here are the costs in USD:

Model	$ / MTok	$ / MChars	$ / hour	Time (s)	Audio (s)	Cost $
GPT-4o mini TTS	74.4	15.9	0.876	46	268	0.0652
TTS-1	70.0	15.0	0.864	44	256	0.0614
TTS-1 HD	140.0	30.0	1.728	62	257	0.1228

The GPT-4o mini TTS audio output cost was USD 0.06471 for the input of 877 tokens, i.e. $73.8 / MTok. Since the actual cost is $12 / MTok, this is a 6.15x multiplier. I guess 1 input text token produces ~6 output audio tokens.

In terms of quality:

GPT-4o mini TTS: Very slightly robotic. Looser interpretation of text (e.g. says "October" when I write "Oct").
TTS-1: Natural. Strict interpretation of text.
TTS-1 HD: Very slightly more natural. Strict interpretation of text.

I will likely use TTS-1 for now given the cost difference is small and the quality is good enough.

Incidentally, the usage API did not show an GPT 4o mini TTS line items even after 20 minutes.

curl "https://api.openai.com/v1/organization/usage/audio_speeches?start_time=$(date -d '1 day ago' +%s)&project_ids=$PROJECT_ID&group_by=model" \
  -H "Authorization: Bearer $OPENAI_ADMIN_KEY" \
  -H "Content-Type: application/json"

When to choose AI over humans

Tue, 28 Oct 2025 04:37:14 +0000

I charted the OpenAI GDPVal paper with industry compensation as the size and AI augmentation as color. Big green areas are we’re paying people where AI does better.

Click here to see the interactive visualization.

Clicking to see some actual tasks compared.

I use this to check whom to ask advice: AI or professional.

AI beats Personal Financial Advisors ~64% of the time. So I invested half my money using ChatGPT’s recommendation. (UTI Nifty 50, if you’re curious.)

Accountants beat AI 76% of the time. So my tax returns are still filed by Ventura Pranas.

AI beats Government - Administrative Services Managers 62% of the time. I haven’t figured out how to bypass them yet. Nor customer service representatives.

Overall, AI beats most managers and clerks, not industrial engineers and pharmacists.

Here’s my current thought where I wouldn’t hire AI if a high portion of their work

Has legal liability (e.g. pharmacist/judge vs shop attendant/lawyer)
Is subjective (e.g. perfumer/auction appraiser vs lab chemist/insurance appraiser)
Needs rapid contextual decisions (e.g. detective/fireman/ER vs parking enforcer)

But this apart, if they charged half, would their demand double? If so, even with AI, they’ll have more demand.

Things I Learned - 10 Aug 2025

Sun, 10 Aug 2025 00:00:00 +0000

This week, I learned:

OpenAI supports a tool "type": "custom" that lets it write code as an argument to a tool call. Great for code / SQL generation. Even more powerfully, you can generate output following specific grammars, e.g. STL files, PostgreSQL dialect, Mermaid/PlantUML diagrams, OpenAPI specs, Vega-Lite JSONs, Cron expressions, GraphQL SDLs, Dockerfiles, Terraform HCLs, or any DSL! # #ai-coding
The OpenAI playground has a GPT-5 Prompt Optimizer that can migrate prompts to GPT-5.
Docsify 4.13.1 is 2 years old and uses marked@1.2.9 which is 5 years old. Newer plugins like marked-directive don’t work with it. Though docsify v5.0.0-rc1 is in development, it may be the better option for modern Markdown plugins. Here’s sample code.
CommonMark has a powerful directive syntax proposal that lets you add classes, attributes, and arbitrary plugins to Markdown. For example, :abbr[MD]{#id .class title="Markdown"} for inline directives. Plugins exist for marked, markdown-it and remark.
biomejs and dprint are gaining traction as prettier alternatives. I’m yet to try them but keen to explore.
- Skip biomejs for now. It uses tabs (not spaces) and does not respect .gitignore by default. Handling these is too much work.
⭐ Code generation is more flexible than tool calling. LLMs can’t write a tool-call loop, for example, but they can write code to run an API in a loop. So, I like telling the LLM to “write code using these APIs” than giving it APIs to tool-call. #ai-coding
npx -y ccusage is an easy way of summarizing your Claude Code usage and cost. My cost so far (since 21 July) is about $10. The median session cost is ~50 cents. Most of it ($7) was from a single temporary coding chat that I kept continuing for way too long, building up the context window. #
defuddle can be used in the browser to get the main content from web pages. A replacement for Mozilla Readability. #
Modern Node.js Patterns for 2025 include these 5 features I’m excited by:
- Single-executable bundling. node --experimental-sea-config sea-config.json builds standalone binaries.
- ES Modules. Use node: prefix for built-in imports. import { createServer } from 'node:http';
- Watch mode. Use node --watch file.js auto-reloads when file.js or dependencies change.
- Env file. Use node --env-file=.env loads .env as environment variables.
- node:test is a full-featured test framework with --watch and coverage.
Concise explanations speed up decisions because they’re faster to read and understand (obvious). They’re also easier to combine with other ideas (less obvious). #
I’ve been uncertain about htmx for some time now. This tutorial, HTMX is hard, so let’s get it right, convinced me that it’s too far from my mental model, so I’m unlikely to ever use it.
⭐ Slow, effortful practice (spaced recall, interleaving topics, self-testing) builds lasting knowledge but looks inefficient and doesn’t help with exams. # #beliefs
GitDoc VS Code extension auto-commits and syncs notes. I dropped gitwatch in favor of this.
It’s interesting that Gemini Deep Research cannot access Google Drive while Gemini can. On the other hand, ChatGPT Deep Research can access Google Drive but ChatGPT cannot.
A trend that AI coding will only accelerate: “It is now possible for tiny teams to make principled software that millions of people use, unburdened by investors. … you need far less money and far fewer employees to reach far more customers. That wave is only just beginning.” # #ai-coding
Typed languages are better suited for vibe coding. This will likely lead to the growth of typed languages (TypeScript, Rust, Go) but also of typing in untyped languages (e.g. Python) # #ai-coding
Instead of Celery, Redis, Kafka, etc. as task queues, we could the file system as a message queue. For example, pending/task-01.json moves to wip/task-01.json to done/task-01.json. Folders for state/tags, files for task details.
Foam is a note-taking VS Code extension. The WikiLinks, tags and backlinking features align naturally with Markdown note-taking. Via Steph Ango who uses Obsidian which nudged me to search for WikiLink-ing features in VS Code.
I’m an open data hawk. But here are things I should remind myself of. #
- Privacy incubates creativity. People self-censor when watched. Privacy shields fragile ideas.
- Power assymetry. Big players can leverage openness more, e.g. Cambridge Analytics + Facebook data.
- Context matters. What’s harmless in one setting can be toxic in another.
- One-way door. Data can’t be unshared. Don’t scrap brakes dreaming of perfect roads. Anticipate tyrannical regimes / cultures.
- Not your call. You don’t share your neighbour’s medical records.
One Punch Man is available as manga. I watched the anime first and assumed that came first. Apparently not.
⭐ In “kind” environments (stable rules, rapid and accurate feedback), specialize. In “wicked” environments (rules shift, feedback is noisy/late), generalize. ChatGPT
Models’ ability to orchestrate longer workflows will improve. Factor that into your application design. Claude Code can already handle over 70 tasks in a workflow
What happens when LLMs play Chinese Whispers / the Telephone Game? Here are learnings. ChatGPT
- Drift increases faster than linear with hops.
- Bigger models do better, but constrained prompts (“Copy the text exactly; change nothing.”) have a bigger impact.
- Low temperature improves copying fidelity.
- But even after “forgetting”, LLMs reproduce rare content if they’re trained on it.
“In fact, React Native looks set to become the most engine-agnostic JavaScript runtime around”. The Many, Many, Many, JavaScript Runtimes of the Last Decade
OMDb (simple) and TMDb (comprehensive) are API-friendly alternatives to the IMDb.
copyparty seems one of the most feature-rich file servers out there. Single Python file, runs on any OS, works with any client, and optimized for speed. Video
Quotes I enjoyed from Linus Torvalds’ TED interview
- I want to not have external stimulation. You can kind of see, on the walls are this light green. I’m told that at mental institutions they use that on the walls. It’s like a calming color. … the main thing I worry about in my computer is – it really has to be completely silent. If the cat comes up, it sits in my lap. And I want to hear the cat purring.
- I did not start Linux as a collaborative project. I started it as one in a series of many projects I had done at the time for myself, partly because I needed the end result, but even more because I just enjoyed programming.
- I’m actually not a people person. But I do love other people who comment and get involved in my project.
- The big point for me was not being alone and having 10, maybe 100 people being involved. Going from 100 people to a million people is not a big deal – to me. Well, I mean, maybe it is if you want to sell your result then it’s a huge deal. But if you’re interested in the technology and you’re interested in the project, the big part was getting the community.
- So Git is my second big project, which was only created for me to maintain my first big project. And this is literally how I work.
- Well, I do code for fun – but I want to code for something meaningful so every single project I’ve ever done has been something I needed.
- Apparently, my sister said that my biggest exceptional quality was that I would not let go.
- I can’t do UI to save my life.
- Good taste is about really seeing the big patterns and kind of instinctively knowing what’s the right way to do things.
- Companies like Google and many others have made, arguably, like, billions of dollars out of your software. Does that piss you off? No. No, it doesn’t piss me off for several reasons. And one of them is, I’m doing fine. But the other reason is – I mean, without doing the whole open source and really letting go thing, Linux would never have been what it is.
- I think one reason open source works so well in code (is that …) Code either works or it doesn’t.
The Uses This site has interviewed professionals for decades. From their repo I scraped the top developer apps post 2020:
CloudFlare has an Iceberg data catalog in R2 Data Catalog. Iceberg is like Parquet but supports metadata, time-travel, and schema edits. But I’m yet to find a single publicly accessible Iceberg catalog. Its open-data adoption is not as high as Parquet’s. Apache Iceberg vs Parquet
Observable Notebook 2 is the new notebook format from Mike Bostock. It is vanilla JS and embeddable into other pages. THis would have been a big deal 2 years ago, but with the LLM ecosystem today, I’m not sure if it matters as much.
To add CORS support to CloudFlare pages protected by Zero Trust, add a _headers file to your repo. (This is different from the Zero Trust CORS which allows automated logins.) Sample _headers that lets logged-in users fetch pages via fetch("...", { credentials: "include" }):
```
/*
  Access-Control-Allow-Credentials: true
  Access-Control-Allow-Origin: https://your-site.example.com
  Access-Control-Allow-Methods: GET, HEAD
  Access-Control-Allow-Methods: *
```
As corporates restrict the use of LLMs, I see employees purchasing personal laptops to use LLMs on. An interesting trend!
openai-python has a CLI. You can run uvx openai api chat.completions.create --stream -m gpt-4.1-nano -g developer 'Translate to Chinese' -g user "Hello" for example
Anthropic has an OpenAI compatible API at https://api.anthropic.com/v1/.
Claude Code tips from Things that didn’t work by Armin Rocher #ai-coding
- Speech-to-text. Cannot stress this enough but talking to the machine means you’re more likely to share more about what you want it to do.
- I maintain some basic prompts and context for copy-pasting at the end or the beginning of what I entered.
- I ended up preloading executables on the PATH that override the default ones, steering Claude toward the right tools, e.g. running python asks it to use uv.
- I use the task tool frequently for basic parallelization and context isolation.
- Simply taking time to talk to the machine and give clear instructions outperforms elaborate pre-written prompts.
- Forcing myself to evaluate the automation has another benefit: I’m less likely to just blindly assume it helps me.
Research indicates that we don’t know in advance which prompts will help. Evals beat prompt engineering. Ethan Mollick

Are LLMs any good at mental math?

Sun, 27 Apr 2025 09:52:10 +0000

I asked 50 LLMs to multiply 2 numbers:

12 x 12
123 x 456
1,234 x 5,678
12,345 x 6,789
123,456 x 789,012
1,234,567 x 8,901,234
987,654,321 x 123,456,789

LLMs aren’t good tools for math and this is just an informal check. But the results are interesting:

Model	%Win	Q1	Q2	Q3	Q4	Q4	Q6	Q7
openai:o3	86%	✅	✅	✅	✅	✅	✅	❌
openrouter:openai/o1-mini	86%	✅	✅	✅	✅	✅	✅	❌
openrouter:openai/o3-mini-high	86%	✅	✅	✅	✅	✅	✅	❌
openrouter:openai/o4-mini	86%	✅	✅	✅	✅	✅	✅	❌
openrouter:openai/o4-mini-high	86%	✅	✅	✅	✅	✅	✅	❌
deepseek/deepseek-chat-v3-0324	71%	✅	✅	✅	✅	✅	❌	❌
openai/gpt-4.1-mini	71%	✅	✅	✅	✅	✅	❌	❌
openai/gpt-4.5-preview	71%	✅	✅	✅	✅	✅	❌	❌
openai/gpt-4o	71%	✅	✅	✅	✅	✅	❌	❌
openrouter:openai/o3-mini	71%	✅	✅	✅	✅	✅	❌	❌
anthropic/claude-3-opus	57%	✅	✅	✅	✅	❌	❌	❌
anthropic/claude-3.5-haiku	57%	✅	✅	✅	✅	❌	❌	❌
anthropic/claude-3.7-sonnet:thinking	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.0-flash-001	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.0-flash-lite-001	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.5-flash-preview	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.5-flash-preview:thinking	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.5-pro-preview-03-25	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-flash-1.5	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-pro-1.5	57%	✅	✅	✅	✅	❌	❌	❌
google/gemma-3-12b-it	57%	✅	✅	✅	✅	❌	❌	❌
google/gemma-3-27b-it	57%	✅	✅	✅	✅	❌	❌	❌
meta-llama/llama-4-maverick	57%	✅	✅	✅	❌	✅	❌	❌
meta-llama/llama-4-scout	57%	✅	✅	✅	✅	❌	❌	❌
openai/gpt-4-turbo	57%	✅	✅	✅	✅	❌	❌	❌
openai/gpt-4.1	57%	✅	✅	✅	❌	✅	❌	❌
amazon/nova-lite-v1	43%	✅	✅	✅	❌	❌	❌	❌
amazon/nova-pro-v1	43%	✅	✅	✅	❌	❌	❌	❌
anthropic/claude-3-haiku	43%	✅	✅	✅	❌	❌	❌	❌
anthropic/claude-3.5-sonnet	43%	✅	✅	✅	❌	❌	❌	❌
meta-llama/llama-3.1-405b-instruct	43%	✅	✅	❌	✅	❌	❌	❌
meta-llama/llama-3.1-70b-instruct	43%	✅	✅	❌	✅	❌	❌	❌
meta-llama/llama-3.2-3b-instruct	43%	✅	✅	❌	✅	❌	❌	❌
meta-llama/llama-3.3-70b-instruct	43%	✅	✅	❌	✅	❌	❌	❌
openai/gpt-4.1-nano	43%	✅	✅	✅	❌	❌	❌	❌
openai/gpt-4o-mini	43%	✅	✅	✅	❌	❌	❌	❌
qwen/qwen-2-72b-instruct	43%	✅	✅	✅	❌	❌	❌	❌
anthropic/claude-3-sonnet	29%	✅	✅	❌	❌	❌	❌	❌
deepseek/deepseek-r1	29%	✅	✅	❌	❌	❌	❌	❌
google/gemini-flash-1.5-8b	29%	✅	✅	❌	❌	❌	❌	❌
google/gemma-3-4b-it	29%	✅	✅	❌	❌	❌	❌	❌
meta-llama/llama-3-8b-instruct	29%	✅	✅	❌	❌	❌	❌	❌
meta-llama/llama-3.1-8b-instruct	29%	✅	❌	❌	✅	❌	❌	❌
openai/gpt-3.5-turbo	29%	✅	✅	❌	❌	❌	❌	❌
amazon/nova-micro-v1	14%	✅	❌	❌	❌	❌	❌	❌
meta-llama/llama-2-13b-chat	14%	✅	❌	❌	❌	❌	❌	❌
meta-llama/llama-3-70b-instruct	14%	✅	❌	❌	❌	❌	❌	❌
meta-llama/llama-3.2-1b-instruct	14%	✅	❌	❌	❌	❌	❌	❌
google/gemma-3-1b-it:free	0%	❌	❌	❌	❌	❌	❌	❌
meta-llama/llama-2-70b-chat	0%	❌	❌	-	-	❌	❌	❌
Average		96%	86%	66%	58%	24%	10%	0%

OpenAI’s reasoning models cracked it, scoring 6/7, stumbling only on the 9-digit multiplication.

Models use human-like mental math tricks. For example, O3-Mini-High calculated 1234567 × 8901234 using a recursive strategy.

```markdown To calculate 1234567 × 8901234, one useful method is to break the second number into two parts: 8901234 = 8900000 + 1234 Then the product becomes: 1234567 × 8901234 = 1234567 × 8900000 + 1234567 × 1234 Step 1. Compute 1234567 × 8900000: Notice that 8900000 = 89 × 100000. So first compute 1234567 × 89. Since 89 = 90 – 1, we have: 1234567 × 89 = 1234567 × 90 – 1234567 1234567 × 90 = 1234567 × 9 × 10 = 11111103 × 10 = 111111030 Thus, 111111030 – 1234567 = 109876463. Now multiply by 100000: 109876463 × 100000 = 10987646300000 Step 2. Compute 1234567 × 1234: Break 1234 into parts: 1234 = 1000 + 200 + 30 + 4 Now multiply each and add: 1234567 × 1000 = 1234567000 1234567 × 200 = 246913400 1234567 × 30 = 37037010 1234567 × 4 = 4938268 Add these together: 1234567000 + 246913400 = 1481480400 1481480400 + 37037010 = 1518517410 1518517410 + 4938268 = 1523455678 Step 3. Add both products: 10987646300000 + 1523455678 = 10989169755678 Thus, the product of 1234567 and 8901234 is 10989169755678 ```

OpenAI’s other models and DeepSeek V3 were next, getting the first 5/7 right. Notably: GPT 4.1 Mini beat GPT 4.1. DeepSeek V3 beat DeepSeek R1.

DeepSeek V3 double-checks results and hallucinates a “reliable computational tool”.

To calculate (1234567 × 8901234), let's break it down using the **distributive property of multiplication over addition**:

1. **Express 8901234 as (9000000 - 98766)**:
   [8901234 = 9000000 - 98766]

2. **Multiply 1234567 by each part**:
   [1234567 × 9000000 = 1234567 × 9 × 10^6 = 11111103000000]
   [1234567 × 98766 = 1234567 × (100000 - 12234) = 123456700000 - 15095721278 = 108360978722]

   _Note_: Further breakdown of (1234567 × 12234) was calculated separately and subtracted here.

3. **Combine the results**:
   [11111103000000 - 108360978722 = 1020274222278]

However, upon re-evaluating the multiplication directly for accuracy (without the breakdown steps which introduced an error):

[1234567 × 8901234 = 10989343694778]

But correcting the previous breakdown:

1. **Express 8901234 as (9000000 - 98766)**:
   [1234567 × 9000000 = 11111103000000]
   [1234567 × 98766 = 1234567 × 90000 + 1234567 × 8000 + 1234567 × 700 + 1234567 × 60 + 1234567 × 6]
   [= 111111030000 + 9877653600 + 864196900 + 74074020 + 7407402 = 121616470922]

2. **Combine the results**:
   [11111103000000 - 121616470922 = 1098934369078]

It appears there was an inconsistency in the intermediate steps. For precision, let's verify with another method:

**Using the standard multiplication method**:

[1234567 × 8901234]

Calculating each part:
[1234567 × 8000000 = 9877653600000]
[1234567 × 900000 = 1111110300000]
[1234567 × 100 = 123456700]
[1234567 × 200 = 246913400]
[1234567 × 30 = 37037010]
[1234567 × 4 = 4938268]

Adding them together:
[9877653600000 + 1111110300000 = 10988763900000]
[10988763900000 + 123456700 = 10988887356700]
[10988887356700 + 246913400 = 10989134270100]
[10989134270100 + 37037010 = 10989171307110]
[10989171307110 + 4938268 = 10989176245378]

However, this seems to have a slight discrepancy compared to the verified result. Upon cross-checking with a reliable computational tool:

**Final Accurate Calculation**:

[1234567 × 8901234 = 10989343694778]

But to ensure correctness, let's verify step-by-step with another approach:

**Breakdown of 8901234**:

[8901234 = 8000000 + 900000 + 100 + 200 + 30 + 4]

Now, multiply each by 1234567:

[1234567 × 8000000 = 9877653600000]
[1234567 × 900000 = 1111110300000]
[1234567 × 100 = 123456700]
[1234567 × 200 = 246913400]
[1234567 × 30 = 37037010]
[1234567 × 4 = 4938268]

Adding these up:

[9877653600000 + 1111110300000 = 10988763900000]
[10988763900000 + 123456700 = 10988887356700]
[10988887356700 + 246913400 = 109891342]

O3 Mini reframes 8901234 as (9000000 − 98766) to simplify the calculation.

To calculate 1234567 times 8901234, follow these steps:

1. Express 8901234 as 9000000 − 98766.
     So, 1234567 × 8901234 = 1234567 × 9000000 − 1234567 × 98766.

2. Calculate the first part:
     1234567 × 9000000 = 1234567 × 9 × 1000000.
     Since 1234567 × 9 = 11111103,
     it becomes 11111103 × 1000000 = 11111103000000.

3. Calculate the second part:
     Write 98766 as 100000 − 234.
     Then, 1234567 × 98766 = 1234567 × 100000 − 1234567 × 234.
     - 1234567 × 100000 = 123456700000.
     - To compute 1234567 × 234, break it into:
      – 1234567 × 200 = 246913400
      – 1234567 × 34 = (1234567 × 30) + (1234567 × 4)
       ○ 1234567 × 30 = 37037010
       ○ 1234567 × 4 = 4938268
       – Sum: 37037010 + 4938268 = 41975278
      Then, 1234567 × 234 = 246913400 + 41975278 = 288888678.
     Thus, 1234567 × 98766 = 123456700000 − 288888678 = 123167811322.

4. Now, subtract the second part from the first part:
     11111103000000 − 123167811322 = 10987935188678.

So, 1234567 times 8901234 equals 10987935188678.

16 models, including the latest Gemini, Anthropic, Grok, and Llama models get 4/7 right.

The Amazon models, older Llama, Anthropic, Google, OpenAI models get 3 or less right.

View the results at https://sanand0.github.io/llmmath/. Hover over the cells to see the reasoning traces (where available).

Things I Learned - 27 Apr 2025

Sun, 27 Apr 2025 00:00:00 +0000

This week, I learned:

OpenAI’s reasoning models are much ahead of other models when multiplying two numbers in their heads. Ref
⭐ Promptfoo may be the most mature open source LLM evals tool. Simon Willison
Dyson Sphere.
LemonSlice showcases real-time audio-video models (avatars) that are close enough to real.
Notes from Latent Space ICLR 2025, Singapore
- Daniel: Menlo’s ReZero. A model that keeps searching till it finds the answer.
  - There are multiple search techniques: Multi-step retreival, Iterative retrieval, Query rewriting. Also, reasoning.
  - The LLM token generation sequence is normally: <think>, <search>, <answer>.
  - Insight: “If we explicitly reward LLMs for retrying after a failed search, they out-perform one-attempt systems.” So <think>, <search>, <think>, <search>, <think>, <search>, <answer>.
  - ⭐ Prompt reasoning models, e.g. “Keep searching till you find the best answer.”
- Roger, Nous Research
  - Supervised learning is limited because accuracy is piece-wise linear, i.e. it’s broken up. Continuous optimization is meaningless.
  - Reinforcement learning works better because rewards can be discrete. (But it converts things back into differentiable loss functions behind the scenes.)
    - Rewards can be good/bad. Single or multi-step. Whatever.
  - We’re in the “Era of experience”, i.e. models gain experience from the environment themselves.
  - ⭐ So, we need environments models can learn in. This is the next thing after training data. That needs a standard for environments.
  - We’d need a model, a trainer, and the environment.
  - The environments whatever capabilities. Run code. Browser. A game. … With an exposed interface
- Eugene Cheah (Featherless.ai)
  - Transformer architectures need n-square GPUs as # of tokens grow. Featherless is exploring an RWKV architecture that scales linearly. THere are other such architectures. Performer, Linformer, Reformer, Hyena.
  - Mistral-Nemo-12b-ic is one of the most popular fine-tuned model. It’s small enough to run on a server.
- Justus Mattern (Prime Intellect)
  - Intellect-2 is a continously learning (RL) model that uses decentralized training on peer-to-peer GPUs.
  - Solving problems on bandwidth, verifiable contributions, etc.
ChatGPT Deep Research now also has an O4-Mini version to serve smaller reports. Free users get 0 original + 5 lightweight 5 tasks / month. $20 version gets 10 + 15. $200 version gets 100 + 150. The month begins on first use of Deep Research and runs on a 30 day “window”. Ref
O4-Mini-High is great at going through an under-documented repo and finding things. For example, here’s how I configured cmdg.
ChatGPT is my new Jupyter Notebook :-)
Google announced new AI capabilities at Google Next APAC 2025. Blog. Interesting ones are:
- @Gemini in chat
- Google Meet support for “Catch me up”
- Google Vids: Create short video clips
- Google Sheets: does better analysis
- Google Slides: image generation
- Google Docs: Create Audio Clips (like NotebookLM in Google Docs)
- Google Docs: “Help me refine” is better than before
- Google Workspace Flows
gcalcli is a convenient way to export Google Calendar. Example: uvx gcalcli agenda --tsv 2025-01-01 2025-01-05
cmdg is a command line GMail client that I’ve now switched to for quick email checks. 80% of my email is spam and this is good enough to scan and delete those. It also avoids running a 200-500 MB tab in the browser that constantly shows me how many unread emails I have.
From Worklife with Adam Grant: Cancelling cancel culture with Loretta Ross
- “Lighten up! Fighting Nazis should be fun. It’s being a Nazi that sucks. If you’re not having fun fighting for hope and joy and human rights, maybe you’re doing the fight wrong. We are the ones who should be having fun.”
- “You can say what you mean. But you don’t have to say it mean.” There is always a way to put it across better. Refusing to say mean things is about to discover these approaches.
- “The true mark of a lifelong learner is knowing that you can learn something from every single person you meet.” If you remember that, you can’t be a know it all.
semantic-text-splitter could be the go-to text splitter. It’s Rust-based, supports MarkdownSplitter, and multiple tokenizers. Alternatives like semchunk, advanced-chunker, chonkie, etc. seem clunkier.
ULID is like UUID but time-sortable. That’s an improvement over timestamp IDs (definitely) and potentially even UUIDs. They can be generated by clients as a globally unique ID. Try pip install python-ulid and npm install ulid.
The Consumer Product Safety Commission Data has thousands of reports of product safety over time
You can run xclip -sel clip -o | pandoc -f markdown -t html --no-highlight | xclip -sel clip -t text/html -i to convert Markdown in the clipboard to rich text. But xclip doesn’t support multiple selections, so the text is lost. ChatGPT
DuckDB UI & Notebooks will potentially be a good alternative to Datasette, DBeaver, etc. But for now, there are still glitches. It crashes with a SIGSEGV (Address boundary error) when connecting to SQLite databases.
Ollama limits MAX_TOKENS to 2K by default.
AI assisted search helps wherever I would have used Google, e.g.
- Debugging. “Fix CUDA initialization: CUDA unknown error”
- Tool search. “Find an online word counter tool.”
- Library search. “Find a JS micro library to render Markdown.”
OpenAI API capabilites lag ChatGPT features. For example:
- o4-mini via the API does not search the web natively as part of its reasoning.
- o4-mini, o3, o3-mini, o1, gpt-4.1-nano don’t yet support the web_search_preview tool. Only gpt-4.1 and gpt-4.1-mini do. Limitations
- Search results are NOT visible via the API. They’re fed directly to the model. The number of searches or results is unknown. Each search costs 0.25-0.5 cents. Pricing
- For reasoning traces (e.g. .reasoning.summary: "medium") you need to verify your organization via withpersona.com which failed with my Indian passport AND Singapore work permit.
The ChatGPT Plus plan ($20) gives you 50 O4 mini messages a day, which I exceeded! It’s supposed to reset at midnight UTC Ref but might operate on a rolling window ChatGPT. “Currently, there is no way to check how many messages you have used in your usage budget.” OpenAI
SignalBloom reads SEC filings and writes analyst reports on it using LLMs
“Evaluation in the loop” or “Evals-in-the-loop” is a new term I learnt. SignalBloom’s Hallucination Bechmark
If AI interacts with the world and generates data from its own experience and learns from that, we have a new scaling mechanism. DeepMind podcast
OpenAI’s search API is fairly expensive at $30+/1K calls. Typically, to read interesting HN articles, I will make 30 calls which is about 75c. Instead I should use the app and summarise HM news across different days manually based on my interests!
Finally! t-strings land in Python. They’re like JavaScript template literals.
DuckDB’s CSV parser might be one of the most forgiving parsers. Even better than Pandas or SQLite3. Ref
Good managers will probably make good AI managers. AI agents can probably substitute humans in business experiments. Ethan Mollick
If Windsurf stops working, reload the extension. GitHub
TLS certificates will start expiring in 47 days from 15 Mar 2029, forcing automated domain renewals. Digicert
Nix flakes are a reliable alternative to DevContainers that don’t need Docker - but don’t work on Windows.
Ink is like React for the CLI.
The Unsure Calculator is a great tool to calculate formulas with multiple uncertainties, like:
- My office is 9-11 km away and it takes me 45-55 min to reach. So I cycle at 9~11 / 45~55 * 60 ~ 10-14 kmph (12 most likely).
- I spend $6-15 on lunch and eat out 80-120 days a year. So I spend 6~15 * 80~120 ~ $600~1550 ($1000 most likely) eating out yearly.
- I take 30-120 min to prepare a quiz question. Each exam has 6-12 questions. So I need 30~120 * 6~12 / 60 = 4~20 hours (11 most likely)
Using Kiran’s macOS setup for dev I enabled colorized less and mouse options for tmux.
time fish -i -c exit prints the time taken for fish startup. fish --profile-startup ~/fish.profile -i -c exit prints the time taken by each command on fish startup to ~/fish.profile. I used this to speed up my fish startup.
The 8 top features of the OpenAI Responses API that are an improvement over the Completions API (IMHO) are:
- Link to previous response rather than sending history
- Uploading files directly
- Swappable system instructions while retaining the chat history
- Customisable reasoning effort AND reasoning summary detail
- Truncation in the middle option
- Web search context size option
- File search filters by file attributes
- Flex service tier for lower cost
OpenAI doesn’t charge for file storage but does charge 10 cents / GB-day for vector storage beyond 1 GB. The first 1GB is free
Augment Code is an AI code editor that’s growing popular on Reddit. #ai-coding
The GPT 4.1 models have a 75% discounted prompt caching (instead of the usual 50%), making them particularly suited for repetitive tasks. OpenAI
chatgpt.com shortcut keys are revealed via Ctrl + /. Here’s my ranking on usefulness:
- Ctrl + Shift + C: Copy last response as Markdown!
- Ctrl + Shift + ;: Copy last code block
- Ctrl + Shift + S: Sidebar toggle
- Ctrl + Shift + O: Open new chat
- Shift + Esc: Focus chat input
- Ctrl + Shift + I: Ccustom instructions
- Ctrl + Shift + X: Delete chat

The Magic of Repeated ‘Improve It’ Prompts

Fri, 18 Apr 2025 10:02:37 +0000

What if you keep ask an LLM Improve the code - dramatically!?

We used the new GPT 4.1 Nano, a fast, cheap, and capable model, to write code for simple tasks like “Draw a circle”.

The we fed the output back and asked again, Improve the code - dramatically!

Here are the results.

Draw a circle rose from a fixed circle to a full tool: drag it around, tweak its size and hue, and hit “Reset” to start fresh.
Animate shapes and patterns turned simple circles and squares into a swarm of colored polygons that spin, pulse, and link up by distance.
Draw a fully functional analog clock grew from a bare face to one that builds all 60 tick marks in code—no manual copy‑paste needed.
Create an interactive particle simulation went from plain white dots on black to hundreds of bright, color‑shifting balls that bounce, die, and come back to life.
Generate a fractal changed from a single Mandelbrot image to an explorer you can zoom, drag, and reset with sliders and the mouse wheel.
Generate a dashboard jumped from static charts to a live page with smooth card animations, modern fonts, and a real‑time stats box.

A few observations.

Models are getting much more reliable. Even a low cost model like GPT 4.1 Nano wrote error-free code in ~100 retries.

When pushed, they tend to brag. They attach grand titles like “Ultimate Interactive Circle” or “Galactic Data Universe”. They sin out flash descriptions like “This dramatically upgraded clock features a pulsating neon glow, animated pulsing background glow, highly stylized tick marks, …”

A simple prompt like Improve it can spark new ideas, revealing features such as:

Things I Learned - 24 Nov 2024

Sun, 24 Nov 2024 00:00:00 +0000

This week, I learned:

OpenAI lets you download GPT instructions and execute arbitrary code in their containerized environment. This is not a bug. Ref
BM25 works as follows: Ref
- For each query term in the query, sum up the product of:
  - Inverse document frequency = LN(% of docs without the query term + 1) – with a small tweak
  - Term frequency = freq / (freq + k) – where k is usually between 1.2 to 2. Returns 0-1 with diminishing frequency benefit
    - k is multiplied by Document length normalization = 1 - b(1- DocLength/AvgDocLength). Longer documents have larger k, dampening frequency benefits.
- Some implications:
  - The actual BM25 score has no meaning. It’s just useful for ordering
  - BM25 scores for 2 queries can be compared ONLY IF the document sets don’t change
A list of Markdown to Website converters on this thread:
- Jekyll - Ruby - 2008
- MkDocs - Python - 2014
- GitBook - JavaScript (Node.js) - 2014
- MkDocs Material - Python (MkDocs-based) - 2016
- Docsify - JavaScript - 2016
- MdBook - Rust - 2017
- Antora - JavaScript (Node.js) - 2017
- Docusaurus - JavaScript (React) - 2017
- JupyterBook - Python - 2019
- Keenwrite - Java - ~2019
- Honkit - JavaScript (GitBook fork) - 2019
- Nextra - JavaScript (Next.js) - 2020
- Astro - JavaScript/TypeScript - 2021
- Hugo Book - Go (Hugo-based) - ~2020
- Clowncar - JavaScript/Node.js - ~2021
- Quarto - R and Python - 2022
- Starlight - JavaScript/TypeScript - 2023
DuckDB has an LLMs.txt. Today, 38 repos on GitHub support it
When identifying LLM use cases, it helps to tell LLMs what they can do. I use one or more of a list like below:
- Core capabilities:
  - Text Generation: Produce coherent and contextually relevant text across various domains.
  - Image Generation: Create realistic images that match the style and content of a given reference image.
  - Text to Speech: Convert text into natural-sounding speech with appropriate intonation and rhythm.
  - Speech to Text: Transcribe and interpret spoken language.
  - Vision: Analyze and describe visual content from images.
  - Video Analysis: Summarize and extract information from video content.
  - Text to Video: Generate realistic (and surrealistic) videos from text descriptions.
  - Function Calling: Execute predefined functions or access external tools to perform specific tasks.
  - Structured Output: Generate structured outputs like JSON, XML, HTML, YAML, DSLs, etc.
  - Tool Use: Utilize external applications or APIs to enhance functionality.
  - Code Generation: Write and debug code snippets in various programming languages.
- Cross-domain use cases:
  - Summarization: Understand and condense lengthy documents into concise summaries.
  - Translation: Convert text between multiple languages with high accuracy.
  - Question Answering: Provide precise answers to user queries based on provided information.
  - Reasoning and Planning: Solve complex problems and develop step-by-step plans.
  - Personalization: Tailor responses based on user preferences and historical interactions.
  - Dialogue Management: Engage in context-aware, multi-turn conversations.
  - Data Analysis: Interpret and generate insights from structured data.
  - Content Moderation: Identify and filter inappropriate or harmful content.
  - Sentiment Analysis: Detect and interpret emotions and opinions in text.
  - Robotics Integration: Interface with robotic systems for control and decision-making.
  - Knowledge Retrieval: Access and present information from vast datasets or knowledge bases.
  - Creative Writing: Generate poetry, stories, and other creative content.
  - Educational Assistance: Provide explanations and tutoring across various subjects.
  - Ethical Reasoning: Assess scenarios for ethical considerations and implications.
  - Accessibility Support: Assist users with disabilities through tailored interactions.
  - Simulation and Modeling: Create predictive models and simulate scenarios.
- Domain-specific use cases:
  - Legal and Medical Assistance: Offer information and guidance within legal and medical domains.
  - Gaming: Generate narratives, dialogues, and scenarios for interactive entertainment.
  - Scientific Research: Aid in literature reviews, hypothesis generation, and data interpretation.
  - Financial Analysis: Analyze market trends and provide investment insights.
  - Cultural Competence: Understand and respect diverse cultural contexts in interactions.
  - Security Applications: Detect and respond to potential cybersecurity threats.
  - Environmental Monitoring: Analyze data related to environmental changes and sustainability.
  - Healthcare Support: Assist in patient monitoring, diagnostics, and personalized treatment plans.
  - Supply Chain Optimization: Enhance logistics and inventory management through predictive analysis.
  - Customer Service: Provide automated support and resolve customer inquiries.
  - Market Research: Analyze consumer behavior and market trends for business insights.
  - Content Creation: Generate articles, blogs, and marketing materials.
  - Virtual Assistance: Manage schedules, reminders, and personal tasks.
  - Social Media Management: Craft posts and engage with audiences across platforms.
  - Human Resources: Assist in recruitment, training, and employee engagement strategies.
  - Event Planning: Organize and coordinate events, including logistics and communication.
  - Travel Planning: Provide itineraries, booking assistance, and destination information.
  - Real Estate: Analyze property markets and assist in buying or selling decisions.
  - Agriculture: Monitor crop health and optimize farming practices through data analysis.
  - Energy Management: Optimize energy consumption and monitor renewable energy sources.
  - Transportation: Enhance route planning and traffic management systems.
  - Urban Planning: Assist in designing sustainable and efficient urban infrastructures.
  - Disaster Response: Provide real-time information and coordination during emergencies.
  - Public Policy: Analyze data to inform policy decisions and predict societal impacts.
  - Art and Design: Generate visual art concepts and assist in creative design processes.
  - Music Composition: Create original music pieces and assist in songwriting.
  - Language Learning: Facilitate language acquisition through interactive exercises and feedback.
  - Historical Analysis: Interpret historical data and provide insights into past events.
  - Philanthropy: Identify charitable opportunities and assess the impact of donations.
  - Sports Analytics: Analyze player performance and game strategies.
  - Fashion: Predict trends and assist in clothing design and merchandising.
  - Culinary Arts: Generate recipes and provide cooking guidance.
  - Astronomy: Analyze celestial data and assist in space exploration research.
  - Psychology: Offer insights into human behavior and mental health support.
  - Linguistics: Analyze language patterns and assist in translation studies.
  - Archaeology: Assist in artifact analysis and historical site interpretations.
  - Literature Analysis: Interpret literary works and provide critical analyses.
  - Philosophy: Engage in discussions on ethical dilemmas and existential questions.
  - Mathematics: Solve complex equations and assist in theoretical research.
  - Physics: Model physical phenomena and assist in experimental design.
  - Chemistry: Analyze chemical compounds and predict reactions.
  - Biology: Assist in genetic research and ecological studies.
  - Geology: Analyze geological data and assist in natural resource exploration.
  - Meteorology: Predict weather patterns and analyze climate data.
  - Oceanography: Study marine ecosystems and assist in ocean exploration.
  - Anthropology: Analyze cultural data and assist in ethnographic research.
Style of writing impacts output style a lot. E.g. Adding an evil laugh makes Claude more creative. Ethan Mollick
For good structured mode output, we need good prompting.
- Mentioning examples and schema and “JSON” helps. When providing examples, using (user, assistant) message pairs helps (I think it’s because it’s easier for the LLM to parse).
- Using a {reasoning, answer} schema (with reasoning first) helps. Make reasoning concise and relevant Ref Arxiv
- We already know code in JSON is not a great idea. Ref
Just adding 3 real examples and regurgitation helped GPT 4o play chess much better. Both techniques may have more general use in prompting. Simon Willison
With Deno 2.0, the same .js file can run in Node.js as well as Deno. Example
jspm lets you generate import maps against any CDN.
You can click on htop columns on the terminal to sort by that column! Mouse events work on command line apps. Julia Evans
Alt Text will very likely be a browser feature. It’s important for the Alt text to flow as part of the content when listening to the page. Perhaps even become a part of the browser APIs like speechRecognition.
Langchain suggests multiple levels of agentic behaviour. LLM Call < LLM Chain < LLM Rounter < State Machine < Autonomous Langchain
A HTML quine: A page that, when rendered as HTML, shows the HTML source code of the page!
You can enable syntax highlighting just using fonts. Ref
HTML is all you need shows examples of using HTML for notebooks instead of Jupyter, Observable, etc.
Straive evaluated Gemini 1.5 Flash 002 and GPT 4o Mini for translation.
- Portugese: Flash is better than GPT 4o Mini. BLEU Word Overlap is 65.5% > 64.6% and METEOR (Semantic) is 84.9% > 78.9%
- Mandarin: Flash is better than GPT 4o Mini. BLEU Word Overlap is 25.0% > 15.9% and METEOR (Semantic) is 54.7% > 51.1%
The problem with Accept headers is that you can’t link to them. Simon Willison
Recraft v3 supports vector (SVG) generation Simon Willison. The output is 100% <path> elements (even for text). You get 50 free credits daily. Creating 1 image is ~2 credits. The API costs $1 per 1K credits. Some things I can create with it are:
- Base data visualizations that I can animate with code
- Icons in a specific style
- Comic strips
- Explainers for talks or student material
- Featured images for blog posts
- Architecture diagrams?

Fri, 20 Sep 2024 04:07:16 +0000

Looks like XML tags are the best way to structure prompts and separate sections for an #LLM. It’s the only format that all of Anthropic, Google, and OpenAI LLMs encourage.

For example:

… … … …

Anthropic Docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/use-xml-tags OpenAI Docs: https://platform.openai.com/docs/guides/prompt-engineering/strategy-write-clear-instructions Google Docs: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/structure-prompts

Alternatives are using JSON, Markdown, templating formats like Mustache/Jinja, etc.

Even Llama’s system tokens seem a little XML-like. https://github.com/meta-llama/llama3/blob/main/llama/tokenizer.py#L61-L74

Personally, I’ve been using Markdown so far. But it’s time to switch over. (Only on the prompt side. On the generation side, Markdown still seems the best.)

How fast are LLMs in production?

Sun, 01 Sep 2024 05:29:54 +0000

At Straive, we use an LLM Router. Since ChatGPT, etc. are blocked for most people, this is the main way to access LLMs.

One thing we measure is the speed of models, i.e. output tokens per second. Fast models deliver a much smoother experience for users.

This is a different methodology than ArtificialAnalysis.ai. I’m not looking purely at the generation time but the total time (including making the connection and the initial wait time) for all successful requests. So, if the provider is having a slow day or is slowing down responses, these numbers will be different.

Hopefully this gives you a realistic sense of speed in a production environment.

Here’s the speed of models with at least 500 requests over the last 2 weeks. I’ve grouped the models based on speed grades

Grade 1: 100+ Tokens / second. Groq is clearly serving the Llama 3 models at blazing speed. No surprises there – except why Groq still doesn’t let me pay. The free tier is open with generous rate limits and the Pay per Token model has been “Coming Soon” for several months now (and I’ve no complaints 🙂).

Grade 2: 70+ Tokens / second. Anthropic’s Claude 3 Haiku is the next fastest class of models, but Claude 3.5 Sonnet is surprisingly fast, almost as fast as Haiku and over 70 tokens per second. This is impressive.

Grade 3: 50-60 Tokens / second. OpenAI’s GPT 4o models are almost as fast. It’s interesting that GPT 4o and GPT 4o mini are at about the same speed! GPT 3.5 Turbo is not far behind either. Perhaps OpenAI increases capacity for slower models?

Grade 4: 30-50 Tokens / second. Gemini 1.5 Flash is a much, much slower than the benchmarks - maybe we’re doing something wrong. Azure’s GPT 4o service is about twice as slow as OpenAI’s, and comparable is speed with Gemini 1.5 Pro.

Grade 5: <20 Tokens / second. Azure’s GPT 3.5 Turbo and Google’s Claude 3 Sonnet are among the slowest ones. These are older models on third-party infrastructure, so I suspect they’ve been given weaker infrastructure (unlike OpenAI which is serving GPT 3.5 Turbo at 3X the speed Azure does.)

Drivers of speed

Here’s what I’m taking away (informally):

GPU architecture is the biggest driver of speed. Groq is FAST! Hopefully, the fact that they won’t let us pay isn’t a red flag that the service will vanish.
How companies operate seems the next biggest driver. Anthropic’s models are consistently faster than OpenAI’s which are faster than Google’s.
Companies run their own models faster than cloud providers. OpenAI is faster than Azure, and Anthropic is faster than Google for the same models.

Sat, 10 Aug 2024 10:59:35 +0000

Fascinating to see the how LLM cost-quality frontier moves. Recent fights were mostly on cost.

Yesterday, #OpenAI halved the GPT-4o cost. At $2.5/MTok (and with GPT-4o-min at 15 cents/MTok), the best and cheapest models are back with OpenAI, IMHO.

Sigh, time to move all our stuff back from #Anthropic. For now…

https://gramener.com/llmpricing/

Things I Learned - 04 Feb 2024

Sun, 04 Feb 2024 00:00:00 +0000

This week, I learned:

Alzhara is one of the VFX companies that worked on Leo’s hyena scene. Their 3D modeling is incredible.
Enterprise scenarios leaderboard. Mistral 7B leads.
Veda Srinivasan.
- How does Google manage culture?
  - AMA sessions
  - Manager feedback. Entirely anonymous. Avoid taking feedback for teams less than 5
  - Workplace concerns team exists. Put managers on watch
- Books
  - Mohammad Younus. Three zeroes book. Read about his social business theme
  - Pluriverse. Anti fragile. Aurobindo Vedas.
  - Barry Oshry. Seeing systems. Runs workshops but book is better
  - Raghu Anantanarayana has written about Indian archetypes based on Mahabharatha
  - India that is Bharath. Sai Deepak.
- Podcasts
  - Listen to Nilesh Oak. Sugreeva’s Atlas.
  - Pankaj Tripathi podcast on geography influences acting
- Areas of focus
  - “I’m an Expert on synthesis and implementation”
  - Intersectionality is another word for complex failures. Also for deep segmentation. Swiss cheese model.
  - Dialogic self theory is about multiple voices in the head. How do we make meaning? Psychological rupture is when cognitive activity is maximum. At any point there are MULTIPLE voices in our heads that are sources of action. We don’t listen to them.
  - Epistemology. Language determines thought. like the word productivity. How does appreciation of a rose become productive? Words from other languages may have incredible power. From other cultures.
Paul Sloan. Lateral thinking podcasts from multiple sources
- Deliberately engage with topics randomly.
- Deliberately engage with random people
- Read a random book from the library
- Watch a random film in a different language
- Consciously where the six thinking hats or look hard for the silent voices in your head and express them
- Ask children. They tend to think of more creative and childlike solutions
- He converted a hiring process into a contest
- Constantly ask yourself. What if every assumption I’m making about this is wrong?
- Scenario planning is really about this. List a few scenarios. They’d have high impact or high probability. What happens in this scenario? Ideate
You can @mention GPTs to ask a specific GPT a question in ChatGPT. This is really powerful.
Hidden brain podcast. Making the most of your mistakes
- FIX every small mistake. You never know how they might line up in the future
- You also never know how small little things done well might line up to give you a boost in the future
- The Toyota cord does not actually stop the production line. It brings a team lead over who quickly diagnoses the problem with you. The responsiveness of the league is a critical factor and so is encouragement
- That isn’t always a single bottleneck to stop that is the case of a simple failure. There can be a series of holes that happen to align perfectly.
- These are events that lead to catastrophic failures or successes
- Do as little as possible, waste as little as possible, until you know that the outcome is worthwhile.
- Figure out what is the value of the outcome and the most important piece of information you need to discover that
- Do full research before you try and fail. The aim of failure is learning at the least possible cost
How I write podcast. 2023 summary
- Ask for feedback from friends in a specific way.
- What 20% should I retain no matter what? What 20% should I cut? This allows them to compliment while providing genuine feedback
- Hire lawyer interns to proofread. They are the ones that find fault the best
- Be in a segment of one. Where there is zero competition. Something only you can do
- Don’t try to do stuff faster. Try to do stuff you don’t want to stop doing
- Read books older than 50 years
- Read Michael Collins book on things that sustain
- Temp service make sure he has some energy to spare. Cuz Riley does the opposite. She waits till she can’t stand it anymore and then writes like crazy until she drops dead. The former leads to thoughtful writing. The latter is emotionally powerful. Be able to do that
Vanna is a SQL generation LLM. An alternative to SQLCoder. This thread has a detailed discussion on SQL generation and BI
Intel developer cloud has a liberal GPU in the free tier.
OpenAI releases text-embedding-3-large which can be truncated. The embedding values have descending importance, so picking the first n is a good approximation. Also, gpt-3.5-turbo-0125 is 50% cheaper.
AppAgent is an LLM that can navigate mobile / web apps
Retrieval Centric Generation is an emerging alternative to RAG, where the LLM is explicitly built to leverage external knowledge. SimplyRetrieve is an early implementation.
Big Code Models Leaderboard is a leaderboard for open source code models.

Embeddings similarity threshold

Sat, 03 Feb 2024 03:21:02 +0000

text-embedding-ada-002 used to give high cosine similarity between texts. I used to consider 85% a reasonable threshold for similarity. I almost never got a similarity less than 50%.

text-embedding-3-small and text-embedding-3-large give much lower cosine similarities between texts.

For example, take these 5 words: “apple”, “orange”, “Facebook”, “Jamaica”, “Australia”. Here is the similarity between every pair of words across the 3 models:

For our words, new text-embedding-3-* models have an average similarity of ~43% while the older text-embedding-ada-002 model had ~85%.

Today, I would use 45% as a reasonable threshold for similarity with the newer models. For example, “apple” and “orange” have a similarity of 45-47% while Jamaica and apple have a ~20% similarity.

Here’s a notebook with these calculations. Hope that gives you a feel to calibrate similarity thresholds.

Comments

The LLM Psychologist - S Anand 6 Oct 2024 11:04 am (pingback): […] Over the last few months, several things changed. Most of my time is spent researching LLMs. […]

Things I Learned - 28 Jan 2024

Sun, 28 Jan 2024 00:00:00 +0000

This week, I learned:

⭐ OpenAI’s prompt engineering strategies are an excellent start for prompt engineering. A few lessons:
- Use detailed system prompts, often containing the entire instruction set, if it won’t change over the course of a conversation.
- “… summary of the prior conversation could be included as part of the system message” is an interesting history compression tactic.
OpenAI summarizes books by recursively summarizing sections and maintaining a running commentary of the summary so far.
Dan sends Google documents with essays instead of emails. This allows people to comment on it. But commenting is a culture and not many people do it. Adriano does it a lot and we’ll. Dan and Adriano actively converse on GitHub issues
llm-guard is an LLM content validation tool.