A colleague told me a near-miss horror story. As Google began deprecating Gemini 2.0, we moved to Gemini 2.5 Pro. But reasoning is enabled by default and cannot be turned off. For our specific problem statement, reasoning was not required. Token costs increased 10x and speeds were 3-4x slower. We moved the client to Gemini 2.5 Flash Lite, which has reasoning turned off by default and offers much lower latency. ...
This week, I learned: LiteParse is a PDF to text library that you can run via npx --package=@llamaindex/liteparse lit parse document.pdf. Simon Willison Always add indecisiveness, inaction, “other”, “not applicable”, etc. as an option to LLMs. They are trained for decisive responses and pattern matching, so we need to guide the the other way. Martin Fowler GPT 5.5 is priced twice that of GPT 5.4. No wonder my Codex usage is much higher than last month. Simon Willison. I am better off sticking to medium effort instead of the xhigh I usually use - it may not be required. OpenAI “… the eigenquestion is the question where, if answered, it likely answers the subsequent questions as well.” Shishir Mehrotra & Matt Hudson Claude Code stores the logged in OAuth token at ~/.claude/.credentials.json. We can use that to fetch https://api.anthropic.com/api/oauth/usage and retrieve Claude usage and reset times. uvx ccusage does this automatically, but I prefer my own script. Ontology matters in the AI era. But some stuff matters more, and some less. 🟢 MORE: Definitions: what “customer” means 🟢 MORE: Constraints: e.g. “don’t reclassify loans” 🟢 MORE: Interactions: how to verify, coordinate, delegate, … 🔴 LESS: Creating ontologies: agents can do that. 🔴 LESS: Completeness and rigor: agents tolerate uncertainty. 🔴 LESS: Proprietary: agents can reverse-engineer. There are several industries / markets that MBA case studies rarely cover (ChatGPT): Kirana stores; Care (child care, elder care, domestic work); Faith (finance, food, media, education); Remittances; Gambling (lottery, sports betting, gacha); Scams & organized fraud; Counterfeiting; …
In Feb 2024, Claude 3 Opus was the best model, at $15/MTok. In Jul 2024, GPT 4o Mini reached that quality at 10% of the price. In Dec 2024, DeepSeek v3 reached that quality at 1% of the price. Video See the interactive version If the price continues to fall 10x every 11-12 months or so (and it has been), then in a year, a Claude 4.6 Opus like model will cost 1/10th of the $5/MTok today, and in 2 years, 1/100th of that. ...
Yesterday, I discovered how much reasoning improves model quality. My Tools in Data Science assignment asks students to draft an llms.txt file for ipify and auto-checks with GPT-5 Nano - a fast, cheap reasoning model. I set reasoning_effort to minimal and ran this checklist: 1. Starts with "# ipify" and explains ipify. 2. Markdown sections on API access, support (e.g. GitHub, libraries). 3. Covers API endpoints (IPv4, IPv6, universal) and formats (text, JSON, JSONP). 4. Mentions free, no-auth usage, availability, open-source, safeguards. 5. Has maintenance metadata (e.g. "Last updated: <Month YYYY>"). 6. Mentions robots.txt alignment. Stay concise (no filler, <= ~15 links). If even one checklist item is missing or wrong, fail it. Respond with EXACTLY one line: PASS - <brief justification> or FAIL - <brief explanation of the first failed item>. With a perfect llms.txt, it claimed “Metadata section is missing” and “JSONP not mentioned” – though both were present. ...
How much does an LLM charge per hour for its services? If we multiple the Cost Per Output Token with Tokens Per Second, we can get the cost for what an LLM produces in Dollars Per Hour. (We’re ignoring the input cost, but it’s not the main driver of time.) Over time, different models have been released at different billing rates. New powerful models like O3 cost ~$7/hr – Poland’s minimum wage rate. Gemini 2.5 Pro costs ~$12/hr – France’s minimum wage rate. The latest Claude 4 Sonnet costs ~$2/hr – India’s minimum wage rate. ...
How much does an LLM charge per hour for its services? If we multiple the Cost Per Output Token with Tokens Per Second, we can get the cost for what an LLM produces in Dollars Per Hour. (We're ignoring the input cost, but it's not the main driver of time.) Over time, different models have been released at different billing rates. Most new powerful models like O3 or Gemini 2.5 Pro cost ~$7 - $11 per hr. ...
With the Gemini 2.5 Flash release, Google envelopes the entire cost-quality frontier of LLMs. In other words, at any cost or quality level, today, the best model to use according to the LM Arena score is a Gemini model. Results for O3, O4 Mini, and GPT 4.1 are not yet on LM Arena. But until then, #Google dominates. Nice work! Link: https://sanand0.github.io/llmpricing/ LinkedIn
Fascinating to see the how LLM cost-quality frontier moves. Recent fights were mostly on cost. Yesterday, #OpenAI halved the GPT-4o cost. At $2.5/MTok (and with GPT-4o-min at 15 cents/MTok), the best and cheapest models are back with OpenAI, IMHO. Sigh, time to move all our stuff back from #Anthropic. For now… https://gramener.com/llmpricing/ LinkedIn