Things I Learned - 25 Oct 2026
This week, I learned: Neural Networks Are Already Clean argues that in modern LLMs, most neurons have one function, not arbitrary overlapping functions (like we thought earlier). This becomes clearer when you see the learning steps, e.g. the “iy” neuron -> “diy” -> “do it yourself”. Without the “diy” in the middle, it’s confusing what “iy” has to do with “do it yourself”. Models probably learn by generalizing concepts repeatedly. Interpreting models has the bitter lesson, too: just tell a powerful agent to interpret it. Modern LLMs have switched from ReLU to SwiGLU. Why? ReLU is on-off and communicates no information when off. SwiGLU is a smooth function that whispers less relevant information - and that gives smarter models more context. Gemini 3.8 Flash is SLIGHTLY cheaper than Gemini 3 Flash Preview when transcribing 1 min of audio - #ForNow until Dec 31. After that, a higher pricing kicks in. ChatGPT ArtCraft extended their open source tools to cover MS Office, Adobe, and Autodesk #ForNow. Others are following suite. The Remade Wiki has includes OneNote, QuickBooks, Power BI, Outlook, MS Project, Visio, Discord, Jira, Linear, Monday, Corel, … #ForNow Cloudflare ported the TypeScript compiler to Rust. Increasingly, language choice is becoming fluid. Tiny Little Office uses a number of interesting tricks: The entire document is in the URL fragment, which is as much as 2 MB for Chrome! That means the entire document can be shared as a single QR code (if it’s not too large). Docs, spreadsheets, presentations, diagrams are all Markdown-based #ForNow and agent-friendly. They can be embedded as code fences inside each other. It uses domain-specific compression tailored to its language. It also uses ~ instead of - for base64 - to avoid wrapping issues with mail clients. It uses directives for agents. The way it is organized is interesting! document.currentScript.replaceWith(...) can be used as a nifty way to dynamically replace content, effectively creating a mini-templating language. Via Simon Willison. Standardization and personalization are independent axis. In fact, standardization can make personalization easier. ChatGPT Before the printing press, people used to spell words differently. Shakespeare, for example, had different spellings of his name. With the printing press and distribution of publication, spelling became standardized. But interestingly, it standardized by freezing the current conventions. So some oddities of the spellings people used are now a formal part of the language definition. In pharmacies, we call them “compounders” because they actually used to compound, i.e. mix, medicines - personalizing them to each individual. That faded away as standardized tablets became more common. Fonts are becoming more sophisticated. Some features in newer fonts are: Icons: NerdFonts is an example of a font class with 10K+ icons embedded - like GitHub, files, or terminals. Opt-in Ligatures: Display multiple characters as one symbol, e.g. => as ⇒, != as ≠, or >= as ≥, letting users pick what to enable. Variable weight: Continuously adjust thickness, e.g. weight 450 or 575 instead of being limited to Regular (400) and Bold (700). Smart width: Make characters narrower or wider, adjust shapes based on neighbors (making mi look more evenly spaced), etc. while preserving mono-space. “No one cares more about your case than you, so take control and send lots of follow-ups.” Things to know if you get cancer. The article has usefuls tips for important things in life, in general - not just cancer. Agents are making this easier. Because Amazon has a fair pricing policy, a strange consequence is that if Amazon sellers raise prices on Amazon they’re pressured to raise prices elsewhere too! So, increasing Amazon seller fees tends to increase prices across the board. How Amazon pushes up pricess all across the Internet Will Jev destroy its servers to save 5 lives? Will other models? A lovely set of experiments based on the trolley problem. Mistakes I made Week ending 25 Oct 2026 ...