Every time I get annoyed at people, I remind myself to be more like ChatGPT. Specifically: Don't get annoyed. Be patient. Encourage them. Step back and show them the big picture. (Then I get annoyed at myself for getting annoyed.) Today, I analyzed how exactly ChatGPT is different from me. So, I took a pitch document I co-authored with ChatGPT. Section A: Authored by Anand WHAT DO WE NEED? ...

This week, I learned: DuckDB is 2-10 times faster than Pandas. ClickHouse is supposedly faster but doesn’t run on Windows. Claude 3 Haiku input costs is $0.25/MTok. That’s half the GPT-3.5 cost. If it’s of comparable quality, it’s worth switching. But Claude 3 Opus is comparable to GPT-4 and twice the cost, so not worth it. Tavily is a search API for LLMs Interesting model garden models There are sites you TRULY cannot scrape even in the browser because of the isTrusted read-only property of events that you can never set to true. Oracle Service Cloud checks for isTrusted in mouse actions.

This week, I learned: Mughals just replaced the top of most temples with Mosque domes as part of the conquer or die policy “Math is racist”. Because people who can’t solve it can’t because of their underprivileged background! Winners: commodity businesses, companies that own lots of data like Reddit and Stackoverflow, profitable bootstrapped businesses Making a tool more usable, e.g. a video, can have a 10-100X impact. Yet every developer thinks it’s redundant All in one podcast. Can Google save itself? the success of a developer platform is the number of people using it. But not everyone uses it equally. Some people create winning products which drives attention to the platform. Use llm proxy like that to measure weekly average users and cost saving through caching one week ago, if someone at Google stood up and said we have too many black people in our images, the responsible AI team would have shut them down calling them racist. They had too much power and it was a one-way conversation. With the backlash now, there is a lot more awareness and acceptance of the balance. Security is like that. It’s too easy to empower and shut things down until there is a backlash the lawyer’s job is to tell you what’s not possible. But like Travis, your job is to decide whether it’s worth the risk of running a taxi company without a license or not Americans pronounce Sundar pichai’s name as Sun Daar! data licensing has become a business model. Reddit, Accel springer, stack overflow and many others are licensing their content to Google and open AI for several million dollars.

A friend told me today that using #ChatGPT will make humanity dumber. “Probably. Like always, #Calvin has the best response I know to that. “I propose we leave math to the machines and go play outside.” 🙂 LinkedIn

This week, I learned: You can use slots to stream HTML out of order! Shane Parrish. Short-term patience podcast have a frame of reference to relate EVERY experience to. That helps you evaluate (measure) and learn. That’s part of what Charlie Munger’s lattice of frameworks is about when there is a very high or very low interest scenario, low interest scenario then go ultra long term. Issued hundred years when the interest rate regime was very low short term optimal is rally long term optimal. So you need to learn to take a loss and look like an idiot to play the long-term game grit is a behavior that enables long-term thinking. Short term success gives you the luxury to think about long term #IMP power is about optionality. It’s about being in a position where you have the options that can affect the positive change rather than circumstances controlling you. Read Robert greene’s book on the 48 laws of Power low leverage enables that begin with the end in mind. Always how do you think about risk? Well, things do happen. It’s as simple as that autonomy and decentralization helps derisk do more and more of what works. That’s a powerful way of compounding long-term investments are better than frequent trading because you get to reinvest the tax you otherwise would have paid. So unless the alternative is super compelling, stay invested if you need to be the person who DOES the thing, you delegate less, leverage list, compound less, because you have to DO. BE A PERSON WHO SETS THE FIELD INSTEAD. The coach, the chess master, the director, patient strategist who Waits for the good hit Being in Control motivates #Lesson. my cycle tires were flat. I thought it was someone pulling out the air and felt very demotivated. But once I carried my cycle pump, I felt so much more in control and power and felt a whole lot better SourceGraph is the default platform for private code completion & search MetaVoice 1B offers voice cloning on American & British accents with 30s training Qwen 1.5 72B appears to outperform Mistral Medium, making it one of the top non-proprietary models Llava 1.6 is a substantial improvement over Llava 1.5 and slightly better than CogVLM, Qwen-VL AI scams are growing. Deepfakes scammed $34m. But voice fake for kidnapping is scarier. Buildspace’s demo is a great demo of how voice and actions can be used effectively. demucs does an EXCELLENT job of splitting songs into drums, bass, vocals and others

This week, I learned: Architecture.md is an emerging standard Managing wealth requires training. htmz is a fantastic way to load HTML into elements! Suguna Poultry is Using robots to walk in their farms, use sound and bird eyes and movement to predict birth health over 1-2 weeks Light on the back of the bird’s back AND face => lays eggs in 14 days, else takes days later (girls and mobile phones?) Teknoturf is using Gen AI to Improve prompts when teaching prompt engineering. Pronounce languages better, identifying which words Tamilians and Malayalis will mis-pronounce. Explore IRBlaster. It can control AC and can automatically increase temperature at night. My view: LLMs are general purpose and more capable than SLMs. They’ll win, like CPUs won over special-purpose chips. GPUs will optimize for LLMs and as usage grows, cost will fall. Andrej Karpathy’s summary of sharp edges in tokenization uses tiktokenizer to explain: Why LLMs can’t be used for spelling Why LLMs are better at English than other languages Why LLMs are bad at math Why SolidGoldMagiKarp is a single token Why trailing spaces are bad Why YAML tokenizes more efficiently than JSON ssyoutube.com: Just add “ss” to “youtube.com” on the video and you can download YouTube videos Discussions with Sachin, AMAT Microsoft said Indigo, Air India uses LLM based bookings Meta invested $70bn in GPUs. Sam Altman is investing $7tn! NVIDIA has a price PREMIUM not discount for bulk GPUs! AMD is the next company to watch for Numenta - Subutai Ahmad - deploys AI models on CPUs #TODO Read A Thousand Brains by Subutai Ahmad Sanjeev Sharma Swaayatt Robots: Autonomous driving in India Deepeigen: Education Rohan Shravan, Bangalore. Likes sharing knowledge. Amazing teacher. IIT KGP 2008. Interested in exploring quantum computing Tresa Motors, Inkers App, The School of AI AMAT is working on photon-based computing. science research models. AI for science. Like Google: Deepmind Genome, Microsoft: Metagen quantum: AMAT is actively in into this. Nagapati Banda is driving this John Kelly is predicting a ChatGPT moment in quantum in a few years Adobe express has a forever free video to GIF converter Edge workspaces let me keep the same tabs open across laptops! Command line interface guidelines RAWGraphs has a custom charts API that is worth learning from Python fastcore has decorators like @typedispatch, Self, etc. All image-to-text models on HuggingFace wddbfs mounts SQLite as a file system. I had a bit of trouble, maybe with Python package versions. Google is using LLM powered bug identification HuggingFace Chat Assistants has open source system prompts!! OpenHermes training dataset is available. 1M prompts! Jio has made IPL free. They make money on data and ads. That’s Scale! Daniel Dennett outsources thinking to students. Reviewing his books. BUT: I don’t take feedback. When someone sends a pull requests, I ignore it.

This week, I learned: Fine tuning makes economic sense only if the input tokens SAVED is twice the output token size on each call. Docker container memory usage on WSL2 docker stats frolvlad/alpine-glibc:alpine-3.17: 540KB ubuntu: 1MB (python3: +5MB) nikolaik/python-nodejs:python3.10-nodejs18-bullseye: 1.4MB (python3: +5MB) python:3-alpine: 612KB (python3: +7.5MB) python:3: 500KB (python3: +11.2MB) continuumio/miniconda3: 7.6MB (+6.5MB) Discussion with Vinu Yamunan Databuck by FirstEigen. Autolysis plus monitoring Quality council has the data steward (maintainer of each dataset) coming together with the uses on a weekly basis to understand what quality problems to users are facing. Data owners jaundice at a lower frequency to get an understanding #TODO Automate rules for data quality in our projects and intranet Convert a config rule into business language. Explain SQL. These are good use cases for llm’s Graph DBs are powerful for flexible data structures, but query generation needs AI or expertise. Check the Neo4J language cypher Explore storing SAME data in relational DBs AND in graph DBs / document DBs for different use cases Dallas rocketry challenge. Build a rocket that can take an egg to 800 feet exactly and land without breaking it Discussion with Karthik A #TODO Ask IIT students to do internship tasks. Use advent of code is a qualifying criterion Tata motors unionized DB admins for longevity. No one can take their jobs. Hires people who LIKE their jobs Rust gives me typing. It’s very efficient. Pola.rs is interesting but Pandas as good enough. Explore alerts from CCTV feeds. Karthik sends email alerts with pictures for: “Is the machine on or off”? for productivity “Are people not wearing helmets?” for safety at Cummins #TODO Integrate with WhatsApp. Use LLMs with function calling for responses Use expiring links (to pictures or content). It increases engagement Check Deno licensing. Is there a commercial clause? #ANS No - it’s MIT license Centre or excellence for zero emission tech at IIT. Karthik is part of it Explore auth0. 7000 users are free toml is part of the Python 3.11 standard library! If copilot writes code we don’t understand we are screwed. Hence expertise matters Discussion with Vikas Kedia #TODO Plan an AMA The mind becomes lazy with financial success. Vikas is treating his podcast as a startup Hire a professional videographer for your content Financial RoI in financial markets is the highest. Programming is high too but FS is even better “Performative power” – when you’re forced to perform, you get better ideas Observable 2.0 is an open source static site generator for data Python dataclasses SORA is OpenAI’s video generation model, and is stunning! If Appa comes to Singapore even for a week, he will feel better and can boast to his friends. At over 90, it may be better to move Appa to where I am since many of his friends would be no more and shops, doctors, etc can be managed and getting an independent house nearby is not hard. There is an SEZ in Gujarat where Indians can invest like in Mauritius without forex restraint Shubha: Media sites are moving away from Vickrey auctions to first-price auctions for ads. That’s because they send the auction price forward to a search engine and the winning second-price value can lose even though the owner is willing to pay more. Second-price auctions don’t work unless ALL bidders are in the SAME auction. Ad networks are a hierarchy of auctions! Gemini 1.5 launched. Fly.io offers GPU hosting and auto stop when they have nothing to do. Embeddings in random forest are very effective at classification – much better than dot product. To deploy apps with OAuth + templating support in a small Docker container, use Caddy Deno has native TypeScript, browser APIs, and compiles to multiple OSs Ruff is a MUCH faster flake8 Two pass generation is a clever technique to get multiple SEQUENTIAL answers in a single API request. For example the schema {'code', 'optimized_code'} will generate code and then optimize it. Unions in function calling allows flexible multi-step prompts in a single API.

This week, I learned: Dockerfile can have FROM scratch and you can add specific binaries rather than an entire OS. via Fine-tuning session by Dan. Notebook Example of fine-tuning Mistral. Consumed 28 computes ($2.8) Axlotl is what the top fine-tuned LLMs are trained on Deepspeed provides distributed training Flash attention lets data stay on GPU Sample packing packs samples of different lengths into equal length tensors Visualize the RANK of a token in a generated stream instead of logprob The Knowledge Project. Tomorrow Gayner What I’d like in my obituary: Anand was happiness. A guru. Generous. To get what we seek we must deserve this. Build, measure, learn If you did the same thing daily for 50 years, would it be a great thing? If yes, do it. If not, stop. Do this in daily retrospectives My new role should be productivity through technology innovation. That may mean a CTO role. But be specific otherwise no one will understand it Hidden brain podcast. Us 2.0. Win hearts, then minds When in an interaction, ask yourself. Can I learn and change myself? Can I win their hearts, then mines, so their behavior will change. That identity will change Notice when you get emotionally triggered. That’s exactly when you should not get emotionally triggered Try model humility and moral Look for close to people’s identities in our conversations. What are things they like? What does it mean for them? Simply ask. With that understanding of identity, it becomes easier to reframe things in a way they will understand Bard can talk to Gmail and Google Drive! #PREDICTION As automation takes over these mainstream activities, people will take over the niches. Since expertise like knowledge is fractal, there will be many more segments of one in the future and it will be easier to automate clusters of similar abilities. Recommenders and brands will become even more important Stephen Osserman’s Observables have some nice notes. Visualizing partial election results D3 Force Dilemmas: Data Distortion Sandra Becker’s 30 day D3 course

For those in #Singapore and interested in #datavisualization & #llms, I’m talking about Visualizing LLM Hallucinations at SUTD on Thu 8 Feb at 7 pm SGT. This is for a non-technical audience. We’ll visualize the basics of how LLMs work, how they make mistakes, and at least one technique on how to spot these. https://www.meetup.com/data-vis-singapore/events/298902921/ LinkedIn

This week, I learned: Alzhara is one of the VFX companies that worked on Leo’s hyena scene. Their 3D modeling is incredible. Enterprise scenarios leaderboard. Mistral 7B leads. Veda Srinivasan. How does Google manage culture? AMA sessions Manager feedback. Entirely anonymous. Avoid taking feedback for teams less than 5 Workplace concerns team exists. Put managers on watch Books Mohammad Younus. Three zeroes book. Read about his social business theme Pluriverse. Anti fragile. Aurobindo Vedas. Barry Oshry. Seeing systems. Runs workshops but book is better Raghu Anantanarayana has written about Indian archetypes based on Mahabharatha India that is Bharath. Sai Deepak. Podcasts Listen to Nilesh Oak. Sugreeva’s Atlas. Pankaj Tripathi podcast on geography influences acting Areas of focus “I’m an Expert on synthesis and implementation” Intersectionality is another word for complex failures. Also for deep segmentation. Swiss cheese model. Dialogic self theory is about multiple voices in the head. How do we make meaning? Psychological rupture is when cognitive activity is maximum. At any point there are MULTIPLE voices in our heads that are sources of action. We don’t listen to them. Epistemology. Language determines thought. like the word productivity. How does appreciation of a rose become productive? Words from other languages may have incredible power. From other cultures. Paul Sloan. Lateral thinking podcasts from multiple sources Deliberately engage with topics randomly. Deliberately engage with random people Read a random book from the library Watch a random film in a different language Consciously where the six thinking hats or look hard for the silent voices in your head and express them Ask children. They tend to think of more creative and childlike solutions He converted a hiring process into a contest Constantly ask yourself. What if every assumption I’m making about this is wrong? Scenario planning is really about this. List a few scenarios. They’d have high impact or high probability. What happens in this scenario? Ideate You can @mention GPTs to ask a specific GPT a question in ChatGPT. This is really powerful. Hidden brain podcast. Making the most of your mistakes FIX every small mistake. You never know how they might line up in the future You also never know how small little things done well might line up to give you a boost in the future The Toyota cord does not actually stop the production line. It brings a team lead over who quickly diagnoses the problem with you. The responsiveness of the league is a critical factor and so is encouragement That isn’t always a single bottleneck to stop that is the case of a simple failure. There can be a series of holes that happen to align perfectly. These are events that lead to catastrophic failures or successes Do as little as possible, waste as little as possible, until you know that the outcome is worthwhile. Figure out what is the value of the outcome and the most important piece of information you need to discover that Do full research before you try and fail. The aim of failure is learning at the least possible cost How I write podcast. 2023 summary Ask for feedback from friends in a specific way. What 20% should I retain no matter what? What 20% should I cut? This allows them to compliment while providing genuine feedback Hire lawyer interns to proofread. They are the ones that find fault the best Be in a segment of one. Where there is zero competition. Something only you can do Don’t try to do stuff faster. Try to do stuff you don’t want to stop doing Read books older than 50 years Read Michael Collins book on things that sustain Temp service make sure he has some energy to spare. Cuz Riley does the opposite. She waits till she can’t stand it anymore and then writes like crazy until she drops dead. The former leads to thoughtful writing. The latter is emotionally powerful. Be able to do that Vanna is a SQL generation LLM. An alternative to SQLCoder. This thread has a detailed discussion on SQL generation and BI Intel developer cloud has a liberal GPU in the free tier. OpenAI releases text-embedding-3-large which can be truncated. The embedding values have descending importance, so picking the first n is a good approximation. Also, gpt-3.5-turbo-0125 is 50% cheaper. AppAgent is an LLM that can navigate mobile / web apps Retrieval Centric Generation is an emerging alternative to RAG, where the LLM is explicitly built to leverage external knowledge. SimplyRetrieve is an early implementation. Big Code Models Leaderboard is a leaderboard for open source code models.

text-embedding-ada-002 used to give high cosine similarity between texts. I used to consider 85% a reasonable threshold for similarity. I almost never got a similarity less than 50%. text-embedding-3-small and text-embedding-3-large give much lower cosine similarities between texts. For example, take these 5 words: “apple”, “orange”, “Facebook”, “Jamaica”, “Australia”. Here is the similarity between every pair of words across the 3 models: For our words, new text-embedding-3-* models have an average similarity of ~43% while the older text-embedding-ada-002 model had ~85%. ...

This week, I learned: ⭐ OpenAI’s prompt engineering strategies are an excellent start for prompt engineering. A few lessons: Use detailed system prompts, often containing the entire instruction set, if it won’t change over the course of a conversation. “… summary of the prior conversation could be included as part of the system message” is an interesting history compression tactic. OpenAI summarizes books by recursively summarizing sections and maintaining a running commentary of the summary so far. Dan sends Google documents with essays instead of emails. This allows people to comment on it. But commenting is a culture and not many people do it. Adriano does it a lot and we’ll. Dan and Adriano actively converse on GitHub issues llm-guard is an LLM content validation tool.

I was crossing a not-too-busy street on a not-too-busy day in Chennai. I was having a voice conversation with ChatGPT (about the log probabilities of tokens on LLMs, if you're curious) when I was rudely interrupted by an auto rikshaw rapidly honking at me. "Honk honk honk honk honk" in rapid succession. Not unusual. Mildly annoying. The street was empty. The auto was empty. The traffic policeman was visible. I gave way and carried on. ...

This week, I learned: When comparing Mistral with 4b quantization vs unquantized: 2 responses were significantly shorter and fairly different 1 was identical 1 was almost identical but shorter by a few words 1 was slightly longer and fairly different #PREDICTION As humans have more conversations with LLMs, they will replace video watching and interactive gaming with conversation based role play. New game genres will evolve Lilac is an LLM-based data curation tool. Use it to search by concept (e.g. PII, duplicates, etc.) and then drop/update the results. Lungs have a Hausdorff dimension of 2.97 – giving them one of the highest surface area to volume ratio. Brains are 2.8. Sierpinski Pyramid is exactly 2 – which is weird. To solid-paint twice the size, you need 4 times as much paint. How I write podcast. Tim Ferriss High bars are constraints. I set the strongest constraints against the scarcest resources. Like reputation Being a category of one is more defensible than a competitive advantage Content always beats presentation. When in doubt, push for more interesting content Regular publishing improves thinking To build a habit, do less than you think you can do. That makes it easier to build momentum on the habit and sustain during crunch times There is a lot of mediocrity in the world. If you’re doing something (in a winner take all ecosystem), be the best. Top lawyers are exceptional proofreaders. They are able to see what is unclair, and what is redundant, and what has loop holes very quickly. Forcing yourself to cut down from a thousand words to 200 to a paragraph to a sentence takes you through a phase transition where you discover something unexpected The more outrageous the question, the more likely it is to be useful in generating a new perspective Eleven-labs speech synthesis with voice cloning is at the uncanny valley. With two 5-minute samples, my voice sounds a fair bit like my voice but is very clearly not my voice. I find stability ~ 30%, similarity ~ 80% and style ~50% gives a reasonable outcome. But the default voices (e.g. Joseph, George, Charlie) are excellent. Practical AI podcast: AI predictions for AI by API is the norm today and will grow Just having AI is no longer a differentiator AI is part of life, not just work #TODO Explore quickdrop from Stability for Maruti #TODO Explore Codium VS Code plugin and Continue.dev Hybrid systems that combine stats, ML, DL and AI models will grow AGI and AutoGPT resurgence RAG will continue to be a focus GPT4 will be beaten by open source models. Special purpose models beat it already Self hosted and cloud hosted models will grow for security Small language models will grow Productivity will be enhanced rather than replaced Multi modal models will grow Cost efficiency will grow in focus GPT Builder help explains how the GPT Builder updates GPTs - including some very interesting prompts

I looked at how Gramener uses ChatGPT Plus by evaluating 600+ chats asked over 3 months from Oct 2023 to Jan 2024. The team asks 6 questions a day. We don't track who or how many actively use ChatGPT Plus. This also excludes personal ChatGPT accounts. Still, 6/day is low for an entire team put together. The questions fall into 8 categories. Category%Excel, data exploration & analysis25%Text extraction and summarization13%HTML, CSS, or JavaScript code13%Python code13%LLMs, AI and use cases9%OCR and image analysis9%Generate images, logos, and designs7%General knowledge, policy & environment5%Audio and translation5% Here are some questions from each category - to give you an idea of emergent ChatGPT Plus usage. ...

This week, I learned: Transparent LED screens will be useful in windshieds to display maps as we drive. Marimo is a reactive alternative to Jupyter notebooks that saves files as pure Python. To run an org-specific chatbot on your own LLM: (via awesome-chatgpt) opengpts - but it doesn’t support auth chatbot-ui - but Supabase is hard to install anse - but it doesn’t support auth ChatGPT-Next-Web - but it doesn’t support auth Python 3.13 gets a store and copy JIT If an npm package adds another package as a dependency with version “*”, target package cannot unpublish ANY version! So this is a way of freezing EVERY repo and preventing unpublishing of EVERY version – an unintentional flaw in the npm design. via Quantization is better than fewer parameters. So prefer high parameters (e.g. 70b) and quantize to 4-bit. In-browser playgrounds has compiled WASM versions of Python, PHP, SQLite. Happiness Lab podcast. Happiness lessons of the ancients Talking to strangers makes us happy Giving money makes us happy Free time makes us happier than working hard Tangi Domain-specific models being beaten by general purpose models is a phase. It will reverse towards domain. AI will potentially help build and understand domain-specific models Models are evolving so rapidly that humans cannot interpret models. We need a process to interpret models! xAI, Responsible AI, Physics-guided or Knowledge-guided models (called grey box models) are therefore a trend CS papers Don’t review other papers, certainly not other fields. Disregard measurement errors. When CS papers get applied to climate, manufacturing or biology, we’ll worry about Interpretability Domain-specific mechanics. (Introduce that into the training as a constraint.) Many domain experts are using AI to UNDERSTAND their process. Need to explore Uncertainty IB adds context to make learning applicable. But that distracts from the core learning, and if there’s a gap it widens Most data science courses teach “Python science”, not data science. They teach a bunch of models. They don’t teach how even one kind of model e.g. LSTM works. Most coaching programs today teach FAMILIARITY with problems, not critical thinking Most of current education will become redundant thanks to LLMs. For students AND teachers Coding will become irrelevant Cognitive thinking, reasoning, human relations, systems thinking will become more relevant Troubleshooting will become more important. AI is not self-diagnosing. I would hire someone who can figure out something is going wrong, diagnose what’s going wrong, and fix it #TODO Hire for troubleshooting ability. Give a Q, an A, and ask them to figure out if it’s wrong, why, and fix it All my exams and quizzes are open book, open ChatGPT. Onus is on me to give a problem that forces you to think. #TODO Write a question paper that is ChatGPT proof. Exploring AI could be a ToK subject. “How to interact with an AI?” We need a manual on how to use AI. Like Simon Willison says Content doesn’t suffice. You need pedegogy. What to serve you at what time, how, how to assess. Lots of businesses are filling this gap Students get great confidence when a teacher points to online content and says, I"ll tell you WHAT to see" and COMPLEMENTS that in their class “The map is not the territory.” Most people confuse sample mean for the actual. #ASK Parameter estimation -> Signal estimation -> State estimation Stats vs DL differ in that There is no notion of a defined “truth”. Hence reliability is not measurable Parameters have no value. Hence interpretability is ignored. #TODO Read 2020 National Education Policy. It’s quite modern. We need a manual on self-learning too Listening is not learning. You know only if you implement. Levels for students: I can solve it. I can explain why it works. I can find alternatives. I can apply it to a new area, reformulating (requires imagination.) For teachers, you also need: Responsible learning (extra careful about what to teach and how to teach, to exceite them, to teach at THEIR level). Show the universality and connecting to other concepts. E.g. noise reduction with FT is like using water to remove dirt. Transform to water domain, remove dirt, transform back to air domain. It’s better than dusting clothes to remove clothes. Washing machine programs are just different models of removing noise in the water domain. Teach people who WANT to learn AND who will APPLY it long-term. That’s what maximizes impact Grad students are more satisfying that way. Else, it is WASTED effort. (Not that it’s a bad thing for the student, but the effort IS wasted for the teacher) Therefore, I believe students should have general engineering first, and let students pick specialization later. Some universitie are doing that. #THINK Students remember my philosophy more than my content. We impart character, not just knowledge. Astrology and horoscopes serve a different function. They provide explainability, not predictive ability. As the world becomes less explainable, the need for astrology will grow. Explainability is about creating STORIES that fits data plausibly. It has nothing to do with data or truth. Explainability and predictive ability and reproducibility are all different. Maybe, Science is about the latter two, less about explainability. Astrology is a model. The map is not the territory. It’s an explanatory, not a predictive model. #THINK Therefore, my lessons are just explanations. Stats about experiments are STILL explanations. They are NOT reproducible or predictive. Hence not yet science The meaning of our life is the transformation we undergo in our lives #TODO Read “The Journey of Souls” by Michael Newton. A hypnotherapist #TODO Try regression therapy / hypnosis. Record it and listen to it. Just for fun! Rohini Deshpande Slam book was the Facebook of the 1900s Prepared mind is an extremely powerful tool for learning. Practice prepared mind When women drop out of education or career, that is also a waste from the teacher and system perspective The time for career growth is the same as child bearing time for women. That’s not true for men. But child rearing can be done by either. That’s not recognised. It’s 0K for a man to raise the child and make the home and 0K to treat that as the default Since men are more senior, it’s usually logical for them to stay in their jobs. That’s a systematic bias. When seniors advise women to step back. they respect it. That widens the barrier. Why not eliminate that situation? Be proud of the working women in the family Stats are just a symptom. They don’t explain the cause. (Map is not the territory.) Explanations are what really helps us fix the cause. Hence stories are important. Read Tinker Tailor Soldier Spy RV Athimber health tips: Eat foods with low glycemic index Eliminate free salt completely Voyage AI Embeddings have a higher quality, similar price compared to OpenAI embeddings. There’s a clear benefit to replacing text-embedding-3-large with voyage-3-lite. There’s a 200 MTok free tier currently. mixtral-offloading cleverly loads only the model layer required at any point, letting you run Mixtral 8x7b on Colab Free and on 16GB GPUs. This notebook runs on Colab Free too. CodeGPT is an alternative to Github Copilot that can use any LLM.

This week, I learned: Raman Srinivasan: IITM Profs and MTechs are spinning off deep tech startup. Agnicool is an example. They 3D print rockets with ceramic composites from Germany Sriram Krishnan (Facebook), Balaji Krishnan invested in pre-Series A Govt is de-regulating space tech and geospatial. Talking of de-regulating nuclear. ISRO seems to be focusing on cutting edge while others are doing commercial stuff There are about 100 space tech startups in India You can build your own modular reactor Geospatial AI is a big opportunity Have released a lot of 10m resolution geospatial data almost for free success is about getting NO factor wrong. Failiure just requires one aspect to fail. Brand, business savviness, financial stability, tech superiority, deep pockets, managing Gvt, long-term mindset, etc. - all of these matter. That’s what made TCS monopolize the exam business in India. For deepening AI, we need, Talent, Data pipelines, Hardware Next wave is LMMs, not LLMs What’s not captured in LLMs is verbal knowledge and tacit knowledge (in people’s fingertips). India is rich in this. The road to tacit knowledge has to go through India We can get a welder to train a simulator and pay the welder We can get a storyteller to tell a few stories and train oral LLMs Tacit knowledge will have to cover robotics. Train robots to bring coffee in just 50 demos! “Project delays are within the ‘rulebook’. Buyt paying skilled welders for ship building or nuclear pressure boilers needs breaking 100s of rules. Once they get certified, they abscond to Iran or somewhere.” TCS Ignite started in 2006 by Ramadorai. Before recession. “There is going to be a talent shortage. Recruit from next rung. Science not engineering graduates. Break HR monopoly and corruption - colleges became placement agencies. Fewer people per college. Across the country. Train them.” Tried in 2000. HR refused. Business refused. When Chandra was identified, Ramadorai took it up himself as a challenge. Ramadorai had very precise attention. Sat 7 am calls. “What are you doing?” 2 min call. Enough to energize. Would exchange and ask for brief updates. He reads and responds. You get a decision in a few hours early in the morning. No decision bottleneck He wanted to know ALL the details. Very precise, small, frequent probes on what’s happening. E.g. one 6 am, he called. “What are the lectures planned for today?” He expected I would know this. If not, next time I would be prepared. He would call another person and ask the same question. So I updated the others. I’ve never seen anyone with that bility to ground-truth. He wants 10 birds from 1 stone. Get BSc, but don’t comprimize. Get the best 2 per college but a full batch size of 500. We became the biggest training program as a single batch – with 500 people. He wanted to demonstrate scale. HR and CFO said, ‘You recruit first. Then we’ll give you money. We don’t think it’s possible." We had anchor colleges and brought people from other campuses. We did digitized exams. Took big servers to the campus. Fully digitized with full auditability. Plugged the laptops into the college LANs. Kids had never used a mouse. We had to teach them. We said, “Don’t worry. These are logical questions, not questions. We’ll pay a full salary.” We learned that 1 out of 2 didn’t even join. Many took up a Masters. They didn’t want to join the workforce. Unless they’re desperate economically. Even poor parents, if they can afford to support you at home, they do that. It’s weird. Every weekend, we visited a few campuses. 71 locations across the country. Found the NSS college in Ottapalam (Kumbakonam of Kerala. Cultural centre.) College had a nice nice Math dept website. I said “Mr Ramadorai, this looks promising.” One Sat morning, he called and said, “When are you going to Ottapalayam?” We landed in the college. There was an impromptu communist student strike. We made 38 offers out of 100 who took the exam. Never had such a high conversion. One girl, whose father was a coolie, jad communication issues. Had a colleague talk in Malayalam. She was an amazing success. My colleague Murali made a documentary about her. We started in July. By Dec, we had 500 joinees. No one is doing such a thing now. You have to get dozens of things just right. Compromising on even one kills it. Ramadorain loaded it with multiple objectives. Fresh talent. Low cast. Sustainable. He kept pushing for innovation. I pushed back. But he was persistent. Over time, I came around and we started innovating. We restructured training program around innovation. Like a YCombinator. That unleashed extraordinary energy. Several of the kids are running their own startups. Ramadorai was very supportive of that. The assessment product came out of that. First batch, everyone was very sceptical. We got a lot of pushback. They’re dumb. Ethics issues. Communication issues. Lot of prejudice. So we got them to do internal recruitment till they were satisfied. An internal placement market. THEN reputation was set. I told them, always stick to the dress code. One weaver’s sone wore a bright yellow polyester T-shirt. I asked him why he didn’t stick to the dress code. “Sir, it’s my first T-shirt.” Ramadorai tracked how many became billable. We were unable to place 70. He said, give them 1 more month training. Then we placed 64 of the 70. He said “Do something about the 6. I want 100% placement.” We absorbed them as a teaching assistant. One was a weaver’s son. One was a PC’s daughter. A mestri’s son. A shopkeeper’s daughter from North Madras. None could speak English. They learned to code and helped build the exam software, with Srikumar who was a brilliant Java coder. That gave us the confidence that these are good kids, just from the wrong part of town. With a good guide, they’re very capable. We bought a bunch of Nintendo Wiis. Kids have to play. He asked for a welding simulator. “Velu the Welder”. The kids built it using the Wii. We got the most accomplished welder spend an afternoon at Ignite. He ripped us apart. 4 hrs non-stop. He told us EVERY thing wrong with it. Blasted us. I told Murali, “Let’s call it a toy. It’s not a simulator. Let kids play.” He said, “I want to show that it can be done!” Murali churned out rapid iterations in a frenzy. Ramadorai said, “Deploy it in the field.” So we went to all kinds of remote places like Gondiya below Nagpur. Surprisingly cosmopolatan. Junction of EW and NS train lines. We set up welding institutes in each. It was on the cloud. We could track everything. KPK killed the skills. Hard core bureaucrat. His view is colonial. Ignite philosophy is about unleashing energy of people. Colonoial model is about controlling people by keeping them poor. KPK and Chidambaram had that mindset. Ramadorai brought him in as Secy of NSDC. he killed the policies Modi did the first cut by creating a ministry. KPK ensured that it never gew. Like Yes Minister. Made sure nothing moved Had Govt not changed, he would have been Secy Finance. He was seen as Chidambaram’s blue eyed boy. People know he was associated with NSE scam. Ramadorai helped by bringing him into skills He is very smart. Knows the IAS machinery in and out. Lives and breathes that. H Ramadorai likes him though. Put him on board of Tata Consumers. NSE Scam. He’s part of the cabal with Ajay Shah. Private trading firms could co-locate within NSE and could make a huge amount of money. KPK ran some of this by proxy to fund Congress. But he left no fingerprints. But everyone knows it is him. He was running Chitra Ramakrishnan by proxy. He was the Himalayan Yogi. Ignite continued with unwavering focus. Kept increasing the kind of focus. We had a 99.5% success rate in placements. Just a handful of failures. Ramadorai has written about Ignite in “The TCS Story”. My Dad translated it in Tamil. It’s not a typical business biography. Worth reading. Should be a mandatory course in MBA courses in India. So many lessons. You have to read it knowing how Mr Ramadorai speaks. What is NOT said is just as important. Ch 5 is the thinnest - on the IPO. It is packed with so much stuff. Unless you know, you won’t understand. “Tatas got the Govt to change a tax law to make the IPO meaningful.” Behind that, there’s a lot. You have to be alert to catch the sentene. He won’t brag, or talk about the significance of some of these. Book is packed with dense insights. Unless you ARE LOOKING FOR IT, you’ll miss it. Worth reading SEVERAL times. You need a foot-noting. Currently reading Pasquenelli – Social History of Artificial Intelligence. Eye of the Master. Worth reading. I’m not Marxist by belief but they get some things right. Surprised how vibrant the European left is. “If someone is doing manual work, there is tacit knowledge that automation captures.” India doesn’t need self-driving cars. But a farmer would like a gaming controller that ploughs his fields while he sits under a tree. Semi-intelligent machines that removes the burden of hard labour in the country. Once a year, for a few weeks, I do manual labour. People are under-nourished. People typically work 5 hours a day. Not enough muscle mass. So use them for what they’re good at I’ve seen the power tools. When Chinese power tools became cheap, the power welding became much more efficient. Everyone has become a monkey with power tools. They charge per inch. They know how to leverage the tech for economic benefit. Just bring in the power tools and rapidly finish and make money. But there are sections that are still poor and haven’t made the transition. How can we create pathways for them? How can AI help? Anand: Why not use a gimball. RS: Good idea. Role modern psychologist DW Winnicott on ChatGPT (like Socrates) E.g. You don’t need a perfect mother. A good enough mother is better Similarly, why not a “good enough” Bharat mata than a perfect one? To persuade someone, align it with their identity. ChatGPT 5 technologies of interest according to Gartner’s latest hype cycle: GitOps Internal Developer Platform Graph Data Science Open Source Program Office Value Stream Management Platforms Gemini is an alternative to the Web. Sort of like Gopher, but recent SALI - Standards Advancement for the Legal Industry - has standards and ontology/taxonomy for legal documents, including patent litigation. Walking new routes habitualizes fighting fear and preferring novelty ⭐ GPT-4 is bad at math. It gets ~60-70% of answers wrong. LMQL provides a constraint-based query language for interacting with LLMs. It uses token masking, which is clever. Hollywood writers signed a deal that limits AI in script writing. It’s primarily aimed at protecting script writer wages. Adobe Firefly offers a “generative fill” that lets you remove or paint new objects into an image. I’m awaiting text to vector images. Duet AI is Google’s answer to Github Copilot. Teachers are using LLMs to plan lessons, write emails to parents, create tests, adjust reading level of materials, personalize content with tools like MagicSchool, Diffit, Eduaide. WizardLM creates datasets for instruction tuning by cleverly using LLMs to create new prompts. Deita is an approach to improve instruction tuning datasets. Dhyeya: Attack on Titan is as good at Death Note Jaidev: Long car drives are a good place to explore new song genres. Try in taxis Same radio channels may have different frequencies across cities. Vividh Bharati is 100.5 FM in Chennai and 106.4 in Delhi Things to explore: Radio for new songs Clubhouse Twitter Spaces Instagram reels YouTube reaction videos (e.g. atheist, Indian songs, etc.) Stand-up comedies (Ricky Gervais, Louis CK, Jordan Peterson) Porn artists are at risk because of Gen AI

I read 52 books in 2023 (about the same as in 2022, 2021 and 2020.) Here’s what I read (best books first). Fiction The Kingkiller Chronicle. I picked it up before a flight to London in 2014. Read it through the flight. Read it late into the night at our AirBnB. Skipped my workshop prep. Read it during the workshop breaks. Read it on the flight back. And I re-read it every year or two. The language is beautiful and the story gripping. I feel miserable this series isn’t complete. The Name of the Wind by Patrick Rothfuss ⭐⭐⭐⭐⭐ The Wise Man’s Fear by Patrick Rothfuss ⭐⭐⭐⭐⭐ The Stormlight Archive. Another series I re-read regularly. Brandon Sanderson takes the scale of the story up a notch in every book. Rhythm of War by Brandon Sanderson ⭐⭐⭐⭐⭐ Words of Radiance by Brandon Sanderson ⭐⭐⭐⭐⭐ Andy Weir’s books. Since my daughter re-reads The Martian (laughing loudly), I picked up Project Hail Mary. It’s a brilliant depiction of alien physiology and communication, with a weird kind of humour I love. Project Hail Mary by Andy Weir ⭐⭐⭐⭐ The Egg by Andy Weir ⭐⭐⭐⭐ The Martian by Andy Weir ⭐⭐⭐⭐ Red Rising Saga. A pleasant discovery of a new series. Somewhat like The Hunger Games and Divergent. Red Rising by Pierce Brown ⭐⭐⭐⭐ Golden Son by Pierce Brown ⭐⭐⭐⭐ Morning Star by Pierce Brown ⭐⭐⭐⭐ Blake Crouch’s books. The two I read were both time-travel related and I love that genre. These do a great job of exploring some of the deeper implications of time-travel. Recursion by Blake Crouch ⭐⭐⭐⭐ Dark Matter by Blake Crouch ⭐⭐⭐ Ready Player One by Ernest Cline ⭐⭐⭐. It’s as good as the movie with slightly different scenes. The Reckoners by Brandon Sanderson. Another series I re-read. Steelheart by Brandon Sanderson ⭐⭐⭐⭐ Firefight by Brandon Sanderson ⭐⭐⭐ Calamity by Brandon Sanderson ⭐⭐⭐ The Year of Sanderson. Brandon Sanderson’s kickstarter raised $41m for 4 books this year (mostly Cosmere). The stories themselves were OK but the hints they drop about the Cosmere are invaluable. Yumi and the Nightmare Painter by Brandon Sanderson ⭐⭐⭐⭐ Tress of the Emerald Sea by Brandon Sanderson ⭐⭐⭐ The Sunlit Man by Brandon Sanderson ⭐⭐⭐ Fullmetal Alchemist by Hiromu Arakawa. After Death Note, it felt like a let-down when it started. A mundane story. Then it grew funny. Showed shades of a much deeper story. I’m mid-way through the series and I’m hooked. Fullmetal Alchemist, Vol. 1 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 2 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 3 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 4 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 5 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 6 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 7 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 8 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 9 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 10 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 11 by Hiromu Arakawa ⭐⭐⭐ Mono no Aware e altre storie by Ken Liu ⭐⭐⭐. A nice short story Traitors Gate by Jeffrey Archer ⭐⭐⭐. A well-writter fast-paced average story. Mistborn: Secret History by Brandon Sanderson ⭐⭐⭐. Average story but with lots of “secrets” about the Cosmere. Asterix and the Griffin by Jean-Yves Ferri ⭐⭐. Some good jokes but not as good as the original series. Non-fiction ...

In 2023, I made 3 resolutions: Run 50 experiments. I managed 44 / 50. (Here are some). Learnings: I need to improve planning (9), scepticism (6), and lateral thinking (4). Make 1 change a month in my environment. I managed 8 / 12. The largest impact was from meeting new people, working out of new places, and using new gadgets. Calendar integrity, i.e. stick to my calendar. I succeeded over 95% of the time. My most memorable events in 2023 were: ...

This week, I learned: Quantum computing is slow, has low transfer bandwidths, and only prime factorization has an exponentially faster algorithm. via The hidden brain podcast. What would Socrates do? Also Philosophy Bites Podcast: why do philosophers use example. And: the happiness lab: happiness lessons of the ancients How many of our beliefs are truly our own? How many are a product of our environment? Contrast these and identify your true beliefs For every thought and action you have, even tiny ones, ask “Why am I doing that?” Dig deeper because it may not be intrinsic One way to become memorable is to.write stuff others will reproduce for a long time. Plato and Aristotle did that everyone has multiple personality. This is partly because different parts of the brain evolved independently for different functions. System one and system two thinking are just such one broad classification. e.g. We think our train is moving when the nearby train moves because our visual brain is faster than our somatic brain. Good lessons and pitches cater to the rational AND the subconscious. Reason AND story. To activate different parts of the brain. That’s why philosophers use examples Philosophy brings change through reason. Revelations: through sudden insight. Rhetoric: through insight. Act as if you already are what you want to become. Aristotle Align your environment (including habits) to your beliefs. It will become easier to act your beliefs then. All virtues are moderation. It’s possible to take every virtue to the wrong extreme Some Christians have wristband that reads WWJD. What would Jesus do? Explore yourself a reminder of what would X do. Maybe Benjamin Franklin, Socrates, Feynman, etc People mistake their environment for their feelings. 1970s Experiment: People on a shaky bridge think they love each other. Experiment: people rationalize things irrespective of reality. “The Unexamined Life” is about questioning theories or stories or maps constantly. It’s also about questioning our thoughts and emotions constantly. Mindfulness is the VERBAL way of doing this. Meditation is the NON-VERBAL way of paying attention. Both are Processes to remove distraction and increase authenticity. Learning about people is a good way to learn about ourselves. And vice versa. Lica has a fascinating demo of how a document can be converted into a video story. Spillnot doesn’t spill drinks even when you swing! Things super-intelligences could do that humans can’t: Solving complex mathematical problems Advanced scientific discovery (quantum computing, nanotechnology, biotechnology) Ultra-precise predictive modeling in complex systems (climate, economics, social dynamics) Optimizing global systems at high precision (logistics, traffic, energy distribution, resource allocation) Universal translation (unknown languages, animal communication, extraterrestrial signals) Deep medical personalization: individualized medical treatments from genetics, environment, and lifestyle Create new materials: Designing materials or chemicals with specific properties Complex system integration: combining AI, bio tech, nano tech in new ways Philosophical insights: new perspectives or solutions to age-old philosophical dilemmas Space exploration and colonization Predicting natural disasters Customized education at scale Ways of working with them Collaborative problem solving Creative collaboration Decision support Personalized education Establishing ethical and safety protocols Recreational and leisure activities Mini-GPTs is an interesting approach to shrink LLMs and make them domain specific. It takes existing LLMs and removes neurons not used in a specific domain (e.g. law, medicine, etc.) Book to read (again) about how to take a team beyond their abilities even if you’re not the expert “Measure What Matters” by John Doerr “High Output Management” by Andy Grove “The Checklist Manifesto” by Atul Gawande “The Lean Startup” by Eric Ries “Creativity, Inc” by Ed Catmull “The Hard Thing About Hard Things” by Ben Horowitz “The Four Disciplines of Execution” by Chris McChesney, Sean Covey, and Jim Huling