Things I Learned - 07 Jan 2024

This week, I learned: Raman Srinivasan: IITM Profs and MTechs are spinning off deep tech startup. Agnicool is an example. They 3D print rockets with ceramic composites from Germany Sriram Krishnan (Facebook), Balaji Krishnan invested in pre-Series A Govt is de-regulating space tech and geospatial. Talking of de-regulating nuclear. ISRO seems to be focusing on cutting edge while others are doing commercial stuff There are about 100 space tech startups in India You can build your own modular reactor Geospatial AI is a big opportunity Have released a lot of 10m resolution geospatial data almost for free success is about getting NO factor wrong. Failiure just requires one aspect to fail. Brand, business savviness, financial stability, tech superiority, deep pockets, managing Gvt, long-term mindset, etc. - all of these matter. That’s what made TCS monopolize the exam business in India. For deepening AI, we need, Talent, Data pipelines, Hardware Next wave is LMMs, not LLMs What’s not captured in LLMs is verbal knowledge and tacit knowledge (in people’s fingertips). India is rich in this. The road to tacit knowledge has to go through India We can get a welder to train a simulator and pay the welder We can get a storyteller to tell a few stories and train oral LLMs Tacit knowledge will have to cover robotics. Train robots to bring coffee in just 50 demos! “Project delays are within the ‘rulebook’. Buyt paying skilled welders for ship building or nuclear pressure boilers needs breaking 100s of rules. Once they get certified, they abscond to Iran or somewhere.” TCS Ignite started in 2006 by Ramadorai. Before recession. “There is going to be a talent shortage. Recruit from next rung. Science not engineering graduates. Break HR monopoly and corruption - colleges became placement agencies. Fewer people per college. Across the country. Train them.” Tried in 2000. HR refused. Business refused. When Chandra was identified, Ramadorai took it up himself as a challenge. Ramadorai had very precise attention. Sat 7 am calls. “What are you doing?” 2 min call. Enough to energize. Would exchange and ask for brief updates. He reads and responds. You get a decision in a few hours early in the morning. No decision bottleneck He wanted to know ALL the details. Very precise, small, frequent probes on what’s happening. E.g. one 6 am, he called. “What are the lectures planned for today?” He expected I would know this. If not, next time I would be prepared. He would call another person and ask the same question. So I updated the others. I’ve never seen anyone with that bility to ground-truth. He wants 10 birds from 1 stone. Get BSc, but don’t comprimize. Get the best 2 per college but a full batch size of 500. We became the biggest training program as a single batch – with 500 people. He wanted to demonstrate scale. HR and CFO said, ‘You recruit first. Then we’ll give you money. We don’t think it’s possible." We had anchor colleges and brought people from other campuses. We did digitized exams. Took big servers to the campus. Fully digitized with full auditability. Plugged the laptops into the college LANs. Kids had never used a mouse. We had to teach them. We said, “Don’t worry. These are logical questions, not questions. We’ll pay a full salary.” We learned that 1 out of 2 didn’t even join. Many took up a Masters. They didn’t want to join the workforce. Unless they’re desperate economically. Even poor parents, if they can afford to support you at home, they do that. It’s weird. Every weekend, we visited a few campuses. 71 locations across the country. Found the NSS college in Ottapalam (Kumbakonam of Kerala. Cultural centre.) College had a nice nice Math dept website. I said “Mr Ramadorai, this looks promising.” One Sat morning, he called and said, “When are you going to Ottapalayam?” We landed in the college. There was an impromptu communist student strike. We made 38 offers out of 100 who took the exam. Never had such a high conversion. One girl, whose father was a coolie, jad communication issues. Had a colleague talk in Malayalam. She was an amazing success. My colleague Murali made a documentary about her. We started in July. By Dec, we had 500 joinees. No one is doing such a thing now. You have to get dozens of things just right. Compromising on even one kills it. Ramadorain loaded it with multiple objectives. Fresh talent. Low cast. Sustainable. He kept pushing for innovation. I pushed back. But he was persistent. Over time, I came around and we started innovating. We restructured training program around innovation. Like a YCombinator. That unleashed extraordinary energy. Several of the kids are running their own startups. Ramadorai was very supportive of that. The assessment product came out of that. First batch, everyone was very sceptical. We got a lot of pushback. They’re dumb. Ethics issues. Communication issues. Lot of prejudice. So we got them to do internal recruitment till they were satisfied. An internal placement market. THEN reputation was set. I told them, always stick to the dress code. One weaver’s sone wore a bright yellow polyester T-shirt. I asked him why he didn’t stick to the dress code. “Sir, it’s my first T-shirt.” Ramadorai tracked how many became billable. We were unable to place 70. He said, give them 1 more month training. Then we placed 64 of the 70. He said “Do something about the 6. I want 100% placement.” We absorbed them as a teaching assistant. One was a weaver’s son. One was a PC’s daughter. A mestri’s son. A shopkeeper’s daughter from North Madras. None could speak English. They learned to code and helped build the exam software, with Srikumar who was a brilliant Java coder. That gave us the confidence that these are good kids, just from the wrong part of town. With a good guide, they’re very capable. We bought a bunch of Nintendo Wiis. Kids have to play. He asked for a welding simulator. “Velu the Welder”. The kids built it using the Wii. We got the most accomplished welder spend an afternoon at Ignite. He ripped us apart. 4 hrs non-stop. He told us EVERY thing wrong with it. Blasted us. I told Murali, “Let’s call it a toy. It’s not a simulator. Let kids play.” He said, “I want to show that it can be done!” Murali churned out rapid iterations in a frenzy. Ramadorai said, “Deploy it in the field.” So we went to all kinds of remote places like Gondiya below Nagpur. Surprisingly cosmopolatan. Junction of EW and NS train lines. We set up welding institutes in each. It was on the cloud. We could track everything. KPK killed the skills. Hard core bureaucrat. His view is colonial. Ignite philosophy is about unleashing energy of people. Colonoial model is about controlling people by keeping them poor. KPK and Chidambaram had that mindset. Ramadorai brought him in as Secy of NSDC. he killed the policies Modi did the first cut by creating a ministry. KPK ensured that it never gew. Like Yes Minister. Made sure nothing moved Had Govt not changed, he would have been Secy Finance. He was seen as Chidambaram’s blue eyed boy. People know he was associated with NSE scam. Ramadorai helped by bringing him into skills He is very smart. Knows the IAS machinery in and out. Lives and breathes that. H Ramadorai likes him though. Put him on board of Tata Consumers. NSE Scam. He’s part of the cabal with Ajay Shah. Private trading firms could co-locate within NSE and could make a huge amount of money. KPK ran some of this by proxy to fund Congress. But he left no fingerprints. But everyone knows it is him. He was running Chitra Ramakrishnan by proxy. He was the Himalayan Yogi. Ignite continued with unwavering focus. Kept increasing the kind of focus. We had a 99.5% success rate in placements. Just a handful of failures. Ramadorai has written about Ignite in “The TCS Story”. My Dad translated it in Tamil. It’s not a typical business biography. Worth reading. Should be a mandatory course in MBA courses in India. So many lessons. You have to read it knowing how Mr Ramadorai speaks. What is NOT said is just as important. Ch 5 is the thinnest - on the IPO. It is packed with so much stuff. Unless you know, you won’t understand. “Tatas got the Govt to change a tax law to make the IPO meaningful.” Behind that, there’s a lot. You have to be alert to catch the sentene. He won’t brag, or talk about the significance of some of these. Book is packed with dense insights. Unless you ARE LOOKING FOR IT, you’ll miss it. Worth reading SEVERAL times. You need a foot-noting. Currently reading Pasquenelli – Social History of Artificial Intelligence. Eye of the Master. Worth reading. I’m not Marxist by belief but they get some things right. Surprised how vibrant the European left is. “If someone is doing manual work, there is tacit knowledge that automation captures.” India doesn’t need self-driving cars. But a farmer would like a gaming controller that ploughs his fields while he sits under a tree. Semi-intelligent machines that removes the burden of hard labour in the country. Once a year, for a few weeks, I do manual labour. People are under-nourished. People typically work 5 hours a day. Not enough muscle mass. So use them for what they’re good at I’ve seen the power tools. When Chinese power tools became cheap, the power welding became much more efficient. Everyone has become a monkey with power tools. They charge per inch. They know how to leverage the tech for economic benefit. Just bring in the power tools and rapidly finish and make money. But there are sections that are still poor and haven’t made the transition. How can we create pathways for them? How can AI help? Anand: Why not use a gimball. RS: Good idea. Role modern psychologist DW Winnicott on ChatGPT (like Socrates) E.g. You don’t need a perfect mother. A good enough mother is better Similarly, why not a “good enough” Bharat mata than a perfect one? To persuade someone, align it with their identity. ChatGPT 5 technologies of interest according to Gartner’s latest hype cycle: GitOps Internal Developer Platform Graph Data Science Open Source Program Office Value Stream Management Platforms Gemini is an alternative to the Web. Sort of like Gopher, but recent SALI - Standards Advancement for the Legal Industry - has standards and ontology/taxonomy for legal documents, including patent litigation. Walking new routes habitualizes fighting fear and preferring novelty ⭐ GPT-4 is bad at math. It gets ~60-70% of answers wrong. LMQL provides a constraint-based query language for interacting with LLMs. It uses token masking, which is clever. Hollywood writers signed a deal that limits AI in script writing. It’s primarily aimed at protecting script writer wages. Adobe Firefly offers a “generative fill” that lets you remove or paint new objects into an image. I’m awaiting text to vector images. Duet AI is Google’s answer to Github Copilot. Teachers are using LLMs to plan lessons, write emails to parents, create tests, adjust reading level of materials, personalize content with tools like MagicSchool, Diffit, Eduaide. WizardLM creates datasets for instruction tuning by cleverly using LLMs to create new prompts. Deita is an approach to improve instruction tuning datasets. Dhyeya: Attack on Titan is as good at Death Note Jaidev: Long car drives are a good place to explore new song genres. Try in taxis Same radio channels may have different frequencies across cities. Vividh Bharati is 100.5 FM in Chennai and 106.4 in Delhi Things to explore: Radio for new songs Clubhouse Twitter Spaces Instagram reels YouTube reaction videos (e.g. atheist, Indian songs, etc.) Stand-up comedies (Ricky Gervais, Louis CK, Jordan Peterson) Porn artists are at risk because of Gen AI

Books in 2023

I read 52 books in 2023 (about the same as in 2022, 2021 and 2020.) Here’s what I read (best books first). Fiction The Kingkiller Chronicle. I picked it up before a flight to London in 2014. Read it through the flight. Read it late into the night at our AirBnB. Skipped my workshop prep. Read it during the workshop breaks. Read it on the flight back. And I re-read it every year or two. The language is beautiful and the story gripping. I feel miserable this series isn’t complete. The Name of the Wind by Patrick Rothfuss ⭐⭐⭐⭐⭐ The Wise Man’s Fear by Patrick Rothfuss ⭐⭐⭐⭐⭐ The Stormlight Archive. Another series I re-read regularly. Brandon Sanderson takes the scale of the story up a notch in every book. Rhythm of War by Brandon Sanderson ⭐⭐⭐⭐⭐ Words of Radiance by Brandon Sanderson ⭐⭐⭐⭐⭐ Andy Weir’s books. Since my daughter re-reads The Martian (laughing loudly), I picked up Project Hail Mary. It’s a brilliant depiction of alien physiology and communication, with a weird kind of humour I love. Project Hail Mary by Andy Weir ⭐⭐⭐⭐ The Egg by Andy Weir ⭐⭐⭐⭐ The Martian by Andy Weir ⭐⭐⭐⭐ Red Rising Saga. A pleasant discovery of a new series. Somewhat like The Hunger Games and Divergent. Red Rising by Pierce Brown ⭐⭐⭐⭐ Golden Son by Pierce Brown ⭐⭐⭐⭐ Morning Star by Pierce Brown ⭐⭐⭐⭐ Blake Crouch’s books. The two I read were both time-travel related and I love that genre. These do a great job of exploring some of the deeper implications of time-travel. Recursion by Blake Crouch ⭐⭐⭐⭐ Dark Matter by Blake Crouch ⭐⭐⭐ Ready Player One by Ernest Cline ⭐⭐⭐. It’s as good as the movie with slightly different scenes. The Reckoners by Brandon Sanderson. Another series I re-read. Steelheart by Brandon Sanderson ⭐⭐⭐⭐ Firefight by Brandon Sanderson ⭐⭐⭐ Calamity by Brandon Sanderson ⭐⭐⭐ The Year of Sanderson. Brandon Sanderson’s kickstarter raised $41m for 4 books this year (mostly Cosmere). The stories themselves were OK but the hints they drop about the Cosmere are invaluable. Yumi and the Nightmare Painter by Brandon Sanderson ⭐⭐⭐⭐ Tress of the Emerald Sea by Brandon Sanderson ⭐⭐⭐ The Sunlit Man by Brandon Sanderson ⭐⭐⭐ Fullmetal Alchemist by Hiromu Arakawa. After Death Note, it felt like a let-down when it started. A mundane story. Then it grew funny. Showed shades of a much deeper story. I’m mid-way through the series and I’m hooked. Fullmetal Alchemist, Vol. 1 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 2 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 3 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 4 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 5 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 6 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 7 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 8 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 9 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 10 by Hiromu Arakawa ⭐⭐⭐ Fullmetal Alchemist, Vol. 11 by Hiromu Arakawa ⭐⭐⭐ Mono no Aware e altre storie by Ken Liu ⭐⭐⭐. A nice short story Traitors Gate by Jeffrey Archer ⭐⭐⭐. A well-writter fast-paced average story. Mistborn: Secret History by Brandon Sanderson ⭐⭐⭐. Average story but with lots of “secrets” about the Cosmere. Asterix and the Griffin by Jean-Yves Ferri ⭐⭐. Some good jokes but not as good as the original series. Non-fiction ...

My Year in 2023

In 2023, I made 3 resolutions: Run 50 experiments. I managed 44 / 50. (Here are some). Learnings: I need to improve planning (9), scepticism (6), and lateral thinking (4). Make 1 change a month in my environment. I managed 8 / 12. The largest impact was from meeting new people, working out of new places, and using new gadgets. Calendar integrity, i.e. stick to my calendar. I succeeded over 95% of the time. My most memorable events in 2023 were: ...

Things I Learned - 31 Dec 2023

This week, I learned: Quantum computing is slow, has low transfer bandwidths, and only prime factorization has an exponentially faster algorithm. via The hidden brain podcast. What would Socrates do? Also Philosophy Bites Podcast: why do philosophers use example. And: the happiness lab: happiness lessons of the ancients How many of our beliefs are truly our own? How many are a product of our environment? Contrast these and identify your true beliefs For every thought and action you have, even tiny ones, ask “Why am I doing that?” Dig deeper because it may not be intrinsic One way to become memorable is to.write stuff others will reproduce for a long time. Plato and Aristotle did that everyone has multiple personality. This is partly because different parts of the brain evolved independently for different functions. System one and system two thinking are just such one broad classification. e.g. We think our train is moving when the nearby train moves because our visual brain is faster than our somatic brain. Good lessons and pitches cater to the rational AND the subconscious. Reason AND story. To activate different parts of the brain. That’s why philosophers use examples Philosophy brings change through reason. Revelations: through sudden insight. Rhetoric: through insight. Act as if you already are what you want to become. Aristotle Align your environment (including habits) to your beliefs. It will become easier to act your beliefs then. All virtues are moderation. It’s possible to take every virtue to the wrong extreme Some Christians have wristband that reads WWJD. What would Jesus do? Explore yourself a reminder of what would X do. Maybe Benjamin Franklin, Socrates, Feynman, etc People mistake their environment for their feelings. 1970s Experiment: People on a shaky bridge think they love each other. Experiment: people rationalize things irrespective of reality. “The Unexamined Life” is about questioning theories or stories or maps constantly. It’s also about questioning our thoughts and emotions constantly. Mindfulness is the VERBAL way of doing this. Meditation is the NON-VERBAL way of paying attention. Both are Processes to remove distraction and increase authenticity. Learning about people is a good way to learn about ourselves. And vice versa. Lica has a fascinating demo of how a document can be converted into a video story. Spillnot doesn’t spill drinks even when you swing! Things super-intelligences could do that humans can’t: Solving complex mathematical problems Advanced scientific discovery (quantum computing, nanotechnology, biotechnology) Ultra-precise predictive modeling in complex systems (climate, economics, social dynamics) Optimizing global systems at high precision (logistics, traffic, energy distribution, resource allocation) Universal translation (unknown languages, animal communication, extraterrestrial signals) Deep medical personalization: individualized medical treatments from genetics, environment, and lifestyle Create new materials: Designing materials or chemicals with specific properties Complex system integration: combining AI, bio tech, nano tech in new ways Philosophical insights: new perspectives or solutions to age-old philosophical dilemmas Space exploration and colonization Predicting natural disasters Customized education at scale Ways of working with them Collaborative problem solving Creative collaboration Decision support Personalized education Establishing ethical and safety protocols Recreational and leisure activities Mini-GPTs is an interesting approach to shrink LLMs and make them domain specific. It takes existing LLMs and removes neurons not used in a specific domain (e.g. law, medicine, etc.) Book to read (again) about how to take a team beyond their abilities even if you’re not the expert “Measure What Matters” by John Doerr “High Output Management” by Andy Grove “The Checklist Manifesto” by Atul Gawande “The Lean Startup” by Eric Ries “Creativity, Inc” by Ed Catmull “The Hard Thing About Hard Things” by Ben Horowitz “The Four Disciplines of Execution” by Chris McChesney, Sean Covey, and Jim Huling

One Year of Transforming Thoughts by Changing Environments

From The Extended Mind I learnt that our environment shapes our thinking more than I’d expected. That we can arrange our environment to extend our thoughts. In 2023, each month I changed something in my environment to see: What does “changing my environment involve”? What can I change? Will I succeed? Does it affect my thoughts? Can I track this? Here are the results. ...

Things I Learned - 24 Dec 2023

This week, I learned: DPO is a simpler alternative to RLHF for fine-tuning. Several HuggingFace models use DPO for training Name2Vec is a potential embedding for names. Google Knowledge Graph ID powers the Knowledge Graph. If it begins with /m/ it’s the same as the FreeBase ID. This is now available as WikiData. e.g https://www.wikidata.org/wiki/Property:P2671 I tried running Mixtral-8x7b locally (via Llamafile) and on together.ai. It’s good, but far from GPT 4. Generic computate-intensive algorithms eventually beat domain-specific tuning, because of Moore’s law. Ref The hidden brain podcast. the mystery of beauty Evolution drove us to beauty as an efficient survival mechanism. Understanding the world is one such mechanism. Hence we enjoy maths and chess ⭐ This leaderboard included paid models like GPT4 and Claude and compared them with open models on HUMAN + system benchmarks Lez Friedman Podcast: Jeff Bezos Build stuff that is is ubiquitous that other people take it for granted. The initial idea needs to be that obvious and easy. Like one click purchase or customer reviews Build stuff that other people can build on. Internet makes startups possible. Infrastructure is about enabling others at scale Decision making approaches: single person decides on two way doors. Deliberate as a team on one way doors Conflict resolution: disagree and COMMIT. NO sniping, I told you so, malicious compliance. Avoid compromise. Avoid decision by attrition (most persistent wins). People are inherently biased towards hierarchy. So the senior most person should speak last We have a happiness bias. Contracted by choosing the unhappier options first The map is not the territory. The metric is not the objective. We need metrics. But make sure you know why See the world through the eyes of the customer. Use your own product. It’s living their lives that makes customer obsession real. Jeff Bezos called their own customer care to see how long the actual wait time was. It was much longer than the metric reported How to prioritize. whatever problems customers will still face in 10 years are the big problems. These are worth putting time into because they are stable in time People working on big problems will never get down to the small problems. So have a dedicated team that works only on the paper cuts. It should be a dedicated team We co evolve with our tools. We build tools and then our tools change us. It reprograms our brains Cut out 10 minutes to the beginning of each meeting for people to read the material. They never reread anyway. This makes the meetings more productive Powerpoint is designed for persuasion, not truth seeking. It is also easier for the author than for the reader. Prefer narratives that are focused on finding the truth and are easier for the audience though tougher for the author ⭐ whisper-standalone-win provides a Windows binary for Faster-Whisper. It just needs CUDA and cuDNN installed. Then whisper-faster.exe video.mkv --language=English --model=medium generates the transcript. LLM use cases by Benedict Evans “Every text box on the internet will get an LLM” “Infinite interns” “Every UNIX function has become a company.” “Every ChatGPT suggestion…” llm360 publishes models along with training datasets. In The Age of AI has begun, Mar 2023, Bill Gates says, “In my lifetime, I’ve seen two demonstrations of technology that struck me as revolutionary.” The GUI (1980) and ChatGPT (2022). Rubeus is a HTTP proxy for multiple LLMs with load-balancing, fallbacks and retries. GPTRouter is a Python interface for multiple LLMs with fallbacks and retries. ⭐ Token Tally has an LLM Cost Tool that estimates GPU memory required and token cost across cloud providers.

Things I Learned - 17 Dec 2023

This week, I learned: Grab. Improving last mile delivery in maps. When did people pick up the phone, when should driver be allocated to minimize waiting time, layer on top of OSM. Singapore developers the Sea Lion 7b model Try VLLM with AWQ format. Can do batch inferencing. Needs a good GPU Amex prediction whether they can pay back in 1 year or 18 months. That choice is a business decision. In real time. Precompute individual score and use it as input to another model. Model must be explainable by regulation. Creates decision tree models therefore. Compliance team must agree if I can use a feature. Can’t use gender. Age (in US, Canada);- high age is more risk. Can’t use edu level in the US. Capture information from camera and use LLMs. Like traffic cameras mapping. Explore GIS from video cameras Grab tracks road closures and road accidents and whether a cycle can go on a road vs a bike vs a car All drivers have a front facing camera Drivers report road accidents by pressing a button Amex prices individual loans when selling to a collection agency #TODO buy a bike head camera! Playwright is a browser-based test framework. Supports recording. OpenAI provides logprobs for tokens! This can be a used to create cool visualizations of the likelihood of the each tokens. Github Copilot’s new features makes your entire workspace or a specific file its context. It also auto-writes your commit messages and PR descriptions. Mixtral-8x7b-Instruct “… really does seem to be equivalent in quality to ChatGPT 3.5.” Ref Practical AI podcast Advent of Gen AI is going on. Explore add to tools in data science course. Model validation write a book as an open source to github repository. Easier to evolve and easier to get feedback on.. Explore utterances as a GitHub commenting platform automatically give credits to contributors who have center pull request that was accepted or an issue that was fixed. This encourages contribution Visit book.premai.io ast-grep is a semgrep alternative that focuses on code refactoring rather than security. Comby is another such tool Serply is a Google Search API alternative to Google CSE ⭐ Generate textbooks! ChatGPT is good at generating questions or training datasets. It genuinely creates them rather than replicating from memory. Ref v0.dev creates web pages from code. Example. LIDA from Microsoft is an LLM based data visualization tool.

Things I Learned - 10 Dec 2023

This week, I learned: Bard supports extensions that include @Gmail – i.e. converse with your email. llama-cpp-python works with other GGUF models like Mistral and allows constrained output - JSON, function calling, etc. Ref 12 Tuning Strategies for RAG Llama Datasets are RAG datasets created mostly using GPT-4. Mostly small datasets. ⭐ Intuitions about large language models Bigger models (70b) are much better at learning from few-shot examples. They really learn. Bigger models will keep getting better! Chain of Thought prompting is a way of providing more compute to complex problems that require more compute Models will show emergent (completely new) behaviors that can’t be predicted from extrapolation. These may not be intentional. CodeAnt.ai is a VS Code plugin to detect code smells, refactor for modularity, to write docstrings and unit tests Anyscale prices the 7b Llama2, Zephyr, Mistral models at 15 cents per 1M tokens. Roughly 1/10th of GPT-3.5 Turbo’s ~$1.5 per 1M tokens Tools to identify personally identifiable information: galactic can use LLMs to detect PII Presidio by Microsoft Sherlock is a generic sematic type matching DL model pii-extractor-llm was trained on Indian names GLiNER is a Lightweight Generalist model for NER Tools to explore ElevenLabs speaks in your voice Cutout Pro removes backgrounds and parts of images Vocal Remover removes vocals from songs CapCut video editor TheBloke’s $35/month Patreon might be one of the least expensive ways to set up quantized LLMs in production. Microsoft released table-transformer to extract tables from PDFs. Sample usage Convert PDF to markdown with marker - an improvement over nougat. JupyterLab has a %%ai magic to use LLMs within notebooks. Ref Telling ChatGPT that the year is 2123 makes it bypass copyright. Ref Meta released SeamlessExpressive which preserves emotions in speech-to-speech translations Unsloth offers faster lower-memory LLM QLoRA finetuning DeepSeek is an open-source high-quality LLM Scalable Extraction of Training Data from (Production) Language Models extracts training data by repeating a token infinitely. SkyPilot lets you run LLMs on any cloud provider. vLLM lets you deploy LLMs with a single command. llamafile lets you run LLMs locally as a single file executable!

Things I Learned - 03 Dec 2023

This week, I learned: Gwern Branwen says LLMs nudge his “… making heavier use of the languages I don’t know well (Emacs Lisp & Python) since I increasingly trust that an LLM can help me maintain them.” Undetectable.ai checks for AI content. But it had false positives AND negatives in the 5 checks I ran. GPTZero got 2/2 right and seems better at detecting AI content. CoVA scrapes web pages via OCR When coding with LLMs, have SHORT, RELIABLE feedback loops. Ref

ChatGPT Custom Instructions

I speak with ChatGPT ~20 times a day. That’s more than I speak with most of my colleagues. ChatGPT is clearly my favorite team member. I conduct trainings, reviews and mentoring sessions with my colleagues. How to write code. How to write slides. How to communicate. That last bit is particularly important. With ChatGPT Custom Instructions, I can guide ChatGPT on how to work better with me. Currently, I have 10 custom instructions. They evolved over time and will continue to evolve. ...

Things I Learned - 26 Nov 2023

This week, I learned: This is an interesting GPT Vision API prompt from Simon Willison: “given this event flyer, create a link to add it to my Google Calendar”. Ref Quote from Jerry Liu: “GPT 4 is really good at complex reasoning”. It’s worth exploring what that means. Quote from Jerry Liu: “RAG is a hack”. It’s engineered, not machine learnt, so it’s suboptimal. We need an ML way of creating the context. Maybe fine tuning can be a way of CREATING the right context. But RAG can handle deterministic stuff like access control. Open AI fine tuning API is not good at memorizing info the way it is exposed. But the Gorilla paper shows that fine tuning can actually memorize well. Learn ML optimization approach - LLMOps. Have an evaluation framework with metrics like weights and biases or tensorboard. Helps figure out where fine tuning helps and where RAG does. Soon, this will become important. Flat indexing of chunks is not the only way to store embeddings. LlamaIndex allows you to create hierarchies that you can traverse for retrieval Agents mimic programming primitives. Switch. While. Call a function. Print. OpenRouter hosts several models and offers them as APIs! Ragas metrics evaluate quality of a RAG pipeline Orca 2 was trained on different reasoning techniques (e.g. step-by-step) and is as good as larger models Embeddings can help just re-rank regular search results. Ref Claude 2 Anthropic has a 200K context window but is still crap. Video-Llava can understand videos too. CoVA scrapes web pages using LLMs and visual information. jsonrepair can fix JSON fairly well. jsonformer wraps HuggingFace models to produce JSON. Ref Google has a model garden with lots of pre-trained and trainable models. Gorilla LLM specializes in APPI calls: Torch Hub, TensorFlow Hub, HuggingFace GPT-4 does not do abstraction at human levels Each of the GPTs / Prompts we create could be like a UNIX command prompt, and become a startup of its own Llava Plus extends LlaVA with pre-trained vision models that make image editing better Ollama runs local LLMs

Winning the alphabetical race

Since my name (Anand) begins with “A”, I used to get called on fairly early at school. In attendance. Answering questions. Classroom exercises. Quizzes. Even the distribution of test results. A few people later told me that it is good training, since I’d always be prepared. (Maybe. I’ve no idea.) At IBM and IIMB, Ajit was the only one ahead of me, alphabetically. Then he went a step ahead and named his son Aadi. I thought that’s impossible to beat. ...

Things I Learned - 19 Nov 2023

This week, I learned: XOT - Everything of Thought is a new prompt from Microsoft but I don’t understand it Creating Fine-Tuning datasets WITHOUT inputs Tamil-Llama Voyager plays Minecraft! Langchain supports evaluators. Pydantic is all you need drives towards code = data = text!

Things I Learned - 12 Nov 2023

This week, I learned: Julius.ai queries structured data. TODO: Explore https://github.com/microsoft/TaskMatrix microsoft/autogen enables multi-agent conversations. Architecture of today’s LLMs is similar to the A16Z architecture Stanford Foundational Model Transparency index was critiqued as misleading vLLM runs HuggingFace transformers models faster. So does DeepSpeed

LLMs can teach experts

I am a fairly good programmer. So, when I see a problem, my natural tendency is to code. I’m trying to break that pattern. Instead, I ask ChatGPT. For example, I asked: Write a compact 1-line Python expression that checks if user.id ends with @gramener.com or @straive.com user.id.endswith(("@gramener.com", "@straive.com")) After 15 years of using Python, I learnt that .endswith() supports tuple suffixes. This has been around since Python 2.5 (released in 2006 – before I knew Python.) The documentation has a tiny sentence in the middle saying “suffix can also be a tuple of suffixes to look for.” ...

Father of the bride

In 2012, I started Gramener with half a dozen friends. This week, we were acquired by Straive, a part of Barings Private Equity Asia. How do you feel? I feel like the father of the bride. Gramener was registered on 26 Feb. A day before my daughter’s birthday. I’ve spent more time with Gramener than my daughter. That makes Gramener my elder child. Who’s moving into a new household. Along with me. (I feel like சகலகலா சம்மந்தி.) ...

Scraping

I was at Cream Centre with my father on a Sunday afternoon. We’d finished a light lunch and were debating dessert. (He has triglycerides. I have cholesterol.) This was my fifth visit this year, and I had abstained so far. I couldn’t any longer. I ordered a Sizzling Brownie Sundae. But not for reasons you might think. Expertise comes from experience. I scrape food more than 99% of the people I know. So, I consider myself an expert. Here’s a guide on the art of scraping. ...

PyCon 2023 Talk

My PyCon talks are a way for me to learn. I usually pick topics I don’t know about. But at PyCon India 2023 the organizers picked “Programming Minecraft with Python” - a talk I’d given before. So, I started exploring ways to game it. (I like gaming things. It’s boring otherwise. Once, Infosys had me write a 400-page document. I began each page with a letter that spells out a poem.) ...

Webinar Impact of LLMs in Pharma

Ashwini Mathur and I are conducting a webinar on the impact of LLMs in Pharma. It’s online at 10 am Eastern on Mon Sep 11. Simon Willison described LLMs as alien technology we’re still discovering. I couldn’t agree more - and it helps to see it from different perspectives. So, we’re pairing the tech research at Gramener with the domain research Ashwini Mathur is doing at Novartis to explore the good, the bad, and the surprising uses of generative AI. ...

Always use value= for dynamic HTML options

Even after 30 years of HTML, I learn new things about it. This Monday morning, I woke up to a mail from Sundeep saying requests for a Data Engineer - AWS/Azure/GCP in our internal fulfilment portal raised an error. My guess was one of these: The “/” in the role is causing a problem. (Developer mistake.) The role exists in one table but not the other. (Recruitment team mistake.) The application wasn’t set up / restarted properly. (IT mistake.) All three were wrong. So I dug deeper. ...