In 2023, I made 3 resolutions: Run 50 experiments. I managed 44 / 50. (Here are some). Learnings: I need to improve planning (9), scepticism (6), and lateral thinking (4). Make 1 change a month in my environment. I managed 8 / 12. The largest impact was from meeting new people, working out of new places, and using new gadgets. Calendar integrity, i.e. stick to my calendar. I succeeded over 95% of the time. My most memorable events in 2023 were: ...
This week, I learned: Quantum computing is slow, has low transfer bandwidths, and only prime factorization has an exponentially faster algorithm. via The hidden brain podcast. What would Socrates do? Also Philosophy Bites Podcast: why do philosophers use example. And: the happiness lab: happiness lessons of the ancients How many of our beliefs are truly our own? How many are a product of our environment? Contrast these and identify your true beliefs For every thought and action you have, even tiny ones, ask “Why am I doing that?” Dig deeper because it may not be intrinsic One way to become memorable is to.write stuff others will reproduce for a long time. Plato and Aristotle did that everyone has multiple personality. This is partly because different parts of the brain evolved independently for different functions. System one and system two thinking are just such one broad classification. e.g. We think our train is moving when the nearby train moves because our visual brain is faster than our somatic brain. Good lessons and pitches cater to the rational AND the subconscious. Reason AND story. To activate different parts of the brain. That’s why philosophers use examples Philosophy brings change through reason. Revelations: through sudden insight. Rhetoric: through insight. Act as if you already are what you want to become. Aristotle Align your environment (including habits) to your beliefs. It will become easier to act your beliefs then. All virtues are moderation. It’s possible to take every virtue to the wrong extreme Some Christians have wristband that reads WWJD. What would Jesus do? Explore yourself a reminder of what would X do. Maybe Benjamin Franklin, Socrates, Feynman, etc People mistake their environment for their feelings. 1970s Experiment: People on a shaky bridge think they love each other. Experiment: people rationalize things irrespective of reality. “The Unexamined Life” is about questioning theories or stories or maps constantly. It’s also about questioning our thoughts and emotions constantly. Mindfulness is the VERBAL way of doing this. Meditation is the NON-VERBAL way of paying attention. Both are Processes to remove distraction and increase authenticity. Learning about people is a good way to learn about ourselves. And vice versa. Lica has a fascinating demo of how a document can be converted into a video story. Spillnot doesn’t spill drinks even when you swing! Things super-intelligences could do that humans can’t: Solving complex mathematical problems Advanced scientific discovery (quantum computing, nanotechnology, biotechnology) Ultra-precise predictive modeling in complex systems (climate, economics, social dynamics) Optimizing global systems at high precision (logistics, traffic, energy distribution, resource allocation) Universal translation (unknown languages, animal communication, extraterrestrial signals) Deep medical personalization: individualized medical treatments from genetics, environment, and lifestyle Create new materials: Designing materials or chemicals with specific properties Complex system integration: combining AI, bio tech, nano tech in new ways Philosophical insights: new perspectives or solutions to age-old philosophical dilemmas Space exploration and colonization Predicting natural disasters Customized education at scale Ways of working with them Collaborative problem solving Creative collaboration Decision support Personalized education Establishing ethical and safety protocols Recreational and leisure activities Mini-GPTs is an interesting approach to shrink LLMs and make them domain specific. It takes existing LLMs and removes neurons not used in a specific domain (e.g. law, medicine, etc.) Book to read (again) about how to take a team beyond their abilities even if you’re not the expert “Measure What Matters” by John Doerr “High Output Management” by Andy Grove “The Checklist Manifesto” by Atul Gawande “The Lean Startup” by Eric Ries “Creativity, Inc” by Ed Catmull “The Hard Thing About Hard Things” by Ben Horowitz “The Four Disciplines of Execution” by Chris McChesney, Sean Covey, and Jim Huling
From The Extended Mind I learnt that our environment shapes our thinking more than I’d expected. That we can arrange our environment to extend our thoughts. In 2023, each month I changed something in my environment to see: What does “changing my environment involve”? What can I change? Will I succeed? Does it affect my thoughts? Can I track this? Here are the results. ...
This week, I learned: DPO is a simpler alternative to RLHF for fine-tuning. Several HuggingFace models use DPO for training Name2Vec is a potential embedding for names. Google Knowledge Graph ID powers the Knowledge Graph. If it begins with /m/ it’s the same as the FreeBase ID. This is now available as WikiData. e.g https://www.wikidata.org/wiki/Property:P2671 I tried running Mixtral-8x7b locally (via Llamafile) and on together.ai. It’s good, but far from GPT 4. Generic computate-intensive algorithms eventually beat domain-specific tuning, because of Moore’s law. Ref The hidden brain podcast. the mystery of beauty Evolution drove us to beauty as an efficient survival mechanism. Understanding the world is one such mechanism. Hence we enjoy maths and chess ⭐ This leaderboard included paid models like GPT4 and Claude and compared them with open models on HUMAN + system benchmarks Lez Friedman Podcast: Jeff Bezos Build stuff that is is ubiquitous that other people take it for granted. The initial idea needs to be that obvious and easy. Like one click purchase or customer reviews Build stuff that other people can build on. Internet makes startups possible. Infrastructure is about enabling others at scale Decision making approaches: single person decides on two way doors. Deliberate as a team on one way doors Conflict resolution: disagree and COMMIT. NO sniping, I told you so, malicious compliance. Avoid compromise. Avoid decision by attrition (most persistent wins). People are inherently biased towards hierarchy. So the senior most person should speak last We have a happiness bias. Contracted by choosing the unhappier options first The map is not the territory. The metric is not the objective. We need metrics. But make sure you know why See the world through the eyes of the customer. Use your own product. It’s living their lives that makes customer obsession real. Jeff Bezos called their own customer care to see how long the actual wait time was. It was much longer than the metric reported How to prioritize. whatever problems customers will still face in 10 years are the big problems. These are worth putting time into because they are stable in time People working on big problems will never get down to the small problems. So have a dedicated team that works only on the paper cuts. It should be a dedicated team We co evolve with our tools. We build tools and then our tools change us. It reprograms our brains Cut out 10 minutes to the beginning of each meeting for people to read the material. They never reread anyway. This makes the meetings more productive Powerpoint is designed for persuasion, not truth seeking. It is also easier for the author than for the reader. Prefer narratives that are focused on finding the truth and are easier for the audience though tougher for the author ⭐ whisper-standalone-win provides a Windows binary for Faster-Whisper. It just needs CUDA and cuDNN installed. Then whisper-faster.exe video.mkv --language=English --model=medium generates the transcript. LLM use cases by Benedict Evans “Every text box on the internet will get an LLM” “Infinite interns” “Every UNIX function has become a company.” “Every ChatGPT suggestion…” llm360 publishes models along with training datasets. In The Age of AI has begun, Mar 2023, Bill Gates says, “In my lifetime, I’ve seen two demonstrations of technology that struck me as revolutionary.” The GUI (1980) and ChatGPT (2022). Rubeus is a HTTP proxy for multiple LLMs with load-balancing, fallbacks and retries. GPTRouter is a Python interface for multiple LLMs with fallbacks and retries. ⭐ Token Tally has an LLM Cost Tool that estimates GPU memory required and token cost across cloud providers.
This week, I learned: Grab. Improving last mile delivery in maps. When did people pick up the phone, when should driver be allocated to minimize waiting time, layer on top of OSM. Singapore developers the Sea Lion 7b model Try VLLM with AWQ format. Can do batch inferencing. Needs a good GPU Amex prediction whether they can pay back in 1 year or 18 months. That choice is a business decision. In real time. Precompute individual score and use it as input to another model. Model must be explainable by regulation. Creates decision tree models therefore. Compliance team must agree if I can use a feature. Can’t use gender. Age (in US, Canada);- high age is more risk. Can’t use edu level in the US. Capture information from camera and use LLMs. Like traffic cameras mapping. Explore GIS from video cameras Grab tracks road closures and road accidents and whether a cycle can go on a road vs a bike vs a car All drivers have a front facing camera Drivers report road accidents by pressing a button Amex prices individual loans when selling to a collection agency #TODO buy a bike head camera! Playwright is a browser-based test framework. Supports recording. OpenAI provides logprobs for tokens! This can be a used to create cool visualizations of the likelihood of the each tokens. Github Copilot’s new features makes your entire workspace or a specific file its context. It also auto-writes your commit messages and PR descriptions. Mixtral-8x7b-Instruct “… really does seem to be equivalent in quality to ChatGPT 3.5.” Ref Practical AI podcast Advent of Gen AI is going on. Explore add to tools in data science course. Model validation write a book as an open source to github repository. Easier to evolve and easier to get feedback on.. Explore utterances as a GitHub commenting platform automatically give credits to contributors who have center pull request that was accepted or an issue that was fixed. This encourages contribution Visit book.premai.io ast-grep is a semgrep alternative that focuses on code refactoring rather than security. Comby is another such tool Serply is a Google Search API alternative to Google CSE ⭐ Generate textbooks! ChatGPT is good at generating questions or training datasets. It genuinely creates them rather than replicating from memory. Ref v0.dev creates web pages from code. Example. LIDA from Microsoft is an LLM based data visualization tool.
This week, I learned: Bard supports extensions that include @Gmail – i.e. converse with your email. llama-cpp-python works with other GGUF models like Mistral and allows constrained output - JSON, function calling, etc. Ref 12 Tuning Strategies for RAG Llama Datasets are RAG datasets created mostly using GPT-4. Mostly small datasets. ⭐ Intuitions about large language models Bigger models (70b) are much better at learning from few-shot examples. They really learn. Bigger models will keep getting better! Chain of Thought prompting is a way of providing more compute to complex problems that require more compute Models will show emergent (completely new) behaviors that can’t be predicted from extrapolation. These may not be intentional. CodeAnt.ai is a VS Code plugin to detect code smells, refactor for modularity, to write docstrings and unit tests Anyscale prices the 7b Llama2, Zephyr, Mistral models at 15 cents per 1M tokens. Roughly 1/10th of GPT-3.5 Turbo’s ~$1.5 per 1M tokens Tools to identify personally identifiable information: galactic can use LLMs to detect PII Presidio by Microsoft Sherlock is a generic sematic type matching DL model pii-extractor-llm was trained on Indian names GLiNER is a Lightweight Generalist model for NER Tools to explore ElevenLabs speaks in your voice Cutout Pro removes backgrounds and parts of images Vocal Remover removes vocals from songs CapCut video editor TheBloke’s $35/month Patreon might be one of the least expensive ways to set up quantized LLMs in production. Microsoft released table-transformer to extract tables from PDFs. Sample usage Convert PDF to markdown with marker - an improvement over nougat. JupyterLab has a %%ai magic to use LLMs within notebooks. Ref Telling ChatGPT that the year is 2123 makes it bypass copyright. Ref Meta released SeamlessExpressive which preserves emotions in speech-to-speech translations Unsloth offers faster lower-memory LLM QLoRA finetuning DeepSeek is an open-source high-quality LLM Scalable Extraction of Training Data from (Production) Language Models extracts training data by repeating a token infinitely. SkyPilot lets you run LLMs on any cloud provider. vLLM lets you deploy LLMs with a single command. llamafile lets you run LLMs locally as a single file executable!
This week, I learned: Gwern Branwen says LLMs nudge his “… making heavier use of the languages I don’t know well (Emacs Lisp & Python) since I increasingly trust that an LLM can help me maintain them.” Undetectable.ai checks for AI content. But it had false positives AND negatives in the 5 checks I ran. GPTZero got 2/2 right and seems better at detecting AI content. CoVA scrapes web pages via OCR When coding with LLMs, have SHORT, RELIABLE feedback loops. Ref
I speak with ChatGPT ~20 times a day. That’s more than I speak with most of my colleagues. ChatGPT is clearly my favorite team member. I conduct trainings, reviews and mentoring sessions with my colleagues. How to write code. How to write slides. How to communicate. That last bit is particularly important. With ChatGPT Custom Instructions, I can guide ChatGPT on how to work better with me. Currently, I have 10 custom instructions. They evolved over time and will continue to evolve. ...