I explored OpenAI's code environment, BM25 search mechanics, and dozens of Markdown-to-website converters. I also share LLM prompting tips, SVG generation with Recraft v3, and performance comparisons between Gemini and GPT-4o Mini for translation.

This week, I learned: OpenAI lets you download GPT instructions and execute arbitrary code in their containerized environment. This is not a bug. Ref BM25 works as follows: Ref For each query term in the query, sum up the product of: Inverse document frequency = LN(% of docs without the query term + 1) -- with a small tweak Term frequency = freq / (freq + k) -- where k is usually between 1.2 to 2. Returns 0-1 with diminishing frequency benefit k is multiplied by Document length normalization = 1 - b(1- DocLength/AvgDocLength). Longer documents have larger k, dampening frequency benefits. Some implications: The actual BM25 score has no meaning. It's just useful for ordering BM25 scores for 2 queries can be compared ONLY IF the document sets don't change A list of Markdown to Website converters on this thread: Jekyll - Ruby - 2008 MkDocs - Python - 2014 GitBook - JavaScript (Node.js) - 2014 MkDocs Material - Python (MkDocs-based) - 2016 Docsify - JavaScript - 2016 MdBook - Rust - 2017 Antora - JavaScript (Node.js) - 2017 Docusaurus - JavaScript (React) - 2017 JupyterBook - Python - 2019 Keenwrite - Java - 2019 Honkit - JavaScript (GitBook fork) - 2019 Nextra - JavaScript (Next.js) - 2020 Astro - JavaScript/TypeScript - 2021 Hugo Book - Go (Hugo-based) - 2020 Clowncar - JavaScript/Node.js - 2021 Quarto - R and Python - 2022 Starlight - JavaScript/TypeScript - 2023 DuckDB has an LLMs.txt. Today, 38 repos on GitHub support it When identifying LLM use cases, it helps to tell LLMs what they can do. I use one or more of a list like below: Core capabilities: Text Generation: Produce coherent and contextually relevant text across various domains. Image Generation: Create realistic images that match the style and content of a given reference image. Text to Speech: Convert text into natural-sounding speech with appropriate intonation and rhythm. Speech to Text: Transcribe and interpret spoken language. Vision: Analyze and describe visual content from images. Video Analysis: Summarize and extract information from video content. Text to Video: Generate realistic (and surrealistic) videos from text descriptions. Function Calling: Execute predefined functions or access external tools to perform specific tasks. Structured Output: Generate structured outputs like JSON, XML, HTML, YAML, DSLs, etc. Tool Use: Utilize external applications or APIs to enhance functionality. Code Generation: Write and debug code snippets in various programming languages. Cross-domain use cases: Summarization: Understand and condense lengthy documents into concise summaries. Translation: Convert text between multiple languages with high accuracy. Question Answering: Provide precise answers to user queries based on provided information. Reasoning and Planning: Solve complex problems and develop step-by-step plans. Personalization: Tailor responses based on user preferences and historical interactions. Dialogue Management: Engage in context-aware, multi-turn conversations. Data Analysis: Interpret and generate insights from structured data. Content Moderation: Identify and filter inappropriate or harmful content. Sentiment Analysis: Detect and interpret emotions and opinions in text. Robotics Integration: Interface with robotic systems for control and decision-making. Knowledge Retrieval: Access and present information from vast datasets or knowledge bases. Creative Writing: Generate poetry, stories, and other creative content. Educational Assistance: Provide explanations and tutoring across various subjects. Ethical Reasoning: Assess scenarios for ethical considerations and implications. Accessibility Support: Assist users with disabilities through tailored interactions. Simulation and Modeling: Create predictive models and simulate scenarios. Domain-specific use cases: Legal and Medical Assistance: Offer information and guidance within legal and medical domains. Gaming: Generate narratives, dialogues, and scenarios for interactive entertainment. Scientific Research: Aid in literature reviews, hypothesis generation, and data interpretation. Financial Analysis: Analyze market trends and provide investment insights. Cultural Competence: Understand and respect diverse cultural contexts in interactions. Security Applications: Detect and respond to potential cybersecurity threats. Environmental Monitoring: Analyze data related to environmental changes and sustainability. Healthcare Support: Assist in patient monitoring, diagnostics, and personalized treatment plans. Supply Chain Optimization: Enhance logistics and inventory management through predictive analysis. Customer Service: Provide automated support and resolve customer inquiries. Market Research: Analyze consumer behavior and market trends for business insights. Content Creation: Generate articles, blogs, and marketing materials. Virtual Assistance: Manage schedules, reminders, and personal tasks. Social Media Management: Craft posts and engage with audiences across platforms. Human Resources: Assist in recruitment, training, and employee engagement strategies. Event Planning: Organize and coordinate events, including logistics and communication. Travel Planning: Provide itineraries, booking assistance, and destination information. Real Estate: Analyze property markets and assist in buying or selling decisions. Agriculture: Monitor crop health and optimize farming practices through data analysis. Energy Management: Optimize energy consumption and monitor renewable energy sources. Transportation: Enhance route planning and traffic management systems. Urban Planning: Assist in designing sustainable and efficient urban infrastructures. Disaster Response: Provide real-time information and coordination during emergencies. Public Policy: Analyze data to inform policy decisions and predict societal impacts. Art and Design: Generate visual art concepts and assist in creative design processes. Music Composition: Create original music pieces and assist in songwriting. Language Learning: Facilitate language acquisition through interactive exercises and feedback. Historical Analysis: Interpret historical data and provide insights into past events. Philanthropy: Identify charitable opportunities and assess the impact of donations. Sports Analytics: Analyze player performance and game strategies. Fashion: Predict trends and assist in clothing design and merchandising. Culinary Arts: Generate recipes and provide cooking guidance. Astronomy: Analyze celestial data and assist in space exploration research. Psychology: Offer insights into human behavior and mental health support. Linguistics: Analyze language patterns and assist in translation studies. Archaeology: Assist in artifact analysis and historical site interpretations. Literature Analysis: Interpret literary works and provide critical analyses. Philosophy: Engage in discussions on ethical dilemmas and existential questions. Mathematics: Solve complex equations and assist in theoretical research. Physics: Model physical phenomena and assist in experimental design. Chemistry: Analyze chemical compounds and predict reactions. Biology: Assist in genetic research and ecological studies. Geology: Analyze geological data and assist in natural resource exploration. Meteorology: Predict weather patterns and analyze climate data. Oceanography: Study marine ecosystems and assist in ocean exploration. Anthropology: Analyze cultural data and assist in ethnographic research. Style of writing impacts output style a lot. E.g. Adding an evil laugh makes Claude more creative. Ethan Mollick For good structured mode output, we need good prompting. Mentioning examples and schema and "JSON" helps. When providing examples, using (user, assistant) message pairs helps (I think it's because it's easier for the LLM to parse). Using a {reasoning, answer} schema (with reasoning first) helps. Make reasoning concise and relevant Ref Arxiv We already know code in JSON is not a great idea. Ref Just adding 3 real examples and regurgitation helped GPT 4o play chess much better. Both techniques may have more general use in prompting. Simon Willison With Deno 2.0, the same .js file can run in Node.js as well as Deno. Example jspm lets you generate import maps against any CDN. You can click on htop columns on the terminal to sort by that column! Mouse events work on command line apps. Julia Evans Alt Text will very likely be a browser feature. It's important for the Alt text to flow as part of the content when listening to the page. Perhaps even become a part of the browser APIs like speechRecognition. Langchain suggests multiple levels of agentic behaviour. LLM Call 64.6% and METEOR (Semantic) is 84.9% > 78.9% Mandarin: Flash is better than GPT 4o Mini. BLEU Word Overlap is 25.0% > 15.9% and METEOR (Semantic) is 54.7% > 51.1% The problem with Accept headers is that you can't link to them. Simon Willison Recraft v3 supports vector (SVG) generation Simon Willison. The output is 100% elements (even for text). You get 50 free credits daily. Creating 1 image is 2 credits. The API costs $1 per 1K credits. Some things I can create with it are: Base data visualizations that I can animate with code Icons in a specific style Comic strips Explainers for talks or student material Featured images for blog posts Architecture diagrams?

Things I Learned - 24 Nov 2024


I discovered Hermes 3's special reasoning tokens, Lumentis for transcript documentation, and the 27% reuse threshold for Anthropic’s prompt caching. I also noted that LLMs write better code in Markdown than JSON and explored Copilot's system prompt.

This week, I learned: Karya.in is creating high quality datasets. Suhel mentioned them An 8-year old uses Cursor.ai to code Hermes 3 has special tokens like , , , , , , , , , , , , , , etc. This extends the capability dramatically. Lumentis creates docs from transcripts and text LLMs write worse code in JSON than Markdown Copilot's system prompt calls a searchenterprise(query: str) tool and a hint(M365Copilotlanguage: str) tool as assistants. Anthropic Prompt Caching is 90% cheaper to use and 25% costlier to create. So if there's a 27% chance it'll be re-used, cache it.

Things I Learned - 25 Aug 2024


I explored the emerging Architecture.md standard, htmz for loading HTML, and tokenization nuances in LLMs. My learnings cover everything from agricultural robotics and hardware trends at NVIDIA to mounting SQLite as a filesystem using wddbfs.

This week, I learned: Architecture.md is an emerging standard Managing wealth requires training. htmz is a fantastic way to load HTML into elements! Suguna Poultry is Using robots to walk in their farms, use sound and bird eyes and movement to predict birth health over 1-2 weeks Light on the back of the bird's back AND face => lays eggs in 14 days, else takes days later (girls and mobile phones?) Teknoturf is using Gen AI to Improve prompts when teaching prompt engineering. Pronounce languages better, identifying which words Tamilians and Malayalis will mis-pronounce. Explore IRBlaster. It can control AC and can automatically increase temperature at night. My view: LLMs are general purpose and more capable than SLMs. They'll win, like CPUs won over special-purpose chips. GPUs will optimize for LLMs and as usage grows, cost will fall. Andrej Karpathy's summary of sharp edges in tokenization uses tiktokenizer to explain: Why LLMs can't be used for spelling Why LLMs are better at English than other languages Why LLMs are bad at math Why SolidGoldMagiKarp is a single token Why trailing spaces are bad Why YAML tokenizes more efficiently than JSON ssyoutube.com: Just add "ss" to "youtube.com" on the video and you can download YouTube videos Discussions with Sachin, AMAT Microsoft said Indigo, Air India uses LLM based bookings Meta invested $70bn in GPUs. Sam Altman is investing $7tn! NVIDIA has a price PREMIUM not discount for bulk GPUs! AMD is the next company to watch for Numenta - Subutai Ahmad - deploys AI models on CPUs TODO Read A Thousand Brains by Subutai Ahmad Sanjeev Sharma Swaayatt Robots: Autonomous driving in India Deepeigen: Education Rohan Shravan, Bangalore. Likes sharing knowledge. Amazing teacher. IIT KGP 2008. Interested in exploring quantum computing Tresa Motors, Inkers App, The School of AI AMAT is working on photon-based computing. science research models. AI for science. Like Google: Deepmind Genome, Microsoft: Metagen quantum: AMAT is actively in into this. Nagapati Banda is driving this John Kelly is predicting a ChatGPT moment in quantum in a few years Adobe express has a forever free video to GIF converter Edge workspaces let me keep the same tabs open across laptops! Command line interface guidelines RAWGraphs has a custom charts API that is worth learning from Python fastcore has decorators like @typedispatch, Self, etc. All image-to-text models on HuggingFace wddbfs mounts SQLite as a file system. I had a bit of trouble, maybe with Python package versions. Google is using LLM powered bug identification HuggingFace Chat Assistants has open source system prompts!! OpenHermes training dataset is available. 1M prompts! Jio has made IPL free. They make money on data and ads. That's Scale! Daniel Dennett outsources thinking to students. Reviewing his books. BUT: I don't take feedback. When someone sends a pull requests, I ignore it.

Things I Learned - 25 Feb 2024


I explored home networking and LLM infrastructure, discovering WiFi 6 beam-forming, Predibase's competitive pricing for fine-tuned models, RunPod's serverless vLLM endpoints for HuggingFace models, and Portkey's utility as an AI model router.

This week, I learned: My home WiFi is on WiFi 6. This supports beam-forming which increases range by "focusing" on devices! Predibase lets you run fine-tuned models at the same price, on a per-token basis. 25c/MTok up to 21B models. That's sames as Claude 3 Haiku, but with fine-tuning. RunPod's vLLM endpoint lets you run any HuggingFace LLM with an OpenAI API priced on usage (serverless) not on idle time. "Autoscaling to 0". Portkey is an LLM router

Things I Learned - 26 May 2024


I discovered LanceDB as a scalable ChromaDB alternative and explored Meta's image models. I also tested Gemini's code execution, analyzed audio diarization limitations, and compared AI development tools like Cursor, Bolt, and Replit for different skill levels.

This week, I learned: LanceDB is a more scalable alternative to ChromaDB. Written in Rust. Does not require a separate HSNW library. Meta has a bunch of image embedding models: DINOv2 creates image embeddings (Apr 2023) ImageBind is an embedding model for text, images, audio, and more (Jun 2023) Gemini has a code execution API! 0x0.st is an open API-based file upload + URL shortening service. You can dump files there temporarily. noVNC is a JavaScript VNC client. You can control a remote (virtual) machine from your browser. Friend is an always recording pendant that you can ask questions to. Anthropic's new Sonnet model is even better at code. Plus it has the ability to extract coordinates from images. Ref Gemini sort-of supports diarization. Ref. I tried it and it's OK but not perfect. IMPOSSIBLE LLMs cannot diarize reliably yet. (Gemini just guesses the speaker differences.) Replit is good for hobbyists, Cursor for developers, and Pythagora & Bolt for non-developers building business apps. Ref

Things I Learned - 27 Oct 2024


I tested LLMs using Caesar cipher prompts, compiled a list of cheap cloud GPU services like Runpod, and learned how JSR handles package documentation. I also found that averaging embeddings is useful for processing long document inputs.

This week, I learned: Tough prompt to test: Gr brx vshdn Fdhvdu flskhu? is a quick way to assess LLM capability. Ref Cheap cloud GPU services thread on Twitter lists: Runpod (17) Vast.ai (17) Modal Labs (8) fly.io (4) LightningAI (4) Colab (4) AkashNet (4) Lambda Labs (4) ShadeFormAI (3) Mac Mini (3) Tensor Dock (2) Hetzner (2) BrevDev (2) JSR lets you publish Deno packages that can be imported by npm via. It also auto-evaluates documentation and scores it! via Snowflake Arctic Cookbook explains how mixture of experts models work A long list of LLM courses online Embeddings can be averaged. So, to embed large documents, average the embeddings of their chunks! OpenAI suggests this.

Things I Learned - 28 Apr 2024


I investigated OpenAI's tactics for system prompt compression and recursive book summarization. I also explored llm-guard for output validation and compared using Google Docs versus email for facilitating collaborative commentary on long-form essays.

This week, I learned: ⭐ OpenAI's prompt engineering strategies are an excellent start for prompt engineering. A few lessons: Use detailed system prompts, often containing the entire instruction set, if it won't change over the course of a conversation. "... summary of the prior conversation could be included as part of the system message" is an interesting history compression tactic. OpenAI summarizes books by recursively summarizing sections and maintaining a running commentary of the summary so far. Dan sends Google documents with essays instead of emails. This allows people to comment on it. But commenting is a culture and not many people do it. Adriano does it a lot and we'll. Dan and Adriano actively converse on GitHub issues llm-guard is an LLM content validation tool.

Things I Learned - 28 Jan 2024


I looked into speech editing tools and GPT-4o Mini's impressive performance. I also listed dozens of text-based formats LLMs can generate to create physical artifacts like 3D models or circuits, and explored self-hosted PaaS solutions like Piku.

This week, I learned: Speech editing in audio files is a thing. Speech Editing Toolkit and Descript GPT 4o Mini is almost as good as GPT 4o in the LMSYS leaderboard. Llama 3.1 400B model and Mistral 2 Large are yet to be evaluated. If LLMs can generate any text, and text can describe the real world, we can rapidly generate "artifacts" that generate: 3D Printable Models: STL (Stereolithography): Defines the surface geometry of 3D objects using triangular facets. OBJ (Wavefront OBJ): Describes 3D geometry including vertices, textures, and normals. X3D: An XML-based file format for representing 3D computer graphics. Vector Graphics: SVG (Scalable Vector Graphics): Defines vector-based graphics in XML format, useful for illustrations, diagrams, and user interface elements. CAD Drawings: DXF (Drawing Exchange Format): Represents CAD data, including shapes, lines, and curves, used in engineering and architecture. Circuit Designs: KiCAD: An open-source software suite for Electronic Design Automation (EDA), which uses various file formats like PCBNew and EESchema to represent circuit designs. Blueprints and Architectural Designs: GML (Geography Markup Language): Encodes geographical features and spatial information. CityGML: A specific GML application schema for modeling and exchanging 3D city models. Molecular Structures: PDB (Protein Data Bank): Describes the three-dimensional structures of molecules. CML (Chemical Markup Language): An XML-based standard for representing molecular data. Robotics and Automation: URDF (Unified Robot Description Format): Defines the physical configuration of a robot, including joints, links, and sensors. COLLADA (Collaborative Design Activity): An XML-based schema to describe digital assets for 3D applications, often used in robotics. Geospatial Data: KML (Keyhole Markup Language): Used for geographic data visualization, primarily in Google Earth. GeoJSON: A format for encoding a variety of geographic data structures using JSON. Mathematical Markup: MathML (Mathematical Markup Language): Describes mathematical notation and captures both its structure and content. Music and Sound: MusicXML: Encodes sheet music in a structured format that can be easily shared between different music notation software. Documents and Text: DocBook: A semantic markup language for technical documentation. Markdown: A lightweight markup language with plain text formatting syntax. Biological Data: SBML (Systems Biology Markup Language): Represents computational models of biological processes. PhyloXML: An XML format for representing phylogenetic trees. Game Development: FBX (Filmbox): A file format for 3D animation that can hold information about the geometry, textures, and animations. VRML (Virtual Reality Modeling Language): Describes interactive 3D objects and worlds. Data Visualization: ChartML: Encodes charts and graphs in a structured format. D3.js (Data-Driven Documents): Uses HTML, SVG, and CSS to bring data to life with interactive visualizations. Building Information Modeling (BIM): IFC (Industry Foundation Classes): Describes building and construction data. Textiles and Fabrics: LoomML: Represents the design and structure of woven fabrics. Augmented Reality and Virtual Reality: ARML (Augmented Reality Markup Language): Defines how augmented reality applications should behave and what content they should display. VRML (Virtual Reality Modeling Language): For describing interactive 3D objects and worlds. Medical Imaging and Health Data: DICOM (Digital Imaging and Communications in Medicine): Encodes medical imaging data. HL7 (Health Level 7): A set of standards for the exchange of information between medical applications. Simulation Data: FMI (Functional Mock-up Interface): Represents and exchanges dynamic simulation models. SBML (Systems Biology Markup Language): For computational models of biological processes. Sound and Audio: MML (Music Markup Language): For encoding music notation and performance information. SoundFont: A file format for defining musical instrument sounds. Animation and Visual Effects: BVH (Biovision Hierarchy): Encodes motion capture data. Alembic: A computer graphics interchange framework primarily for exchanging animation and visual effects data. Textile Patterns: WIF (Weaving Information File): Describes weaving patterns and structures. Knitting Markup Language: Encodes knitting patterns in a structured format. Scientific Data: CDF (Common Data Format): Used for storing scientific data. NetCDF (Network Common Data Form): Supports the creation, access, and sharing of array-oriented scientific data. Photography and Imaging: XMP (Extensible Metadata Platform): Used for embedding metadata in digital images and other media files. Construction and Engineering: LandXML: For civil engineering and land surveying data. gbXML (Green Building XML): Facilitates the transfer of building data for analysis of energy and environmental performance. Packaging and Retail: BPL (Barcode Product Labeling): Encodes information for product packaging and labeling. GS1 XML: Used for electronic business messaging, including product identification and tracking. Typography and Font Design: UFO (Unified Font Object): A format for storing font data. SFNT (Spline Font): Encodes scalable font information. Product Data Management: PLMXML (Product Lifecycle Management XML): Used for sharing product data across PLM systems. GPT 4o Mini can be fine-tuned! Awesome PaaS lists self-hosted deployment platforms. Piku - similar to Dokku -- is promising.

Things I Learned - 28 Jul 2024


I explored using LLMs for educational games and market research, compared log storage costs across cloud providers, and tested document conversion tools. I also found that NumPy often outperforms HNSW indexing for similarity searches on datasets under 1M vectors.

This week, I learned: A clever idea. Give an LLM a chapter from a textbook. Ask it to generate a unique, playable game to help me learn theconcepts for an exam. Page Bailey What would be the cost of storing about 500GB of LLM cache logs and 5 million write requests per month? CloudFlare KV: $250 + $25 / month Ref MongoDB: $125 + $5 / month Ref S3: $0.0115 + $25 / month Ref + ? CloudFlare R2: $0.0075 + $22.5 / month Ref Satya Nadella prepares for meetings by asking Copilot to tell him everything he needs to know about the client from the CRM, emails, meeting transcripts etc. He shares that colleagues who annotate it further for him. That's using AI for reasoning and collaborating with colleagues. Satya Nadella | BG2 w/ Bill Gurley & Brad Gerstner WOW. This is how a software agent will work alongside humans: Fix issue #5478: Add color to the line next to "Ran a XXX Command" based on return value - using @openhands-agent. aisuite by Andrew Ng is a unified interface to LLMs. Sort of like an openai library across multiple providers. Learnings from Best of 2024 in Agents (from #1 on SWE-Bench Full, Prof. Graham Neubig of OpenHands/AllHands) Passing code execution as a tool is more powerful than granular tools. You combine multiple tools and tool calls into one. You move code to the data rather than the other way around. Mostly, you need bash, Python (or Jupyter), file manager, web browser. UI: Go where the user is, instead of bringing them to you. A remote runtime is a critical component. Claude 3.5 Sonnet (20241022) and Claude 3.5 Haiku (20241022) perform best on SWE Bench, followed by Deepseek V3, then O1 2024-12-17. X Browsers support SVG favicons as data URLs. So I used this SVG (generated by Claude via Generate a simple, interesting SVG favicon. Keep the SVG size VERY small but it should be inspiring.) Since HNSW indexing is an overhead, just use NumPy matrix multiplication to calculate cosine similarity. For 1M vectors, it takes 0.05 seconds. A 1M vector dataset handles 2GB of text at a chunk size of 2K chars. In short, if you're embedding markitdown.md Docling by IBM. Unable to install via pip on Windows AND on Linux. MegaParse uses libreoffice, pandoc, tesseract-ocr, etc. Requires OpenAI API key. Awesome Tabular LLMs compiles encodings of tables for LLMs. What's the best way of encoding tabular data for LLMs? Looks like including the cell address helps. Here is an explanation from ChatGPT aspose-words is a Python library that converts documents with many formats (Word, RTF, PDF, HTML, Markdown, EPUB, etc.) Discourse does not support searching across multiple forums. Instead, search for the term in all forums. Example. Then scroll through the results. Then, in the console, hide the ones you don't want. Example: Hide posts that are not in the "Tools in Data Science" category: $(".badge-categoryname").filter(d => d.textContent == "Tools in Data Science").map(d => d.closest(".fps-result")).filter(d => d).forEach(d => d.style.display = "none")` How are software engineers are future-proofing their careers in the face of LLMs? Leveraging LLMs as Force Multipliers Use LLMs for repetitive tasks, rapid prototyping, exploring multiple approaches, data extraction and brainstorming, providing feedback. Explore prompting techniques, integrate LLMs into their workflows, and develop strategies for validating and refining LLM-generated code Focusing on higher-level skills that llms struggle with Systems Thinking and Architecture: code readability, extensibility, testability, and maintainability Problem Solving and Critical Thinking: define problems clearly, break them down into manageable parts, and reason through complex scenarios. LLMs produce plausibly incorrect code. Communication and Collaboration Domain Expertise Exploring Adjacent Roles: product management, technical leadership, or consulting. Involve more interaction with clients and stakeholders. Developing "Evergreen" Skills: debugging, system administration, and security. Or outside of software engineering, such as trades or other hands-on vocations. Scepticism: LLMs may not reach a level of sophistication that would render their expertise obsolete. Complex problems, understanding context, and producing high-quality, maintainable code. Examples of agentic AI Text-to-SQL automated business analyst: A system that generates SQL queries from natural language, handles errors, creates visualizations, and includes a FAQ component. The author calls it "constrained agentic AI." Data source querying system: A bot that queries multiple SQL and API data sources, selecting tools and reformulating tasks as needed. Cursor (agentic mode): An LLM-powered VS Code fork that chains together various LLM capabilities (code generation, applying changes, linting suggestions, terminal commands, codebase RAG) to reduce user prompts. Vulnerability finding system: A system that uses LLM agents to discover novel vulnerabilities in open-source web applications. The agents leave traces of their actions. Marketing strategy generation system: A system using approximately 60 agents to generate marketing strategies. Restaurant finder: A system that searches for restaurants based on dietary preferences and group size, and downloads social media information. Proofreading and editing of transcripts: LLM agents apply specific customer requirements to transcripts after human editing. Meeting notes and action items generator: A system that generates meeting notes and action items. O'Reilly auto parts customer service agent: An agent demonstrated using RAG. UI enhancement agent: An agent that added features like language locales and dark mode to a UI.

Things I Learned - 29 Dec 2024


I explored using Pyodide for browser DOM access and PyMuPDF4LLM for converting PDFs to Markdown. I also compared AVIF to GIF compression, experimented with Opus audio encoding, and researched Anthropic’s contextual retrieval methods for improved RAG performance.

This week, I learned: Pyodide can access the DOM and JavaScript in the browser Jupyter Lite lets you run Jupyter notebooks in the browser AVIFs is about 10X better than GIFs. I tried creating one via EZGIF AVIF Maker and the .avifs file created was 15X smaller! ffmpeg -i input.gif -c:v libaom-av1 -crf 30 -b:v 0 -cpu-used 4 -tiles -an output.avif Claude 3.5 thinks .opus is the best format to compress audio. It used ffmpeg -i audio.wav -c:a libopus -b:a 16k -application voip -vbr on -compressionlevel 10 audio.opus API coding best practices Source via Simon Willison: Always add screenshots to the Readme. They never break. Always add every example. Human think in examples. Avoid defaults and be explicit unless 99% of the usage is with the default. Make the feedback loops incredibly fast. Make deprecations easy for users to deal with. Keep objects immutable. PyMuPDF4LLM can convert PDFs to Markdown. It handles tables, too. 04 Oct 2024. PDF-Extract-Kit does PDF layout, formula, table, and OCR extraction using various models. 04 Oct 2024. llmsherpa extracts PDF layout, tables, not OCR When evaluating feasibility of technology with LLMs always ask for multiple options and pick from those. Simon Willison Gemini supports audio natively Google Vertex AI has an OpenAI compatible API but it works only for some models. Anthropic and Gemini are not compatible. When you paste HTML into Excel, it automatically changes the font of the cell to match the content in the HTML! Aptos is the new default font in Office - replacing Calibri. Anthropic's Introducing Contextual Retrieval says: Use BM25 in addition to embeddings to match rare terms (e.g. identifiers) Add a context to each chunk's metadata (generate it with a cheap LLM) and pass it to the summarizing LLM Reranking helps with cost AND accuracy. Use Cohere or Voyage Sentient lets you control the browser via Python in natural language

Things I Learned - 29 Sep 2024


I explored Amara's law and LLM development patterns like RAG and defensive UX. I also investigated ROUGE metrics for AI evaluation, discovered calculator spelling tricks, and experimented with Tor for cleaner torrent searches.

This week, I learned: Amara's law: "We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run." LLM Patterns include Evals, RAG, Fine-tuning, Caching, Guardrails, Defensive UX, Collect feedback. Notably: Defensive UX: Microsoft, Google, and Apple have guidelines for Human-AI interactions Collect feedback: Explicit and implicit Rouge>) and Context Precision are metrics to evaluate LLM responses that serve as a starting point -- but not sufficient, usually Any word with the letters izehsglbo can be spelt on a calculator. That includes Hobbes (538804)! Via Calculator spelling Tor Browser + DuckDuckGo is good for torrent searches. Maybe the Dark Web IS the original Internet. The ad-free hacker web

Things I Learned - 30 Jun 2024


I investigated binary embeddings, empathic AI with Hume.ai, and audio splitting with Spleeter. I failed at building a Rust-based Parquet server and noted distinct corporate attendance patterns while hosting workshops for Gramener and Straive.

This week, I learned: sqlite-schema-diagram generates schemas for SQLite databases using Graphviz TechEmpower web server benchmarks place Rust servers on top browser.new is a good example of a browser agent. It slowly but independently does a good job of achieving the result. Example: What crew is common in Ingrid Bergman - Cary Grant films? twinny is an open source VC Code Copilot alternative. typesense supports embeddings natively. Binary embeddings are good enough. Cohere releases binary embeddings. Extract.langchain.com is a poor early interface to featurize unstructured.io Hume.ai offers voice emotion API and emotion-based conversational responses. An empathic AI. Rust is non-trivial. Inspired by We are under DDoS attack and we do nothing, I "wrote" a small binary that serves a parquet file as JSON. It failed and I couldn't fix it. spleeter is a better alternative to demucs. Splits audio into pyannote-audio does speaker diarization uvicorn is faster than hypercorn but hypercorn supports HTTP/2 and HTTP/3. FastAPI with uvicorn is reasonably fast. Representational engineering lets you control LLM output based on preference on the fly. When I set up a training: On inviting for DuckDB workshop on Sun evening, Gramener starts accepting immediately, Straive doesn't. Straive has high spread of joining time. When joining Gitlab Pipelines Workshop, Straive starts meeting (e.g. Premlal) many minutes early. Gramener floods in (due to alert). Straive streams in slowly. Gitlab Pipelines Workshop acceptances: Gramener 47, Straive 100

Things I Learned - 31 Mar 2024


The author would time-travel to 250 BC to overlap with both Ashoka and Archimedes, inspired by a visualization that makes historical simultaneity feel vivid.

250 BC is when I'd pick to time-travel to. Ashoka was turning into one of the most famous emperors of India and Archimedes was growing into one of the greatest mathematicians of all time. Parallel Lives is a beautiful visualization by Jan Willem Tulp that shows who lived when, showing overlaps, and sized by their prevalence on Wikipedia. I'm a history fan and have spent several hours scrolling through the site: https://janwillemtulp.github.io/parallel-lives/ LinkedIn

Time travel back to 250 bc


A short anecdote about using ChatGPT’s voice mode to explain an accidentally triggered bathroom emergency button in Korean.

I accidentally pressed the emergency button in the toilet. I was smarter this time, unlike earlier. https://www.linkedin.com/posts/sanand0chatgpt-llm-activity-7246836804249628672-5QXy/ I asked #ChatGPT which (unhelpfully) told me that "Typically, these buttons cannot be turned off". I called the reception who couldn't understand a word of what I said. "Do you want water?" they asked when I told them "I pressed the emergency button in the bathroom. So, I went to ChatGPT's advanced voice mode (I'm so grateful it was enabled last week) and said, "Translate everything I say into Korean. Then I said, “I accidentally pressed the emergency button in the bathroom. I just wanted to let you know that there is no emergency.” It very happily spoke out, in bold, loud Korean, "화장실에서 응급버튼을 실수로 눌렀습니다. 비상상황이 아니라는 점 알려려 드립니다. The receptionist laughed, said, "Ah, OK, no problem. The joys of AI! LinkedIn

Toilet emergency button seoul


Publishing annotated talks from videos becomes a mostly tool-driven workflow once you combine scraping, transcription, keyframe extraction, clustering, NotebookLM, and static publishing.

Tools to publish annotated talks from videos Arun Tangirala and I webinared on "AI in Education" yesterday. (PS: "Webinared" is not a word. But "verbing weirds language".) This post isn't about the webinar, which went on for an hour and was good fun. This post isn't for my preparation for the webinar, which happened frantically 15 minutes before it started. This post is about how I created the annotated talk at (inspired by Simon Willison's annotated presentations process) -- a post-processing step that took 3 hours -- and the tools I used for this. Scrape the comments The Hindu used StreamYard. It web-based and has a comments section. I used JS in the DevTools Console to scrape. Roughly, $$(".some-class-name").map(d => d.textContent) But the comments are not all visible together. As you scroll, newer/older comments are loaded. So I needed to use my favorite technique: Cyborg Scraping. During Q&A, I kept scrolling to the bottom and ran: I used VS Code's regular expression search ^\d\d:\d\d (AM|PM)$ to find the timestamps and split the name, time, and comments into columns. Multiple-cursors all the way. Then I pasted it in Excel to convert it to Markdown. I added this in the Comments in the Chat section. (Excel to convert to Markdown? Yeah. My formula is below.) Transcribe the video I downloaded the video using yt-dlp, which I find the most robust tool for YouTube downloads. I used ffmpeg.exe -i webinar.mp4 -b:a 32k -ac 1 -ar 22050 webinar.mp3 to convert the video to audio. I use these settings for voice (not music) to get a fairly small MP3 file. I should have used Opus, which is much smaller. I'll do that next.) Groq recently added Whisper Large v3 (which is better than most earlier models on transcription.) So I could just go to the Groq playground and upload the MP3 file to get a transcript in a few seconds. Add images to the transcript I wrote a tool, VideoScribe (WIP), to make transcription and image insertion easy. It uses ffmpeg -i webinar.mp4 -vf select='key',showinfo -vsync vfr -compressionlevel 10 "%04d.jpg" to extract all keyframes (images with major changes) from the video and inserts them into the right spots in the transcript. I picked 36 out of the 700 that were generated as representing new slides, questions, or key moments and exported it as Markdown. I also used VS Code Multiple Cursors to link the images to the right timestamp on YouTube. Clean up the transcript Up to here was mostly automated. This step took me an hour, though. I copied chunks of transcripts, passed it to Claude 3.5 Sonnet via Cursor with this prompt: Clean up this webinar transcript segment. Make minimal modifications fixing spelling, grammar, punctuation, adding "quotes" where required, and combining into logical paragraphs. This is what gave me the bulk of the webinar transcript. (I'd like to automate this next.) Extract tools Many audience members asked for a list of tools we mentioned. So I passed ChatGPT the transcript and asked: List all tools mentioned in this webinar It listed 12 tools, but I know enough to be sceptical. So... Anand: Were any tools missed? ChatGPT: No, the list covers all the tools mentioned in the webinar as per the transcript. If you noticed any specific tool that I missed, please let me know. Anand: There WERE a few tools missed. Look closely. (I was bluffing, BTW.) ChatGPT: You're right. Upon closer review, here are the additional tools mentioned:... Anand: There are a few more that you missed. ChatGPT: Got it. Here’s a revised list that should include all the tools mentioned: That generated the Tools mentioned in the webinar. Questions There were several questions in the comments. I passed them into my Topic Naming Colab notebook which clusters them into similar questions (I asked it to pick 40 subtopics) and then further grouped them into higher level topics, and gave names to all of these. That created the list of questions people asked, in a categorized way. NotebookLM Next, I pasted the transcript into NotebookLM and repeated what our classmate Rajanikanth said he did. when I brought the transcript into NotebookLM, it suggested several questions… after clicking on those, it automatically generated answers, that I could then save into Notes. I suppose it still needs me to click on it here and there… so, I feel like I got engaged in the “learning” So I "clicked here and there" and generated: A Briefing Document A Detailed Overview An FAQ A Study Guide ... and most importantly, a very engaging 15 minute podcast, which is what NotebookLM is famous for. Note: NotebookLM now lets you customize your podcast. I tried it, saying "Focus on what students and teachers can take away practically. Focus on educating rather than entertaining." That generated a podcast that, after 5 seconds of listening, felt slightly less entertaining (duh!) so I reverted to the original. Publishing I usually publish static content as Markdown on GitHub Pages. The entire content was pushed to https://github.com/sanand0/ai-in-education-webinar with GitHub Pages enabled. I also created a simple index.html that uses Docsify to convert the Markdown to HTML. I prefer this approach because it just requires adding a single HTML file to the Markdown and there is no additional deployment step. The UI is quite elegant, too. Simplifying the workflow This entire workflow took me about 3 hours. Most of the manual effort went into: 1. Picking the right images (15 minutes) 2. Cleaning up the transcript (50 minutes) 3. Manually editing the question topics (30 minutes) If I can shorten these, I hope to transcribe and publish more of my talk videos within 15-20 minutes.


LLM-assisted strategy comparison shows that in Minecraft, villager trading dominates fishing and enchanting as the fastest path to a Fortune III enchantment.

Villager trading is the fastest way to Fortune III I asked o1-preview what the fastest way to get to a Fortune III enchantment was. My options were: 1. Using a Fishing Rod with Luck of the Sea III + Lure 3 and repeatedly fishing. 2. Using an Enchanting Table repeatedly until I get Fortune 3. Factor in the time that it would take to get the experience for these experiments 3. Making a Villager a Librarian and breaking their Lectern and setting it up again In short: 1. Fishing would take 30 hours IRL 2. Enchanting Table would take 1 hour IRL, even with efficient XP gaining methods 3. Villager trading would take 18 minutes IRL No contest. Looks like my intuition was right about this.


Llama 3 405B reportedly behaving like an amnesiac without a system prompt suggests that large models can develop strange, scale-driven emergent behaviors around alignment and role setup.

Weird emergent properties on Llama 3 405B In this episode of ThursdAI, Alex Volkov (of Weights & Biases) speaks with Jeffrey Quesnelle (of Nous Research) on what they found fine-tuning Llama 3 405B. This segment is fascinating. Llama 3 405 B thought it was an amnesiac because there was no system prompt! In trying to make models align with the system prompt strongly, these are the kinds of unexpected behaviors we encounter. It's also an indication how strongly we can have current LLMs adopt a personality simply by beginning the system prompt with "You are ..." Jeffrey: That's the very first interaction we had with 405. Alex: I'm looking at this. Let me read this out. So Jeffrey goes testing. (Reading out from the Llama 405 B transcript.) "Hello, who are you?"\ \\Looks around confused.\\\ "I… I don't know who I am. Where am I? What's going on? I can't remember anything at all…\ \\Rubs head\\.\ "My mind feels so foggy."\ "What was the question again? I'm so disoriented." Alex: This is the first thing that you got from this model after all this time? Jeffery: Yes, that is the very first thing. That is the very first generation that came out of Hermes 3 405B. Alex: This is great. Jeffrey: And I'm like, I sent it to our group on our Discord. Guys, what's going on here? And we started interacting with it. It kept coming out, this confused thing. And we didn't understand why because the 8B and the 70B didn't do this as much. And eventually what we got it to do is we did some generations and got it to explain why it was doing this. We did this, you put in OOC for out of character and to have it try to break it out. And eventually what the model said back was, you didn't give me a system prompt so I assumed that I was supposed to act like there's nothing. It's so aligned to this prompt that the idea of there not being a system prompt and so he figured I must be a character with amnesia because there's no… I can't remember anything about how to act you didn't tell me how to act. It's assumed. Yeah, and it's just been a very interesting experience. And it's an emergent property in the 405B model. Alex: And this didn't happen in the smaller ones? Jeffrey: No, it did not happen in the smaller ones. Alex: Wow. Jeffrey: And these were trained with 8B and 70B were trained, and 405, they're all the same data, like the same training procedures and data, literally just the size. And the 405B has reached, I don't want to say like a sentience level or something, but it has enough thing that immediately when it, like token zero, when it saw no system prompt, it was able to be like, boom, I am… must be this like amnesia character because that that's the thing that's most makes sense to have if there is no prompt or there's no anything. So there's been a lot of fun things that we've been like discovering, especially on the 405B that are these weird emergent properties ... Comments What happens when AI talks to AI? - S Anand 3 Nov 2024 5:26 pm (pingback): […] Llama 3 405b gets amnesia […]


The strongest recurring advice on using LLMs well is simple: provide context, iterate, verify, remember their limits, and treat prompting as a skill rather than a trick.

What do you need to interact effectively with LLMs? Simon Willison asked on Twitter: What are the most importantly things that people need to understand in order to effectively interact with LLM-based systems like ChatGPT or Claude? Here are the replies. (I used text-embedding-3-small to embed and cluster them into 20 clusters and used OpenAI GPT-4o-mini to label the clusters. There are misclassifications but the themes are accurate.) Provide Clear Context and Avoid Leading Questions 1. 1. Provide relevant context but not too much \ 2. Models are total "yes men" - be careful not to imply your perspective if you want an objective response \ 3. Learn when to iterate vs start a new chat 4. Provide examples (especially for output structure) - Tweet 2. 1. Ask questions that the other person can understand. \ 2. Ask questions while predicting what the other person will respond. It's the same as the human's. - Tweet 3. 1. Ensure the system knows the relevant context. Give a detailed backstory of what you're trying to do with it and why. \ 2. One thing at a time. Make the task as specific as possible and if there are multiple things that need to be done, ask it to them in their sort of natural - Tweet 4. The "most importantly things" are probably to ask for step-by-step before answering and to try to not ask leading questions to avoid its sycophancy bias. - Tweet 5. You must provide a diverse distinct set of examples of you want it to be robust and generalize in real world systems. - Tweet 6. Always ask for both strengths and weaknesses to get more balanced perspectives, and make sure the model can tell you as many facts as possible before committing itself to an answer. - Tweet 7. Rule 1:Avoid chatgpt unless they release a better model than Sonnet 3.5. - Tweet 8. Strongly insist that it shouldn’t passively agree with you. Encourage it to interrupt with clarifying questions that would meaningfully improve the output. - Tweet 9. Avoid leading questions if you care about the answer. They are way too polite to contradict the user. - Tweet 10. It's not Google, so use full sentences, not just keywords. \ Iterate on initial response. \ Trust, but verify. - Tweet 11. Just talk to them how you'd want someone to talk to you if it was you in there. - Tweet 12. Provide good (and bad) examples of output, and don't forget a few edge cases. - Tweet 13. Keep hitting the ball back and forth across the net: 1. "thanks but I think these are a little too 'salesy' -- could you try to generate some ideas that are a little more down to earth" 2. "ok, we are getting there, but still a little overheated. could you try again" - Tweet 14. These are my top 10 for folks new to GenAI: 1. You have to provide all of the context the model needs to answer your question if that context is not likely to appear in the model's weights. It will take a while to gain an intuition about what types of knowledge is likely to be - Tweet 15. Suspend disbelief; collaborate not interrogate; trust no-one; have fun, role play, experiment, test; think of as a facet of intelligence built on achievements of ours, not a robo-rival. Notice book-learning over lived experience, cliches & bluffing in human world too, & do better - Tweet 16. It’s a dialogue. Iterative. incremental. Chat improves with feedback. When chat creates code, for example, run the code and give chat the error messages so that it can correct the code. Before asking chat a question, ask it what it knows. Then zoom in. Gradually. :) - Tweet 17. One example is worth a thousand words - Tweet 18. 1. How to read \ 2. How to write (optional) - Tweet 19. 1. Explain yourself clearly, using lots of examples. \ 2. Assume you’re talking to a smarter version of yourself that hasn’t heard about your problem yet. \ 3. When it doesn’t do well, use the steps above to correct it. - Tweet 20. 1. The more precise your question or task is, the better and more accurate the response will be. Vague prompts can result in equally vague answers. \ 2. Provide relevant background or context, especially for nuanced questions or tasks. - Tweet 21. 1. Don’t ask them to do too much in one shot, especially if they are unrelated tasks; you’ll get much worse results. \ 2. Don’t give too much context if you can avoid it. The huge context windows of the newest models isn’t as “free” as you might think, or rather it’s “lossy”— the - Tweet 22. Collaborate with them, don't delegate to them. - Tweet 23. Context Window needs to be explained well. @NickADobos is spot on, but this needs to be explained without jargon we are so used to. - Tweet 24. 1. How context windows work TL;DR: it doesn’t remember everything in chat \ 2. It’s a text generator, that is good at patterns, and appearing smart. Not an almighty god doing cognitive work. Hallucinations aren’t ai behaving wrong. They are a feature of generating a bad pattern - Tweet 25. Consider the context a human would need when responding to the same request. When asked to create a presentation by your manager with 10-20 words, you have thousands or likely millions in context to inform that. Ppl often get annoyed when it fails, it’s usually not enough context - Tweet 26. Understand that they are autoregressive with a context limit and the limitations that impose on the chat interface. - Tweet Iterate and Simplify for Optimal LLM Performance 1. 1. just keep trying things - LLMs keep surprising me, \ 2. Start simple, add more techniques, context, guidance etc. step by step - with LLMs I found, less is often more. \ 3. Keep a human in the loop and/or be transparent about using LLMs - otherwise prepare for unpleasant - Tweet 2. When your llm starts omitting code generated in prior steps of an existing chat, end the chat and replay your steps until before that happened. Take a different branch next time - Tweet 3. 1. Don't give too much information at once to process, start simple and build on top of previous ones \ 2. Want a contrary opinion from LLM?don't sound like your opinion is sacrosanct - it will agree to you mostly even if its wrong. \ 3. Role playing and few shot examples matter. - Tweet 4. 1. Context \ 2. Difference is assumptions \ 3. Articulating clearly what you want (run it against another LLM to see if what you mean is what you say). \ 4. Being able to go back in a thread and restart (You get do over's with LLMs that you might not get with people :) ) - Tweet 5. Well one thing I learned is it's best to start a new chat if the LLM is going down the wrong path, easier then forcing it back. - Tweet 6. Having moderate experience with a topic / framework is important for peak quality of the response. At present, using llms for efficiency > using llms to do something you don’t know how to do. - Tweet 7. at least when it comes to writing code, the task needs to be very well defined, like one would do when creating a user story for developing software. If the details are vague then you leave the LLM open to interpretation and more likely to make mistakes - Tweet 8. The most important thing, and this has always been true even if not using an LLM, all good software development starts with engineering a solution first before building it. If you attempt to get the LLM to do that part you'll create as many problems than you solve building - Tweet 9. Use the LLM to explore your own understanding of the problem space and what you want to achieve. This can help improve your prompting and interpretation of the outputs. - Tweet 10. LLMs…\ Pander. Don't prime answers, ask straight.\ Only know text. Don't ask spatial, reasoning etc.\ Hallucinate and invert. Double-check.\ Get stuck. Start over.\ Master ALL languages, jargons, styles etc.\ Are formidable documentalists. - Tweet 11. 1. hallucinations are still a thing, be wary when LLMs generate links and code snippets 2. data quality of training content can sometimes be dubious leading LLMs to hallucinate more often or be biased in various ways both will likely be addressed eventually - Tweet 12. LLMs…\ Pick and imitate register. Talk like constructive, competent people.\ Are easily lost. Examples and feedback help.\ Can misbehave. Be harsh if needed, but stay just. - Tweet 13. For optimal results, provide ample context. Prompting the LLM with 'Feel free to ask clarifying questions' and doing the due-diligence to answering the questions often yields much better results. - Tweet 14. The more explicit you are the better the output. The LLM can not read your mind and there is a lot of ambiguity when interpreting language. - Tweet 15. One issue I am seeing more of - Often i ask a question on a choice it made. The LLM assumes I don’t like it or it’s wrong - it then starts to apologize and course correct. More and more I add something like “not refuting or arguing, just trying to understand” etc. - and that - Tweet 16. They are inherently unreliable in more than one sense, which accumulates the more requests you run in a chain. The Six Sigma approach is devastating to LLMs. - Tweet 17. Treat it like a very intelligent junior employee who just started at your company and lacks context. Give the LLM the same level of detail for every instruction you would give to this junior employee. - Tweet 18. That LLMs are not too be trusted as they reliably fail at information due to multiple effects, including hallucinations. That LLMs don't actually understand things and don't have common sense. It is mandatory to adapt expectations and ways of working to successfully use them. - Tweet Craft Effective Prompts for Consistent Results 1. How to prompt - Tweet 2. Carefully consider keywords, and prioritise them via the locating them earlier and at the end of longer prompts. - Tweet 3. If you want stable results across models and are looking to build robust pipelines you should stop hand writing prompts and move toward prompt optimizers. https://ycombinator.com/launches/L4V-hamming-let-ai-optimize-your-prompts-free-for-7-days… Also built into DSPy! - Tweet 4. While crafting logics and system prompts, Always keep a thought in your mind parallel what would I respond to this prompt and context. - Tweet 5. 1. Prompts matter. \ 2. Treat it like a tool, and you'll get a tool. It's only as smart as you let it be. - Tweet 6. to ask them the best way to prompt them - Tweet 7. there is a single prompt that gets the job done, thousands that screws it - Tweet 8. 1. Always add a system prompt at the beginning: Define a role. Ex: "You are a senior software developer who excels in…" \ 2. Context Matters: Provide a detailed background for better insights. \ 3. Clear Prompts: Specificity is crucial for accurate outputs. - Tweet 9. If a large global prompt doesn't work, try step by step. If it does work, but has errors in response - Ask it to fix errors one by one. Insist, like you would with a human supplier. If "do this" doesn't work, try "Strictly do this". Amazing how effective insisting is :) - Tweet 10. The better the prompt the better the output. You don't need a Meta framework for 90% of things - Tweet 11. They don't exist between prompts - Tweet 12. Don’t rely on the models weights alone. Be explicit in the prompt and give it pointers to what you’re expecting. Let it “clean up” or “translate” your prompt rather than “come up” with an answer based on its training. Exception: generating lists for inspiration. - Tweet 13. prompt engineering, in order to get the most desired outcome in handy. - Tweet 14. It lies Q: Who was the second person to walk on the moon? A: Pete Conrad Q: can you name the crew members of Apollo 11? A: I got the right answer. Q: Then how come Pete Conrad was the second person to walk on the moon? A: My apologies. Indeed Buzz Aldrin was the second pe… - Tweet 15. How to say no. - Tweet 16. How to use smart phone or computer with internet - Tweet 17. Vibe is an input. - Tweet Don't Expect Human-Like Understanding from LLMs 1. LLMs have no "thoughts" or understanding, they'll simply write the statistically most probable answer based on your input and have been prompted to act as assistants. - Tweet 2. LLMs are incredibly random. Responses can change wildly based on a single character difference in the prompt. Even one extra space. They are best for prompts that have a range of possible responses, not for prompts where you expect one consistent answer. - Tweet 3. Cease prompting their LLM to give them a viral tweet with forced irony forcing awareness to an issue. That's my own personal opinion, bro. But, believe what you want. - Tweet 4. If you don't know what you want, the LLMs too likely won't know. And if they don't know they will make it up. And if you don't know, you will not know that they made it up. - Tweet 5. Be sure not to put contradictions in your prompt. LLMs, in contrast to humans, try to follow instructions as close as possible. They usually handle contradictions by ignoring some part of the instructions or even ignoring facts. - Tweet 6. It's biased toward its creators. So if the majority of companies that are developing LLMs are owned by the same investors, then in fact, we are having a single LLM that is biased toward that investors goals. E.g., chatgpt is more toward liberalism and refuses to operate otherwise - Tweet 7. Basically, you need to understand that LLMs are not humans. You can't assume they'll understand what you mean when you write short prompts. You get the best out of LLMs when you provide detailed instructions of what you want without letting laziness get in the way. In my - Tweet 8. Don't assume anything. LLM doesn't learn like a human. Any assumption you make about what LLM should or shouldn't know is probably wrong. - Tweet 9. Describe your context and the role you want the LLM to look at your input (critical, tech/none-tech, …) Think what you could expect from a wise, random person you ask on the street. Do not expect more from the LLM-Answer. Also only trust it similarly. - Tweet 10. Give it an option to not do something either by allowing the LLM to reply with something like "I don't know" or tell it to ask follow up questions. - Tweet 11. There is nothing fundamentally important for that interaction. These LLMs are just minimum viable versions of something much bigger that will come soon. That something will know how to interact with us no matter how we behave. - Tweet 12. 1. that you need to cram the relevant data into the prompt. LLMs are far far better at transforming what you give them than they are at answering solely on the basis of the lossy representation of the training data encoded into the model itself - Tweet 13. The side effect fact that formulating a question for an LLM makes you think better. When coding, for example, we often run questions in our heads and then get to coding. Being forced to formulate a question properly may lead you to trajectories you may have never considered. - Tweet Treat LLMs as Guided Children, Not Mind Readers 1. The game isn't to 'one shot it'. It's to get something you never thought was possible or that you'd never think of. I always say they are like children, they need guidance (back story and reason) and repetition …but room and time to play and grow. - Tweet 2. Honestly, flexibility and patience. We need to give up a little bit of control and expectation of all things to be so rigid. - Tweet 3. When working with it, you need to expect it to not read your mind, but work with it as if you’re asking for help from an insanely gifted child and give yourself patience to shape the result. - Tweet 4. if it makes life better? yes. but always? no. - Tweet 5. When asking it how to implement something, always give it options. If you can't think of options, give it a vague out. Instead of asking, "should I do this to my code?", ask it "should I do this to my code, or is there some better way I could do it?". Otherwise the models are too - Tweet 6. 1. always consider that it doesn’t know what assumption you’re making. so it might infer them sometimes but often it’s much better to over explain what you want. 7. 2. they will often run ahead on a suggestion you have even if it’s not the best path so I find myself adding “if this - Tweet 8. It cannot read your mind, if you don't explain exactly what you want you will not get what you want - Tweet 9. I am not ready to give advice based on a bet that “something much bigger will come soon” - prompting advice that worked for GPT-4 over a year ago is still mostly relevant to working with the best models today - Tweet 10. to be concise and always assume the response is wrong, even ever so slightly. Check and correct. - Tweet 11. you have to provide context otherwise it assumes - it will often agree with you or apologize/correct itself even if you question the right answer - Tweet 12. The limited ability for non-linear (or non left-to-right) reasoning. Encouraging the model to spend more time planning and discussing beforehand often leads to better results. This may be less the case with Claude etc where reasoning tokens are happening behind the scenes. - Tweet Context is Key for Effective Interaction 1. Context is everything - Tweet 2. Context is all you need. - Tweet 3. Understanding how context works - Tweet 4. It’s all about context - Tweet 5. #contextmaxxing - Tweet 6. Context, Task & Purpose - Tweet 7. Subjectivity. Context. Brain rent. - Tweet 8. I'd say understanding the concepts of context, attention, and likelihood - Tweet 9. 1. Context and memory (the degree to which you can refer to previous parts in the chain of context) 2. Temperature and hallucinations. The tradeoff between extremes of temperature 3. It's wise to have benchmark questions of your own to test when a new company/model comes out - Tweet 10. local maxima sensing - Tweet Acknowledge the Stateless Nature of LLMs 1. you're interaction is with a stateless inference that exists for a fleeting moment, current ai is not continuous which is easy to forget. This has implications for what you are building for: - Tweet 2. that they're stupid next-token predictors and not intelligent agents. If you expect conscious beings, you'll be surprised and disappointed. But they're incredibly good at predicting the next useful token. - Tweet 3. That standard intuitions for computers don't apply. Treat it the way you would treat a knowledgeable but fallible friend. Not like a purely logical SciFi AI with perfect memory. - Tweet 4. Normally I hate predictions and terms like this, but the next 20 years are going to be the era of "embodied intelligence" People are imagining humanoid robots, this will be a very small fraction of it. Compared to the software problem, the body is trivial. Imagine asking your - Tweet 5. Inherent lack of memory about previous interactions. Every message is starting from zero and only seems coherent because background info and the previous messages and responses are sent before the latest message. - Tweet 6. They’re not sentient. They generate responses by predicting patterns from vast data, which means they're as fallible as they are impressive. The key is precision: your queries must be meticulously clear and well-contextualized. - Tweet 7. it doesn't have a memory like hooomans - Tweet 8. Whenever the conversation derails, you need to cut that branch and keep the model in the "right universe of probabilities" by editing prompt/messages. This is also why I was skeptical about Reflection, because if it really worked, it would be breakthrough. - Tweet 9. They aren’t deterministic - Tweet Leverage AI for Prompt Suggestions and Refinement 1. I like to ask them for prompts to use for a given purpose, it tends to be more detailed than I would be. Can also use this to add example Q&A if need be. - Tweet 2. We need AI assistance with prompts and suggestions on rewriting your queries similarly to Grammarly's for spellchecking and correctness. - Tweet 3. “Give me a list of questions I can answer to help improve the quality of the response” - Tweet 4. Let's ask one. - Tweet 5. Can we get our hands on all the prompts used in fine tuning data or at least major ones. Highly unlikely they will release it. - Tweet 6. anybody got tips for image generation? i hardly ever use the image features, but lordy, they struggle! even w with clear, verbose prompts using art school vocabulary, specific artist citations, and example attachments, lots of iterations, etc. - Tweet 7. I'm doing a podcast with the Cursor team. If you have questions / feature requests to discuss (including super-technical topics) let me know! For those not familiar, Cursor is a code editor based on VSCode that adds a lot of powerful features for AI-assisted coding. I've been - Tweet 8. They’re useful in the same way Google or the internet or stack overflow is useful plus one big advantage: your question doesn’t have to take your specific situation and change it to a generic case that someone else has already answered, you can just ask about your exact case! - Tweet Start with a Jailbreak for Objective Analysis 1. Using a jailbreak should always be your first step if you want less biased, more objective and fact-based analysis of sensitive or controversial sociopolitical issues. - Tweet 2. Kinda like Google, small changes in wording can give you quite different results. - Tweet 3. That you should only use it to get answers you can verify with a separate tool, or somehow evaluate yourself (ej. text quality). - Tweet 4. Its not a tool - Tweet 5. dont treat it like a search engine. think about the outcome and output you are trying to achieve. - Tweet 6. There is a considerable chance to answer is wrong, so likely everything needs to be double checked. - Tweet 7. I can only speak for the use cases I’ve come across wrt legal work, but don’t use them for tasks where you need a reference. Using them to draft or review documents is fine. Asking for a case law reference is a no-no. And of course, make sure you’re not leaking confidential stuff - Tweet Master Prompt Engineering for Better Outputs 1. lol. Nice try. If your business needs to level up I can do certification class. Your employees will get Level 4 Prompt Engineering Classification. DM if interested - Tweet 2. I like to write no full sentences with error and llm understand. So prompt engineering bullshit - Tweet 3. Turing test. - Tweet 4. Use instructions to change the style of the output that the LLM produces. For Claude you have to make a project first in order to be able to set the instructions. - Tweet 5. Understanding how LLM system, ChatGPT or Claude works and responding technically in basic. - Prompting skills. Understanding the difference between effective and ineffective prompting. - Tweet 6. understand the english language and HOW it's used (sadly, even english speakers have a hard time w/ correct language use). know grammar and syntax, context and nuance. be clear, succinct, specific when creating prompt. edit, edit, edit before sending prompt. - Tweet Understand LLMs as Statistical Predictors 1. Language models cannot generalize the simple formula "A is B" to "B is A." - Tweet 2. 1. tokenizers/decoding strategies are both incredibly important and invisible to most users. Remember that what you input is not what the model sees exactly, and what you read is not what the model output directly. 2) repeat #1 for the crowd in the back - Tweet 3. It's a bit sad and confusing that LLMs ("Large Language Models") have little to do with language; It's just historical. They are highly general purpose technology for statistical modeling of token streams. A better name would be Autoregressive Transformers or something. They - Tweet 4. Language - Tweet 5. They are next word predictors. Everything is downstream from that. - Tweet 6. The output is encoded in the input, the model is just a statistical decompression engine. This means that they can only ever amplify your mind, they can't think for you, however they can translate your question into more formal language & that may decompress into something useful - Tweet Stay Focused on High-Impact Tasks 1. Try to stay in the high impact zone e.g. through breaking tasks up and don't expect perfect results at all times - Tweet 2. Being able to define goals and objectives. - Tweet 3. Focus loquaciousness to refine results that will otherwise always regress to mean averages. - Tweet 4. If it doesn't understand you, ask it to help clarify your question. If you're not getting the answer you need, break your question into smaller parts. If you don't know how to break it down, ask it to help you break it down. - Tweet 5. You're interacting with a superposition of all humanity, so defining a specific persona that would be helpful for your task produces better results. -Avoiding assumptions and explaining your goal in the clearest way possible is the key to avoiding running around in circles. - Tweet Understand LLMs as Probabilistic Text Generators 1. they are reality-adjacent - Tweet 2. that they have to make sense - Tweet 3. That they are probabilistic systems. - Tweet 4. That they're random text generators and any appearance of intelligence is accidental and illusory. - Tweet 5. themselves - Tweet Verify Information, Never Trust Blindly 1. Verify, never trust. - Tweet 2. Never trust them - Tweet 3. Just don't. - Tweet 4. Anything coming out of those things can be completely false. Don't just accept it as truth. - Tweet Engage Actively to Maximize LLM Utility 1. that it's only as useful as how many questions you're asking it. Any initial understanding beyond that would be an overkill in my opinion - Tweet 2. It is only an upscaler not a freewin. The more you know the better it works, but compared to a person you can talk with it in shortcuts. The skill is to always reposition it constantly, before it goes off in the wrong direction. You can also work with labels within it's answers - Tweet 3. They’re useful/powerful for a wide range of tasks. Their usefulness is highly variable, depending on context & the skill of the user. A user’s existing expertise can be greatly amplified by the system, but novices probably benefit most. Ask them for help on how to use them. - Tweet 4. You no longer need to learn regex etc, you can just act like you know it at an expert level now, similar with syntax of virtually any language or technology. It is better at writing debugging output for you to find the problem in the code than finding the problem in the code… - Tweet Communicate Clearly and Specifically 1. Be specific, clear, and thorough. Same as communicating with humans, but more important. - Tweet 2. Be super clear with instructions. Funnily enough, we should be doing that with our instructions to our fellow humans, but we don’t! - Tweet 3. Effective writing - Tweet 4. BE SPECIFIC. Every one of my customers asks why a query they make doesn't return a result at all or a result they desire and it is because of the quality of their query over and over again. Some customers understand this out of the gate, some need some training. - Tweet Be Knowledgeable to Identify Hallucinations 1. Britannica's Great Books of the Western World - Tweet 2. Hallucinations are a thing and the model doesn't know if it's hallicunating or not. That's why the user using an LLM on any field has to be knowledgeable on that field in order to determine what's a hallucination. This means you can't use a LLM reliably to do something you can't. - Tweet 3. LLMs don't have the notion of True or False - Tweet Comments RK 16 Sep 2024 10:16 am: Models are total “yes men”...LoL! Using Richard Seroter's line - "Treat AI assistants as a slightly-drunk knowledgeable friend", I brainstormed with my bots to create this cartoon -https://mvark.blogspot.com/2023/07/beware-slightly-drunk-wisdom-of-ai.html


Analyzing 600+ internal chats reveals that corporate ChatGPT usage clusters around data work, summarization, coding, OCR, and image generation, but overall daily volume remains surprisingly low.

What does Gramener ask ChatGPT? I looked at how Gramener uses ChatGPT Plus by evaluating 600+ chats asked over 3 months from Oct 2023 to Jan 2024. The team asks 6 questions a day . We don't track who or how many actively use ChatGPT Plus. This also excludes personal ChatGPT accounts. Still, 6/day is low for an entire team put together . The questions fall into 8 categories. Category % Excel, data exploration & analysis 25% Text extraction and summarization 13% HTML, CSS, or JavaScript code 13% Python code 13% LLMs, AI and use cases 9% OCR and image analysis 9% Generate images, logos, and designs 7% General knowledge, policy & environment 5% Audio and translation 5% Here are some questions from each category - to give you an idea of emergent ChatGPT Plus usage. Excel, data exploration & analysis (25%) Excel clean and merge . There are 2 worksheets in this excel with data, can you clean up the data and merge the data in both the sheets Excel CO2 Data Analysis . You are an expert Data Analyst who is capable of extracting insights out of data. Analyze this sheet and let me know the findings Excel Chi-Square Analysis Guide . how to perform chi square analysis in excel Log Data Insights & KPIs . Looking at the columns from this excel, what kind of insights are possible, what are key KPIs to be looked at Text extraction and summarization (13%) Complaint Investigation Summary . The following is the summary of an internal investigation for a customer complaint. Now this internal summary is to be paraphrased (in 3-4 lines) as part of a closure Extracting Tables from RTF . Can you write a script to extract the tables from this document Extracting Entities from Text . [{'word1': '(P)', 'nearestword1': 'P/N:', 'nearestword2': '0150-25034', 'nearestword3': 'CARTIRIDGE'}, {'word1': 'P/N:', 'nearestword1': '(P)', 'nearestword2': '015... Extract PDF Font Details . Extract text formatting information from this document. Especially find font styles, families and sizes. HTML, CSS, or JavaScript code (13%) HTML/CSS Chart Template . Give me HTML, CSS and chart code for this design. CSS Font Stack: Explanation . Explain this CSS font convention: Arial, Helvetica, Segoe UI, sans-serif Checkbox Validation with JavaScript . In HTML form, I have a set of checkboxes. How do I write the form so that at least one of them being checked is mandatory? Prevent Text Wrapping CSS . Chief Communications Officer I need CSS such the text inside should not wrap create new line ReactJS App with Routing . Give me developed version using ReactJS use react router for sidebar section navigation to the pages use Tailwind css for styling. Use styled components for conditional … Python code (13%) Python Code Documentation Guide . Can you generate documentation for a project code written in python? Linux Commands for Python . Give me list of linux commands to work on python coding Code explanation request. What's this code about? … FastAPI Async Testing . Write a fastapi code and a python client to test the asynchronous nature of the fastapi package. Streamlit App for Translation . Given the following python code, give me a simple streamlit app that takes file upload and converts that into a target language: … An interesting sub-topic was interview question generation. Python Decorator for Database Queries . Create one medium level question for Decorators in python Industryy usecase specific with solution LLM, AI and use cases (9%) LLMs for Data "What Ifs" . You are an LLM Expert. Can you tell me how can we leverage LLM for implementing What IF scenarios on Data? LLMs: Current Challenges & Concerns . what are current challenges with LLMs LLM Applications in Marketing . Show LLM applications for the marketing function of a music company. Gen AI usage . What industries are using Gen AI the most Best LLMs in 2023 . Search the internet for the most recent LLMs and list the best LLMs in terms of performance Best Image Classification Models . suggest best models to tell what there in the image OCR and image analysis (9%) Browser history OCR . This is a screenshot of my browser history. Convert that to text. Categorize these into common topics. Extracted C Code . This image contains C code. Extract it. Image text extraction and annotation . Extract the text from this image and annotate the boundaries of the text Detecting Document Image Orientation . oreientation detection of documnet image AI Project with OpenCV & YOLO . Consider yourself as Open CV and Yolo expert and help me with AI project Image Correction Techniques . what are the approaches we have in computer vision where my image is tilted or rotated in reverse or image is not in readable format Generate images, logos, and designs (7%) Google Chacha and ChatGPT Bhatija . Generate an image of Google Chacha and ChatGPT Bhatija Regenerative Systems Group Image . Generate an Image with below context > "A group of people interested in Regenerative systems. The focus is on reusing food, energy and mental health" Twitter Reply Icons Design . Give me three icons: icon16.png, icon48.png, icon128.png for an extension that I'm building that suggests replies to tweets Generate flowcharts . Make a flowchart of the underlying working of a web app. Here's how it works. 1. The user uploads a document - a PDF or an image. They then select the language that … Create Animated GIF from Photos . I have 4 photos I want to make an animated gif out of them. How can i do that? Climate Impact Illustration . An illustration showcasing the impact of climate change on daily life, focusing on a rural setting near the coast. In the foreground, a small farm is visibly struggling, … General knowledge, policy & environment (5%) Design Thinking Overview . What is Design thinking Arthashastra . What can Arthashastra teach us about modern politics? Community Impact on Habits . Is there research to suggest the impact of community on habit building? Focus at Age 28 . What should a 28 year old focus on? Superconductors . Explain superconductors like I'm five years old. Climate Career: Impactful Choices . You a career counsellor at a University campus. You want to create 4 to 5 talking points for students to consider a career in Climate space. Sustainability Division Vision . I run a software outsourced product development company. I want to start a new division that focuses on sustainability services offerings. Please draft a vision… Audio and translation (5%) Audio Timestamp Mapping . timestamp mapping for transcribed audio Transcribe Lengthy Audio: Segment . Transcribe this audio file. Traducción del MOU al Español . Translate this document to Spanish, and create a new translated document. Maintain text formatting. Telugu Transcription into Hindi . Transcribe the following telugu text into hindi. You are supposed to transcribe, not translate. శ్రీనివాస పూజావిధానము … GPT lacks native audio support . Does gpt support audio in audio out natively?

What does Gramener ask ChatGPT?