reasoning-models on S Anand

Are LLMs any good at mental math?

Sun, 27 Apr 2025 09:52:10 +0000

I asked 50 LLMs to multiply 2 numbers:

12 x 12
123 x 456
1,234 x 5,678
12,345 x 6,789
123,456 x 789,012
1,234,567 x 8,901,234
987,654,321 x 123,456,789

LLMs aren’t good tools for math and this is just an informal check. But the results are interesting:

Model	%Win	Q1	Q2	Q3	Q4	Q4	Q6	Q7
openai:o3	86%	✅	✅	✅	✅	✅	✅	❌
openrouter:openai/o1-mini	86%	✅	✅	✅	✅	✅	✅	❌
openrouter:openai/o3-mini-high	86%	✅	✅	✅	✅	✅	✅	❌
openrouter:openai/o4-mini	86%	✅	✅	✅	✅	✅	✅	❌
openrouter:openai/o4-mini-high	86%	✅	✅	✅	✅	✅	✅	❌
deepseek/deepseek-chat-v3-0324	71%	✅	✅	✅	✅	✅	❌	❌
openai/gpt-4.1-mini	71%	✅	✅	✅	✅	✅	❌	❌
openai/gpt-4.5-preview	71%	✅	✅	✅	✅	✅	❌	❌
openai/gpt-4o	71%	✅	✅	✅	✅	✅	❌	❌
openrouter:openai/o3-mini	71%	✅	✅	✅	✅	✅	❌	❌
anthropic/claude-3-opus	57%	✅	✅	✅	✅	❌	❌	❌
anthropic/claude-3.5-haiku	57%	✅	✅	✅	✅	❌	❌	❌
anthropic/claude-3.7-sonnet:thinking	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.0-flash-001	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.0-flash-lite-001	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.5-flash-preview	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.5-flash-preview:thinking	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-2.5-pro-preview-03-25	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-flash-1.5	57%	✅	✅	✅	✅	❌	❌	❌
google/gemini-pro-1.5	57%	✅	✅	✅	✅	❌	❌	❌
google/gemma-3-12b-it	57%	✅	✅	✅	✅	❌	❌	❌
google/gemma-3-27b-it	57%	✅	✅	✅	✅	❌	❌	❌
meta-llama/llama-4-maverick	57%	✅	✅	✅	❌	✅	❌	❌
meta-llama/llama-4-scout	57%	✅	✅	✅	✅	❌	❌	❌
openai/gpt-4-turbo	57%	✅	✅	✅	✅	❌	❌	❌
openai/gpt-4.1	57%	✅	✅	✅	❌	✅	❌	❌
amazon/nova-lite-v1	43%	✅	✅	✅	❌	❌	❌	❌
amazon/nova-pro-v1	43%	✅	✅	✅	❌	❌	❌	❌
anthropic/claude-3-haiku	43%	✅	✅	✅	❌	❌	❌	❌
anthropic/claude-3.5-sonnet	43%	✅	✅	✅	❌	❌	❌	❌
meta-llama/llama-3.1-405b-instruct	43%	✅	✅	❌	✅	❌	❌	❌
meta-llama/llama-3.1-70b-instruct	43%	✅	✅	❌	✅	❌	❌	❌
meta-llama/llama-3.2-3b-instruct	43%	✅	✅	❌	✅	❌	❌	❌
meta-llama/llama-3.3-70b-instruct	43%	✅	✅	❌	✅	❌	❌	❌
openai/gpt-4.1-nano	43%	✅	✅	✅	❌	❌	❌	❌
openai/gpt-4o-mini	43%	✅	✅	✅	❌	❌	❌	❌
qwen/qwen-2-72b-instruct	43%	✅	✅	✅	❌	❌	❌	❌
anthropic/claude-3-sonnet	29%	✅	✅	❌	❌	❌	❌	❌
deepseek/deepseek-r1	29%	✅	✅	❌	❌	❌	❌	❌
google/gemini-flash-1.5-8b	29%	✅	✅	❌	❌	❌	❌	❌
google/gemma-3-4b-it	29%	✅	✅	❌	❌	❌	❌	❌
meta-llama/llama-3-8b-instruct	29%	✅	✅	❌	❌	❌	❌	❌
meta-llama/llama-3.1-8b-instruct	29%	✅	❌	❌	✅	❌	❌	❌
openai/gpt-3.5-turbo	29%	✅	✅	❌	❌	❌	❌	❌
amazon/nova-micro-v1	14%	✅	❌	❌	❌	❌	❌	❌
meta-llama/llama-2-13b-chat	14%	✅	❌	❌	❌	❌	❌	❌
meta-llama/llama-3-70b-instruct	14%	✅	❌	❌	❌	❌	❌	❌
meta-llama/llama-3.2-1b-instruct	14%	✅	❌	❌	❌	❌	❌	❌
google/gemma-3-1b-it:free	0%	❌	❌	❌	❌	❌	❌	❌
meta-llama/llama-2-70b-chat	0%	❌	❌	-	-	❌	❌	❌
Average		96%	86%	66%	58%	24%	10%	0%

OpenAI’s reasoning models cracked it, scoring 6/7, stumbling only on the 9-digit multiplication.

Models use human-like mental math tricks. For example, O3-Mini-High calculated 1234567 × 8901234 using a recursive strategy.

```markdown To calculate 1234567 × 8901234, one useful method is to break the second number into two parts: 8901234 = 8900000 + 1234 Then the product becomes: 1234567 × 8901234 = 1234567 × 8900000 + 1234567 × 1234 Step 1. Compute 1234567 × 8900000: Notice that 8900000 = 89 × 100000. So first compute 1234567 × 89. Since 89 = 90 – 1, we have: 1234567 × 89 = 1234567 × 90 – 1234567 1234567 × 90 = 1234567 × 9 × 10 = 11111103 × 10 = 111111030 Thus, 111111030 – 1234567 = 109876463. Now multiply by 100000: 109876463 × 100000 = 10987646300000 Step 2. Compute 1234567 × 1234: Break 1234 into parts: 1234 = 1000 + 200 + 30 + 4 Now multiply each and add: 1234567 × 1000 = 1234567000 1234567 × 200 = 246913400 1234567 × 30 = 37037010 1234567 × 4 = 4938268 Add these together: 1234567000 + 246913400 = 1481480400 1481480400 + 37037010 = 1518517410 1518517410 + 4938268 = 1523455678 Step 3. Add both products: 10987646300000 + 1523455678 = 10989169755678 Thus, the product of 1234567 and 8901234 is 10989169755678 ```

OpenAI’s other models and DeepSeek V3 were next, getting the first 5/7 right. Notably: GPT 4.1 Mini beat GPT 4.1. DeepSeek V3 beat DeepSeek R1.

DeepSeek V3 double-checks results and hallucinates a “reliable computational tool”.

To calculate (1234567 × 8901234), let's break it down using the **distributive property of multiplication over addition**:

1. **Express 8901234 as (9000000 - 98766)**:
   [8901234 = 9000000 - 98766]

2. **Multiply 1234567 by each part**:
   [1234567 × 9000000 = 1234567 × 9 × 10^6 = 11111103000000]
   [1234567 × 98766 = 1234567 × (100000 - 12234) = 123456700000 - 15095721278 = 108360978722]

   _Note_: Further breakdown of (1234567 × 12234) was calculated separately and subtracted here.

3. **Combine the results**:
   [11111103000000 - 108360978722 = 1020274222278]

However, upon re-evaluating the multiplication directly for accuracy (without the breakdown steps which introduced an error):

[1234567 × 8901234 = 10989343694778]

But correcting the previous breakdown:

1. **Express 8901234 as (9000000 - 98766)**:
   [1234567 × 9000000 = 11111103000000]
   [1234567 × 98766 = 1234567 × 90000 + 1234567 × 8000 + 1234567 × 700 + 1234567 × 60 + 1234567 × 6]
   [= 111111030000 + 9877653600 + 864196900 + 74074020 + 7407402 = 121616470922]

2. **Combine the results**:
   [11111103000000 - 121616470922 = 1098934369078]

It appears there was an inconsistency in the intermediate steps. For precision, let's verify with another method:

**Using the standard multiplication method**:

[1234567 × 8901234]

Calculating each part:
[1234567 × 8000000 = 9877653600000]
[1234567 × 900000 = 1111110300000]
[1234567 × 100 = 123456700]
[1234567 × 200 = 246913400]
[1234567 × 30 = 37037010]
[1234567 × 4 = 4938268]

Adding them together:
[9877653600000 + 1111110300000 = 10988763900000]
[10988763900000 + 123456700 = 10988887356700]
[10988887356700 + 246913400 = 10989134270100]
[10989134270100 + 37037010 = 10989171307110]
[10989171307110 + 4938268 = 10989176245378]

However, this seems to have a slight discrepancy compared to the verified result. Upon cross-checking with a reliable computational tool:

**Final Accurate Calculation**:

[1234567 × 8901234 = 10989343694778]

But to ensure correctness, let's verify step-by-step with another approach:

**Breakdown of 8901234**:

[8901234 = 8000000 + 900000 + 100 + 200 + 30 + 4]

Now, multiply each by 1234567:

[1234567 × 8000000 = 9877653600000]
[1234567 × 900000 = 1111110300000]
[1234567 × 100 = 123456700]
[1234567 × 200 = 246913400]
[1234567 × 30 = 37037010]
[1234567 × 4 = 4938268]

Adding these up:

[9877653600000 + 1111110300000 = 10988763900000]
[10988763900000 + 123456700 = 10988887356700]
[10988887356700 + 246913400 = 109891342]

O3 Mini reframes 8901234 as (9000000 − 98766) to simplify the calculation.

To calculate 1234567 times 8901234, follow these steps:

1. Express 8901234 as 9000000 − 98766.
     So, 1234567 × 8901234 = 1234567 × 9000000 − 1234567 × 98766.

2. Calculate the first part:
     1234567 × 9000000 = 1234567 × 9 × 1000000.
     Since 1234567 × 9 = 11111103,
     it becomes 11111103 × 1000000 = 11111103000000.

3. Calculate the second part:
     Write 98766 as 100000 − 234.
     Then, 1234567 × 98766 = 1234567 × 100000 − 1234567 × 234.
     - 1234567 × 100000 = 123456700000.
     - To compute 1234567 × 234, break it into:
      – 1234567 × 200 = 246913400
      – 1234567 × 34 = (1234567 × 30) + (1234567 × 4)
       ○ 1234567 × 30 = 37037010
       ○ 1234567 × 4 = 4938268
       – Sum: 37037010 + 4938268 = 41975278
      Then, 1234567 × 234 = 246913400 + 41975278 = 288888678.
     Thus, 1234567 × 98766 = 123456700000 − 288888678 = 123167811322.

4. Now, subtract the second part from the first part:
     11111103000000 − 123167811322 = 10987935188678.

So, 1234567 times 8901234 equals 10987935188678.

16 models, including the latest Gemini, Anthropic, Grok, and Llama models get 4/7 right.

The Amazon models, older Llama, Anthropic, Google, OpenAI models get 3 or less right.

View the results at https://sanand0.github.io/llmmath/. Hover over the cells to see the reasoning traces (where available).

How to Create a Data Visualization Without Coding

Sun, 27 Apr 2025 09:45:21 +0000

After seeing David McCandless’ post “Which country is across the ocean?” I was curious which country you would reach if you tunneled below in a straight line (the antipode).

This is a popular visualization, but I wanted to see if I could get the newer OpenAI models to create the visual without me 𝗿𝘂𝗻𝗻𝗶𝗻𝗴 any code (i.e. I just want the answer.) After a couple of iterations, O3 did a great job with this prompt:

𝙱𝚞𝚒𝚕𝚍 𝚊 _𝚜𝚒𝚗𝚐𝚕𝚎_ 𝙶𝚎𝚘𝙹𝚂𝙾𝙽 (𝙴𝙿𝚂𝙶:𝟺𝟹𝟸𝟼) 𝚝𝚑𝚊𝚝 𝚜𝚑𝚘𝚠𝚜, 𝚏𝚘𝚛 𝚎𝚊𝚌𝚑 𝚌𝚘𝚞𝚗𝚝𝚛𝚢, 𝚘𝚗𝚕𝚢 𝚝𝚑𝚎 𝚙𝚊𝚛𝚝𝚜 𝚘𝚏 𝚒𝚝𝚜 𝚊𝚗𝚝𝚒𝚙𝚘𝚍𝚎 𝚝𝚑𝚊𝚝 𝚕𝚒𝚎 𝚘𝚟𝚎𝚛 𝚘𝚌𝚎𝚊𝚗. 𝙲𝚊𝚛𝚎𝚏𝚞𝚕𝚕𝚢 𝚑𝚊𝚗𝚍𝚕𝚎 𝚌𝚘𝚞𝚗𝚝𝚛𝚒𝚎𝚜 𝚝𝚑𝚊𝚝 𝚜𝚝𝚛𝚊𝚍𝚍𝚕𝚎 𝚝𝚑𝚎 𝚙𝚛𝚒𝚖𝚎 𝚖𝚎𝚛𝚒𝚍𝚒𝚊𝚗 - 𝚄𝙺, 𝙵𝚛𝚊𝚗𝚌𝚎, 𝙰𝚕𝚐𝚎𝚛𝚒𝚊, 𝚎𝚝𝚌.

Here is the output and here is the ChatGPT conversation that generated it.

I learnt a few things:

Ask for the output, not the code. Models like O3 and O4 Mini can run code while thinking. Let’s stop asking for code to run. Just ask for the output directly. Let it figure out how.
Edge cases are everywhere. I had a problem with UK, France, Algeria, etc. straddling the prime meridian. If all goes well, you get AI-speed results. But it never does, and fixing it takes an expert and human-speed results. Programmers under-estimate edge cases, so compensate for this.

If you want to run this yourself, the code is at https://github.com/sanand0/antipodes

Things I Learned - 09 Feb 2025

Sun, 09 Feb 2025 00:00:00 +0000

This week, I learned:

Lessons from discussions at IIT Madras:
- Even in recorded video tutorials, asking students a question and pausing to give them time to think can be effective.
- When you put students in front of real clients, engagement increases dramatically.
- Most teaching assistants would like to help diligent students among the bottom half (more than the top decile of students).
- However, there is a fraction of poor performers who do not care, and are best ignored. Their engagement and effort is a good measure of their interest.
- Defining a minimal set of principles that we want to teach helps us measure if we’ve helped the bottom half at least meet those objectives.
- Teaching is hard. Even after explanations, students, even ENGAGED students, tend to make basic mistakes
ChatGPT does a good job of spotting errors in architectural and structural diagrams. In fact, the whole theme of spotting errors in large diagram is a theme that can have potential use cases. Source: Dan Becker.
R1 seems good at text-to-CAD. Even better than Sonnet. Source: Dan Becker
OpenAI advices a few different prompting techniques for reasoning models. OpenAI:
- Avoid examples unless zero-shot prompting fails.
- Avoid chain-of-thought. These models do that internally anyway.
- Short, direct prompts are better than detailed prompts.
GitHub models is free for anyone to try. The model catalog us extensive and even includes o3-mini which was launched this week (though in limited preview).
The data catalog space is led by proprietary solutions:
- Alation Data Catalog: Market leader; growing steadily in enterprise use
- Collibra Data Catalog: Widely adopted with steady growth
- AWS Glue Data Catalog: Growing rapidly as AWS expands its data services
- Informatica Enterprise Data Catalog: Long established and stable, though facing newer alternatives
- Microsoft Purview Unified Catalog: Experiencing fast growth driven by cloud momentum
- Atlan Data Catalog: Relatively new but gaining fast traction among tech-forward organizations
OpusClip automatically creates short clips from long videos. I ran it on Programming Minecraft with WebSockets in Python to get this short 30-second clip. 30 minutes. 100% automated.
Alternatives to Postman:
- Hoppscotch – A web‑based/desktop API client supporting REST, GraphQL, and WebSockets. It’s lightweight, open-source, and self‑hostable.
- HTTPie – A web-based API along with a friendly command-line tool for API interaction.
- Insomnia (or its fork Insomnium) – A popular cross‑platform API client with a minimal interface and plugin ecosystem.
- Bruno – A desktop open-source API client that stores collections as files (ideal for Git versioning).
- Milkman – A desktop open‑source workbench for managing API requests.
Here is the summary of DuckCon #6 on 31 Jan 2025 in Amsterdam. I copied the transcript from YouTubeTranscript and passed it through Gemini 2.0 Flash Exp with the system prompt: “Summarize this transcript from the DuckDB conference without missing any points. Cover every point mentioned. A lot of spelling errors that sound like DuckDB are likely to be DuckDB”.
- Introduction & Welcome:
  - DuckCon #6: This is the 6th DuckDB conference, held in their hometown. The first DuckCon was online due to the pandemic.
  - Live Streaming: This is the first time DuckCon is being live-streamed, chosen to accommodate global time zones (especially China and the US).
  - Global Reach: The live stream is intended to reach users in areas where in-person DuckCons are unlikely.
  - Q&A: Slido (qa.duckdb.org) will be used for Q&A, with upvoting to prioritize questions.
  - Sponsors: Thanks to gold sponsor monday.com and silver sponsors Real and Crunchy Data.
  - DuckCon Purpose: DuckCon is a place for users to connect, share experiences, and provide feedback to the DuckDB team.
  - Inspiration: The team is inspired by the community’s use of DuckDB and how far the project has come.
  - Mission Statement: DuckDB aims to make large datasets less intimidating and more accessible, moving away from fear of data to confidence in handling it.
  - Motivation: The project was born from seeing people struggle with data that didn’t fit in Excel and the lack of user-friendly tools.
  - Industry Trends: Single-node processing capabilities have grown faster than the size of useful datasets.
  - Data Singularity: A prediction that most data analysis queries can run on a single node is now a reality.
  - Real-World Data Sizes: Analysis of Snowflake and Redshift data shows that 99.9% of datasets are under 300GB.
  - Raspberry Pi Benchmark: The industry-standard TPCH benchmark (scale factor 300, ~300GB) can run on a Raspberry Pi using DuckDB.
  - Single Node Growth: Single-node processing power is rapidly increasing, allowing for larger datasets to be handled.
  - Adoption Numbers:
    - 32 Million Extension Installs: 32 million DuckDB extension installs in the last month.
    - 1.8 Million Unique Website Visitors: 1.8 million unique visitors per month to the DuckDB website.
    - Blue Sky Community: Growing community on Blue Sky, with the hashtag #dataBS.
- Technical Updates (Mark):
  - Extension Ecosystem: Focus on enabling the community to build and share extensions.
  - Community Extensions: Making it easier to create and use community-built extensions.
  - DuckDB v1.2 (Harlequin Duck): Releasing next week, named after the Harlequin duck.
    - CSV Reader Improvements: Significant improvements to the CSV reader.
    - Friendlier SQL: Improvements to the SQL experience.
    - CLI Autocomplete: Reworked and improved CLI autocomplete.
    - Performance Optimizations: Many queries are now faster due to performance work.
    - C API for Extensions: Introducing a C API to make building extensions easier.
    - Logging Features: Improved logging for production use.
  - Lakehouse Focus: The main focus for the year is on lakehouse formats and related features.
- Q&A (Mark & Hanis):
  - Doubling Team: If the team doubled, they would focus on client integrations and other projects, not a major architectural change.
  - Partitioning: Near-term plans to add support for partitioning, related to lakehouse formats.
  - DuckDB WASM: The WASM ecosystem is evolving, with exciting possibilities for in-browser use.
  - Financial/Pharmaceutical Industries: DuckDB could replace some SAS workflows due to its cost-effectiveness and capabilities.
  - Lakehouse & MotherDuck: Lakehouse work is separate from MotherDuck, though MotherDuck will likely support lakehouse features.
  - Contributing to Extensions: Plans to make it easier to contribute to extensions, including support for Rust and Go.
- Airport Extension (Rusty):
  - Analogy: The airport extension allows DuckDB to “fly” to remote servers using Apache Arrow Flight.
  - Functionality: Supports select, insert, update, and delete operations on remote data sources.
  - Motivation: To reduce the burden of writing extensions and enable faster development using existing code.
  - Arrow Flight: Uses Arrow Flight for communication, enabling connections to various data sources.
  - Demo 1: Delta Lake:
    - Attaches to a flight server for Delta Lake access.
    - Allows creating schemas, tables, and performing standard SQL operations.
    - Uses Python and deltars (Rust implementation of Delta Lake).
    - Supports predicate pushdown and C integration with the DuckDB catalog.
  - Demo 2: AutoGluon:
    - Integrates the AutoGluon AutoML package.
    - Predicts Hacker News post votes using a trained model.
    - Demonstrates table-returning functions for model fitting and prediction.
    - No C++ code required, just Python.
  - Demo 3: Geocoding:
    - Uses a geocoder service to convert addresses to coordinates and vice versa.
    - Demonstrates scalar UDFs for vectorized requests.
    - Uses a Python example for a simple uppercase function.
  - Features:
    - List flights, take flights.
    - Catalog integration.
    - Select, update, delete.
    - Scalar UDFs.
    - Table in/out functions.
    - Authentication for row/column filtering.
  - Availability: Requires DuckDB 1.2, MIT licensed, available on GitHub.
- Q&A (Rusty):
  - Most Proud Extension: Airport is the most fun, but the AWS API wrapper also brings joy.
  - Extension Resources: The GitHub DuckDB extension template and reading others’ source code are helpful.
  - Airport & Other Extensions: Airport is separate and can be used alongside other extensions like spatial or httpfs.
  - Graph Support: Graph database support is planned, with examples like Kuzu, Neptune, and Neo4j.
  - Licensing: Airport is MIT licensed, compatible with Apache license.
  - Scaling Out: Airport can be used to query multiple DuckDB instances on different machines.
- Ibis & Geospatial (Nati):
  - Nati Clementi: Senior software engineer at Nvidia, working on open-source projects like Ibis.
  - Ibis: Open-source Python library for data wrangling, with a DataFrame API and interfaces to 15+ engines, including DuckDB.
  - DuckDB for Geospatial: DuckDB is fast, has a geospatial extension, and supports various geospatial formats.
  - Geop Parquet: Becoming a standard for geospatial data, enabling cloud data warehouse interoperability and compression.
  - Geo Arrow: A way of representing geospatial vector data in memory for faster processing.
  - Ibis Benefits: Allows writing Python instead of SQL, with deferred execution determined by the engine.
  - Demo:
    - Uses OverTour Maps data in geop parquet format.
    - Filters data using bounding boxes.
    - Demonstrates geospatial operations like ST_Distance and ST_Transform.
    - Plots data using Lumber.
    - Shows how to find points of interest near a location (e.g., the Van Gogh Museum).
  - Ibis & DuckDB: Ibis uses DuckDB for the parquet reader and lets DuckDB do the heavy lifting.
  - Ibis Optimizations: Ibis does type checking but doesn’t do query optimization, leaving that to the engine.
  - Ibis in Browser: Ibis works in the browser through DuckDB WASM.
- Q&A (Nati):
  - Linear Interpolation: Ibis ML module can help with regression-related tasks.
  - Missing Features: No major features are missing in the DuckDB/Ibis geospatial setup, with minimal overhead.
  - Parquet Reader: Ibis uses DuckDB’s parquet reader.
  - Query Optimization: Ibis does not optimize SQL queries, leaving that to DuckDB.
  - Ibis in Browser: Ibis works in the browser through DuckDB WASM.
- Rill & Metrics Layer (Mike):
  - Rill: A BI tool optimized for DuckDB, with instant slicing and dicing, BI as code, and a metrics-first philosophy.
  - Metrics-First: Design metrics models, and Rill autogenerates dashboards and user experiences.
  - Live Demo:
    - Downloaded Rill using a curl command.
    - Created a new project called “DuckCon 6”.
    - Imported a parquet file of GitHub commit data.
    - Used AI to generate a metrics model and dashboard.
    - Showed the dashboard with trends and filtering.
  - Metrics as Building Blocks: Metrics are flexible, fast, and intuitive.
  - SQL for Metrics: Metrics should be defined in SQL, not other languages.
  - Visual Metrics Editor: Rill has a visual editor for defining metrics using DuckDB SQL.
  - Metric Stack:
    - Legacy: Data warehouses, traditional BI tools, inconsistent metrics, full table scans.
    - DuckDB Powered: Consistent metrics, fast olap queries, SQL everywhere.
  - Challenges: Data modeling is hard, metric changes can be expensive, single-node scale has limits.
  - AI & Metrics: AI can assist in metrics modeling, optimization, and conversational data exploration.
- Q&A (Mike):
  - Complex Metrics: Rill works well with complex metrics involving multiple sources and transformations by joining tables in DuckDB.
  - 60 FPS Dashboards: Users can feel the difference with faster dashboards.
  - Defining Metrics: Metrics are defined in the Rill UI using SQL expressions.
  - Replacing ChatGPT: Considering locally run self-hosted models for privacy.
- Stock Data Analysis (Ryan):
  - Two Takeaways: Simple finance data flows with trade data and a tool called Q Studio.
  - Ryan Hamilton: 14 years building large data platforms in banks.
  - Bank Data: Data from exchanges, market data providers, and internal systems.
  - Use Cases: Backtesting, data analysis, and report generation.
  - Q Studio: A Java desktop application that connects to 30 databases, including DuckDB.
  - Demo:
    - Loaded a 6GB CSV file of trade data into DuckDB.
    - Showed basic queries, pivoting, and Candlestick charts.
    - Demonstrated time-based aggregation and moving averages.
    - Showed a basic trading strategy using window functions.
  - DuckDB Benefits: Fast, easy to use, great for time-based analysis.
- Q&A (Ryan):
  - KDB+ vs. DuckDB: KDB+ is for large data, DuckDB is more approachable with strong Python integration.
  - XML Files: Offloading processing to DuckDB, not planning XML integration.
- Lightning Talks:
  - Zuk (Jared):
    - Search engine research using DuckDB.
    - Python-based experiments with SQL.
    - Removing document lengths for faster search engines.
  - DuckPGQ (Daniel):
    - Graph analytics in DuckDB using SQL property graph queries (pgq).
    - Visual graph syntax for pattern matching and path finding.
    - Outperforms Neo4j on analytical queries.
  - Yat (Kristoff):
    - Smallest DuckDB SQL orchestrator.
    - Runs SQL queries in a folder in the correct order.
    - Generates a mermaid diagram for lineage.
  - Grafana & DuckDB (Sam):
    - Lessons learned from using DuckDB in Grafana.
    - Security incident due to shell commands and file access.
    - Importance of reading the documentation.
  - Cloud Slur (Adam):
    - Syncing query engine for bank transaction data.
    - Uses LLM to convert human language to SQL.
    - Uses DuckDB in the browser, Node.js, and Python.
  - Healthcare Data (Tony):
    - Data engineering use cases in healthcare.
    - Dynamic data masking system using DuckDB and Snowflake.
    - Data integration pipeline using DuckDB and Arrow streams.
- Closing Remarks:
  - Michel Simmons: Author of the DuckDB in Action book, will be signing books.
  - Poster Session: A poster session will follow the talks.
  - Sponsors: Thanks again to the sponsors.
  - Social Event: The conference will now move to the social event.
ibis is a Python library that works with multiple dataframe backends like DuckDB, Polars, and Pandas.
With just 3 annotators and 50-100 samples, you can figure out if an LLM can replace human annotators systematically.Arxiv ChatGPT explanation
Curiosity and agency may be the differentiator in a world of LLMs (not experience, knowledge, or ability), since LLMs will democratize expertise. Jack Clark
“AI/human combined work can be copyrighted as long as a human is adding, changing or selecting elements. Prompts alone do not usually produce copyrighted work.” - Copyright and Artificial Intelligence, Jan 2025, US Copyright Office via Ethan Mollick
- Human Authorship is Essential: Works created solely by AI are not copyrightable.
- AI can be used as a Tool: Using AI as a tool does not negate copyright protection, as long as the final work reflects sufficient human creativity.
- Prompts Alone are Insufficient: Simply providing prompts to an AI system, even detailed ones, is generally not enough to establish authorship. Prompts are considered instructions or ideas, which are not copyrightable.
- Expressive Inputs: When a human author provides their own expressive content (like a drawing, photo, or text) as input to an AI system, and that content is perceptible in the output, the human author can claim copyright in that portion of the output.
- Modifying and Arranging AI-Generated Content: Humans can claim copyright in the creative selection, coordination, and arrangement of AI-generated material, as well as in creative modifications to AI-generated outputs.
- No Need for New Legislation: The report concludes that existing copyright law is adequate to address the copyrightability of AI-generated works, and no new legislation is needed at this time.
- Case-by-Case Analysis: Copyrightability will be determined on a case-by-case basis, considering the specific facts of each work and the extent of human contribution.