pandoc on S Anand

Things I Learned - 26 Apr 2026

Sun, 26 Apr 2026 00:00:00 +0000

This week, I learned:

mdq is pretty useful to extract Markdown sections. For example cat *.md | mdq '# Title' extracts all sections where the header contains ‘Title’ (case-insensitive).
CloudFlare Browser Run is, roughly, a browser as a service. Pricing: 10 hours free per month, then 9c per hour. I had Codex run a small research to explore it, and it seems simple to set it up and use it.
GPT 5.5 seems to be especially better than GPT 5.4 and running for long, with tool calls, without losing focus. That’s something OpenAI models are good at anyway, so this takes it a step further. ChatGPT
I added gpt-image-2 to my LLM Art Style gallery. It is notably better with text accuracy. For example, on Rock - Paper - Scissors - Lizard - Spock it consistently lists all 10 rules, which Nano Banana 2 does not.
World leaders do keep us entertained.
- Saparmurat Niyazov (Turkmenistan) renamed the months of the year and days of the week after himself and his mother. He built a towering, gold-plated statue of himself in the capital that rotated so it would always face the sun. He also banned lip-syncing at concerts, outlawed gold teeth, and banished dogs from the capital because he found their smell unappealing.
- Idi Amin (Uganda) declared himself the “Uncrowned King of Scotland” and sent baffling, unsolicited telegrams to world leaders - advising Richard Nixon to recover from Watergate, or offering food aid to a struggling Britain.
- François “Papa Doc” Duvalier (Haiti) reportedly ordered all black dogs in Haiti to be put to death and claimed his personal Vodou curse was responsible for the assassination of John F. Kennedy.
- Francisco Macías Nguema (Equatorial Guinea) banned the word “intellectual”, banned the use of lubricants in the power plant (claiming his magic would keep it running, which promptly broke the generators), and stored the nation’s remaining foreign currency under his bed.
- Kim Jong-il (North Korea) claimed he invented the hamburger (calling it “double bread with meat”) and shot 11 holes-in-one his first time playing golf.
- Donald Trump (United States) used late-night tweets to announce major policy shifts and fire his own cabinet members. He altered an official government hurricane map with a Sharpie to match a previous erroneous statement, and publicly mused during a press briefing about the injection of household disinfectants as a medical treatment.
Git repositories inside git repositories (without using sub-modules) don’t seem to work well. I need this because I have mono-repos for research and I want to use git in a sub-folder to iterate, then commit just the final version to the parent folder. Looks like I need to remove the child .git/ (e.g. rename to .git.bak/, which I’ve added to my ~/.config/git/ignore) for this to work. Gemini
To run a script in the background (without logs) and detach / disown it, use nohup your-script >/dev/null 2>&1 & disown
Running /insights on Claude Code helped me add these two instructions to my code skill:
- Test web pages with screenshots (for layout, overlaps, contrast) AND CDP (for interactions, navigation) before finalizing
- Prefer icon libraries over unicode/emoji icons.
Sending an entire PDF/PPTX to Gemini costs ~40% of sending PDF/PPTX + images. The quality is fine for small files, but for large files adding images reduces error rate from ~5% to 0.5%.
Pandoc Markdown to Word DOCX supports sidebar comments. You can use this Markdown: Here is [comment in sidebar]{.comment-start id="c1" author="Anand" date="2026-01-01T12:00:00Z"}commented text[]{.comment-end id="c1"} inline. Gemini. In fact, Pandoc supports lots of other things, like:
- Custom styles via block ::: {custom-style="Custom Style Name"}
- Track changes via [inserted text]{.insertion author="Name" date="2026-04-20T12:00:00Z"} and [deleted text]{.deletion author="Name"}
- Page breaks via \newpage (a LaTeX command that Pandoc supports in Markdown)
- CSS styles via ![Alt Text](image.png){width="5.5in" height="3in"}
Offpunk is a CLI offline-first browser. Interesting idea, but installation is a problem. After sudo apt uninstall offpunk running offpunk failed with ImportError: lxml.html.clean module is now a separate project lxml_html_clean. After a git clone it reported HTML document detected. Please install python-bs4 and python-readability. These are easy to fix, but I wasn’t inclined.
Creating an authenticated MCP Server for ChatGPT is complex. It requires OpenID Connect (for which library support is weak and requires a provider like Auth0), dynamic client registration (which is hard to implement though Auth0 supports it), and after half a day of experiments, I still couldn’t connect. An easier option is to run temporary tunnels with cloudflared or ngrok or localtunnel.

Things I Learned - 23 Nov 2025

Sun, 23 Nov 2025 00:00:00 +0000

This week, I learned:

Here are some new CLI tools I installed:
- vd (visidata): Terminal spreadsheet viewer & editor for CSV, Excel, JSON, SQL, Parquet, etc.
- qsv: Fast CSV command line toolkit for slicing, filtering, aggregating, and analyzing CSV files.
- rga (ripgrep-all): ripgrep that searches PDFs, Office docs, EPUBs, zip files.
- pdfcpu: PDF processor for splitting, merging, optimizing, and manipulating PDF files.
- gum: Stylish CLI tool for creating interactive prompts, confirmations, and more.
Models read pretty fast, consuming input tokens at ~4K-20K words per second. It’s the “speaking” (output token rate) that is the bottleneck. So shortening input doesn’t matter as much as shortening output for latence. ChatGPT
When building agents, as of now, prefer native provider SDKs (OpenAI Agents SDK, Anthropic SDK) over even light abstractions like Vercel AI SDK or Pydantic. There are subtle issues related to error messages, response handling, cache handling, etc. that trip up abstractions given how early things are. Armin Ronacher
Gone are the times when LLMs couldn’t do mental math. Now they’re computing base64 and SHA256 from memory, without needing code! Example
Organizing a round table event in Singapore costs ~$75-150. Here’s what drives the cost variation #
- 50%: brand/location.
- 25%: food and beverage.
- 15%: duration (full day is only slightly more expensive than half day)
- 10%: date, demand, etc.
- 10%: add-ons: AV, etc.
OpenRouter supports embedding models. BGE base seems pareto optimal with 0.5 cents / MTok and a good MTEB ranking.
TOON vs JSON. Early days, and TOON seems to be marketing a lot, so I’m wary, but for large tabular data where input tokens are crunched, it seems a readable alternative to multiple CSVs, but not worth the hype. 0 19 Nov 2025. Always use GPT-5.1-Codex-Max instead of GPT-5.1-Codex. At every thinking level, it takes fewer tokens for similar or higher accuracy. Tibo

ug -i --smart-case --bool 'word1 word2 ...' seems the cleanest way to find files that have all words. –smart-case uses case-insensitive if all words are lowercase, else case-sensitive. Examples:

ug --bool '"exact phrase" word2'  # exact phrase + other tokens anywhere
ug --bool 'word1 word2 -word3'    # must contain word1 AND word2, but NOT word3
ug --bool '("foo bar") OR baz'    # grouped expressions and OR
ug --bool 'word1 NEAR/5 word2'    # match when words are within 5 tokens/words
ug -Z2 'word'                     # allows up to 2 typos in 'word'

⭐ ug -i --smart-case --bool -Q lets you interactively search within files. This is the coolest feature!
Fixing laptop issues is clearly a whole lot easier with an AI chatbot. I fixed these Ubuntu issues purely using Claude. It told me what to run. I ran it, shared the output, it diagnosed, told me what to do next, etc. until the issues were fixed. For example:
- My keyboard shortcuts stopped working. It turned out I edited my media-keys.dconf and removed the trailing slash. #
- A 3-finger tap mapped to a middle click and I couldn’t remove it. It turned out my touchegg.conf explicitly had this mapping. I disabled it. #
- My gnome extensions would get disabled every time the screen went to sleep. It turned out my extension cache was corrupted or stale. sudo apt install --reinstall gnome-shell-extension-manager and rm -rf ~/.cache/gnome-shell/ fixed it. #
GhostScript seems the best way to compress PDFs via the CLI. Example: gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/screen -dNOPAUSE -dQUIET -dBATCH -sOutputFile=output.pdf input.pdf
Pandoc supports Lua filters which are a powerful way to customize the document conversion process. Here is a Lua filter that converts horizontal rules in a markdown document to page breaks and preserve in a Word document (OpenXML format)
```
function HorizontalRule()
  return pandoc.RawBlock('openxml', '<w:p><w:r><w:br w:type="page"/></w:r></w:p>')
end
```
readpst - via sudo apt install pst-utils - extracts emails from Outlook PST files to mbox format. Useful for email migrations.
Write tutorials or blog posts as you learn. Steve Klabnik
Running a coding agent post mortem, e.g. “what worked well, what didn’t, and why? Next time, what are a few bullets I could include that will avoid these problems?” helps me prompt better next time. For example, Claude Code suggested:
- Use Firefox for headless browser automation (Chromium often crashes)
- Set HOME=/root when running Playwright with Firefox
- Start a local HTTP server rather than using file:// protocol
- External images may not load in screenshots due to network isolation

Things I Learned - 18 May 2025

Sun, 18 May 2025 00:00:00 +0000

This week, I learned:

Birds navigate using quantum entanglement! Guardian ChatGPT
DeerFlow is an open source Deep Research MCP. Lets you run deep research outside of the standard chatbots.
⭐ Today, if I had to store a bunch of data files (e.g. parquet) under 1GB, I would use GitHub Releases. Here are options:
- GitHub Releases. 2 GiB per file, unlimited total & bandwidth. 🟢 Immortal URL, versioning, easy CI publish. 🔴 Each file must stay < 2 GiB; no built-in SQL.
- Zenodo (CERN). 50 GB per record; one-off bumps to 200 GB. 🟢 DOI assignment, archival mandate. 🔴 Occasional throttled bandwidth; no API for partial file reads.
- Hugging Face Hub. 300 GB per repo; 50 GB per file. 🟢 Git-based, dataset tooling, lively ML community. 🔴 Large files need git-LFS; pushes via LFS can be slow.
- Cloudflare R2. 10 GB storage & 1 M ops / month. 🟢 S3 API, zero-egress to Cloudflare Workers, fast. 🔴 10 GB cap below your 50 GB target.
- Kaggle Datasets. 20 GB per dataset, public only. 🟢 Built-in notebooks & GPU. 🔴 No programmatic SQL API; quotas sometimes change.
- data.world (free). 1 GB total, 100 MB per dataset. 🟢 Nice social features. 🔴 Too small for your size.
If I had to query a bunch of data files in an external Parquet or SQLite file, here are SQL engines-as-a-service:
- MotherDuck. 10 GB storage + 10 CU-hrs/mo compute. Native DuckDB; no credit card; GA June 2024; monthly feature drops.
- Datasette Cloud. Two-month trial (or 1-yr for non-profits). SQLite backend. Great UX; but not free forever for general use.
- AWS Athena. Pay-per-TB scanned; no free tier; S3 fees after 12 mo. Costs creep quickly; free-tier S3 ends after a year.
Bootstrap has a .stretched-link that makes a link cover the containing block. A clever trick that I discovered when Claude 3.5 Sonnet wrote my code.
Discovered spray and peel paints at ArtFriend. I had no idea that was a thing.
Gemini Live API is the real-time equivalent from Gemini. It supports tools, search, and code execution.
mcp-mem0 is an MCP for memory
llm-min.txt compresses docs for LLMs to read optimally. Like a compressed llms.txt or context7. Usage GEMINI_API_KEY=... uvx llm-min -i $DIR #ai-coding
There’s a lot of action on encrypted LLM operations.
- Responses API allows reasoning tokens to be encrypted if organizations don’t want their reasoning data to persist. Ref
- Tinfoil (YC X25) offers an OpenAI-compatible inference API where data is encrypted from the client to the NVIDIA Hopper/Blackwell GPUs in confidential computing mode. Prompts, model weights, outputs are encrypted in transit and memory, with verifiable privacy on code running in GPU.
- Modelyo (Israel) offers VMs/K8 clusters with encrypted GPUs across multiple cloud providers with continuous attestation, managed on Modelyo’s portal.
⭐ LLMs are able to do things independently longer and longer. That’s a useful metric to track. METR: Measuring AI Ability to Complete Long Tasks.
If you’re looking for datasets / APIs related to research publications (especially funding), then explore:
- Crossref API and snapshots
- OpenAlex API and snapshots which is funded by OurResearch. OpenAlex is like CrossRef but includes some disambiguation
- OpenAIRE Graph 2024 / 2025
- Europe PMC dataset
To avoid Ubuntu 24 suspending on closing the laptop lid use one of these and restart:
- /etc/systemd/logind.conf: Set HandleLidSwitch=ignore
- etc/UPower/UPower.conf: Set IgnoreLid=true
UV_TORCH_BACKEND=auto uv pip install torch torchvision torchaudio installs the most appropriate PyTorch version. Ref
Cog is a Python based templating language. It is embedded as comment chunks in any file and replaced itself with the output of the Python code you write.
CloudFlare Zero Trust seems the easiest way to enable auth on static websites, especially if your DNS is already on Cloudflare. No cost
We could “fine-tune” system prompts automatically with evals, creating a “system prompt learning” paradim – like my promptevals. Andrej Karpathy
I was asked how to improve speed when building an enterprise ChatGPT clone using an API. Here’s what I’d suggest, in order:
- Streaming. High impact, low effort.
- Caching RAG retrieval as well as generation. High impact, low effort.
- UI tweaks. Loading / streaming icons and progress hints ()“Retrieving context”, “Generating answer”, etc.)
- Parallelize, if possible
- Use model options where available, e.g. speculative decoding, models with higher speed, models with closer CDN, etc.
- Shorten prompts
- Persistent HTTP/2 Keep-Alive. Low impact, low effort (tweak server settings).
Cloudflare Vectorize, at 768 dimensions / embedding, is free for ~6.5K chunks storage at ~1,000 queries / day. For a light load like 1M 768d chunks queried 1K times a day, the cost is: ChatGPT
NVIDIA parakeet is a lightweight speech to text model that leads benchmarks. Installing such packages continues to be a nightmare due to PyTorch (despite uv).
I explored the real-time avatar space. Heygen seems to be the easiest to use, but even that is complex and expensive ($99/mo). We may need to wait a few months for avatars to explode.
⭐ Model reliability is a huge enabler for performance. As models become more reliable, they can work autonomously for longer and that is another kind of scaling. Vending Bench
ChatGPT, Gemini, etc. have become lead generation engines. Chat Bot Optimization (CBO), is it? WhatsApp + ChatGPT
⭐ Never live delete data. Mark it for deletion and schedule a deletion task. That way you have time to react to mistakes. Simon Willison
Pandoc has several options useful when converting Markdown to HTML (cat file.md | pandoc -f markdown -t html). My favorites:
- --no-highlight skips code-highlighting. --highlight=pygments adds Pygments styling
- --wrap=none doesn’t wrap the content in a single block
- --number-sections adds section numbering (<h2>1. Introduction</h2>)
- --shift-heading-level-by=NUM – shift all headings by NUM levels (e.g., start at <h2> instead of <h1>)
- pandoc -f markdown-auto_identifiers drops the auto-identifiers extension that generates id=... for each heading
- pandoc -f gfm uses GitHub flavored Markdown. Run pandoc --list-extensions=gfm to identify the extensions it uses.
- Pandoc’s Markdown extension examples are quite extensive.
- Auto-enabled GFM extensions:
  - alerts: GitHub-style callouts (info, tip, warning) via > [!TYPE] blocks.
  - autolink_bare_uris: Turns bare URLs into links, without needing <...>.
  - emoji: Parses :smile:-style codes into Unicode emoji characters.
  - footnotes: Enables footnote syntax with [^id] and definitions at the bottom.
  - gfm_auto_identifiers: Uses GitHub’s heading-ID algorithm: spaces → dashes, lowercase, removes punctuation.
  - pipe_tables: Enables table.
  - raw_html: Raw HTML is unchanged.
  - strikeout: Enables strikethrough with ~~text~~.
  - task_lists: Parses - [ ] and - [x] items as checkboxes.
  - yaml_metadata_block: YAML front matter for document metadata, e.g. <title>
- GFM extensions worth enabling:
  - ascii_identifiers: Strips accents/non-Latin letters in automatically generated IDs.
  - bracketed_spans: [Warning]{.alert} becomes <span class="alert">
  - definition_lists: Term\n: Definition text becomes a definition list
  - fenced_divs: ::: {.note} block creates a <div class="note">...</div>
  - implicit_figures: Standalone images become <figure> with <figcaption>.
  - implicit_header_references: [Section] is treated as [Section][#section]
  - raw_attribute: <b>bold</b>{=html} is inserted as HTML
  - smart: Converts straight quotes to curly, -- to en-dash, --- to em-dash, ... to ellipsis.
  - subscript & superscript: E.g. H~2~O and E = mc^2^

How to publish an eBook in 60 minutes

Mon, 31 Mar 2025 12:47:13 +0000

I published an eBook on Amazon. It takes an hour if you have the content ready.

STEP 1 (10 min): Set up a Kindle Direct Publishing account with your address, bank details, and tax info.

STEP 2 (15 min): Export my London 2000 blog archive and convert to Markdown.

STEP 3 (10 min): Reformat the Markdown by writing a script in Cursor. Here’s the prompt:

Write a Python script that reads *.md including the YAML frontmatter, adds the YAML title as H1, date (yyyy-mm-dd) like Sun, 01 Jan 2000 in a new para after the frontmatter and before the content.

STEP 4 (15 min): Convert it to an ePub using pandoc.

pandoc *.md -o book.epub --toc \
  --metadata title="An LBS Exchange Program" \
  --metadata author="Anand S" \
  --metadata language=en \
  --metadata date="31 Mar 2025"

STEP 5 (10 min): Generated a cover page with ChatGPT (5 min) and compressed it into JPEG via Squoosh.

Draw a comic-style book cover page that covers the experiences of an Indian exchange student (picture attached) from IIM Bangalore at London Business School and exploring London. The book title is “An LBS Exchange Program”.

STEP 6 (10 min): Publish the book on KDP. It’s priced at $0.99 / ₹49 because Kindle doesn’t allow free downloads.

That’s it. Here’s the book: https://www.amazon.in/dp/B0F3D55R2Z/

The three things that made publishing in 1 hour possible are:

Amazon’s publishing process is simple.
Open-source tooling (WordPress, Markdown, ePub, pandoc) has built a big part of the infrastructure.
LLMs make the rest (figuring out the steps, generating the cover) very easy.

(An eBook takes 72 hours of review before going live on the Kindle store.)