self-hosting on S Anand

SearXNG and Vane

Thu, 19 Mar 2026 15:40:25 +0530

While exploring resonant computing tools, I discovered SearXNG, a self-hostable metasearch engine, which aggregates results from multiple search engines.

It lets you search using APIs without needing to buy API keys and without being tracked. Pretty useful for research, people discovery, etc. when combined with LLMs.

Setting it up for API use seems easy (thought Gemini got it wrong twice):

cat <<EOF > settings.yml
use_default_settings: true
server:
  secret_key: "local_dummy_secret_key_987654321"
search:
  formats:
    - html
    - json
EOF

docker run -d \
  -p 8080:8080 \
  --name searxng \
  -v "$(pwd)/settings.yml:/etc/searxng/settings.yml" \
  -e "SEARXNG_BASE_URL=http://localhost:8080/" \
  -e "SEARXNG_SERVER_LIMITER=false" \
  searxng/searxng

Now, you can run:

curl -s "http://localhost:8080/search?q=best+sci-fi+books&format=json"

… which returns a JSON like this:

{
  "query": "best sci-fi books",
  "number_of_results": 0,
  "results": [
    {
      "template": "default.html",
      "url": "https://www.goodreads.com/list/show/19341.Best_Science_Fiction",
      "title": "Best Science Fiction (3733 books) - Goodreads",
      "content": "Other SF Lists of Note: ...",
      "publishedDate": null,
      "thumbnail": "https://imgs.search.brave.com/...",
      "engine": "brave",
      "parsed_url": ["https", "www.goodreads.com", "/list/show/19341.Best_Science_Fiction", "", "", ""],
      "img_src": "https://imgs.search.brave.com/...",
      "priority": "",
      "engines": ["startpage", "duckduckgo", "brave"],
      "positions": [2, 1, 1],
      "score": 7.5,
      "category": "general",
    },
    /// ...
  ],
  "suggestions": [
    "top 10 best sci-fi books of all time",
    // ...
  ],
  "unresponsive_engines": [
    ["google", "access denied"],
    ["wikidata", "timeout"],
  ],
}

There are many public SearXNG instances. Google seems to be blocked on many of these, but Brave, DuckDuckGo, and others seem to work fine.

That leads me to Vane which is a self-hostable Perplexity clone! A hosted version is available at Perplexica.io.

docker run -d -p 3000:3000 -v vane-data:/home/vane/data --name vane itzcrazykns1337/vane:latest

I tried a few questions and compared with Perplexity / Gemini using GPT-5 nano – a really cheap model.

🟢 It solves multi-hop queries. Which company acquired the data visualization firm Gramener in late 2023, and what is the current public stock ticker and market capitalization of that acquiring company?
- A: The data-visualization firm Gramener was acquired in November 2023 by Straive. Straive is a private company owned by BPEA EQT, part of the EQT group. Because Straive is privately held, it does not have a public stock ticker, and its market capitalization is not publicly disclosed.
🟢 It catches wrong questions. Summarize the findings of the April 2025 paper ‘Agile Methodologies in High-Latency LLM Architectures’ by Dr. Aris Thorne.
- A: Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?
🔴 It doesn’t get the latest data. Summarize the latest blog post on my website s-anand.net.
- A: I can do that. To ensure accuracy, I’ll need to fetch the latest post from s-anand.net and summarize it with key takeaways and insights. Would you like me to fetch the latest post directly from the site now, or you provide the post URL or paste the content here?
- Q: Fetch the latest post directly and give me a 1 para summary
- A: The latest post on s-anand.net is “Education Blog embeddings map,” published on Tue, 17 Mar 2026…

Though Vane got the last question wrong, Perplexity and Gemini got it wrong as well! So I guess it’s a good, comparable alternative.

LLM GPU or API? The Cost Will Surprise You

Sun, 30 Mar 2025 10:24:02 +0000

Say you want to use Llama 3.3 70b Instruct. You could:

Use it via an API. OpenRouter offers it at ~12 cents / MTok. Azure offers it at 71 cents. Your price may vary.
Self-host it on a major cloud provider. Azure offers A100 80GB at ~$3.67 / hour. In a day, you could generate ~0.5-1M tokens.
Self-host it on an emerging cloud provider. Lambda Labs offers A100 80GB at ~$1.79 / hour. Again, ~0.5-1M tokens a day.

Clearly, self-hosting is cheaper if you run it continuously. Let’s say we run for 1 million tokens every day. Then:

APIs cost 12 - 71 cents
Azure costs $3.67 x 24 = $88
Lamba Labs costs $1.79 x 24 = $43

So, the API is between 60 - 700 times cheaper than running it yourself. 60 times cheaper if you move from Lambda Labs to Azure AI Foundry. 700 times cheaper if you move from Azure servers to OpenRouter.

Not 60-700% cheaper. 60 - 700 times cheaper. Instead of spending a million dollars, you can get away with $1,500 - $15,000.

But what if…

What if you need higher scale? APIs typically offer better scalability than most organizations can configure.
What if you need higher uptime? Again, APIs typically offer higher uptime than most organizations can handle.
What if you need lower latency? Check the throughputs. APIs are typically much faster throughput than self-hosting.
What if you need edge computing? Then you’re GPU is one-time cost. My comparison is irrelevant.

So, are there any reasons to self-host? I’ve seen only a few.

Fine-tuned models. No one else offers an API version of your model.
Ultra-high security. You can’t trust even Microsoft, Google, or Amazon with your data. Your chats must remain in your own data center. (In this case, you probably run your own email and cloud services instead of Office 365 or Google Workspace.)
Learning. You’re curious about what it takes to self-host these models or want to build this skill.

If you don’t have one of the above needs, remember: your GPU costs can shrink from $1,000K to $1.5 - $15K.

Source: OpenAI Deep Research.

Comments

Sankar Ganesh 8 Apr 2025 5:50 pm: I asked ChatGPT: Other than extraordinary scale, what other possible reasons will make self-hosting the right choice for the customers, like customization, personalization, security, on-time support for unexpected failures, etc.? The answer as below Great follow-up. While cost is often the primary consideration, there are several compelling reasons why self-hosting a language model might be the better choice for some customers—even at small to moderate scales. Here’s a breakdown:

🛠 1. Customization & Fine-Tuning - You can fine-tune or RAG-augment open models on your domain-specific data. - Adjust the tokenizer, prompt structure, or system instructions in ways not possible on shared APIs. - Example: A legal-tech company might self-host Llama 3.3 fine-tuned on case law to improve legal document reasoning.

👤 2. Personalization & Memory - Full control over session memory and user data enables long-term personalization. - API services might limit how much context or history you can store, or restrict memory features. - You can implement user embeddings, persistent profiles, or behavioral adaptation directly.

🛡 3. Data Privacy & Compliance - Some orgs (especially in healthcare, finance, or defense) need full control over where data is stored and processed. - Self-hosting ensures no data ever leaves your infrastructure—critical for HIPAA, GDPR, or SOC2 compliance. - This also helps meet residency requirements (e.g., EU or on-prem data sovereignty).

🔐 4. Security & Access Control - You control who accesses the model and how. - You can implement your own authentication layers, audit trails, rate limits, and internal RBAC policies. - This reduces the attack surface and risks of using public APIs.

🚨 5. Reliability & SLAs - For mission-critical applications, self-hosting allows you to: - Ensure high uptime with redundancy and failover strategies. - Implement real-time monitoring, autoscaling, and hotfixes. - You’re not dependent on third-party API outages or usage limits.

🧩 6. Architectural Flexibility - Custom batching, streaming, quantization, or hardware-specific optimizations are possible when you self-host. - You can deploy quantized models (e.g., 4-bit or GGUF variants) on CPU/GPU hybrids, reducing cost without relying on API providers to support those formats.

🧪 7. Experimentation & Research - If you’re developing new LLM applications, agents, or model architectures, having full control over inference is a must. - You can benchmark across variants, simulate failure scenarios, or modify decoding strategies (e.g., contrastive search, top-k sampling) as needed.

🌍 8. Offline or Air-Gapped Environments - Self-hosting is the only viable option for offline environments (e.g., submarines, satellites, isolated R&D labs, or government-secured zones).

So even if you’re not pushing 100M tokens/day, self-hosting might be the right call if: You need strict control over data. You’re building a custom experience. You want architectural or security flexibility. You’re doing R&D or long-horizon experimentation. Want a quick decision checklist or flowchart to help folks figure this out more systematically?

Things I Learned - 28 Jul 2024

Sun, 28 Jul 2024 00:00:00 +0000

This week, I learned:

Speech editing in audio files is a thing. Speech Editing Toolkit and Descript
GPT 4o Mini is almost as good as GPT 4o in the LMSYS leaderboard. Llama 3.1 400B model and Mistral 2 Large are yet to be evaluated.
If LLMs can generate any text, and text can describe the real world, we can rapidly generate “artifacts” that generate:
- 3D Printable Models:
  - STL (Stereolithography): Defines the surface geometry of 3D objects using triangular facets.
  - OBJ (Wavefront OBJ): Describes 3D geometry including vertices, textures, and normals.
  - X3D: An XML-based file format for representing 3D computer graphics.
- Vector Graphics:
  - SVG (Scalable Vector Graphics): Defines vector-based graphics in XML format, useful for illustrations, diagrams, and user interface elements.
- CAD Drawings:
  - DXF (Drawing Exchange Format): Represents CAD data, including shapes, lines, and curves, used in engineering and architecture.
- Circuit Designs:
  - KiCAD: An open-source software suite for Electronic Design Automation (EDA), which uses various file formats like PCBNew and EESchema to represent circuit designs.
- Blueprints and Architectural Designs:
  - GML (Geography Markup Language): Encodes geographical features and spatial information.
  - CityGML: A specific GML application schema for modeling and exchanging 3D city models.
- Molecular Structures:
  - PDB (Protein Data Bank): Describes the three-dimensional structures of molecules.
  - CML (Chemical Markup Language): An XML-based standard for representing molecular data.
- Robotics and Automation:
  - URDF (Unified Robot Description Format): Defines the physical configuration of a robot, including joints, links, and sensors.
  - COLLADA (Collaborative Design Activity): An XML-based schema to describe digital assets for 3D applications, often used in robotics.
- Geospatial Data:
  - KML (Keyhole Markup Language): Used for geographic data visualization, primarily in Google Earth.
  - GeoJSON: A format for encoding a variety of geographic data structures using JSON.
- Mathematical Markup:
  - MathML (Mathematical Markup Language): Describes mathematical notation and captures both its structure and content.
- Music and Sound:
  - MusicXML: Encodes sheet music in a structured format that can be easily shared between different music notation software.
- Documents and Text:
  - DocBook: A semantic markup language for technical documentation.
  - Markdown: A lightweight markup language with plain text formatting syntax.
- Biological Data:
  - SBML (Systems Biology Markup Language): Represents computational models of biological processes.
  - PhyloXML: An XML format for representing phylogenetic trees.
- Game Development:
  - FBX (Filmbox): A file format for 3D animation that can hold information about the geometry, textures, and animations.
  - VRML (Virtual Reality Modeling Language): Describes interactive 3D objects and worlds.
- Data Visualization:
  - ChartML: Encodes charts and graphs in a structured format.
  - D3.js (Data-Driven Documents): Uses HTML, SVG, and CSS to bring data to life with interactive visualizations.
- Building Information Modeling (BIM):
  - IFC (Industry Foundation Classes): Describes building and construction data.
- Textiles and Fabrics:
  - LoomML: Represents the design and structure of woven fabrics.
- Augmented Reality and Virtual Reality:
  - ARML (Augmented Reality Markup Language): Defines how augmented reality applications should behave and what content they should display.
  - VRML (Virtual Reality Modeling Language): For describing interactive 3D objects and worlds.
- Medical Imaging and Health Data:
  - DICOM (Digital Imaging and Communications in Medicine): Encodes medical imaging data.
  - HL7 (Health Level 7): A set of standards for the exchange of information between medical applications.
- Simulation Data:
  - FMI (Functional Mock-up Interface): Represents and exchanges dynamic simulation models.
  - SBML (Systems Biology Markup Language): For computational models of biological processes.
- Sound and Audio:
  - MML (Music Markup Language): For encoding music notation and performance information.
  - SoundFont: A file format for defining musical instrument sounds.
- Animation and Visual Effects:
  - BVH (Biovision Hierarchy): Encodes motion capture data.
  - Alembic: A computer graphics interchange framework primarily for exchanging animation and visual effects data.
- Textile Patterns:
  - WIF (Weaving Information File): Describes weaving patterns and structures.
  - Knitting Markup Language: Encodes knitting patterns in a structured format.
- Scientific Data:
  - CDF (Common Data Format): Used for storing scientific data.
  - NetCDF (Network Common Data Form): Supports the creation, access, and sharing of array-oriented scientific data.
- Photography and Imaging:
  - XMP (Extensible Metadata Platform): Used for embedding metadata in digital images and other media files.
- Construction and Engineering:
  - LandXML: For civil engineering and land surveying data.
  - gbXML (Green Building XML): Facilitates the transfer of building data for analysis of energy and environmental performance.
- Packaging and Retail:
  - BPL (Barcode Product Labeling): Encodes information for product packaging and labeling.
  - GS1 XML: Used for electronic business messaging, including product identification and tracking.
- Typography and Font Design:
  - UFO (Unified Font Object): A format for storing font data.
  - SFNT (Spline Font): Encodes scalable font information.
- Product Data Management:
  - PLMXML (Product Lifecycle Management XML): Used for sharing product data across PLM systems.
GPT 4o Mini can be fine-tuned!
Awesome PaaS lists self-hosted deployment platforms. Piku - similar to Dokku – is promising.

Goodbye Google

Sat, 16 Mar 2013 12:29:25 +0000

Google Reader was where I spent most of my browsing time, but now, it’s shutting down.

Time for alternatives, but not just for Reader: for all Google products. I’m not sure when one of these might go down, become paid, or become unusable.

I just uninstalled Google Drive and Google Talk. but I don’t use it much (I use Skype), so no loss. I’ll leave Chrome for the while, but I’m hearing reports that Firefox is improving faster than Chrome is. Or there’s Chromium.

I’m not worried much about search services (including image, video, scholar and books). When needed, I can switch. Scholar might be a bit sad to lose, but I don’t use it much. Google Translate, too, isn’t essential.

Likewise for content. YouTube’s not a problem. There’re enough other video services. Trends are useful, but not critical. Maps might be, so I’ll try and switch to OpenStreetMap. I don’t use News or Picasa much.

I don’t care much for social media anyway, so Blogger, Orkut and Plus can die any time.

Google’s apps are the worrying ones. Mail and Calendar, in particular. I’ll probably migrate away from them last, but the attempt is on. I’ll be documenting the alternatives I find at https://gist.github.com/sanand0/5176161 (safely cloned locally).

Looks like there’s no safe long-term alternative to being able to host your own apps. Pity.

Comments

Hari K T 20 Mar 2013 12:23 pm: You are right. Probably many of them will follow your path :-) .
Dan Murray 16 Mar 2013 2:01 pm: It is annoying…isn’t it.
Ravikumar Venkateswar 18 Mar 2013 6:33 pm: Reader’s the one that bugs me the most, since they basically out-competed all of the alternatives, even the paid ones. I prefer paid apps, personally, because I know where the money’s coming from, and there’s less chance that they’d make them unusable. I switched from Picasa to Smugmug a while ago, and I’m probably going to move Mail and Calendar to free Google Apps or maybe my employer’s service. Why would you host your own apps instead of paying directly?
Hosting options | s-anand.net 1 Jun 2013 6:49 am (pingback): […] means. I can’t just take that code and move it to another server. Besides, I’m a bit wary of Google pulling the plug. Heroku? Same problem. I just want to take the code elsewhere and run […]
Tarun 21 Mar 2013 11:44 am: What about the appengine, did you explore any alternatives for it? I was quite impressed with script.google.com also, its in-line with your style of coding .
Dev E 22 Mar 2013 3:22 am: How about Nokia’s here.com for maps?
Mathew 23 Mar 2013 12:38 am: Feedly if you plan to keep Chrome.
Om Shankar 16 Jul 2013 11:35 am: Dude. I seriously hope you were saying that all out of anger and they are not your actual insights of other Google products. (Some were ridiculous - like Maps, Gmail, +plus, Chrome, etc.) Coz, if it were, you definitely need to come back to internet in today’s age and have a check again on your updates. Looking forward to your conf. at Bangalore JS. :-)