I explored home networking and LLM infrastructure, discovering WiFi 6 beam-forming, Predibase's competitive pricing for fine-tuned models, RunPod's serverless vLLM endpoints for HuggingFace models, and Portkey's utility as an AI model router.
My home WiFi is on WiFi 6. This supports beam-forming which increases range by “focusing” on devices!
Predibase lets you run fine-tuned models at the same price, on a per-token basis. 25c/MTok up to 21B models. That’s sames as Claude 3 Haiku, but with fine-tuning.
RunPod’s vLLM endpoint lets you run any HuggingFace LLM with an OpenAI API priced on usage (serverless) not on idle time. “Autoscaling to 0”.
I learned how Predibase uses LoRAX to serve multiple fine-tuned model adapters on a single GPU. This method leverages LoRA adapters to optimize hardware usage and scale specialized models more efficiently than traditional deployments.
I learned about GPT Vision for calendar extraction, the limitations of RAG versus fine-tuning, and using LlamaIndex for hierarchical retrieval. I also found tools for JSON repair, LLM evaluation metrics, and running local models with Ollama.
I discovered Julius.ai for data queries and AutoGen for multi-agent systems. I also explored high-performance LLM inference using vLLM and DeepSpeed, alongside architectural patterns and critiques of the Foundation Model Transparency Index.
I explored Agentic RAG for complex retrieval, fine-tuning with LoRAX, and practical LLM strategies. Key takeaways include using N-shot prompting before scaling models, automating workflows via disposable apps, and leveraging context caching to significantly reduce inference costs.
I explored various LLM tools, including llamafile for single-executable local models and marker for PDF-to-markdown conversion. I also looked into RAG tuning strategies, PII detection libraries like Presidio, and intuitions regarding scaling and emergent model behaviors.