--- title: "Moving GPUs away from Generally Problematic Usage" newsletter: "MLOps Community" date: 2025-05-29 source: https://aaif.live/newsletters/mlopscommunity/2025-05-29-moving-gpus-away-from-generally-problematic-usage --- # Moving GPUs away from Generally Problematic Usage *Plus - fine-tuning tricks with TAO, graph-based vector databases, building a quick RAG app with Vertex, ML Confessions, and Hidden Gems.* *MLOps Community — Agentic AI Foundation, 2025-05-29* After rolling back its stint as the ultimate hypeman, OpenAI now wants to be your wingman. “Sign in with ChatGPT” is in testing, with a form open [https://go.mlops.community/urysj1] for developers. You can preview it via Codex CLI. Completely unrelated, but feel free to share this [https://go.mlops.community/TrainingT] with anyone it feels relevant to. ## Building Out GPU Clouds The definition shifted a while ago: GPU = Gridlocked, Pricey, Unattainable. This episode covered how new GPU clouds are trying to offer more flexible options, from notebooks-as-a-service to token-based APIs. But behind the scenes, many still rely on manual provisioning and shaky inventory systems. Running infrastructure at scale brings real challenges, especially mid-training: * Failure handling – Multi-GPU clusters can see up to 30% failure rates. You need swap-in strategies to avoid losing full runs. * Network fragility – East-west and storage interconnects are common failure points. Resilience is non-negotiable. * Upgrades – Rolling updates often mean 20% of your fleet is offline just to stay afloat. Click below to make it Great Podcast's Useful. Video [https://go.mlops.community/rom9u9] || Spotify [https://go.mlops.community/qu18tl] || Apple [https://go.mlops.community/k98gst] [https://go.mlops.community/k98gst](https://go.mlops.community/k98gst) [https://go.mlops.community/rom9u9](https://go.mlops.community/rom9u9) [https://go.mlops.community/qu18tl](https://go.mlops.community/qu18tl) ## can LLM agents remember what you told them last week? MEM0: BUILDING PRODUCTION-READY AI AGENTS WITH SCALABLE LONG-TERM MEMORY I know I struggle, but can LLM agents remember what you told them last week? That’s the focus of this month’s MLOps Community Reading Group on Thursday, May 29, where we’re digging into: 📄 “Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory” 🎙️ With special guest Prateek Chhikara, Founding AI Engineer at Mem0 and co-author of the paper. We’ll unpack how Mem0 tackles persistent memory using graph-based structures - and what that unlocks for multi-session, production-grade agents. Register here [https://go.mlops.community/jpnexf] and join the #reading-group [https://go.mlops.community/mzne34] channel in Slack to chat before and after. [Register here](https://go.mlops.community/jpnexf) ## Tricks to Fine-Tuning At risk of a recursive loop, I'm going to label labels as expensive, scarce, and often overrated. So what if you could fine-tune a model without any? That’s what Raj did with TAO (Test-time Adaptive Optimization) - a fine-tuning method that adapts LLMs to domain-specific tasks using reinforcement learning. Users only provide prompts - no labels - and the model learns by generating responses and scoring them with a reward model during training. Inference stays fast and lightweight. A few clever moves make this work: * The model generates multiple varied responses per prompt, using sampling strategies to avoid redundancy. * A reward model scores the outputs, guiding learning without hand-labeled examples. With just prompts, TAO-fine-tuned 8B models matched or beat GPT-4 on finance QA and text-to-SQL tasks. I’ll label this episode a solid listen. Video [https://go.mlops.community/x5ybpd] || Spotify [https://go.mlops.community/g4cwtx] || Apple [https://go.mlops.community/upun6x] If you're curious how to evaluate outputs when your model’s generating its own training data, the AI Evals for Engineers & PMs [https://bit.ly/3EW4Tru] course is a perfect follow-on. It’s an affiliate link – clicking helps support the community. [https://go.mlops.community/upun6x](https://go.mlops.community/upun6x) [https://go.mlops.community/x5ybpd](https://go.mlops.community/x5ybpd) [https://go.mlops.community/g4cwtx](https://go.mlops.community/g4cwtx) ## Job of the Week [https://job-boards.greenhouse.io/gusto/jobs/6865882](https://job-boards.greenhouse.io/gusto/jobs/6865882) [https://go.mlops.community/MLConfess](https://go.mlops.community/MLConfess) ## Should You Use Graph-based Vector Database for Multimodal AI? We’ve had VibeOps but this one made me think more of VibeShops, with its opening about jacket recommendations. It’s not a fashion blog though, it’s about infrastructure: how we retrieve, relate, and reason across multimodal data. Multimodal systems need infrastructure that supports relationships across text, image, audio, and metadata – not just flat similarity search. Graph-based vector databases offer a way to handle this complexity, especially when schemas are fluid or context matters as much as content. They’re especially useful when dealing with evolving or loosely structured data: * New relationships often break rigid schemas – graph models adapt more easily * Multi-hop queries become manageable without tangled JOIN logic * Foreign key-linked or semi-structured data fits more naturally in a graph format Have a read and see if this might be a better fit for your stack – or even a tailor-made one. Read it here [https://go.mlops.community/moxi3l] Looking to build your own multimodal search system? Our Introduction to Vector Databases and Multimodal Semantic Search [https://go.mlops.community/VCourseNL] course covers embeddings, vector databases, and building a working multimodal search system – all for just $20 with code: VECTOR20 [Read it here](https://go.mlops.community/moxi3l) ## Hackathon Speedrun: Build & Deploy a RAG App in Minutes with Vertex AI Studio & Vertex AI Search! This blog is well-timed for our Hackathon with Bright Data [https://gatewaze.mlops.community/offer/brightdata-mcp-hackathon#email], showing how to build a RAG app quickly using Google Cloud tools. Source documents are uploaded to GCS, indexed via Vertex AI Search, and linked to a prompt in Vertex AI Studio using grounding. The assistant is then deployed to Cloud Run or integrated via code export. The data prep section is clear and easy to follow: * Files are cleaned, converted to .txt, and mapped in a metadata.jsonlines file * Each entry includes URI, MIME type, and source for accurate indexing * Uploads are handled with simple gcloud commands Have a read and get building! Read it here [https://go.mlops.community/nflfj4] [Hackathon with Bright Data](https://gatewaze.mlops.community/offer/brightdata-mcp-hackathon#email) ## Hidden Gems Working on something tricky or planning ahead? Here’s how we can help - just hit reply: * Custom workshops tailored to your company’s needs * Hiring? I know some quality folks looking for a new adventure * Want to connect with someone tackling similar problems? I can introduce you Thanks for reading, catch you next time! --- Source: https://aaif.live/newsletters/mlopscommunity/2025-05-29-moving-gpus-away-from-generally-problematic-usage