--- title: "Fixing GPU Waste in LLM Workloads" newsletter: "MLOps Community" date: 2025-04-03 source: https://aaif.live/newsletters/mlopscommunity/2025-04-03-fixing-gpu-waste-in-llm-workloads --- # Fixing GPU Waste in LLM Workloads *Plus, not your typical AI meetup, AI security, ML confessions, why RAG still rules, and Hidden Gems.* *MLOps Community — Agentic AI Foundation, 2025-04-03* https://go.mlops.community/VibeT It's a lifestyle now. Grab your VibeOps tee [https://go.mlops.community/VibeT]. ## Efficient GPU infrastructure at LinkedIn That time you called your 8th grade teacher “mom”, having to explain you’re not a cat on Zoom, the thought of idle GPUs - some things will keep you up at night. This episode can help with one of them. Animesh got into GPU infra challenges with LLM workloads - training, inferencing, and where the real bottlenecks are now. Inferencing costs are ballooning, especially when latency matters, and memory constraints are often worse than compute. LinkedIn’s been tackling this with: * In-memory checkpointing and async restore to reduce downtime * Kernel fusion with Liger for faster, leaner training * Elastic architectures to stop GPUs sitting idle Click below and make room for that 3rd grade show and tell to keep you up at night instead. Video [https://go.mlops.community/x75pl7] || Spotify [https://go.mlops.community/64he30] || Apple [https://go.mlops.community/37vuf5] [https://go.mlops.community/37vuf5](https://go.mlops.community/37vuf5) [https://go.mlops.community/x75pl7](https://go.mlops.community/x75pl7) [https://go.mlops.community/64he30](https://go.mlops.community/64he30) ## AI BEYOND THE MATRIX: AI AND THE FUTURE OF HUMAN CREATIVITY We've all been to AI meetups – but what about AI meetups? Fausto walked me through simulations he's been running to explore how AI could help people make better, faster connections at events. Each agent represents a meetup attendee, negotiating and connecting under constraints: * Each agent is built from user metadata and simulated goals, then dropped into a multi-agent environment * They use negotiation strategies and Nash-style dynamics to prioritize interactions * The system can suggest high-signal introductions – and even offer icebreakers based on shared objectives or complementary roles We also got into how reasoning models can recognize uncertainty and trigger smarter retrieval in complex RAG pipelines. Click below to listen and start making connections! Video [https://go.mlops.community/w7linr] || Spotify [https://go.mlops.community/l9d3t9] || Apple [https://go.mlops.community/o2hqmo] [https://go.mlops.community/o2hqmo](https://go.mlops.community/o2hqmo) [https://go.mlops.community/w7linr](https://go.mlops.community/w7linr) [https://go.mlops.community/l9d3t9](https://go.mlops.community/l9d3t9) ## Fraud Detection in the AI Era If you liked the meme about getting into cyber security for when the vibe-coded apps hit production, this one's especially for you. We got into AI security – everything from prompt injection to the risks of chaining LLMs. Rafael shared how he’s applying lessons from fraud detection at Nubank and building Siri’s guardrails. What really stuck with me was how he’s handling safety for AI assistants in banking apps. Instead of stacking LLMs, he uses: * Traditional classifiers to flag risky inputs * Zero-trust principles with tight access scopes * Red-teaming to uncover real-world vulnerabilities early Give it a listen – unless you want to end up relying on VibeSec. Video [https://go.mlops.community/hmyzxi] || Spotify [https://go.mlops.community/45j8bd] || Apple [https://go.mlops.community/zzg7d8] [https://go.mlops.community/zzg7d8](https://go.mlops.community/zzg7d8) [https://go.mlops.community/hmyzxi](https://go.mlops.community/hmyzxi) [https://go.mlops.community/45j8bd](https://go.mlops.community/45j8bd) ## Job of the Week [https://go.mlops.community/gtmmsu](https://go.mlops.community/gtmmsu) [https://go.mlops.community/MLConfess](https://go.mlops.community/MLConfess) ## RAG Reigns Supreme: Why Retrieval Still Rules! With the undoubted success of Vibe Coding, do we still need things like stacks, pipelines, and RAG? This post explores why RAG still matters, even with long-context LLMs. It’s not just a bolt-on for missing knowledge – the blog digs into retrieval quality, embedding finetuning, and when standard RAG starts to fall short. Agentic RAG gets a detailed look, especially where tasks involve multiple sources or tools. The blog highlights how agents can: * Reformulate queries if initial results aren’t useful * Chain multiple retrieval steps to refine answers * Interact with APIs or external systems mid-query Read it now to get ahead of the next big trend, VibeRAG. Read it here [https://go.mlops.community/lvgf81] [Read it here](https://go.mlops.community/lvgf81) ## Hidden Gems ## CALL FOR SPEAKERS What happens in Vegas, stays in Vegas. Apart from what people will learn from your talk at Ai4, taking place August 11-13. Call for speakers is open - go all in and apply here [https://go.mlops.community/AI4Speakers]. ## QUICK QUESTION Why is nobody talking about Reve AI [https://go.mlops.community/lgm9ij]? It's free, and just as good as chatgpt image gen. Send any others you think are flying under the radar my way. HERE TO HELP Before you go, here are three ways I can help - just hit reply: * Curated intros to other community members * What problems are you dealing with? Let me help you find the best solutions through my network * Looking to augment your staff for an MLOps or AI project? I got you covered Thanks for reading, catch you next time! Interested in partnering with us? Get in touch: partners@mlops.community Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ?view_as=subscriber], and podcast [https://home.mlops.community/public/content/] land. Oh yeah, and we are also on X [https://twitter.com/mlopscommunity] and LinkedIn [https://go.mlops.community/linkedin]. The MLOps Community newsletter is edited by Jessica Rudd [https://www.linkedin.com/in/jmrudd/]. --- Source: https://aaif.live/newsletters/mlopscommunity/2025-04-03-fixing-gpu-waste-in-llm-workloads