--- title: "What’s eating your streaming budget?" newsletter: "MLOps Community" date: 2025-04-10 source: https://aaif.live/newsletters/mlopscommunity/2025-04-10-what-s-eating-your-streaming-budget --- # What’s eating your streaming budget? *Plus, guiding habits with behavioral data, fixing GenAI workflows with classic ML, running distributed training across mixed GPU clusters, ML Confessions, and Hidden Gems.* *MLOps Community — Agentic AI Foundation, 2025-04-10* First they came for the steel imports, and I did nothing because I build in the cloud. Then they came for the car imports, and I did nothing because I use a local drive. Then they came for the CSV imports... Has it hit your repo yet? Hit reply and let me know. ## Streaming Ecosystem Complexities and Cost Management Streaming used to be Netflix and chill, but now it's Netflix and bill as you try to manage Paramount, Prime, Disney+ and more. It's the same story with streaming pipelines. They’ve grown more complex, especially when stitched together with Kafka, Flink, and DynamoDB. Rohit explained how teams often overspend chasing low-latency goals without tailoring systems to actual needs. Reliability suffers too, as responsibilities are split across teams and tools. Cost often creeps in quietly through settings like: * Checkpointing: Frequent checkpointing (e.g. every second) drives up costs fast. Tuning it to 10–30 seconds can help. * Data retention: Keeping all data in stream storage is expensive. Combining recent stream data with batch (e.g. Iceberg) is more efficient. Click below to stream the podcast - don't worry, it’s free! Video [https://go.mlops.community/pnt8kv] || Spotify [https://go.mlops.community/o5za21] || Apple [https://go.mlops.community/ynkrkt] [https://go.mlops.community/ynkrkt](https://go.mlops.community/ynkrkt) [https://go.mlops.community/pnt8kv](https://go.mlops.community/pnt8kv) [https://go.mlops.community/o5za21](https://go.mlops.community/o5za21) ## From Shiny to Strategic: The Maturation of AI Across Industries Picking greens over grease, broccoli over bacon, fruit over fries. They all make sense but sometimes we need a nudge. This chat with David wasn’t as technical as usual but was a great look at behavioral science and reinforcement learning. He shared how AI systems can be shaped by understanding why people act, not just what they do - using data from wearables and daily routines to gently steer habits, and looked at ideas like: * Contingency management: External rewards (like money) can kickstart behavior change, later tapering off as habits stick. * Keystone habits: Some actions (like morning runs) have ripple effects—boosting diet, sleep, and focus. Unsupervised ML plays a key role; LLMs are mostly irrelevant here. Here’s a personalized nudge - click below to watch. Video [https://go.mlops.community/ox7cgx] || Spotify [https://go.mlops.community/scbe93] || Apple [https://go.mlops.community/n2d6f4] [https://go.mlops.community/n2d6f4](https://go.mlops.community/n2d6f4) [https://go.mlops.community/ox7cgx](https://go.mlops.community/ox7cgx) [https://go.mlops.community/scbe93](https://go.mlops.community/scbe93) ## We're All Fine-tuning Incorrectly If you're an investor in AI, you may want to look away from this quote from Tanmay: “You should use as little ML or AI as you feasibly can.” It might feel counterintuitive, but it’s because most GenAI products still fall short in production. Tanmay explained how foundational models, while flexible, often aren’t aligned with real-world tasks - especially in expert workflows. CodeGen works well when users lack domain knowledge, but can frustrate those who need speed and precision. Fine-tuning can help bridge that gap - but only when there’s a clear objective and the infra to support it. One approach he recommends is layering classic ML around LLMs: * Classifiers upstream: A simple “should this be handled by the LLM?” check reduces hallucinations. * Assistants vs supervisors: LLMs help novices learn, but slow down experienced users. I'll finish with a made-up, but accurate, quote: “You should listen to this podcast as much as you feasibly can.” Video [https://go.mlops.community/a43152] || Spotify [https://go.mlops.community/o7848n] || Apple [https://go.mlops.community/zczk2m] Tanmay makes a strong case for being intentional about fine-tuning. If you want to explore it hands-on, DataCamp [https://go.mlops.community/DCLlama3FineTuning]* has a course using Llama 3 and LoRA. *This is an affiliate link – clicking helps support the newsletter. [https://go.mlops.community/zczk2m](https://go.mlops.community/zczk2m) [https://go.mlops.community/a43152](https://go.mlops.community/a43152) [https://go.mlops.community/o7848n](https://go.mlops.community/o7848n) ## Job of the Week [https://go.mlops.community/MLConfess](https://go.mlops.community/MLConfess) ## Distributed Training in MLOps Break GPU Vendor Lock-In: Distributed MLOps across mixed AMD and NVIDIA Clusters Great one-liner: Frankenstein entered a bodybuilding competition and won. Less funny: ending up with a cluster like Frankenstein’s monster because of vendor lock-in after an acquisition. This guide shows how to run distributed PyTorch jobs across mixed AMD and NVIDIA GPUs on AWS using UCX, UCC, and MPI - no need to rewrite model code. Kubernetes orchestration is handled with Volcano for flexible, GPU-aware scheduling. VolcanoJob enables: * Vendor-specific pod templates (AMD/NVIDIA) * Gang scheduling to prevent partial rollouts * MPI coordination via mpirun with UCX/UCC Click below to skip the horror story. Read it here [https://go.mlops.community/yc04h4] [Read it here](https://go.mlops.community/yc04h4) ## Hidden Gems ## CALL FOR SPEAKERS The AI Conference is happening in San Francisco, September 17 - 18. We hosted AIQCON [https://go.mlops.community/AIQCONcollection] there, so we know the speaker list will be good. Now's your chance to join them and share something cool you've been working on - the call for speakers is open now [https://go.mlops.community/5l0zsi]. ## MERCH Good vibes need good data. Show you're doing your part [https://go.mlops.community/TrainingDataTee]. ## HERE TO HELP Before you go, here are three ways I can help - just hit reply: * Curated intros to other community members * What problems are you dealing with? Let me help you find the best solutions through my network * Looking to augment your staff for an MLOps or AI project? I got you covered Thanks for reading, catch you next time! --- Source: https://aaif.live/newsletters/mlopscommunity/2025-04-10-what-s-eating-your-streaming-budget