--- title: "Building infra for AI pipelines" newsletter: "MLOps Community" date: 2025-07-03 source: https://aaif.live/newsletters/mlopscommunity/2025-07-03-building-infra-for-ai-pipelines --- # Building infra for AI pipelines *Plus - a correction, physical AI and data, MLflow 3 release, genAi and the data lake, Hidden Gems, and ML Confessions.* *MLOps Community — Agentic AI Foundation, 2025-07-03* We all know the dangers of bad data, so here’s a quick correction from Tuesday’s Tooling feature: We said LlamaIndex didn’t support high-level multi-agent workflows - that was wrong. Jerry (LlamaIndex’s CEO) reached out to clarify they’ve supported structured multi-agent workflows for over a year. You can read more about them here [https://go.mlops.community/licjul3]. ## ML Engineers Who Ignore LLMs Are Voluntarily Retiring Early This one was special for me, as I played Co-founder Cupid introducing Yoni and Kostas. It was a great chat too, about how inference is becoming a core data transformation step, while most tooling - like Spark - was built for structured, deterministic workloads. AI pipelines introduce non-determinism, GPU bottlenecks, and scaling issues that legacy infra can’t handle. They explored what’s missing from today’s AI infrastructure: * Lineage needs to go deeper - Teams need row-level tracing to debug how specific inputs affect outputs. * Evals lack full context - Single-call evals miss issues that emerge across multi-step chains. * Reliability is its own job now - Expect to see roles focused solely on production-grade AI stability. I’ll play Cupid again - click below to meet your next favorite episode. Video [https://go.mlops.community/ptd3jul] || Spotify [https://go.mlops.community/std3jul] || Apple [https://go.mlops.community/atd3jul] [https://go.mlops.community/atd3jul](https://go.mlops.community/atd3jul) [https://go.mlops.community/ptd3jul](https://go.mlops.community/ptd3jul) [https://go.mlops.community/std3jul](https://go.mlops.community/std3jul) ## Hidden Gems ## The Missing Data Stack for Physical AI Training, weights, pushing the limits... this can be a pretty physical job. But this time we were talking physical AI - systems where machine learning runs in the real world. Robots, spatial computing, sensor-driven setups. Messy, ambiguous environments where things shift constantly and nothing waits for your model to catch up. We talked about how teams are handling that, including: * Visualizing time-based data across modalities – camera, motion, sensors – all synced up * Debugging across online and offline systems – and spotting data bugs before they show up in prod Physical work more rewarding than your gym? Clicking below to listen. Video [https://go.mlops.community/prr3jul] || Spotify [https://go.mlops.community/srr3jul] || Apple [https://go.mlops.community/arr3jul] [https://go.mlops.community/arr3jul](https://go.mlops.community/arr3jul) [https://go.mlops.community/prr3jul](https://go.mlops.community/prr3jul) [https://go.mlops.community/srr3jul](https://go.mlops.community/srr3jul) ## Practitioner-driven talks and panels, coming to you ## WORLD TOUR ML stood for Miami Live [https://go.mlops.community/mwtlin] last week, with talks on building trustworthy GenAI systems, designing scalable agent infra, and real-world agent UX + dev tooling. Coming to a city near you: * Amsterdam [https://go.mlops.community/wtadam] - July 9 - talks from folks at weet.ai, orq.ai, and Pebbling.ai * Munich [https://go.mlops.community/wtmunich] - July 16 - featuring streaming agents, semantic code lookup, real-world failures, and autonomous infra ## MLflow 3: LoggedModel, GenAI workflows, and prompt evaluation ## RELEASE RADAR If you’ve ever had to wade through a sea of baseline_experiment_final_v2_reallyfinal_final MLflow 3 might help clean things up. The old run-centric structure is out. Now there’s a new LoggedModel object that ties together metadata from code, configs, traces, and evals - across both traditional ML and GenAI setups. They’ve also rebuilt evaluation and monitoring with GenAI in mind. No more duct-taped dashboards - you can now track accuracy, latency, and cost out of the box. Prompt engineering gets first-class treatment too: * Prompt Registry: Store, version, and document prompts properly. * Auto-tuning: Use eval feedback and labeled data to improve prompts. * Integrated evals: Built-in tools for measuring prompt performance. And if you’re working with humans in the loop, you can now log annotations next to predictions - useful when you’re getting feedback from domain experts or tracking changes over time. Documentation [https://go.mlops.community/mlfdocjul3] Reslease notes [https://go.mlops.community/mlf3jul3] ## Job of the Week [https://go.mlops.community/rowjob3jul](https://go.mlops.community/rowjob3jul) ## The Great Data Divergence: Why Generative AI Demands a New Approach Beyond the Data Lake Last week's Hot Take said the current approach to RAG and agents is fundamentally broken - this blog lays out the case in more detail. It explains that GenAI systems like RAG need fast, contextual access to live data - something traditional data lakes, with their batch pipelines and delayed curation, just can’t deliver. The post argues for an API-first model where operational systems stay as sources of truth and are accessed directly. Rather than copying everything into a lake just to make it usable, APIs offer a cleaner setup: * Agents query systems like Salesforce or Jira in real time * APIs handle access, monitoring, and governance * The data lake sticks around, but only for historical and analytical workloads It’s not really a hot take to say you should read this one. Read it here [https://go.mlops.community/blogjul3] [Read it here](https://go.mlops.community/blogjul3) [https://go.mlops.community/MLConfess](https://go.mlops.community/MLConfess) Most LLM infra is built by people who’ve never debugged a broken embedding, let alone tracked it through a production pipeline. Feeling seen after chasing down a broken vector? Or quietly offended because you’ve actually done the hard stuff? Let me know. https://go.mlops.community/vibe30jul3 Working on something tricky or planning ahead? Here’s how we can help - just hit reply: * Custom workshops tailored to your company’s needs * Hiring? I know some quality folks looking for a new adventure * Want to connect with someone tackling similar problems? I can introduce you Thanks for reading, catch you next time! --- Source: https://aaif.live/newsletters/mlopscommunity/2025-07-03-building-infra-for-ai-pipelines