--- title: "Catch Breakage Before It Catches You" newsletter: "MLOps Community" date: 2025-10-16 source: https://aaif.live/newsletters/mlopscommunity/2025-10-16-catch-breakage-before-it-catches-you --- # Catch Breakage Before It Catches You *Plus: Terragrunt fixes Terraform pain, NVIDIA’s compact GPU rig, and the State of AI 2025.* *MLOps Community — Agentic AI Foundation, 2025-10-16* We’ve been running a quick pulse on what teams are focused on right now. So far, deployments and infra are taking the lead, but there’s a quiet push for career growth and team structures too. Add your voice before we close the poll - it only takes a few seconds [https://go.mlops.community/poll0910teamtalk]. ## Roleplaying Agents Agents that can’t take action are chatbots with a costume. Be honest, what are you building: Action or Chat? ## Rage against the machine Last week we asked what’s holding you back, and it wasn't the models. ## MLOPS COMMUNITY VIRTUAL CONFERENCE ## Agents in Production - MLOps x Prosus https://go.mlops.community/AIPNov2025 Now in its sixth year, Agents in Production brings together practitioners from OpenAI, NVIDIA, Meta, and Google DeepMind to share what’s actually working when deploying agentic AI systems. Join us on 18 & 19 November for practical sessions on architecture choices, coordination frameworks, and debugging tools - plus lighter moments between talks to keep the live event energy. Check out the agenda and register here [https://go.mlops.community/AIPNov2025]. ## Hidden Gems ## Job of the Week [https://go.mlops.community/jobsnl](https://go.mlops.community/jobsnl) ## Evals Aren't Useful? Really? When your agent starts leaking secrets or handing out free discounts, it’s already too late. The only thing standing between a stable system and chaos? Solid evals. * Building evals from zero: Start with curated test sets and treat them like unit tests - flow-by-flow coverage that catches breakage before users do. * Red-teaming with agents: Use persona-based simulations to push your own systems to failure - persistent, goal-driven attackers reveal weak guardrails fast. * Scaling evaluation like CI/CD: Move from handcrafted tests to automated pipelines and production feedback loops that evolve with real user behavior. It’s a reminder that shipping agents safely isn’t about perfection - it’s about testing like your users are trying to break you. Video [https://go.mlops.community/ccp16oct] || Spotify [https://go.mlops.community/ccs16oct] || Apple [https://go.mlops.community/cca16oct] HOW TO BUILD AGENTS THAT TAKE ACTION In a standout session from our AI Agent Builders Summit, Alex Salazar unpacked why most agents never make it past the demo - 70% fail before reaching production due to security gaps, high costs, latency, or poor accuracy. * Evals first: Treat curated scenarios as unit tests; track tool-use accuracy and regressions before rollout. * Tools over APIs: Build intention-based tools, not raw wrappers; push logic into tools to cut LLM loops, cost, and compounding errors. * Auth that scales: Use delegated authorization (user and app scopes) for Gmail/CRM access; handle token refresh and revocation. Building agents that act means thinking like engineers - testing rigorously, enforcing permissions, and designing for real-world complexity. Watch the talk here [https://go.mlops.community/summit16oct] WHY I USE TERRAGRUNT OVER TERRAFORM/OPENTOFU IN 2025 Terraform breaking your CI/CD at 2 AM? That’s not bad luck - it’s a design flaw. A newer approach fixes the duplication, orchestration, and backend chaos that Terraform users have lived with for years. * Code reuse: Shared configs with include blocks remove the folder-copy mess across environments. * Orchestration: Dependency graphs handle deploy order automatically with a single command. * Stacks: Pattern-level reuse turns repeated setups into reusable infrastructure blueprints. The takeaway: it’s finally possible to manage multi-environment IaC without fighting the tool itself. Read the blog [https://go.mlops.community/blog16oct] IN PERSON EVENTS * Amsterdam [https://luma.com/8dt60z7m] - October 16 * New York [https://luma.com/agentsummitNYC] - October 16 * Barcelona [https://luma.com/xlazzzyw] - October 22 * San Jose [https://luma.com/eqatoz90] - October 30 * Helsinki [https://luma.com/29uac5sx] - November 6 VIRTUAL EVENTS * Reading Group [https://luma.com/ql7n0phx] - October 23 We'll discuss LiveMCP-101 [https://arxiv.org/abs/2508.15760], a new benchmark that stress-tests MCP-enabled agents on real-world, multi-step tasks across web search, data analysis, and more. * Agents in Production - MLOps x Prosus [https://home.mlops.community/home/events/agentsinproduction2025-mlops-prosus] - November 19 Learn how leading teams from OpenAI, NVIDIA, Meta, and Google DeepMind are turning agentic AI experiments into production systems. [Video](https://go.mlops.community/ccp16oct) ## The early days of ML... [https://go.mlops.community/MLConfess](https://go.mlops.community/MLConfess) ## Making the hard stuff simpler Working on something tricky or planning ahead? Here’s how we can help - just hit reply: * Custom workshops tailored to your company’s needs * Hiring? I know some quality folks looking for a new adventure * Want to connect with someone tackling similar problems? I can introduce you Thanks for reading, catch you next time! --- Source: https://aaif.live/newsletters/mlopscommunity/2025-10-16-catch-breakage-before-it-catches-you