---
title: "Agents of Chaos: Debugging LLM Pipelines"
newsletter: "MLOps Community"
date: 2025-06-19
source: https://aaif.live/newsletters/mlopscommunity/2025-06-19-agents-of-chaos-debugging-llm-pipelines
---

# Agents of Chaos: Debugging LLM Pipelines

*Plus - FastAPI and what's next, ML Confessions, and Hidden Gems*

*MLOps Community — Agentic AI Foundation, 2025-06-19*

## Job of the Week

[https://go.mlops.community/GwazeEmail](https://go.mlops.community/GwazeEmail)

## Everything Hard About Building AI Agents Today

I still have to apply cream after the burns Willem and Shreya gave me during this.

In between the roasting, they got into the pain of debugging in production and building LLM-powered data pipelines. “Is this even correct?” comes up far too often, and relying on users to fix things usually makes it worse. That fuzziness came up again and again - in how people write prompts, interpret results, and try (and fail) to improve things:

 * Three UX gaps: Specification, generalization, and comprehension each break in different ways and need different tooling
 * Sim environments beat live infra: Safer, faster, and more reliable for evals - if the agent believes the simulation
 * Prompt tweaking often backfires: Giving users too much control leads to worse performance and wasted time

Click for the insults, stay for the insight.

Video [https://go.mlops.community/660bub] || Spotify [https://go.mlops.community/wkqjpz] || Apple [https://go.mlops.community/e1cq1a]

Want to build your own multi-agent MVP?

Build Multi-Agent Applications - A Bootcamp [https://go.mlops.community/mav19june] gives you 1:1 mentorship and a step-by-step playbook. It's an affiliate link, clicking helps support the community, thanks.

[https://go.mlops.community/e1cq1a](https://go.mlops.community/e1cq1a)

[https://go.mlops.community/660bub](https://go.mlops.community/660bub)

[https://go.mlops.community/wkqjpz](https://go.mlops.community/wkqjpz)

## The Creator of FastAPI’s Next Chapter

Last time I chatted to Sebastián, I got flack for not asking the right questions, so I made sure to cover them this time.

We talked about how FastAPI grew out of his frustration deploying ML models. None of the existing tools offered the developer experience he needed, so he built one - leaning heavily on Pydantic for validation, parsing, and structure. He’s deliberate about abstractions: they need to save more effort than they cost to learn.

He shared how to get the most out of Pydantic:

 * TypedDict for inputs – clean autocomplete, no extra imports
 * Models for outputs – strong typing with minimal boilerplate
 * Editor feedback – better dev flow without added complexity

Click below to listen, then hit reply to tell me what I missed this time.

Video [https://go.mlops.community/8j0vum] || Spotify [https://go.mlops.community/dz8i02] || Apple [https://go.mlops.community/n2llf5]

[https://go.mlops.community/n2llf5](https://go.mlops.community/n2llf5)

[https://go.mlops.community/8j0vum](https://go.mlops.community/8j0vum)

[https://go.mlops.community/dz8i02](https://go.mlops.community/dz8i02)

## Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents

Can AI systems evolve on their own - rewriting and improving themselves without human redesign?

That’s the question at the heart of this month’s MLOps Community Reading Group on Thursday, June 26 (11 AM ET / 5 PM CET).

This paper [https://arxiv.org/abs/2505.22954] explores a system that rewrites its own code through evolutionary search, validating every change through empirical testing. It’s a big swing at building agents that can improve indefinitely.

We’ll cover:

 * How open-ended evolution drives autonomous agent development
 * How self-modification can be tested safely
 * What it might mean for future AI architectures

Join us in the #reading-group [https://go.mlops.community/3x3vy9] channel on the MLOps Community Slack. We meet monthly, come be part of it.


Register now [https://lu.ma/578el3ve]

[This paper](https://arxiv.org/abs/2505.22954)

## Job of the Week

[https://go.mlops.community/ce615g](https://go.mlops.community/ce615g)

[https://go.mlops.community/MLConfess](https://go.mlops.community/MLConfess)

## Hidden Gems

Parameter count has become the new vanity metric in AI, and it's actively stalling real progress.


Agree or disagree: we're chasing leaderboard clout over real-world impact?

Working on something tricky or planning ahead? Here’s how we can help - just hit reply:

 * Custom workshops tailored to your company’s needs
 * Hiring? I know some quality folks looking for a new adventure
 * Want to connect with someone tackling similar problems? I can introduce you

Thanks for reading, catch you next time!

---
Source: https://aaif.live/newsletters/mlopscommunity/2025-06-19-agents-of-chaos-debugging-llm-pipelines