--- title: "What if prompts worked like real code?" newsletter: "MLOps Community" date: 2025-06-05 source: https://aaif.live/newsletters/mlopscommunity/2025-06-05-what-if-prompts-worked-like-real-code --- # What if prompts worked like real code? *Plus - product metrics as LLM evals, when prompt deployment goes wrong, World Tour dates, ML Confessions, and Hidden Gems.* *MLOps Community — Agentic AI Foundation, 2025-06-05* ## $10 billion? Would be cheaper to make it out of flatpack and call it Billy Bitskåp. ## Getting AI Apps Past the Demo Eight open tabs of LangChain docs, flaky prompt outputs, unrealistic deadlines, lack of coffee... there are so many reasons AI apps don’t make it to production. Vaibhav told me how BAML helps with (at least) a couple of those by treating prompts more like real code: structured, testable, and easier to debug. That shift helps teams build more reliable systems without killing iteration speed. It’s built to support smoother workflows through: * Live previews – watch prompts update as you write * Instant test runs – check outputs without leaving your editor * Debuggable views – token-level highlights and prompt diffs I'll give you a prompt to click below to listen. Video [https://go.mlops.community/1h6sac] || Spotify [https://go.mlops.community/x2ebl3] || Apple [https://go.mlops.community/rfaun3] [https://go.mlops.community/rfaun3](https://go.mlops.community/rfaun3) [https://go.mlops.community/1h6sac](https://go.mlops.community/1h6sac) [https://go.mlops.community/x2ebl3](https://go.mlops.community/x2ebl3) ## GenAI in Games, 3D & VFX ## UPCOMING VIRTUAL CONFERENCE JUNE 10 – 10:00–11:30 AM PDT / 19:00–20:30 CEST https://go.mlops.community/aigames MLOps Community and Tulip are hosting a live event with studios and builders using GenAI in production to reshape workflows across pre-vis, camera work, mesh generation, and more. Talks cover: * Fine-tuning small models and running agent-based systems in creative pipelines * AI-assisted animation: from custom keyframes to full motion * Legal Q&A on IP, contracts, and the risks of AI-generated content * Real-world R&D wins (and friction) from production teams Not your usual use case roundup - but packed with lessons about building GenAI tooling and navigating deployment in complex pipelines. Register Here [https://go.mlops.community/aigames] ## Product Metrics are LLM Evals Evals can be humbling - a bit like thinking you’re nailing Duolingo, then your six-year-old casually switches languages and leaves you behind. Raza argued that the best evals are just product metrics in disguise - what really matters is whether the user achieved their goal. In practice, that means building evals around real behavior, not just static labels. For example, teams are using: * Behavior-driven signals: Tracking if users copy text, edit it, or follow up with an action * Prompt iteration workflows: Letting domain experts tweak prompts without redeploying code * CI-integrated evals: Automatically gating releases based on test case performance I'm sure this episode will match your eval expectations when you click below and listen. Video [https://go.mlops.community/i992q6] || Spotify [https://go.mlops.community/4d06qt] || Apple [https://go.mlops.community/7a2tkh] [https://go.mlops.community/7a2tkh](https://go.mlops.community/7a2tkh) [https://go.mlops.community/i992q6](https://go.mlops.community/i992q6) [https://go.mlops.community/4d06qt](https://go.mlops.community/4d06qt) THE LOWEST BUDGET, HIGHEST SIGNAL AI AGENT NON-CONFERENCE CONFERENCE San Francisco - Done! [https://go.mlops.community/is60wo] The tour continues: * Austin [https://lu.ma/53l71vrp] - June 11 * Cape Town [https://lu.ma/x278l3xt] - June 12 * Miami [https://lu.ma/3m9e0na3] - June 18 * Amsterdam [https://lu.ma/m7m5847s] - July 9 Still more dates to be announced, and if you want to get involved, just let us know [https://tally.so/r/mBgW25]. ## Job of the Week [https://go.mlops.community/ajg70q](https://go.mlops.community/ajg70q) [https://go.mlops.community/MLConfess](https://go.mlops.community/MLConfess) ## Prompt Deployment Goes Wrong: xAI Grok's obsession with White Genocide With his black eye and K-hole trance perfectly normal behaviour, Musk’s latest Oval Office drop-in seemed to mirror xAI’s recent state. A late prompt edit caused Grok to inject political framing into neutral queries - something better ops could have flagged. The post makes a case for treating prompts like code: versioned, tested, and cautiously deployed. It adds: * Progressive rollouts: Shadow and canary tests to catch regressions early. * Beyond human feedback: Include automated and long-term reliability checks. * Prompt changes as high-risk deploys: They need real safeguards. Save yourself slipping into an avoidable hole and read the blog instead. Read it here [https://go.mlops.community/994bz7] [Read it here](https://go.mlops.community/994bz7) ## Hidden Gems Working on something tricky or planning ahead? Here’s how we can help - just hit reply: * Custom workshops tailored to your company’s needs * Hiring? I know some quality folks looking for a new adventure * Want to connect with someone tackling similar problems? I can introduce you Thanks for reading, catch you next time! --- Source: https://aaif.live/newsletters/mlopscommunity/2025-06-05-what-if-prompts-worked-like-real-code