---
title: "Putting it all out there: Evaluating your outputs"
newsletter: "MLOps Community"
date: 2024-10-24
source: https://aaif.live/newsletters/mlopscommunity/2024-10-24-putting-it-all-out-there-evaluating-your-outputs
---

# Putting it all out there: Evaluating your outputs

*Plus, making the most of your memory, optimization mini summit, and hidden gems.*

*MLOps Community — Agentic AI Foundation, 2024-10-24*

Enjoy the view - everyone needs a break now and then.

## How to Systematically Test and Evaluate Your LLMs Apps // Gideon Mendels // MLOps Podcast #269

People who’ve seen Megalopolis, Borderlands, or Joker: Folie à Deux know that hard work in production means little if the final output falls short.

This is something Gideon knows through his work evaluating LLM and Gen AI outputs. We talked about traditional metrics like accuracy and F1 scores, plus newer ones like using another LLM as a judge, and how ongoing post-deployment evaluation sets LLMs apart from traditional ML. He also shared how Opik, his open-source tool, tracks pipelines and configurations to boost reproducibility. We touched on data scientists learning engineering, but Gideon flipped it, saying software engineers working with LLMs need to embrace data science methods to deal with the unpredictability.

This episode? A blockbuster of an output.

Video [https://go.mlops.community/pgcv1s] || Spotify [https://go.mlops.community/sro045] || Apple [https://go.mlops.community/14ponr]

[https://go.mlops.community/14ponr](https://go.mlops.community/14ponr)

[https://go.mlops.community/pgcv1s](https://go.mlops.community/pgcv1s)

[https://go.mlops.community/sro045](https://go.mlops.community/sro045)

## Boosting LLM/RAG Workflows & Scheduling w/ Composable Memory and Checkpointing // Bernie Wu // MLOps Podcast #270

Boosting LLM/RAG Workflows & Scheduling w/ Composable Memory and Checkpointing // Bernie Wu // MLOps Podcast #270

This chat made me think of the photo of Margaret Hamilton [https://go.mlops.community/ambwbm] standing next to the Apollo code, and what if they only had a post-it note to store it all?

That’s because Bernie pointed out how memory scarcity is limiting today’s AI systems, like how underutilized GPUs can be bottlenecked by insufficient memory. His team is working on elastic memory solutions to optimize resource usage, especially in cases where GPU utilization drops as low as 30%. We also discussed the need for advanced schedulers and memory composability, essential for scaling AI workloads efficiently.

Use your memory wisely and remember to click below to listen!

Video [https://go.mlops.community/k74lv4] || Spotify [https://go.mlops.community/psmle3] || Apple [https://go.mlops.community/bj0jt5]

[https://go.mlops.community/bj0jt5](https://go.mlops.community/bj0jt5)

[https://go.mlops.community/k74lv4](https://go.mlops.community/k74lv4)

[https://go.mlops.community/psmle3](https://go.mlops.community/psmle3)

## Upcoming MLOps Community Mini-Summit

https://go.mlops.community/lwmjkq

This Halloween, don't let terrifying AI costs send shivers down your spine!

Join us on October 30 for a free mini-summit, "Productionalizing AI: Driving Innovation with Cost-Effective Strategies." We'll dig into tricks for treating your production pipeline better—from compute optimization to deploying lean, efficient models that deliver without the financial fright.


Sign up now [https://go.mlops.community/iw7q2e] for scarily good savings!

## Job of the Week

[https://go.mlops.community/38j158](https://go.mlops.community/38j158)

## Hidden Gems

Interested in partnering with us? Get in touch: partners@mlops.community

Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://go.mlops.community/mgNff7], and podcast [https://go.mlops.community/ttftlf] land. Oh yeah, and we are also on X [https://go.mlops.community/twitter]. The MLOps Community newsletter is edited by Jessica Rudd [https://go.mlops.community/cetjhj].

---
Source: https://aaif.live/newsletters/mlopscommunity/2024-10-24-putting-it-all-out-there-evaluating-your-outputs