--- title: "What's up doc? šŸ°šŸ„•šŸ“„" newsletter: "MLOps Community" date: 2024-05-30 source: https://aaif.live/newsletters/mlopscommunity/2024-05-30-what-s-up-doc --- # What's up doc? šŸ°šŸ„•šŸ“„ *Plus, why being exactly wrong is good, the great migration, I choo choo choose you, and hidden gems.* *MLOps Community ā€” Agentic AI Foundation, 2024-05-30* ## Just when we Started to Solve Software Docs, AI Blew Everything Up // Dave Nunez // MLOps Podcast #235 Orddocsder. If that looks strange, it's because itā€™s been a while since youā€™ve seen docs in order. Which is why itā€™s great to have Dave on talking about creating effective developer docs, as well as Kuba on as co-host, as heā€™s a documentation dynamo. We talk about the essentials, like focusing on strategic structuring, simplicity, and getting feedback to create a positive user experience. Thereā€™s also plenty of actionable insights, like annotating templates with examples, organizing user journey phases with ā€˜escape hatches,ā€™ fostering a strong writing culture, and the impact of AI on practices. Give it a listen and get your docs in a row. šŸ¦†šŸ¦†šŸ¦† Video [https://home.mlops.community/home/videos/retrieval-augmented-generation] || Spotify [https://open.spotify.com/episode/6Q7qRUgMqQgzT86oJuU60o?si=_4hFRjIYQ5G4JV5JVthPpA] || Apple [https://podcasts.apple.com/us/podcast/retrieval-augmented-generation/id1505372978?i=1000655962869] [https://podcasts.apple.com/us/podcast/retrieval-augmented-generation/id1505372978?i=1000655962869](https://podcasts.apple.com/us/podcast/retrieval-augmented-generation/id1505372978?i=1000655962869) [https://home.mlops.community/home/videos/retrieval-augmented-generation](https://home.mlops.community/home/videos/retrieval-augmented-generation) [https://open.spotify.com/episode/6Q7qRUgMqQgzT86oJuU60o?si=_4hFRjIYQ5G4JV5JVthPpA](https://open.spotify.com/episode/6Q7qRUgMqQgzT86oJuU60o?si=_4hFRjIYQ5G4JV5JVthPpA) ## Engineerā€™s Guide to Automatically Identifying and Mitigating Data Leakage by LatticeFlow AI https://latticeflow.ai/news/engineers-guide-to-data-leakage/?utm_source=mlops&utm_medium=email&utm_campaign=data+leakage What do the movie The Perfect Score (2004), LLMs passing the bar exam, and a YOLO model correctly detecting objects have in common? They all achieve impressive results due to data leakage, not true generalization. Whether it's students stealing exam questions or models trained on overlapping datasets, the outcome is the same: misleadingly high performance. Explore the root causes of data leakage, how to spot it, and how to automate safeguards to ensure data integrity and build robust systems. Learn about: * Types of data leakage: exact duplicates, augmentation duplicates, semantic duplicates, and more. * Automated checks for detecting data leakage across various data modalities. * Real-world examples showing how data leakage can degrade performance by up to 50%, as noted in an upcoming CVPR research paper. Ensure your models are reliable and robust with our expert strategies. Don't let data leakage compromise your hard work. Join us at CVPRā€™24 [https://latticeflow.ai/events/cvpr-conference/?utm_source=mlops&utm_medium=email&utm_campaign=cvpr] in Seattle to learn more about how to operationalize your data and model quality. Learn More [https://latticeflow.ai/news/engineers-guide-to-data-leakage/?utm_source=mlops&utm_medium=email&utm_campaign=data+leakage] LatticeFlow AI is an award-winning, ETH Zurich spin-off founded in 2020, providing the first scalable platform that guides AI teams through the AI lifecycle by automatically finding and fixing data and model issues. ## Managing Small Knowledge Graphs for Multi-agent Systems // Tom Smoker // MLOps Podcast #236 There are plenty of times in my life Iā€™ve been "exactly wrong." I just wish I knew then that it was a good thing! Tom Smoker introduces the mind-bending concept of being "exactly wrong" rather than "exactly right," which he explains better than I can here, so itā€™s worth the watch for that alone. He also chats about the importance of knowledge graphs in making RAG systems more reliable, the need for context-specific understanding, and warns about over-optimizing RAG systems. We also get into the challenges of building reliable chatbots, the significance of structured grounding, creating deterministic AI systems, and developing accurate and reliable agents within multi-agent systems. I will say I know itā€™s exactly right to give it a listen. Video [https://home.mlops.community/home/videos/open-standards-make-mlops-easier-and-silos-harder] || Spotify [https://open.spotify.com/episode/7a9fUqZkZzMQ8coQxwuPS6?si=k2oFY-dDQoay1VxxBhVWEg]|| Apple [https://podcasts.apple.com/us/podcast/open-standards-make-mlops-easier-and-silos-harder/id1505372978?i=1000656298533] [https://podcasts.apple.com/us/podcast/open-standards-make-mlops-easier-and-silos-harder/id1505372978?i=1000656298533](https://podcasts.apple.com/us/podcast/open-standards-make-mlops-easier-and-silos-harder/id1505372978?i=1000656298533) [https://home.mlops.community/home/videos/open-standards-make-mlops-easier-and-silos-harder](https://home.mlops.community/home/videos/open-standards-make-mlops-easier-and-silos-harder) [https://open.spotify.com/episode/7a9fUqZkZzMQ8coQxwuPS6?si=k2oFY-dDQoay1VxxBhVWEg](https://open.spotify.com/episode/7a9fUqZkZzMQ8coQxwuPS6?si=k2oFY-dDQoay1VxxBhVWEg) ## Job of the Week ## Vertex AI has Some Limits and I Reached All of Them // Adam Hjerpe // IRL #79 Stockholm ## MLOps Community IRL Meetup As summer approaches, the wonders of nature herald the great migrations; birds, butterflies, fish, and time series modeling from Spark to GCP. Adam shares his experience using Vertex AI to orchestrate machine learning flows with Kubeflow and TensorFlow extensions. Challenges like pipeline artifact limits were managed with custom training jobs and a "fan in, fan out" strategy to control data write rates to BigQuery. He also covers building recommendation systems, identifying model drivers, handling champion models, hyperparameter tuning, retraining frequency, and storing models in GCS, with considerations for vendor lock-in. All that's missing is a David Attenborough narration over the top of it. Watch it here [https://www.youtube.com/watch?v=4nOFGJBXM48] [Watch it here](https://www.youtube.com/watch?v=4nOFGJBXM48) Develop and train large models cost-efficiently with Metaflow and AWS Trainium [https://home.mlops.community/home/blogs/develop-and-train-large-models-cost-efficiently-with-metaflow-and-aws-trainium] It would be great if you could train your models as fast as a montage from Rocky. Unfortunately, thatā€™s not possible, but this blog does looks at how to use AWS Trainium and Metaflow to optimize training models. Metaflow simplifies ML development and integrates with AWS Trainium, which is designed for efficient deep learning training. The post walks you through setting up an environment using AWS services like SageMaker and managing training workflows in Metaflow to use Trainium instances. Key points include smooth scaling, cost efficiency, and using Trainiumā€™s high-performance capabilities to speed up model training while keeping control over infrastructure and deployment. Have a read of this blog instead of running up those steps. With thanks to Ville Tuulos, Eddie Mattia, Vidyasagar Ananthan, Diwakar Bansal, Sadaf Rasool, and Scott Perry for their contribution. [Develop and train large models cost-efficiently with Metaflow and AWS Trainium](https://home.mlops.community/home/blogs/develop-and-train-large-models-cost-efficiently-with-metaflow-and-aws-trainium) ## Hidden Gems ## IRL Meetups Stockholm [https://www.meetup.com/stockholm-mlops-community/events/300881538/] - May 30 Frankfurt [https://www.meetup.com/frankfurt-mlops-community/events/300877018/?utm_medium=referral&utm_campaign=share-btn_savedevents_share_modal&utm_source=link] - June 12 Denver [https://www.meetup.com/denver-mlops-community/events/300432149/?utm_medium=referral&utm_campaign=share-btn_savedevents_share_modal&utm_source=link] - June 12 San Francisco [https://www.meetup.com/mlops-community/events/300375509/?utm_medium=referral&utm_campaign=share-btn_savedevents_share_modal&utm_source=link] - June 25 [Stockholm](https://www.meetup.com/stockholm-mlops-community/events/300881538/) Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ?view_as=subscriber], and podcast [https://home.mlops.community/public/content/] land. Oh yeah, and we are also on X [https://twitter.com/mlopscommunity]. The MLOps Community newsletter is edited by Jessica Rudd [https://www.linkedin.com/in/jmrudd/]. --- Source: https://aaif.live/newsletters/mlopscommunity/2024-05-30-what-s-up-doc