--- title: "LLMs, YAMLs, and CI/CD: A legendary episode!" newsletter: "MLOps Community" date: 2024-11-21 source: https://aaif.live/newsletters/mlopscommunity/2024-11-21-llms-yamls-and-ci-cd-a-legendary-episode --- # LLMs, YAMLs, and CI/CD: A legendary episode! *Plus, AI Agents in Production panel on multimodality, complying with regulations, self-healing systems, MARS, synthetic data, integrating ML models, and testing some ideas.* *MLOps Community — Agentic AI Foundation, 2024-11-21* https://go.mlops.community/azuc4f Seems like someone hasn't seen what it’s like to get agents into production. They can catchup here [https://go.mlops.community/azuc4f]. ## We Can All Be AI Engineers and We Can Do It with Open Source Models We Can All Be AI Engineers and We Can Do It with Open Source Models Luke Marsden - CEO @ HelixML This episode is about the original LLM - the Legendary Luke Marsden! After a little trip down memory lane to kick things off, we got into how CI/CD principles can be applied to generative AI using YAML-based specifications from the AI Spec framework. This approach standardizes models, prompts, integrations, and tests, enabling reproducible, version-controlled workflows. A three-stage API integration pipeline - intent classification, API request construction, and response summarization -was a highlight, showcasing how LLMs orchestrate complex tasks like Jira integration. Automated evaluations validate outputs, catching regressions and ensuring production-readiness. This method bridges rapid prototyping and scalable deployment, empowering both technical and semi-technical teams to integrate AI into applications seamlessly, whether for productivity tools or embedding AI directly into products, with maintainability always in mind. And because you can't have too much of a good thing, don’t miss Luke’s upcoming webinar on Dec 2 [https://go.mlops.community/2bkeez], about Testing & CI for GenAI, where you’ll learn how to prototype, test, and deploy a GenAI app with automation. Both are well worth a listen - after all, you don’t get a nickname like LLM for nothing! Video [https://go.mlops.community/upwu1i] || Spotify [https://go.mlops.community/rjeyxr] || Apple [https://go.mlops.community/o924b3] ## Exploring AI Agents: Voice, Visuals, and Versatility Exploring AI Agents: Voice, Visuals, and Versatility AI Agents in Production // Panel ## Evaluating LLMs for Technical Compliance Evaluating LLMs for Technical Compliance With thanks to Pavol Bielik for their contribution. From insurance forms to mailbox checks for the HOA, you've got enough compliance without the EU adding more. This blog details how to evaluate LLMs for compliance with frameworks like the EU AI Act. It introduces COMPL-AI, an open-source tool that translates broad regulatory principles into measurable technical benchmarks. It explains how to install the framework, run evaluations on models such as GPT-Neo or HuggingFace’s options, and generate detailed compliance reports. It also highlights key considerations like technical requirements, evaluation metrics, and practical examples of benchmarking, including model transparency and interpretability. Only compliance left is to click below and read it. Read it here [https://go.mlops.community/z9ktrn] ## Hidden Gems ## Heal Yourself // Heal Yourself // Gem [https://go.mlops.community/74lmt3] // Song [https://go.mlops.community/h1fgo4] A look at how Netflix engineered a "self-healing" system to handle a severe concurrency bug and keep the cluster alive as CPU usage spiraled. Mars // Gem [https://go.mlops.community/rpucwy] // Song [https://go.mlops.community/21yomo] A paper presenting MARS, a framework combining variance reduction and preconditioned gradients to improve large model training. Synthesizer // Gem [https://go.mlops.community/a24dob] // Song [https://go.mlops.community/hr8ex6] An article that outlines synthetic data strategies for LLMs, addressing bias, diversity, and fidelity while showcasing applications in fine-tuning and evaluation. Go // Gem [https://go.mlops.community/dzk8sj] // Song [https://go.mlops.community/6z5v24] An article on integrating ML models into Go apps via a Python sidecar, using REST APIs to leverage frameworks like TensorFlow or PyTorch. ## Job of the Week [https://go.mlops.community/j6dqm7](https://go.mlops.community/j6dqm7) ## The Sandbox The Sandbox A little place to test some ideas There's been a lot of changes over the past few weeks - huge thanks to everyone who provided feedback! We've implemented what we could and are still working on a few other suggestions. To keep things organized we've got this experimental environment to test out ideas before they go into full production. Let us know what you think here [https://go.mlops.community/hx6agg] or email steve@mlops.community Slack Spotlights Highlighting some of the chat you might have missed An interesting question in #comuter-vision [https://go.mlops.community/aqtibs] about open solutions for pose detection, looking for models with more key points than MediaPipe offers. Suggestions included Papers with Code for 3D pose estimation, checking dataset licenses, and using YOLO for customizable keypoints. If you’re tackling something similar, or have tips to add, join in. There was a shoutout in #datascience [https://go.mlops.community/d0jms9] from someone getting into table structure recognition and looking for insights on metrics like BLEU-4, TEDS, DAR, and GriTS. If you’ve worked with these or have tips to share, it’d be great to jump in and lend a hand! In #discussions [https://go.mlops.community/2jxllp], someone asked for advice on frontend tools, aiming to avoid JavaScript/HTML and stick to Python. They’re leaning toward HTMX + CSS with FastAPI for scalable web apps. Share your tips or experience. Tech Teaser A mini MLOps mindbender Inspired by seeing Mario at the AI Agents in Production conference, you decide to create your own game. To make it more dynamic, you incorporate in-game AI that predicts player strategies, powered by a neural network. Doubling the input dimensions and output dimensions of its neural network doubles the number of neurons per layer. How many times more parameters are there now? Interested in partnering with us? Get in touch: partners@mlops.community Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ?view_as=subscriber], and podcast [https://home.mlops.community/public/content/] land. Oh yeah, and we are also on X [https://twitter.com/mlopscommunity] and LinkedIn [https://go.mlops.community/linkedin]. The MLOps Community newsletter is edited by Jessica Rudd [https://www.linkedin.com/in/jmrudd/]. --- Source: https://aaif.live/newsletters/mlopscommunity/2024-11-21-llms-yamls-and-ci-cd-a-legendary-episode