--- title: "Domino Effect: How Training Got Faster and Smarter" newsletter: "MLOps Community" date: 2024-12-19 source: https://aaif.live/newsletters/mlopscommunity/2024-12-19-domino-effect-how-training-got-faster-and-smarter --- # Domino Effect: How Training Got Faster and Smarter *Plus, governance challenges, a stocking stuffed with Gems, and the sandbox.* *MLOps Community — Agentic AI Foundation, 2024-12-19* https://mlops.community/https://mlops.community/ This is our final weekly newsletter of the year, so we’re leaving you with some inspiration for your office Christmas party playlist: * Jingle Bell ROC * Last Cluster * Feliz Navidata * Have Yourself a Model Little Christmas * Baby, It's Code Outside * The Christmas Song (Vectors Roasting on an Open Fire) * Santa Codes Is Coming to Town * Let it Flow! Let it Flow! Let it Flow! Pair it with our new t-shirt [https://go.mlops.community/7x7jms] and you’ll be the talk of the office party (for the right reasons). Keep an eye out for some seasonal special emails from us. Until then - Happy Holidays! ## Domino: Communication-Free LLM Training Engine Domino: Communication-Free LLM Training Engine Guanhua Wang // Senior Researcher @ Microsoft This time of year, it’s easy to get distracted by all the food and pies - pecan, apple, mince - but let’s stay focused. This is about Phi, not pie, and the Domino we’re talking about delivers speed, not slices. We explored Phi-3, a small language model built for efficiency, powered by high-quality data at its core. Guanhua emphasized the importance of rigorous data preprocessing, cleaning, and sourcing premium datasets. Post-training also got a spotlight, with customized datasets boosting model performance beyond what pre-training alone can achieve. Some highlights from DeepSpeed’s contributions to model training: * Zero Optimizer: A memory-efficient sharding strategy that splits model parameters across GPUs, significantly reducing overhead. * Domino: DeepSpeed’s latest innovation, which hides communication within computation, offering up to 1.3x speedup over NVIDIA’s Megatron in multi-node training. Quantization has taken center stage as the leading optimization technique for smaller models, outperforming distillation, which often compromises accuracy. And with future hardware improvements, like higher inter-node bandwidth, Domino’s benefits are poised to grow even further. Hungry for more? Click below to grab a pizza the Phi. Video [https://go.mlops.community/efllir] || Spotify [https://go.mlops.community/nx0y3f] || Apple [https://go.mlops.community/xqev5m] ## Governance for AI Agents with Data Developer Platforms Governance for AI Agents with Data Developer Platforms With thanks to Brij Mohan Singh, Travis Thompson, and Ritwika Chowdhury for their contribution. I'm looking forward to the day AI agents can handle all my Christmas presents, but I'm worried about how many Octonauts LEGO sets they'd order for my kids. It’s these kinds of governance challenges with autonomous AI agents that are the focus of this blog, as they evolve from simple tools to decision-making systems. It highlights critical risks such as: * Data mismanagement and unauthorized access. * Delayed or diffused impacts that can remain hidden for long periods. * Cascading failures in multi-agent systems, where small issues can escalate across networks. The blog looks at how a Data Developer Platform (DDP) addresses these challenges, emphasizing unified governance. Centralized policy decision points (PDP) and decentralized policy execution points (PEP) are explained as core mechanisms. Key features include: * Role-based access controls ensuring compliance and secure operations. * Secure credential storage to prevent leaks and adhere to best practices. * Real-time observability for tracking agent actions, maintaining accountability, and detecting anomalies. By integrating these governance practices into development workflows, DDPs make it easier to deploy AI agents securely while preserving transparency, compliance, and accountability. An easy gift to sort is the link below for you. Read it here [https://go.mlops.community/zd9p7k] ## Hidden Gems ## Job of the Week [https://go.mlops.community/ifroc5](https://go.mlops.community/ifroc5) ## The Snowbox The Snowbox Featuring: Slack Spotlight // Back to the Feature // Tech Teaser Trying out a few things here - let us know what you think here [https://go.mlops.community/hx6agg] or email steve@mlops.community Slack Spotlight Sharing some of the chat you might have missed An interesting thread in #llmops [https://go.mlops.community/gtgb86] about reducing hallucinations with Mistral. A chat about open source OCRs and which ones to use for parsing handwritten and types text from the same PNG in #mlops-questions-answered [https://go.mlops.community/wf2ypi] There's also a really useful thread in there aboout performing structured extraction on docx/PDFs with little to no images, but needing identification of clauses subclauses and some tables. Back to the Feature A highlight from last week Before investing your Christmas money, check out this episode on the current state of AI and MLOps investment, covering infrastructure, tooling, and applications. Aditya Naganath from Kleiner Perkins shared how sectors like data platforms have flourished, while others such as startups have struggled to break through, and highlighted where value is being created and where gaps remain. Video [https://go.mlops.community/KleinerPod] || Spotify [https://go.mlops.community/DelphinaAd] || Apple [https://go.mlops.community/KleinerApple] Tech Teaser A mini MLOps mindbender Last week's question: To get in the mood for Chistmas and help you binge watch all the Hallmark Christmas movies, you build an AI that recommends videos. Every new video requires your recommendation engine to compare it against all existing videos. If there are currently eee videos and nnn new ones are uploaded, how many comparisons are needed? Answer: Total Comparisons = e × n + (n × (n - 1)) / 2 This week's question: Santa Clause is embracing AI and using drones to deliver presents. His drone AI uses a graph where nodes are delivery locations and edges are routes. Adding one delivery point connects it to all existing nnn locations. If nnn is 10, how many new edges are added? Interested in partnering with us? Get in touch: partners@mlops.community Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ?view_as=subscriber], and podcast [https://home.mlops.community/public/content/] land. Oh yeah, and we are also on X [https://twitter.com/mlopscommunity] and LinkedIn [https://go.mlops.community/linkedin]. The MLOps Community newsletter is edited by Jessica Rudd [https://www.linkedin.com/in/jmrudd/]. --- Source: https://aaif.live/newsletters/mlopscommunity/2024-12-19-domino-effect-how-training-got-faster-and-smarter