--- title: "AI infrastructure: What’s winning, what’s lagging, and why" newsletter: "MLOps Community" date: 2024-12-12 source: https://aaif.live/newsletters/mlopscommunity/2024-12-12-ai-infrastructure-what-s-winning-what-s-lagging-and-why --- # AI infrastructure: What’s winning, what’s lagging, and why *Plus, streamlining your workflow, a bumper edition of Gems, and the sandbox.* *MLOps Community — Agentic AI Foundation, 2024-12-12* https://go.mlops.community/7x7jms Struggling for gift ideas? We've got something for everyone - and I do mean everyone - our new t-shirt [https://go.mlops.community/7x7jms]. And in the spirit of gifting, we've got you $60 off Eric Riddoch's 6-week Python/AWS Cloud Engineering bootcamp [https://go.mlops.community/yxzgyd] starting in January. Use code MLOPSCOMMUNITY at checkout for hands-on labs, live Q&A sessions, and lifetime access to materials. ## AI's Next Frontier AI's Next Frontier Aditya Naganath // Principal @ Kleiner Perkins If your aunt slips you some cash this Christmas, tune in before deciding where to spend it. Aditya shared his take on the current state of AI and MLOps investment, covering infrastructure, tooling, and applications. Some sectors like data platforms have flourished, while others such as startups have struggled to break through. He highlighted where value is being created and where gaps remain: * Data Platforms vs. MLOps: Data platforms thrive on growing enterprise data needs, while many MLOps tools struggle due to enterprise preferences for in-house solutions. * Inference Demand: Inference workloads now drive significant revenue, overtaking training in importance and scale. * Infrastructure Innovation: Compute and networking optimizations are critical to reducing costs and improving efficiency, especially for large-scale training and inference. Despite progress, challenges such as GPU inefficiencies and network optimization underscore the need for innovation. Aditya believes the future hinges on solving these issues and expanding AI applications to deliver tangible results. Have a listen to make sure your aunt’s $5 is well spent. Video [https://go.mlops.community/KleinerPod] || Spotify [https://go.mlops.community/DelphinaAd] || Apple [https://go.mlops.community/KleinerApple] ## The Ultimate Must-Have for MLOps The Ultimate Must-Have for MLOps With thanks to Sophia Rowland for their contribution. Managing your workflow can feel like pulling off Christmas dinner - juggling everything to bring it all together seemlessly. This isn’t a recipe blog with a whole life story attached, but it does show how to bring order to your MLOps kitchen. It explores how to streamline MLOps by combining tools into a unified platform. It highlights the value of version control, where every model update is tracked with timestamps, user histories, and detailed metadata. Orchestration workflows keep processes running smoothly, while the model registry organizes assets with key metrics. Deployment options include: * Serving models as REST APIs. * Containerizing for AWS, Azure, and other platforms. * Batch scoring data for workflows, dashboards, or reporting. The blog also covers monitoring tools to track performance and data drift. Pipelines support experimentation, from data preparation to hyperparameter tuning. Built-in responsible AI features, like bias detection and privacy safeguards, ensure ethical and reliable outcomes. Click here [https://go.mlops.community/polf0n] to sink your teeth into it. ## Hidden Gems ## Behind the Mask // // Behind the Mask // Gem [https://go.mlops.community/RedditChatPromptGem] // Song [https://go.mlops.community/MaskSongGem] A Reddit post about accidentally discovering a prompt revealed the rules ChatGPT operates under, and the original shared chat [https://go.mlops.community/ChatPromptRulesGem]. Use Me // Gem [https://go.mlops.community/tqpnwm] // Song [https://go.mlops.community/UseMeGem] A collection of real-world LLMOps implementations, featuring detailed summaries and technical notes to make them easy to understand and apply. P.E.T. // Gem [https://go.mlops.community/+UberGem1] // Song [https://go.mlops.community/PETGem] Uber’s Prompt Engineering Toolkit: a centralized system for designing, managing, and deploying prompt templates for LLMs with version control and collaboration tools. Streamline // Gem [https://go.mlops.community/623l5b] // Song [https://go.mlops.community/StreamlineGem] A Python framework for streamlining the integration of AI featuring type-safe interaction, structured response validation, and support for multiple LLMs. Design Of The Future // Gem [https://go.mlops.community/4dgndh] // Song [https://go.mlops.community/DesignFutureGem] The first in a five-part series on the future of AI agents, exploring the shift to 'agent-responsive design' and how it’s transforming platforms for seamless AI interaction. Plug in Baby // Gem [https://go.mlops.community/StanfordWebinar] // Song [https://go.mlops.community/PlugInBabyGem] A webinar from Stanford discussing how AI systems built with multiple interacting components can achieve superior results compared to standalone models. Part of the Process // Gem [https://go.mlops.community/GoogleKPIsBlog] // Song [https://go.mlops.community/PartoftheProcessGem] A guide from Google on KPIs for genAI, covering model quality, operational efficiency, adoption, and business impact, with metrics to track success and maximize ROI. ## Job of the Week [https://go.mlops.community/2tlnpa](https://go.mlops.community/2tlnpa) ## The Sandbox The Sandbox Featuring: Slack Spotlight // Back to the Feature // Tech Teaser Trying out a few things here - let us know what you think here [https://go.mlops.community/hx6agg] or email steve@mlops.community Slack Spotlight Sharing some of the chat you might have missed Can you picture yourself helping someone? There's an interesting question on controlled image generation and augmenting conventionally generated charts with generative AI in #generative-ai [https://go.mlops.community/GenAIslack]. Got thoughts or ideas? Jump in and share! Over in #mlops-questions-answered [https://mlops-community.slack.com/archives/C015J2Y9RLM] someone's asked what tools or frameworks people are using to build agentic systems, especially with so many new options emerging. Head over to share your top pick! There's also a good thread in there on tackling a Chessboard Analysis problem in Computer Vision - from handling orientations and lighting variations to managing distortions. Back to the Feature A highlight from last week Last week's episode with Vincent Moens made it clear how things can go wrong when you make assumptions, so I won't assume you listened. Instead, I'll just flag it again here and say it explored pin memory misconceptions, PyTorch optimization tips, and TensorDict’s modularity. Video [https://go.mlops.community/lnfvgj] || Spotify [https://go.mlops.community/48qvgr] || Apple [https://go.mlops.community/cdyzpm] Tech Teaser A mini MLOps mindbender Last week's question: Insired by the openings to the podcasts, you’re training a model to recommend coffee. Each coffee shop adds 3 unique features to the dataset, but doubling the number of shops also doubles the number of users in your training data. How does the size of your dataset grow? Answer: The dataset size grows quadratically because it is the product of the number of features (3 per shop) and the number of users (proportional to the number of shops). This week's question: To get in the mood for Chistmas and help you binge watch all the Hallmark Christmas movies, you build an AI that recommends videos. Every new video requires your recommendation engine to compare it against all existing videos. If there are currently eee videos and nnn new ones are uploaded, how many comparisons are needed? Interested in partnering with us? Get in touch: partners@mlops.community Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ?view_as=subscriber], and podcast [https://home.mlops.community/public/content/] land. Oh yeah, and we are also on X [https://twitter.com/mlopscommunity] and LinkedIn [https://go.mlops.community/linkedin]. The MLOps Community newsletter is edited by Jessica Rudd [https://www.linkedin.com/in/jmrudd/]. --- Source: https://aaif.live/newsletters/mlopscommunity/2024-12-12-ai-infrastructure-what-s-winning-what-s-lagging-and-why