--- title: "Run Rings Around Messy Data with Paranets" newsletter: "MLOps Community" date: 2025-01-30 source: https://aaif.live/newsletters/mlopscommunity/2025-01-30-run-rings-around-messy-data-with-paranets --- # Run Rings Around Messy Data with Paranets *Plus, is it fine to just finetune?, troubleshooting your AI agents, hidden gems, Slack spotlight, and the sandbox.* *MLOps Community — Agentic AI Foundation, 2025-01-30* https://go.mlops.community/hu3wds It's amazing to see how truely human AI is becoming. ## Collective Memory for AI on Decentralized Knowledge Graph 2 min read Collective Memory for AI on Decentralized Knowledge Graph Tomaz Levak // Founder, Core Developers of OriginTrail @ Trace Labs The saying goes, if you liked it, then you should have put a ring on it - so would buying yourself an Oura Ring be the ultimate act of self-affirmation? Just a thought I had after Tomaz shared how he and his co-founder prototyped integrating sleep data into a DKG during our chat on decentralized knowledge graphs (DKGs), their ability to enhance transparency, and how they enable flexible, verifiable data sharing. A key part of the discussion focused on paranets, specialized neighborhoods within the DKG that enable domain-specific applications: * Paranets provide tailored ontologies for specific use cases, such as enterprise data-sharing or decentralized science initiatives. * Users can define rules and data structures within these paranets, allowing seamless collaboration while maintaining private or verifiable data exchanges. * This flexibility supports scenarios from enterprise collaborations (e.g., private data within organizations) to open research networks driving scientific breakthroughs. DKGs’ design emphasizes interoperability while addressing scalability and trust concerns, making them a valuable tool across diverse domains. Have a listen - and don’t worry, I know you’ll like it. No ring required. Video [https://go.mlops.community/olya6t] || Spotify [https://go.mlops.community/l75x2b] || Apple [https://go.mlops.community/lh79b5] ## Navigating Machine Learning Careers: Insights from Meta to Consulting 2 min read Navigating Machine Learning Careers: Insights from Meta to Consulting Ilya Reznik // ML Engineering Thought Leader @ Instructed Machines There’s the trope about companies saying, “Get AI to do it” as a quick fix, but this episode with Ilya made me realize we’ve got our own version in the ML world: “Just fine-tune it.” We explored why fine-tuning LLMs doesn’t always deliver the expected ROI. While it’s sometimes necessary for specific structured outputs or domain-specific language, the process is expensive, time-consuming, and often produces inconsistent results. Prompt engineering or RAG can provide better results for tasks like JSON outputs or specialized applications, especially since fine-tuning can degrade performance on unrelated tasks. We also discussed improving evaluation and training practices, including: * Curriculum learning: Revisiting this staged approach to teach models foundational knowledge, improving data efficiency and reducing reliance on brute-force training. * Evaluation benchmarks: Many metrics fail to reflect real-world utility. High scores often mask poor practical performance, making dynamic and meaningful benchmarks essential. * Data limitations: With concerns about running out of high-quality training data, reinforcement learning and smarter data curation are gaining importance. As someone who quit Meta, it was also interesting hearing him talk about career development in ML, emphasizing the importance of adaptability, iterative learning, and navigating roles like staff engineer. Tune in for a fine time! Video [https://go.mlops.community/e0hw19] || Spotify [https://go.mlops.community/xlqqyy] || Apple [https://go.mlops.community/nuzcjf] ## Troubleshooting AI Agents: Advanced Data-Driven Techniques 1 min read Troubleshooting AI Agents: Advanced Data-Driven Techniques With thanks to Bartosz Mikulski for their contribution. James Bond is the world’s most famous secret agent, which just proves that no agent is perfect. This blog won’t save you if you’ve blown your cover behind enemy lines, but it does break down practical ways to troubleshoot and improve AI agent performance. It focuses on common issues and actionable fixes, like bad query generation, poor data retrieval, and techniques such as chain-of-thought prompting to keep agents on track. It also highlights the importance of good data collection: * Detailed Logging: Capture tool outputs, response times, and document relevancy to pinpoint problems and refine performance. * Tracking Metrics: Prioritize metrics like precision, faithfulness, and retrieval effectiveness to measure progress meaningfully. * Correlation and Context: Use correlation IDs to track interactions across the entire agent process, making it easier to link inputs, outputs, and user satisfaction. By combining these strategies with thoughtful prompt engineering and better retrieval methods, the blog offers a clear path to building smarter, more reliable AI agents. After reading, you’ll be anything but shaken or stirred. Read it here [https://go.mlops.community/noxuaa] ## Hidden Gems ## Deep Pockets // // Deep Pockets // Gem [https://go.mlops.community/2xlhd9] // Song [https://go.mlops.community/k7ovv9] An analysis of how DeepSeek managed to train AI models 30 times cheaper, highlighting breakthroughs in resource allocation, training infrastructure, and model efficiency. Rolling In The Deep // Gem [https://go.mlops.community/hizdkq] // Song [https://go.mlops.community/63hwlf] An exploration of DeepSeek-R1’s function-calling features, focusing on its architecture, accuracy improvements, and practical use cases for developers integrating LLMs with external systems. Can’t Think // Gem [https://go.mlops.community/4kanrf] // Song [https://go.mlops.community/95iqx3] A study exploring how reliance on AI tools affects cognitive offloading and critical thinking, analyzing generational differences and the potential risks of AI dependency in decision-making. Higher State Of Consciousness // Gem [https://go.mlops.community/5y0879] // Song [https://go.mlops.community/4zgdju] An exploration of data-conscious software engineering principles, offering insights into how developers can make better decisions around data modeling, storage, and processing to enhance application performance and robustness. ## Slack Spotlights 2 min read Slack Spotlights Stephen Oladele [https://go.mlops.community/733wfg] shares some of the chat you might have missed ## 🤖 Rethinking AI Agent Architecture This week in #llmops [https://mlops-community.slack.com/archives/C04T55KFV8S], Hamza Tahir started a critical discussion about how AI agents are being treated in production. His take? Treating agents like regular software components is a recipe for disaster. ⏩ Here’s the grab-and-go summary: AI agents are statistical systems, not just APIs. Treating them as such requires adopting proper MLOps practices, including evaluation rigor, observability, and tailored tools to manage their unique challenges. While off-the-shelf solutions like Langfuse or LangChain can help, the community largely agreed that custom-built solutions often provide more flexibility and control. 🔥 More detail on Hamza’s Hot Take: * AI agents aren’t traditional APIs: They’re statistical models, not deterministic software artifacts, yet they're being shipped like microservices. They need unique monitoring, evaluation, and governance processes. * Key challenges Hamza observed: Treating prompts like config files (they’re not). Relying on unit tests for semantic drift (ineffective). Lack of behavioral consistency checks and prompt versioning. * Here’s what’s needed: Behavioral evaluation pipelines. Semantic drift tracking. Robust edge case feedback loops. Version control for prompts. Gradual rollouts with close monitoring. 💬 Community Insights: * Médéric Hurier: "Agents require the same rigorous MLOps processes as ML models, including observability, versioning, and evaluation. * Laszlo Sragner: A basic CRUD app with a Python stack (FastAPI, Pydantic) is often sufficient. However, higher-level tools like LangChain can reduce coding but might not be worth the dependency for simpler use cases. * Bartosz Mikulski: Custom-built CRUD apps for tracking prompts, versions, and feedback offer flexibility and control. However, ensure easy data retrieval for analysis, as some tools make this unnecessarily difficult. * Zach Wallace: Observability remains a major challenge, and existing tools are far from sufficient. 🤔 Build vs. Buy Debate: While Hamza noted the value of tools like LangFuse for advanced tracking and feedback, Laszlo and others emphasized that rolling out custom solutions (CRUD apps with Prisma/Streamlit) provides flexibility without unnecessary abstraction. Got your own take? Join the conversation on #llmops [https://mlops-community.slack.com/archives/C04T55KFV8S/p1737725481128019] and share your experiences building and managing AI agents in production! 🚀 [#llmops](https://mlops-community.slack.com/archives/C04T55KFV8S) [the conversation on #llmops](https://mlops-community.slack.com/archives/C04T55KFV8S/p1737725481128019) ## Job of the Week [https://go.mlops.community/qea8z8](https://go.mlops.community/qea8z8) ## The Sandbox 1 min read The Sandbox A little place to test some ideas Trying out a few things here - let us know what you think here [https://go.mlops.community/hx6agg] or email steve@mlops.community Back to the Feature A highlight from last week A lot of people like living on the edge judging by the reraction to Krishna's episode! He discussed edge AI innovations, Qualcomm’s AI Hub for streamlined model deployment, adaptive chip compatibility, and real-world applications like sports tracking and on-device LLMs, highlighting efficiency and versatility. Video [https://go.mlops.community/7rbtr1] || Spotify [https://go.mlops.community/lwvs8b] || Apple [https://go.mlops.community/l4nazw] Gatewaze Grooves Sharing music picks from our latest members through the Gatewaze [https://go.mlops.community/GwazeEmail]. It's amazing to see just how diverse the musial taste of the community is! Not only does it cross borders, with KK [https://go.mlops.community/wqp39d] from India, La Oreja de Van Gogh [https://go.mlops.community/vh2ii7] from Spain, Dunsin Oyekan [https://go.mlops.community/q91bku] from Nigeria, it also crosses genres from classic pop with Taylor Swift [https://go.mlops.community/5znq4c] to Dutch-Turkish psychedelic rock from Altın Gün [https://go.mlops.community/khp23t]. Enjoy the updated list here [https://go.mlops.community/Grooves], and I’ll close with this video [https://go.mlops.community/gud1lq] that joins genres and place! Tech Teaser A mini MLOps mindbender Inspired by the Gatewaze Grooves feature, you create a smart assistant that builds a playlist by clustering songs based on features like tempo and mood. It generates a final similarity matrix that stores only one similarity score per song pair. If you double the total number of songs from n to 2n and also double the number of features, by what factor does the stored matrix increase in size? Click here [https://go.mlops.community/TechTeasers] for the answer. Interested in partnering with us? Get in touch: partners@mlops.community Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ?view_as=subscriber], and podcast [https://home.mlops.community/public/content/] land. Oh yeah, and we are also on X [https://twitter.com/mlopscommunity] and LinkedIn [https://go.mlops.community/linkedin]. The MLOps Community newsletter is edited by Jessica Rudd [https://www.linkedin.com/in/jmrudd/]. --- Source: https://aaif.live/newsletters/mlopscommunity/2025-01-30-run-rings-around-messy-data-with-paranets