From Earth Day to Earth Data - saving the planet with AI — MLOps Community

If you found this more traumatizing than funny, don’t worry, we can help! Our new course gives a thorough overview of how to set up, use, and automate a bunch of Python tools that are vital for any project. Your guide through this tangle of tech tools is Laszlo Sragner [https://www.linkedin.com/in/laszlosragner/], who's got bags of experience, so you know it's going to be packed with great advice. Learn how to change the default Python version, run tests remotely with GitHub Actions, use a debugger in VSCode, and work with code directly in Jupyter. Plus, you’ll get a set of default configurations that you can easily add to your own projects. So come and learn with Laszlo [https://learn.mlops.community/courses/languages/python/project-essentials/] and get your time back! ## Beyond AGI, Can AI Help Save the Planet? // Patrick Beukema // MLOps Podcast #225 I try to do my bit for the environment, but it’ll take more than just recycling old jokes. Thankfully, Patrick and the folks at AI2 are doing a whole lot more. He gave me a breakdown of the four areas of focus at AI2, what it’s like to work in a non-profit, the unique metrics used by nonprofits compared to businesses, and the importance of being transparent about their models. We talked about how they use satellite imagery for model validation and the necessity for continuous iteration and improvement for these models. One of the key takeaways for me was how quick they look to apply it to real-world situations – as he says, when it comes to the issues the planet is facing there’s no time to waste. But it also gives a chance to iterate on models based on user feedback, highlighting the integral role users play in the model development process. I am a little confused with the saying, "There's only one planet Earth" though, as I'm sure the BBC released Planet Earth III not that long ago? Any way, don't watch that, watch this! And then watch that, it's very good. Video [https://home.mlops.community/home/videos/beyond-agi-can-ai-help-save-the-planet] || Spotify [https://open.spotify.com/episode/5fDNIzZ4fbVJLNIX6qlvui?si=1262be0b11e6407a] || Apple [https://podcasts.apple.com/us/podcast/beyond-agi-can-ai-help-save-the-planet-patrick-beukema-225/id1505372978?i=1000653011203] [https://podcasts.apple.com/us/podcast/beyond-agi-can-ai-help-save-the-planet-patrick-beukema-225/id1505372978?i=1000653011203](https://podcasts.apple.com/us/podcast/beyond-agi-can-ai-help-save-the-planet-patrick-beukema-225/id1505372978?i=1000653011203) [https://home.mlops.community/home/videos/beyond-agi-can-ai-help-save-the-planet](https://home.mlops.community/home/videos/beyond-agi-can-ai-help-save-the-planet) [https://open.spotify.com/episode/5fDNIzZ4fbVJLNIX6qlvui?si=1262be0b11e6407a](https://open.spotify.com/episode/5fDNIzZ4fbVJLNIX6qlvui?si=1262be0b11e6407a) ## The Rise of Modern Data Management // Chad Sanderson // MLOps Podcast #226 I think Taylor Swift must have had a sneak preview of this episode. The Tortured Poets Department sounds like it could be the section for data engineers in Chad's library metaphor for the old data model. He made a good comparison where data is gathered, curated, and controlled by a single central authority (like a librarian), with users (like library users) accessing and using the data under the oversight of that authority, but suggests that this "library" model is shifting towards a more federated approach. Chad also suggests that data engineering shouldn't primarily involve debugging. He recommends upgrading data infrastructure to better serve operational demands in addition to analytics, treating data as a product. His strategies include human involvement in data quality checks, early issue detection, and implementing data contracts for improved management. He also highlights the importance of systems that easily track data changes and enable more autonomous development. Drawing from his extensive experience and current industry trends, Chad aims to assist businesses in various sectors. And to those who say the intro to this was a ridiculous stretch to show I've got my finger on the cultural pulse, I'd have to hold my hands up and say Track 9. 🙌 Video [https://home.mlops.community/home/videos/the-rise-of-modern-data-management] || Spotify [https://open.spotify.com/episode/6vHyv3c6Gm5uTAJw3vKGAV?si=euihLN5rTn2mgYMXQ2DwbQ] || Apple [https://podcasts.apple.com/us/podcast/the-rise-of-modern-data-management-chad-sanderson-226/id1505372978?i=1000653374716] [https://podcasts.apple.com/us/podcast/the-rise-of-modern-data-management-chad-sanderson-226/id1505372978?i=1000653374716](https://podcasts.apple.com/us/podcast/the-rise-of-modern-data-management-chad-sanderson-226/id1505372978?i=1000653374716) [https://home.mlops.community/home/videos/the-rise-of-modern-data-management](https://home.mlops.community/home/videos/the-rise-of-modern-data-management) [https://open.spotify.com/episode/6vHyv3c6Gm5uTAJw3vKGAV?si=euihLN5rTn2mgYMXQ2DwbQ](https://open.spotify.com/episode/6vHyv3c6Gm5uTAJw3vKGAV?si=euihLN5rTn2mgYMXQ2DwbQ) ## Innovate, experiment, and deploy faster than ever before with Outerbounds https://outerbounds.com/blog/obp-on-all-clouds/ Outerbounds, the team behind Metaflow [http://github.com/netflix/metaflow], is now available on all major clouds: hyper scalers like - AWS, Azure, and GCP and GPU clouds like Coreweave, NVIDIA DGX Cloud, and more! Deploy Outerbounds with just a few clicks in your existing cloud infrastructure, no matter your preference. Here's what this means for you: * Effortless Multi-Cloud Operations: Utilize a multi-cloud strategy effectively, maintaining security and cost efficiency. * Focus on Innovation, Not Infrastructure: Streamline your workflow and eliminate infrastructure headaches. * Data to Deployment, Simplified: Maintain agility throughout your projects and accelerate time-to-market. Already running ML/AI on your cloud account? Start using the open-source Metaflow, and you’ll be in good company, joining the likes of Netflix and others known for their long-term ML efforts. Metaflow was designed to help ML/AI developers efficiently use cloud resources like scalable storage and computing. It provides a dedicated platform to quickly and independently develop systems. Ready to unlock the full potential of your ML projects? Check out our blog [https://outerbounds.com/blog/obp-on-all-clouds/] to learn more about leveraging Outerbounds in your multi-cloud environment and discover how it can simplify your path to success. ## Job of the Week [https://www.ycombinator.com/companies/zep-ai/jobs/8mZPsSW-developer-relations-engineer](https://www.ycombinator.com/companies/zep-ai/jobs/8mZPsSW-developer-relations-engineer) ## Stockholm 2024 Community Kick-Off Panel // Rebecka Storm, Anna Maria Modée & Francesca Carminati // IRL #74 Stockholm ## MLOps Community IRL Meetup I propose the ML paradox: the complexity of keeping ML projects simple. This panel looks at the balance between these and looks at the risks of overcomplexity and technical debt, the importance of system validation before scaling, and the use of collaborative tools for complexity management. It also discusses managing ML experiments, traceability, social and ethical implications of AI, the need for clear communication with non-technical teams, and aligning ML projects with business goals. Nothing complex about simply clicking below to watch! Watch it here [https://home.mlops.community/home/videos/stockholm-2024-community-kick-off-panel] [Watch it here](https://home.mlops.community/home/videos/stockholm-2024-community-kick-off-panel) Using tech on medical data sounds like it could be a great way to create a cyborg, if only it wasn't for the cost. Well, this blog looks at budget-friendly methods for fine-tuning the Llama 3 8B instruct model on medical data using Hugging Face, Google Colab, and Unsloth. It covers the benefits of fine-tuning pre-trained language models, especially to enhance their performance for domain-specific tasks. The post explains different strategies of fine-tuning based on data utilization and weight adjustment, such as Supervised Fine-Tuning, Few-Shot Learning, Full Transfer Learning, and Domain-specific Fine-Tuning. It also provides a guide on preparing medical instruction data and details the steps to efficiently utilize GPU resources in Google Colab for this purpose. Seems like the Six Million Dollar man just got cheaper! (Look at that, a reference from 10 years before Taylor Swift was even born, how’s that for range!) Read it here [https://mlops.community/budget-instruction-fine-tuning-of-llama-3-8b-instructon-medical-data-with-hugging-face-google-colab-and-unsloth/] [Read it here](https://mlops.community/budget-instruction-fine-tuning-of-llama-3-8b-instructon-medical-data-with-hugging-face-google-colab-and-unsloth/) ## IRL Meetups Montreal [https://autodb.lt.acemlna.com/Prod/link-tracker?notrack=1&notrack=1&redirectUrl=aHR0cHMlM0ElMkYlMkZ3d3cubWVldHVwLmNvbSUyRm1vbnRyZWFsLW1sb3BzLWNvbW11bml0eSUyRmV2ZW50cyUyRjMwMDI1MjkzOCUyRiUzRnV0bV9tZWRpdW0lM0RyZWZlcnJhbCUyNnV0bV9jYW1wYWlnbiUzRHNoYXJlLWJ0bl9zYXZlZGV2ZW50c19zaGFyZV9tb2RhbCUyNnV0bV9zb3VyY2UlM0RsaW5rJTI2dXRtX3NvdXJjZSUzREFjdGl2ZUNhbXBhaWduJTI2dXRtX21lZGl1bSUzRGVtYWlsJTI2dXRtX2NvbnRlbnQlM0RJZiUyNTIweW91JTI1MjAlMjUyQXN0YXJ0JTI1MkElMjUyMHdpdGglMjUyMGElMjUyMHByb2JsZW0lMjUyQyUyNTIwaXMlMjUyMHRoYXQlMjUyMHdvcmtpbmclMjUyMGJhY2t3YXJkcyUyNTIwZnJvbSUyNTIwaXQlMjUyMCUyNTNGJTI1MjAlMjVGMCUyNTlGJTI1QTQlMjU5NCUyNnV0bV9jYW1wYWlnbiUzRE1MT3BzJTI1MjAtJTI1MjBOZXdzbGV0dGVyJTI1MjAtJTI1MjAyMDI0JTI1MjAtJTI1MjAwNCUyNTIwLSUyNTIwMTElMjUyMCUyNTI4Q29weSUyNTI5&sig=5pQa5Bwk227ffVNdTWMJkFKEm46TbvLdRAjcFvYnth2T&iat=1713522110&a=%7C%7C650082935%7C%7C&account=autodb%2Eactivehosted%2Ecom&email=3UollgB%2B6wRzPn3GY0Dyl8XuOXBOg%2BWeE9VjWwTz4r8BD22WDQ%3D%3D%3AQqmJoWZEbmAq5J1ZQgY6BygwQCein6IR&s=bad97c655476f96a390a72c05a742011&i=463A557A36A17808] - May 2 Stockholm [https://www.meetup.com/stockholm-mlops-community/events/300530812/?utm_medium=referral&utm_campaign=share-btn_savedevents_share_modal&utm_source=link] - May 7 Copenhagen [https://autodb.lt.acemlna.com/Prod/link-tracker?notrack=1&notrack=1&redirectUrl=aHR0cHMlM0ElMkYlMkZ3d3cubWVldHVwLmNvbSUyRmNvcGVuaGFnZW4tbWxvcHMtY29tbXVuaXR5JTJGZXZlbnRzJTJGMzAwNDQ3NDg4JTJGJTNGdXRtX21lZGl1bSUzRHJlZmVycmFsJTI2dXRtX2NhbXBhaWduJTNEc2hhcmUtYnRuX3NhdmVkZXZlbnRzX3NoYXJlX21vZGFsJTI2dXRtX3NvdXJjZSUzRGxpbmslMjZ1dG1fc291cmNlJTNEQWN0aXZlQ2FtcGFpZ24lMjZ1dG1fbWVkaXVtJTNEZW1haWwlMjZ1dG1fY29udGVudCUzRElmJTI1MjB5b3UlMjUyMCUyNTJBc3RhcnQlMjUyQSUyNTIwd2l0aCUyNTIwYSUyNTIwcHJvYmxlbSUyNTJDJTI1MjBpcyUyNTIwdGhhdCUyNTIwd29ya2luZyUyNTIwYmFja3dhcmRzJTI1MjBmcm9tJTI1MjBpdCUyNTIwJTI1M0YlMjUyMCUyNUYwJTI1OUYlMjVBNCUyNTk0JTI2dXRtX2NhbXBhaWduJTNETUxPcHMlMjUyMC0lMjUyME5ld3NsZXR0ZXIlMjUyMC0lMjUyMDIwMjQlMjUyMC0lMjUyMDA0JTI1MjAtJTI1MjAxMSUyNTIwJTI1MjhDb3B5JTI1Mjk=&sig=Dmegwx5NnJ5cyqBVK53DDsWgEt7c8v686gBKsCc8wjzP&iat=1713522110&a=%7C%7C650082935%7C%7C&account=autodb%2Eactivehosted%2Ecom&email=3UollgB%2B6wRzPn3GY0Dyl8XuOXBOg%2BWeE9VjWwTz4r8BD22WDQ%3D%3D%3AQqmJoWZEbmAq5J1ZQgY6BygwQCein6IR&s=bad97c655476f96a390a72c05a742011&i=463A557A36A17809] - May 8 (thanks to Todai 🙌) Bristol [https://www.meetup.com/mlops-community-bristol/events/300590639/?utm_medium=referral&utm_campaign=share-btn_savedevents_share_modal&utm_source=link] - May 9 Denver [https://autodb.lt.acemlna.com/Prod/link-tracker?notrack=1&notrack=1&redirectUrl=aHR0cHMlM0ElMkYlMkZ3d3cubWVldHVwLmNvbSUyRmRlbnZlci1tbG9wcy1jb21tdW5pdHklMkZldmVudHMlMkYzMDA0MzU0ODQlMkYlM0Z1dG1fbWVkaXVtJTNEcmVmZXJyYWwlMjZ1dG1fY2FtcGFpZ24lM0RzaGFyZS1idG5fc2F2ZWRldmVudHNfc2hhcmVfbW9kYWwlMjZ1dG1fc291cmNlJTNEbGluayUyNnV0bV9zb3VyY2UlM0RBY3RpdmVDYW1wYWlnbiUyNnV0bV9tZWRpdW0lM0RlbWFpbCUyNnV0bV9jb250ZW50JTNESWYlMjUyMHlvdSUyNTIwJTI1MkFzdGFydCUyNTJBJTI1MjB3aXRoJTI1MjBhJTI1MjBwcm9ibGVtJTI1MkMlMjUyMGlzJTI1MjB0aGF0JTI1MjB3b3JraW5nJTI1MjBiYWNrd2FyZHMlMjUyMGZyb20lMjUyMGl0JTI1MjAlMjUzRiUyNTIwJTI1RjAlMjU5RiUyNUE0JTI1OTQlMjZ1dG1fY2FtcGFpZ24lM0RNTE9wcyUyNTIwLSUyNTIwTmV3c2xldHRlciUyNTIwLSUyNTIwMjAyNCUyNTIwLSUyNTIwMDQlMjUyMC0lMjUyMDExJTI1MjAlMjUyOENvcHklMjUyOQ==&sig=2EXAVdtegcjbtAv5JkoEkFD6mYX4QJM9BkaP1rrrD9pa&iat=1713522110&a=%7C%7C650082935%7C%7C&account=autodb%2Eactivehosted%2Ecom&email=3UollgB%2B6wRzPn3GY0Dyl8XuOXBOg%2BWeE9VjWwTz4r8BD22WDQ%3D%3D%3AQqmJoWZEbmAq5J1ZQgY6BygwQCein6IR&s=bad97c655476f96a390a72c05a742011&i=463A557A36A17810] - May 14 [Montreal](https://autodb.lt.acemlna.com/Prod/link-tracker?notrack=1&notrack=1&redirectUrl=aHR0cHMlM0ElMkYlMkZ3d3cubWVldHVwLmNvbSUyRm1vbnRyZWFsLW1sb3BzLWNvbW11bml0eSUyRmV2ZW50cyUyRjMwMDI1MjkzOCUyRiUzRnV0bV9tZWRpdW0lM0RyZWZlcnJhbCUyNnV0bV9jYW1wYWlnbiUzRHNoYXJlLWJ0bl9zYXZlZGV2ZW50c19zaGFyZV9tb2RhbCUyNnV0bV9zb3VyY2UlM0RsaW5rJTI2dXRtX3NvdXJjZSUzREFjdGl2ZUNhbXBhaWduJTI2dXRtX21lZGl1bSUzRGVtYWlsJTI2dXRtX2NvbnRlbnQlM0RJZiUyNTIweW91JTI1MjAlMjUyQXN0YXJ0JTI1MkElMjUyMHdpdGglMjUyMGElMjUyMHByb2JsZW0lMjUyQyUyNTIwaXMlMjUyMHRoYXQlMjUyMHdvcmtpbmclMjUyMGJhY2t3YXJkcyUyNTIwZnJvbSUyNTIwaXQlMjUyMCUyNTNGJTI1MjAlMjVGMCUyNTlGJTI1QTQlMjU5NCUyNnV0bV9jYW1wYWlnbiUzRE1MT3BzJTI1MjAtJTI1MjBOZXdzbGV0dGVyJTI1MjAtJTI1MjAyMDI0JTI1MjAtJTI1MjAwNCUyNTIwLSUyNTIwMTElMjUyMCUyNTI4Q29weSUyNTI5&sig=5pQa5Bwk227ffVNdTWMJkFKEm46TbvLdRAjcFvYnth2T&iat=1713522110&a=%7C%7C650082935%7C%7C&account=autodb%2Eactivehosted%2Ecom&email=3UollgB%2B6wRzPn3GY0Dyl8XuOXBOg%2BWeE9VjWwTz4r8BD22WDQ%3D%3D%3AQqmJoWZEbmAq5J1ZQgY6BygwQCein6IR&s=bad97c655476f96a390a72c05a742011&i=463A557A36A17808) Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ?view_as=subscriber], and podcast [https://home.mlops.community/public/content/] land. Oh yeah, and we are also on X [https://twitter.com/mlopscommunity]. The MLOps Community newsletter is edited by Jessica Rudd [https://www.linkedin.com/in/jmrudd/].