--- title: "GPU Cloud: No One's Safe" newsletter: "MLOps Community" date: 2025-07-24 source: https://aaif.live/newsletters/mlopscommunity/2025-07-24-gpu-cloud-no-one-s-safe --- # GPU Cloud: No One's Safe *Plus, scaling outbound agents reliably, tools for vibe coding, Hidden Gems, and ML Confessions.* *MLOps Community — Agentic AI Foundation, 2025-07-24* Can't imagine how much I'd be worth [https://go.mlops.community/intro24jul] if I'd dropped out... ## Cloud, but make it GPU THE LOST EPISODE BENCHMARKING THE GPU CLOUD MARKET Vibe coding, MechaHitler, Coldplay gigs. Things go wrong in AI. Worse than all those were the technical issues we had recording a podcast with Dylan Patel from SemiAnalysis. We couldn’t use the recording, but the chat was too good to let it disappear. So instead, here’s a recap from memory (and rough notes), of what Dylan had to say on GPU infra, the mess of scaling AI, and what’s coming next. Cloud, but make it GPU “There’s 120+ GPU infra companies right now. I don’t think more than 20 will survive.” The old vs new cloud framing came up a few times. Old cloud is all SaaS layers and margin. New cloud’s closer to the metal - built around GPU cost, scarcity, and speed to deploy. It’s less ‘buy a managed service’ and more ‘buy raw horsepower plus a team that keeps it from falling over.’ Some of the better providers now sell themselves on operational maturity. If they can handle failures, rollout updates, rate-limit smoothly across a big cluster - that’s the product. Margins are disappearing “In 2023, GPU rentals were going for $5/hour with $1.50 of actual cost. Those margins were insane. Now they’ve come right down.” The arbitrage era’s done. Prices are falling, utilisation’s climbing, and providers can’t coast on markup anymore. It’s not a race to the bottom on price - but it is a race to efficiency. The ones that are scaling now are doing it by: * Getting customers live faster * Making the infra more reliable * Actually delivering the promised performance Bad infra used to survive on hype. Now even 20% margin might not be enough to keep the lights on. Infra vs Solution “Enterprises don’t want infra. They want working AI. But you can’t do one without the other.” This bit got pretty spicy. Dylan’s take: too many companies try to avoid infra complexity, build something clever on top of someone else’s stack… and then get buried by their own API bills. He told the story of one agent startup that got to a billion tokens/day two weeks post-launch. It worked. Too well. They had to scramble for custom infra to stay alive. There’s also the constant fear that OpenAI drops a new API and suddenly your whole product is redundant. If you’re just calling the same foundation model as everyone else, where’s your edge? Ranking the GPU clouds “We’re not sure how we’ll make money off it yet. But everyone will know us!” ClusterMax started as diligence for an acquisition. Now it’s SemiAnalysis testing 60+ GPU clouds, benchmarking them on performance, reliability, and usability. Dylan described it as “trying to be the Moody’s of GPU cloud,” but with no kickbacks. Interestingly, some of the cheapest clouds performed the worst. CoreWeave scored well - not because they’re cheap, but because they’re consistent. Together got props for their kernel libraries that help squeeze extra performance out of your models. Eventually, Dylan wants the scores to factor into financing: rank Platinum, get better loan terms. It’s early, but makes sense - infra quality is a financial risk. Nobody’s safe “No one has a right to win.” That line came up a few times. Not the hyperscalers, not the tooling startups, not the new clouds. Everyone’s living on uptime, utilisation, and whether they can stay just ahead of the next thing. Some of these providers raised debt at 17% interest to buy GPUs. As Dylan put it: “The GPU’s down for a day - am I making money, or just paying interest?” Infra’s no longer just technical. It’s financial. And it’s brutal. ## Hidden Gems ## AI Agent Development Tradeoffs You NEED to Know Not many people get to interview the person building the AI that’s coming for their old job. Many moons ago, before the Community, I was an SDR. Now 11x have built two AI agents, Alice and Julian, to replace that role. Alice handles everything from lead sourcing to multi-step email outreach. Sherwood explained why they chose LangGraph for its structured control, hosting the agents on LangGraph Cloud and monitoring performance via LangSmith and Arise. Alice’s workflow leans heavily on structure, especially in outreach: * Campaign flow as a graph: Research, sequencing, and message generation are all modular, improving reliability and customization * Eval sampling: About 1% of outputs are checked for hallucinations by comparing against internal research Help me keep this job and click below to listen. Video [https://go.mlops.community/scp24jul] || Spotify [https://go.mlops.community/scs24jul] || Apple [https://go.mlops.community/sca24jul] [https://go.mlops.community/sca24jul](https://go.mlops.community/sca24jul) [https://go.mlops.community/scp24jul](https://go.mlops.community/scp24jul) [https://go.mlops.community/scs24jul](https://go.mlops.community/scs24jul) ## Job of the Week [https://go.mlops.community/usjob24jul](https://go.mlops.community/usjob24jul) ## To Vibe Code or Not to Vibe Code? Things move quickly in AI, but one trend that’s hanging around longer than a Coldplay gig meme is vibe coding. It’s moving away from toy apps and creeping into real workflows, especially for quick iterations and prototyping. This blog looks at tools like Cursor and Bolt that let you describe what you want in plain language, accept AI suggestions, and ship something functional with minimal manual coding. It’s fast, sometimes chaotic, and very different from traditional development. For prototyping, two patterns stood out: * Cursor with SDKs: Useful for sketching agent logic, testing prompts, and letting AI fill in boilerplate Python. * Bolt for UI: Entire sites built via natural language prompts, no design tools needed. Read the blog, then order the t-shirt [https://go.mlops.community/vibet24jul] so you look cool on camera for the meme. Read it here [https://go.mlops.community/blog24jul] [t-shirt](https://go.mlops.community/vibet24jul) [https://go.mlops.community/MLConfess](https://go.mlops.community/MLConfess) Old cloud optimized for cost and convenience. New cloud’s optimizing for “did the kernel panic at 2am?” Feels less like a platform war, more like parallel universes. You old-school or new-school? Working on something tricky or planning ahead? Here’s how we can help - just hit reply: * Custom workshops tailored to your company’s needs * Hiring? I know some quality folks looking for a new adventure * Want to connect with someone tackling similar problems? I can introduce you Thanks for reading, catch you next time! --- Source: https://aaif.live/newsletters/mlopscommunity/2025-07-24-gpu-cloud-no-one-s-safe