--- title: "Five layers, fewer wasted GPUs" newsletter: "User Community" date: 2026-09-10 source: https://aaif.live/newsletters/usercommunity/2026-09-10-five-layers-fewer-wasted-gpus --- # Five layers, fewer wasted GPUs *Plus… open gateways, stateless MCP, and Claude Code hooks* *User Community — Agentic AI Foundation, 2026-09-10* Explaining my daughter’s homework is the real impossible math problem, but good to see they’re making progress [https://openai.com/index/navier-stokes-solution/]. ## Gems ## Building the next voice agents ## FREE VIRTUAL EVENT What does it take to move voice AI from a slick demo to something fast, reliable, and ready for production? Join us September 16 for 90 minutes on voice-agent architecture, infrastructure, and open standards, including UNMUTE, a new MIT-licensed standard for voice agents, plus a live look at how semantic caching, CDNs, and edge infrastructure can cut latency in voice pipelines. September 16 · 08:30 PDT / 17:30 CEST JOIN LIVE [https://home.mlops.community/home/events/voice-agent-virtual-event-bnzc6hiff8] ## Prompt Injection as Role Confusion ## READING GROUP This month’s reading group looked at research showing that models can infer who is speaking from writing style rather than reliably following role tags. With forged reasoning written in a model’s own style, attack success rose from under 4% to over 80% on several of the six models tested. Lucas Pavanelli walked through the ICML 2026 paper, before Sparsh Jain took the discussion into reasoning-trace extraction and agent red-teaming. Read the session notes and checklist [https://learn.mlops.community/wp-content/uploads/2026/09/Reading-Group-Promp-Injection-as-Role-Confusion-asset.pdf] ## What’s happening across the chapters ## LOCAL ORGANIZERS It’s great to see AAIF chapters taking shape around the world, with local organizers bringing people together and trying out their own formats. Melbourne’s first AAIF meetup [https://lnkd.in/p/gjSKhy_V] brought 58 engineers and builders together for four talks. Pittsburgh’s hands-on MCP session drew around 40 people and so many questions that the agentgateway and A2A workshops were saved for another event, while Shanghai opened its Launch Series and Atlanta tried a speaker-free Lean Coffee format where the room chose the agenda. Over in Silicon Valley, Vijay Bohre and Rahul Parundekar brought together three technical sessions on agent memory, covering everything from what should be remembered or forgotten to memory scoping, retrieval and keeping useful context under control. Watch it here [https://youtu.be/dZMYV_6xvYo?si=AyVufTCKSyoALOjB&t=505]. The community map is growing too, with five new organizers joining across Madrid, Santiago, Chennai and Ulaanbaatar. There’s plenty more happening this month - check out the upcoming events near the bottom of the newsletter. ## Coding agents, security and uncertainty ## LUNCH AND LEARN Last week, Ads Dawson joined us to talk about the security scaffolding around coding agents, from least privilege and sandboxing to prompt injection and stopping conditions. Read the session notes [https://learn.mlops.community/wp-content/uploads/2026/09/Lunch_and_Learn_Session_23_Asset.pdf.pdf] This week, Eric Bigelow from Goodfire joins us on September 11 at 09:00 PDT / 18:00 CET to look at uncertainty in LLM reasoning, and how to get useful signals from fewer sampled reasoning chains. Check out his paper on Model Reasoning [https://arxiv.org/abs/2603.05488], and bring your questions! Join us [https://home.mlops.community/home/events/coding-agents-lunch-and-learn-session-24-understanding-uncertainty-in-llm-reasoning-rruhl39k8q?agenda_day=6a99c6311df02a64b5c62bf2&agenda_track=6a99c6321df02a64b5c62c07&agenda_stage=6a99c6311df02a64b5c62bf7&agenda_filter_view=stage&agenda_view=list] ## The Five-Layer Cake Approach to Scaling AI Without Wasting Money Thirty minutes of GPU warm-up across thousands of accelerators can waste expensive capacity. AI efficiency depends on choices across hardware, capacity, inference, models, and routing, where gains and mistakes compound. * Match hardware to workload: coding favors heavy prefill and context processing; multimodal pipelines may need different resources for vision and text. * Treat caching, quantization, fleet partitioning, and engine startup as tradeoffs between throughput, flexibility, and utilization. * Benchmark changes with repeatable workloads: model swaps and lower precision can improve speed, but they can also change output quality and behavior. Stable baselines make it easier to turn efficiency gains into lower latency, more capacity, or larger reasoning budgets. [https://podcasts.apple.com/gb/podcast/the-five-layer-cake-approach-to-scaling-ai-without/id1505372978?i=1000787876792](https://podcasts.apple.com/gb/podcast/the-five-layer-cake-approach-to-scaling-ai-without/id1505372978?i=1000787876792) [https://home.mlops.community/home/videos/the-five-layer-cake-approach-to-scaling-ai-without-wasting-money](https://home.mlops.community/home/videos/the-five-layer-cake-approach-to-scaling-ai-without-wasting-money) [https://open.spotify.com/episode/41z72mvSRsH0YwVSbXE4ia?si=_x6U9acZRFyOPZpjWDeYhw](https://open.spotify.com/episode/41z72mvSRsH0YwVSbXE4ia?si=_x6U9acZRFyOPZpjWDeYhw) ## The Winchester Mystery House Problem in AI development A workflow that cost $1 per 1,000 records at 90% accuracy reached 95% accuracy while costing less, after an optimizer moved 75% of the work out of the model and into code. That result sits inside a broader discussion about when agentic behavior should become a defined workflow. * Models increasingly inherit assumptions from the coding harnesses they were trained around, which can make custom harnesses harder to build. * Repeated agent tasks can often be “crystallized” into smaller models or conventional code once the process is understood. * DSPy separates task definitions from implementation, allowing prompts, models, and even harness code to be optimized without redefining the task. The practical direction is toward identifying which tasks still need open-ended agents and which are mature enough to become cheaper, more reliable workflows. [https://podcasts.apple.com/us/podcast/the-winchester-mystery-house-problem-in-ai-development/id1505372978?i=1000785551986](https://podcasts.apple.com/us/podcast/the-winchester-mystery-house-problem-in-ai-development/id1505372978?i=1000785551986) [https://home.mlops.community/home/videos/the-winchester-mystery-house-problem-in-ai-development](https://home.mlops.community/home/videos/the-winchester-mystery-house-problem-in-ai-development) [https://open.spotify.com/episode/4IjjWOjSq8XQPAXLq6ApRQ?si=TnlufikXRKqxS8ZHSpyYCQ](https://open.spotify.com/episode/4IjjWOjSq8XQPAXLq6ApRQ?si=TnlufikXRKqxS8ZHSpyYCQ) ## LiteLLM is the known option. agentgateway is the open one. AI gateways can sit in front of model keys, MCP servers, and A2A traffic, making licensing and failure modes architectural concerns. Comparing LiteLLM and agentgateway shows how differently they behave once identity, routing, and operations matter. * LiteLLM has broader provider coverage, but SSO, OIDC/JWT, secret managers, and governance controls require Enterprise. * agentgateway keeps JWT, CEL, mTLS, MCP OAuth, and external authorization in its Apache 2.0 tree, with a smaller runtime footprint. * Their attack surfaces differ: LiteLLM production deployments depend on PostgreSQL and Redis, while agentgateway can run as one binary. The better default depends on whether provider breadth or an open, general-purpose control plane is the main constraint. [Read the blog](https://mlops.community/blog/litellm-is-the-known-option-agentgateway-is-the-open-one) ## MCP 2026 changes the rules MCP’s 2026-07-28 revision removes protocol-level sessions, changing how clients scale across load balancers and replicated servers. Requests now carry the metadata they need, while discovery and optional capabilities move out of the old initialization flow. * Stateless requests let servers process calls independently and scale horizontally without shared session state. * Versioned extensions keep the core smaller, so features such as Tasks or MCP Apps can evolve separately and be adopted only where needed. * server/discover lets clients inspect supported versions and capabilities without creating a session or sending notifications/initialized. The main shift is toward a more scalable, modular protocol with less connection state and clearer capability negotiation. [Read the blog](https://aaif.io/blog/mcp-2026-changes-the-rules) ## Authorization in MCP 2026-07-28: Clarifying trust relationships for agentic systems An MCP client may cross several authorization servers, backend services, and organizational boundaries in a single workflow. MCP 2026-07-28 tightens how those trust relationships are identified, validated, and maintained using established OAuth and OIDC mechanisms. * Clients now validate the iss parameter when present, reducing the risk of accepting authorization responses from the wrong authority. * Dynamic registration requires an appropriate OIDC application_type, while credentials remain bound to the authorization server that issued them. * Step-up authorization preserves existing scopes, and refresh-token guidance better supports long-running agent workflows. The key change is clearer, more verifiable authorization across distributed MCP systems without requiring a new IAM model. [Read the blog](https://aaif.io/blog/authorization-in-mcp-2026-07-28-clarifying-trust-relationships-for-agentic-systems) ## IN-PERSON EVENTS ## Come and connect * Toronto [https://luma.com/p6vg7zag] - September 10 * AGNTCon + MCPCon Japan [https://events.linuxfoundation.org/agntcon-mcpcon-japan/] - September 10 * Colombo [https://luma.com/xqn9aig2] - September 10 * Pune, India [https://luma.com/4h0mjfzr] - September 12 * New York [https://luma.com/aaif-kc6u] - September 15 * AGNTCon + MCPCon Europe [https://events.linuxfoundation.org/agntcon-mcpcon-europe/] - September 17 * Luxembourg [https://luma.com/4ciica7u] - September 17 * Singapore [https://luma.com/gspnb7f6] - September 17 Find your city here [https://aaif.io/events?tab=community], or start a chapter if there isn't one yet. ## VIRTUAL EVENTS ## Join from anywhere * Lunch and Learn Session 24 [https://home.mlops.community/home/events/coding-agents-lunch-and-learn-session-24-understanding-uncertainty-in-llm-reasoning-rruhl39k8q?agenda_day=6a99c6311df02a64b5c62bf2&agenda_track=6a99c6321df02a64b5c62c07&agenda_stage=6a99c6311df02a64b5c62bf7&agenda_filter_view=stage&agenda_view=list] - September 10 * 36 Hours Live from AGNTCon + MCPCon Europe [https://luma.com/aaif-afd7] - September 17 - 18 --- Source: https://aaif.live/newsletters/usercommunity/2026-09-10-five-layers-fewer-wasted-gpus