--- title: "Agents That Click Back" newsletter: "MLOps Community" date: 2026-01-08 source: https://aaif.live/newsletters/mlopscommunity/2026-01-08-agents-that-click-back --- # Agents That Click Back *Plus real-time features, sandboxed agents, multimodal queries, and lean browser control.* *MLOps Community — Agentic AI Foundation, 2026-01-08* Microsoft CEO Satya Nadella wants people to stop peddling AI slop and develop “bicycles for the mind [https://go.mlops.community/NL_Intro_Jan8]”. ## Agents are a blast Giving agents more tools doesn’t make them smarter. It just increases the blast radius. Do you scale capability first or containment first? ## Hidden Gems ## Job of the Week [https://go.mlops.community/NL_Jobs_Board](https://go.mlops.community/NL_Jobs_Board) ## Rewriting the script on browser control for AI agents A recent comparison [https://go.mlops.community/NL_AgInf_Vid_Jan8] of browser control via MCP using Playwright versus Playwriter highlighted some real architectural trade-offs that matter if you’re building agent-driven workflows. Playwright’s MCP server mirrors its testing roots: a separate browser instance and a large set of narrowly scoped tools for actions like click, type, navigate, and screenshot. That structure is explicit, but it comes with real overhead - roughly 14–15k tokens can disappear just defining tools and browser state before the agent does any useful work. Playwriter takes a different approach. It attaches to an existing Chrome session via an extension and exposes a single, code-first execution interface, allowing the agent to run arbitrary JavaScript or Playwright commands directly in-page. In practice, that trims MCP overhead to around 5k tokens, preserves logged-in state and open tabs, and reduces multi-step interactions to a single call. It also makes agentbehavior visible and inspectable in the live browser, which is useful for debugging and trust. Once you factor in token cost, latency, and real session state, the “lean interface” approach starts to look less like a design choice and more like a requirement. USEFUL LINKS Playwriter GitHub [https://go.mlops.community/NL_AgInf_Playwriter_Jan8] – The community Playwriter MCP server repository that focuses on a single execute tool to reduce context overhead. Playwriter listing on MCP Market [https://go.mlops.community/NL_AgInf_Playwriter_MCPmkt_Jan8] – Summary of what Playwriter does, its integration style, and key features like minimal context overhead and full Playwright API access. Playwright MCP GitHub [https://go.mlops.community/NL_AgInf_Playwright_Jan8] – Source for the Playwright MCP server implementation and how it exposes browser actions via MCP. ## Computers that think and take actions for you Computers that read intent and act on your behalf are closer than they look - but getting there breaks most of today’s training assumptions. * Models trained on text and video can explain interfaces fluently but fail at real work because they never experience causality or system state changing through action. * Teaching computer use means letting agents operate real software in large sandboxes, collecting long action sequences and learning directly from success, failure, and recovery. * The hardest problems aren’t benchmarks but reality: slow rollouts, shifting UI state, subjective outcomes, and brittle infrastructure at scale. All of this points to a shift from language modelling to systems that learn through interaction, failure, and recovery. Video [https://go.mlops.community/NL_Pod1_G_Jan8] || Spotify [https://go.mlops.community/NL_Pod1_S_Jan8] || Apple [https://go.mlops.community/NL_Pod1_A_Jan8] REAL-TIME FEATURES, AI SEARCH, AGENTIC SIMILARITIES Fraud teams don’t get weeks to ship a new feature - they get hours before money starts leaking. That pressure is why “feature platforms” are shifting from a storage problem to a compute problem. * Cronon frames the job as end-to-end compute: take raw data, build streaming and batch features, and keep online inference and offline training consistent at scale. * The hard bit is point-in-time correctness for training sets - naive SQL explodes into an N-cube join problem, which is why many teams fall back to stale batch features. * The scope is widening from “features” to “features plus embeddings,” where you window event streams, generate embeddings, and serve them in real time for RAG, search, and personalization. What comes through is that streaming features and embeddings force teams to treat training and serving as one system. Video [https://go.mlops.community/NL_Pod2_G_Jan8] || Spotify [https://go.mlops.community/NL_Pod2_S_Jan8] || Apple [https://go.mlops.community/NL_Pod2_A_Jan8] TOOL DEFINITIONS ARE THE NEW PROMPT ENGINEERING A hungry user doesn’t care about your agent architecture - they care whether dinner shows up fast, right, and without fiddling with menus. * iFood built a lightweight “decide for me” ordering agent across app and WhatsApp, then found the real bottlenecks were UX expectations, trust, and latency - not model quality. * Tool design turned into the main engineering lever: stop wrapping raw APIs, encode intent (for example, “get brochure” not “browse Drive”), and move instructions into tool outputs to cut tokens and errors. * Scaling beyond one agent forced a new layer: shared tools, registries, evals, versioning, access control, and ownership - because changing a tool definition is effectively changing prompts for every downstream agent. Fast agents come from tight tool contracts and disciplined evaluation, not longer prompts or bigger multi-agent graphs. Video [https://go.mlops.community/NL_Pod3_G_Jan8] || Spotify [https://go.mlops.community/NL_Pod3_S_Jan8] || Apple [https://go.mlops.community/NL_Pod3_A_Jan8] THE FUTURE OF AI AGENTS IS SANDBOXED If you let an agent run shell commands, clone repos, and execute arbitrary code, one bad step can nuke your server. That’s why “give the agent its own computer” is becoming the default pattern: isolate the blast radius, then widen capability safely. * Sandbox as the execution unit: provision CPU/RAM/disk per job, run lots in parallel, and cap worst-case resource burn when agents go off-script. * Reproducible environments: start with ad-hoc dev boxes, then lock the “recipe” into a blueprint (container build) with tools, repos, agents, and uploaded artifacts baked in. * Production hygiene: audit trails + trajectories, snapshot/suspend for human-in-the-loop, and benchmark-style evals that score end-to-end outcomes, not single LLM calls. This approach makes agent behavior easier to reason about, safer to run, and simpler to improve over time. Video [https://go.mlops.community/NL_Pod4_G_Jan8] || Spotify [https://go.mlops.community/NL_Pod4_S_Jan8_V2] || Apple [https://go.mlops.community/NL_Pod4_A_Jan8] AI REWIND 2025 - READING GROUP YEAR-END S PECIAL The reading group took a step back from papers to look across the past year and surface the shifts people felt in real work, based on input from the community. * Agents in production, moving from isolated demos to systems with memory, orchestration, and operational constraints. * Vibe coding, changing how people prototype and explore ideas while keeping human review essential. * Context, MCP, and evaluation, becoming foundational engineering concerns rather than optional tooling. A solid reflection of how AI practice evolved over the year. Watch it here [https://go.mlops.community/NL_RG_Jan8] BEYOND SQL: THE QUERY LANGUAGE MULTIMODAL AI REALLY NEEDS SQL works well when queries mean filtering rows. It struggles when a single request needs similarity search, metadata constraints, joins, and media preparation in one pass. This piece looks at what breaks when multimodal AI workloads hit traditional query languages. * Multimodal queries behave like pipelines: vector search, filtering, joins, and image or video processing turn into long, fragile chains when handled outside the database. * Relational models leak abstraction: clips, frames, bounding boxes, and overlaps don’t map cleanly to tables without added joins, custom code, and performance tradeoffs. * Schema-light querying fits evolving AI systems: nested, extensible structures handle changing modalities and agent-driven access patterns with less glue code. The examples show how query languages built for tabular data start to strain as AI workloads become more multimodal. Read the blog [https://go.mlops.community/NL_Blog1_Jan8] HOW MULTIMODAL VECTOR DATABASES ARE TRANSFORMING CHALLENGES ACROSS INDUSTRIES A single missing sentence in a healthcare ad can trigger regulatory action, public backlash, and real harm. This piece walks through a concrete system for catching those gaps automatically by combining multimodal vector search with LLM-based reasoning. * Multimodal retrieval in practice: clinical PDFs, tables, and images are embedded in one system so similarity search works across text and visuals, not just keywords. * LLMs as omission detectors: models flag missing risks, contraindications, and limitations, then ground those findings against trusted clinical sources. * Why unified storage matters: linking embeddings back to original documents makes verification, traceability, and audits feasible at scale. The result is a practical pattern for turning regulatory compliance from a manual review burden into something engineers can actually build and maintain. Read the blog [https://go.mlops.community/NL_Blog2_Jan8] [Video](https://go.mlops.community/NL_Pod1_G_Jan8) ## The Gift of a Broken Backfill [https://go.mlops.community/NL_Confess](https://go.mlops.community/NL_Confess) ## Making the hard stuff simpler Working on something tricky or planning ahead? Here’s how we can help - just hit reply: * Custom workshops tailored to your company’s needs * Hiring? I know some quality folks looking for a new adventure * Want to connect with someone tackling similar problems? I can introduce you Thanks for reading, catch you next time! --- Source: https://aaif.live/newsletters/mlopscommunity/2026-01-08-agents-that-click-back