---
title: "Unified Backend Framework for APIs, Background Jobs, and AI Agents"
newsletter: "MLOps Community"
date: 2025-09-02
source: https://aaif.live/newsletters/mlopscommunity/2025-09-02-unified-backend-framework-for-apis-background-jobs-and-ai-ag
---

# Unified Backend Framework for APIs, Background Jobs, and AI Agents

*Building production-ready backends with a single primitive that eliminates infrastructure complexity*

*MLOps Community — Agentic AI Foundation, 2025-09-02*

For some trade secrets that won’t get you sued by xAI [https://sfstandard.com/2025/08/29/xai-elon-musk-openai-stanford-sam-altman-ai-talent-wars/], join us at the AI Agent Builder Summit [https://luma.com/aiagentsummit] in San Francisco this Thursday.

Keynotes from Arcade, Deepgram, Vellum, Databricks, and OpenAI, plus quick-fire talks from teams in the trenches. Hands-on Voice Agent [https://luma.com/voiceagents] and Authentication [https://luma.com/aiagentauthentication] masterclasses, and 30+ booths where you can talk to the engineers actually building this stuff.

Not in SF? Check out the tour stops in:

 * Seattle [https://luma.com/g42ppkok] - Sept 25
 * NYC [https://luma.com/agentsummitNYC] - Oct 16
 * London [https://luma.com/i51hrr7n] - Nov 12

## by

UNIFIED BACKEND FRAMEWORK FOR APIS, BACKGROUND JOBS, AND AI AGENTS

by Rohit Ghumare [https://www.linkedin.com/in/rohit-ghumare/]

Today, backend engineers face several recurring challenges:

 * 🧩 Fragmented Systems: APIs in Express, background jobs in Celery/BullMQ, AI agents in LangChain – each with different deployment, debugging, and scaling patterns
 * 🌐 Multi-Language Barriers: AI tools in Python, business logic in TypeScript – forcing teams to choose between cutting-edge tech and their existing skillset
 * 🔍 Observability Gaps: Tracing requests across multiple frameworks and runtimes is complex and often incomplete
 * ⚖️ Scalability vs. Velocity: Choose between fast development (monolith) or proper scaling (microservices complexity)
 * 🚀 Deployment Complexity: Multiple runtimes mean multiple deploy targets, configs, and failure points

The rapid advancement of AI has made this worse – many cutting-edge AI tools are only available in specific languages, forcing companies to abandon their existing tech stack or miss out on breakthrough technologies.

Motia removes this limitation by unifying your entire backend into a single runtime where everything is a Step:

🎯 Unified vs. Fragmented

 * Before: APIs in Express, jobs in BullMQ, AI agents in LangChain
 * After: All backend patterns as composable Steps with shared state and observability

🌐 Multi-Language Support

 * Before: Choose between Python AI tools OR your existing TypeScript stack
 * After: Each Step can be written in any language while sharing common state – use Python for AI, TypeScript for APIs, JavaScript for workflows

🔍 Built-in Observability

 * Before: Complex tracing setups across multiple frameworks
 * After: Complete observability toolkit available in both cloud and local environments out of the box

⚖️ Scalability Without Complexity

 * Before: Choose between monolith simplicity or microservice complexity
 * After: Each Step scales independently, avoiding bottlenecks while maintaining development velocity

🚀 One-Click Everything

 * Before: Multiple deployment pipelines, configs, and failure points
 * After: Single deployable with atomic blue/green deployments and instant rollbacks

Step-by-Step: Building Event-Driven Architecture for AI Streaming Chatbot

1. Understanding the Step Primitive

This open source framework unifies backend complexity through a single concept: the Step. Every backend operation, whether an API endpoint, background job, scheduled task, or AI agent is defined as a Step with consistent patterns for input validation, error handling, and observability.


QUICK SPIN-UP (AGENT MEMORY TOOL)

TypeScript // Four step types handle all backend scenarios export const config: ApiRouteConfig = { type: 'api', // HTTP endpoints name: 'ChatApi', description: 'Send a message to the AI chatbot', path: '/chat', method: 'POST' } export const config: EventConfig = { type: 'event', // Asynchronous processing name: 'AiResponse', description: 'Generate streaming AI response' } export const config: CronConfig = { type: 'cron', // Scheduled tasks schedule: '0 9 * * *' // Daily at 9 AM } export const config: NoopConfig = { type: 'noop' // Testing and development }

💡 Production tip: Start with noop steps during development to test your workflow logic before implementing actual API or event handlers.

2. Setting Up Real-Time Streaming

Motia's streaming capabilities enable real-time communication between steps and external clients. The streaming AI chatbot example demonstrates how to configure type-safe streams with schema validation.

TypeScript // Stream configuration with Zod schema validation import { StreamConfig } from 'motia' import { z } from 'zod' const conversationSchema = z.object({ message: z.string(), from: z.enum(['user', 'assistant']), status: z.enum(['created', 'streaming', 'completed']), timestamp: z.string() }) export const config: StreamConfig = { name: 'conversation', schema: conversationSchema, baseConfig: { storageType: 'default' } }

3. Implementing API Steps with Event Emission

API steps handle HTTP requests and can emit events to trigger downstream processing. This pattern enables clean separation between request handling and business logic.

TypeScript // API step that receives chat messages and triggers AI processing import { ApiRouteConfig, Handlers } from 'motia' import { z } from 'zod' import { conversationSchema } from '../conversation.stream' const inputSchema = z.object({ message: z.string().min(1, 'Message is required'), conversationId: z.string().optional() }) export const config: ApiRouteConfig = { type: 'api', name: 'ChatApi', description: 'Send a message to the AI chatbot', path: '/chat', method: 'POST', emits: ['chat-message'], bodySchema: inputSchema, responseSchema: z.object({ $on: conversationSchema }), flows: ['chat'] } export const handler: Handlers['ChatApi'] = async (req, { logger, emit, streams }) => { const { message, conversationId } = req.body const userMessageId = crypto.randomUUID() const assistantMessageId = crypto.randomUUID() logger.info('New chat message received', { conversationId, message: req.body.message }) // Emit user message to stream await streams.conversation.set(conversationId, userMessageId, { message: req.body.message, from: 'user', status: 'completed', timestamp: new Date().toISOString() }) // Emit event to trigger AI response const aiResponse = await emit('streams.conversation.set', conversationId, assistantMessageId, { message: '', from: 'assistant', status: 'streaming', timestamp: new Date().toISOString() }) return { message: '', from: 'assistant', status: 'streaming' } }

4. Building Event-Driven AI Processing

Event steps handle asynchronous processing triggered by API calls or other events. The AI response step demonstrates streaming responses with OpenAI integration.

TypeScript // Event step that generates streaming AI responses import { EventConfig, Handlers } from 'motia' import { OpenAI } from 'openai' import { z } from 'zod' const inputSchema = z.object({ message: z.string(), conversationId: z.string(), assistantMessageId: z.string() }) export const config: EventConfig = { type: 'event', name: 'AiResponse', description: 'Generate streaming AI response', subscribes: ['chat-message'], emits: [], input: inputSchema, responseSchema: z.object({ $on: conversationSchema }), flows: ['chat'] } export const handler: Handlers['AiResponse'] = async (input, context) => { const { logger, streams } = context const { message, conversationId, assistantMessageId } = input logger.info('Generating AI response', { conversationId }) const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY, baseURL: process.env.OPENAI_BASE_URL || 'https://api.openai.com/v1' }) try { // Stream AI response with real-time updates await streams.conversation.set(conversationId, assistantMessageId, { message: '', from: 'assistant', status: 'streaming' }) const stream = await openai.chat.completions.create({ model: 'gpt-4', messages: [ { role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: message } ], stream: true }) let fullResponse = '' for await (const chunk of stream) { const content = chunk.choices[0]?.delta?.content || '' fullResponse += content // Update stream with partial response await streams.conversation.set(conversationId, assistantMessageId, { message: fullResponse, from: 'assistant', status: 'streaming', timestamp: new Date().toISOString() }) } // Mark response as completed await streams.conversation.set(conversationId, assistantMessageId, { message: fullResponse, from: 'assistant', status: 'completed', timestamp: new Date().toISOString() }) } catch (error) { logger.error('AI response generation failed', { error, conversationId }) await streams.conversation.set(conversationId, assistantMessageId, { message: 'Sorry, I encountered an error processing your request.', from: 'assistant', status: 'completed', timestamp: new Date().toISOString() }) } }

5. Multi-Language Support and Type Safety

Motia supports JavaScript, TypeScript, Python, and more languages. TypeScript provides the best development experience with full type safety across steps, streams, and events.

TypeScript // TypeScript interfaces for complete type safety export interface MotiaStream { set(key: string, value: T): Promise get(key: string): Promise delete(key: string): Promise list(): Promise } export interface EventHandler { (input: TInput, context: { logger: Logger emit: EmitFunction streams: Record }): Promise } export interface ApiRouteHandler { (req: { body: TInput params: Record query: Record }, context: { logger: Logger emit: EmitFunction streams: Record }): Promise }

Python example for cross-language teams:

Python # Python step implementation from motia import step, StreamConfig from pydantic import BaseModel from typing import Optional class ChatMessage(BaseModel): message: str conversation_id: Optional[str] = None @step( type="api", path="/chat-python", method="POST" ) async def chat_api_python(input: ChatMessage, context): logger = context.logger streams = context.streams logger.info(f"Python API received: {input.message}") # Emit to conversation stream await streams.conversation.set( input.conversation_id, { "message": input.message, "from": "user", "status": "completed" } ) return {"status": "received"}

6. Production Deployment and Scaling

Motia provides zero-configuration deployment with built-in observability, state management, and horizontal scaling.

Run with a single command:

Shell # Install Motia example app npx motia@latest create -i # If you want to access above project, Clone this repo: git clone https://github.com/MotiaDev/motia-examples.git # Change directory to example folder cd examples/streaming-ai-chatbot # Initialize project for local development npx motia dev Shell # Deploy to production motia cloud deploy --api-key --version-name [options]

💡 Scaling strategy: Motia automatically handles load balancing across step instances. Event steps scale independently from API steps, allowing you to optimize resource allocation based on workload patterns.

7. Monitoring and Observability

Motia includes built-in tracing, metrics, and logging that work across all step types and languages.

💡 Community engagement: Star the https://github.com/MotiaDev/motia Repo and Join the Motia Discord community to share production experiences and contribute to the evolving framework. With growing adoption across startups and enterprises, your implementation insights help shape the future of unified backend development.

Official Docs & GitHub Links

Motia Framework:

 * Main Repository: https://github.com/MotiaDev/motia
 * Official Documentation: https://motia.dev/docs
 * Examples Repository: https://github.com/MotiaDev/motia-examples
 * Discord Community: https://discord.gg/motia

Streaming AI Chatbot Demo:

 * Complete Example: https://github.com/MotiaDev/motia-examples/tree/main/examples/streaming-ai-chatbot

Development Tools:

 * Motia NPM: https://www.npmjs.com/package/motia

Interested in partnering with us? Get in touch: partners@mlops.community

Thanks for reading. See you in Slack [https://go.mlops.community/slack], YouTube [https://www.youtube.com/channel/UCG6qpjVnBTTT8wLGBygANOQ?view_as=subscriber], and podcast [https://home.mlops.community/public/content/] land. Oh yeah, and we are also on X [https://twitter.com/mlopscommunity] and LinkedIn [https://go.mlops.community/linkedin].

Thanks again to Rohit Ghumare [https://www.linkedin.com/in/rohit-ghumare/] for his contribution.

---
Source: https://aaif.live/newsletters/mlopscommunity/2025-09-02-unified-backend-framework-for-apis-background-jobs-and-ai-ag