Skip to content
Sachin Rana

Agentic AI & LLMOps

Agents that do the work, and know when to ask.

I build AI agents that plan, call tools against your real systems, reason over live data and digital twins, and stop for a person before anything risky. Then I run them in production, on infrastructure you control.
Brief me on your agent ideaTry the console

01Live walkthrough

Four problems, four agents

Pick a problem and watch the full trace: the plan, every tool call and its result, the guardrail, and the outcome. Each one is drawn from delivered work.

Agent trace · simulated walkthrough

Goal

Find what is driving spend and propose a cut that keeps latency and security intact.

    A scripted walkthrough of how my agent systems reason. No live model runs here, and nothing you click leaves your browser.

    02Architecture

    What sits under a production agent

    A demo agent is a prompt and an API key. A production agent needs six more layers. I design, build and operate all of them.

    • LLM gateway

      One private entry point for every model: routing, fallback, per-team token budgets and cost attribution. Your prompts never hit a public multi-tenant endpoint unless you choose it.

    • MCP tool bus

      Agents act through typed tools (databases, twins, ticketing, IaC) exposed over the Model Context Protocol, each with its own auth and scope.

    • Memory & retrieval

      Exact operational facts, episodic history and hybrid vector search, so agents answer from your data with citations instead of guessing.

    • Guardrails & approval

      Read-only by default. Anything that changes production, money or customers stops at a human approval gate, and every step is logged.

    • Evals & observability

      Traces for every run, regression evals before each change, and latency and cost tracked per tool call.

    • Private inference

      Open-weight models on your own GPUs when data cannot leave, with quantization tuned for throughput.

    Sovereign AI loop: agents call tools through an MCP bus, routed by an LLM gateway to private GPU inference with memory and guardrails.
    Sovereign AI Reasoning & FastMCP Tool Bus. Reference topology; client specifics and addressing omitted.

    Sovereign AI Reasoning & FastMCP Tool Bus

    High-density bare-metal AI infrastructure decoupling model reasoning from tool execution through the Model Context Protocol (FastMCP) with strict token budgets.

    Cost Reduction
    85% vs Cloud API Bills
    Context Window
    128k+ Tokens Sovereign
    Tool Latency
    < 40ms JSON-RPC Handshake
    Security
    OWASP LLM Top 10 Sandboxing

    03In production

    What I have built and run

    Architecting sovereign on-premise AI inference stacks, centralized LLM routing proxies, FastMCP tool ecosystems, and multi-agent autonomous engineering teams with episodic memory.
    • LiteLLM
    • vLLM
    • Ollama
    • FastMCP
    • MemOS
    • Qdrant
    • DeepSeek
    • Qwen 2.5/3.5
    • GGUF IQ4/Q4

    Sovereign AI & Bare-Metal Local Inference Stacks

    High-throughput, private LLM serving infrastructure deployed directly on private bare metal to eliminate cloud API egress, token taxation, and confidentiality leaks.

    Show detail (3)
    • GGUF Quantization Precision: Deep experience selecting optimal quantization profiles (IQ4_XS, Q4_K_M, Q5_K_M, Q8_0) balancing perplexity scores against memory bandwidth limits.
    • PagedAttention & KV Cache Paging: Tuned vLLM serving parameters (gpu_memory_utilization=0.92, max_num_seqs=256) to maximize token throughput for concurrent agent requests.
    • Sub-Second Tool-Calling Models: Championed LiquidAI LFM2-1.2B-Tool-GGUF (1.16 GiB VRAM footprint, 0.24s latency) as the optimal lightweight edge model for instant JSON tool synthesis.

    LiteLLM Gateway & Multi-Model Proxy Architecture

    Production LLM routing gateway managed under systemd and Docker, providing unified OpenAI-compatible endpoints with governance and resilience.

    Show detail (5)
    • Centralized Routing Engine: Deployed LiteLLM Gateway routing across local GPUs (Ollama, vLLM) and hosted foundation models (DeepSeek V4 Flash, MiniMax, Kimi K2.7).
    • Multi-Tenant Quota & Key Management: Backed by PostgreSQL database for persistent API key generation, team budget caps, and audit logs.
    • Real-Time Rate Limiting: Backed by Redis cluster enforcing precise Requests Per Minute (RPM) and Tokens Per Minute (TPM) ceilings per client application.
    • Automated Fallback Cascades: Dynamic router configuration that intercepts 429 rate limits or 500 server timeouts and falls back to secondary models in under 150ms without dropping client SSE streams.
    • Enterprise Guardrail Pipeline: Built custom code guardrails (SqlSafetyGuard) intercepting destructive SQL keywords (DROP, TRUNCATE, ALTER) in prompts before reaching database MCPs, alongside Presidio PII filtering.

    Model Context Protocol (MCP) Server Infrastructure

    Custom FastMCP servers transforming static AI models into autonomous systems capable of executing database queries, system operations, and telemetry inspections.

    Show detail (4)
    • clickhouse-mcp: Executes read-only analytical SQL against ClickHouse production dataset, returning structured column/row JSON arrays.
    • duckdb-mcp: High-speed analytical inspection over local Parquet time-series dumps without database server overhead.
    • postgres-mcp: Transactional data access for multi-tenant LiteLLM metadata and user state management.
    • sqlite-mcp: Direct access to local episodic memory databases and state stores.

    Autonomous Multi-Agent Teams & A2A Orchestration

    Production multi-agent engineering workflows leveraging hierarchical Agent-to-Agent (A2A) REST protocols and specialized role gates.

    Show detail (3)
    • A2A Inter-Agent Mesh: Built multi-container agent teams where Master/Manager agent orchestrates and assigns sub-tasks to Employee worker agents via authenticated A2A protocols.
    • Specialized Agent Role Gates: Strict separation of concerns across agent personas: Engineer (code generation & dynamic patching), Architect (topology design & security policy), Auditor (forensics & automated test validation), and CFO (cloud tax & ROI auditing).
    • Strict Human-in-the-Loop Governance: Built-in safety interlocks requiring explicit manual authorization before any infrastructure mutation, server credential change, or firewall policy update.

    Persistent Episodic & Holographic Memory Architecture

    Multi-tiered memory architecture providing autonomous agents with long-term cross-session recall and verified domain ground truth.

    Show detail (3)
    • MemOS Holographic Fact Store (SQLite WAL Engine (~/.hermes/memory_store.db)): Single source of truth for exact operational facts, service inventories and device mappings. Supports probe, reason, search, and update actions.
    • EverOS Semantic Cloud Recall (EverOS Cloud Vector Recall (api.evermind.ai)): Cross-session contextual memory storing behavioral traits, conversational history, and architectural decisions, synchronized via daily cron at 23:30 IST.
    • Qdrant Vector Database & RAG Studio (Qdrant (self-hosted) · RAG Studio): Dense and hybrid vector search over engineering documentation, skills, and hardware manuals across collections ideaxa-hermes-agent and hermes-production-memory.

    Have a workflow an agent should be doing?

    Send a two-minute brief. You get a reply from me with a first view on approach, timeline and cost.

    Send a project briefEmail directly