Enterprise GenAI Assistant

The Challenge

A global enterprise needed a conversational AI layer over their SAP landscape without exposing raw system access or risking cross-tenant data leakage. The core challenge was building a secure, scalable RAG pipeline capable of ingesting and retrieving tenant-specific data in real time, while keeping response latency low enough for a production chat interface used by thousands of employees.

Technology

SAP AI Core

Generative AI Hub

CAP Service Layer

Cloud Foundry

Architecture

My Role

• Defined the overall solution architecture and led technical design across all workstreams
• Designed the multi-tenant data isolation model and RAG pipeline using LangChain and SAP AI Core vector store capabilities
• Selected and evaluated LLMs available through Generative AI Hub against latency, cost, and accuracy criteria
• Built the CAP service layer exposing AI endpoints with role-based access control and audit logging
• Ran architecture validation workshops with the customer and SAP product stakeholders

Outcomes

• Deployed to production for multiple enterprise tenants within a single BTP subaccount
• Average response latency under 3 seconds for retrieval-augmented queries
• Zero cross-tenant data exposure incidents since go-live
• Architecture pattern adopted as an internal reference design for similar engagements