Querying SAP HANA Vectors with LlamaIndex and hdbcli

This guide demonstrates how to build a Model Context Protocol (MCP) server that acts as a bridge between an AI Agent and SAP HANA Cloud’s Vector Engine.

By retrofitting SAP HANA with an MCP interface, you allow agents (like CrewAI or generic LLM implementations) to perform semantic search (RAG) against your enterprise data without moving it out of SAP’s secure environment.

🏗️ Architecture Blueprint

We will create a Python-based FastMCP server that:

Connects to SAP HANA Cloud using the native hdbcli driver.
Uses LlamaIndex to abstract the Vector Search logic (handling embeddings and SQL generation).
Exposes a tool query_sap_knowledge via the Model Context Protocol (SSE).

Prerequisites

SAP HANA Cloud instance with the Vector Engine enabled.
OpenAI API Key (for generating embeddings for the incoming query).
Docker for containerization.

💻 The Bridge Code (`server.py`)

This server exposes a single tool. When an agent asks a question, the server embeds the query, sends it to SAP HANA to find the nearest cosine-similarity matches, and returns the context.

import os
import logging
from typing import List

# Standard library
from contextlib import asynccontextmanager

# MCP Framework
from mcp.server.fastmcp import FastMCP

# SAP and LlamaIndex
from hdbcli import dbapi
from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.vector_stores.hana import HanaVectorStore
from llama_index.embeddings.openai import OpenAIEmbedding

# Configure Logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# Initialize FastMCP
mcp = FastMCP("sap-hana-vector-bridge")

def get_hana_connection():
    """
    Establishes a connection to SAP HANA using hdbcli.
    # Ensure your container has network access (e.g. via NordLayer)
    """
    try:
        conn = dbapi.connect(
            address=os.environ.get("SAP_HANA_HOST"),
            port=int(os.environ.get("SAP_HANA_PORT", "443")),
            user=os.environ.get("SAP_HANA_USER"),
            password=os.environ.get("SAP_HANA_PASSWORD"),
            encrypt="true",
            sslValidateCertificate="false"  # Adjust for production
        )
        return conn
    except Exception as e:
        logger.error(f"Failed to connect to SAP HANA: {e}")
        raise

@mcp.tool()
def search_sap_knowledge_base(query: str, limit: int = 3) -> str:
    """
    Semantically searches the SAP HANA vector store for relevant documentation or data.

    Args:
        query: The natural language question (e.g., "How do I process a return in SD?").
        limit: Number of results to return.
    """
    conn = None
    try:
        logger.info(f"Processing query: {query}")

        # 1. Establish DB Connection
        conn = get_hana_connection()

        # 2. Initialize LlamaIndex abstractions
        # We assume the table 'SAP_VECTORS' already exists and is populated.
        vector_store = HanaVectorStore(
            connection=conn,
            table_name="SAP_VECTORS",
            content_field="TEXT_CONTENT",
            embedding_field="EMBEDDING_VECTOR"
        )

        # 3. Setup Embedding Model (Must match what you used to ingest data)
        embed_model = OpenAIEmbedding(api_key=os.environ.get("OPENAI_API_KEY"))

        # 4. Construct Query Engine
        index = VectorStoreIndex.from_vector_store(
            vector_store=vector_store,
            embed_model=embed_model
        )

        retriever = index.as_retriever(similarity_top_k=limit)
        nodes = retriever.retrieve(query)

        # 5. Format Output for the Agent
        results = []
        for i, node in enumerate(nodes):
            results.append(f"Result {i+1} (Score: {node.score:.2f}):\n{node.text}\n")

        return "\n---\n".join(results)

    except Exception as e:
        return f"Error querying SAP HANA: {str(e)}"
    finally:
        if conn:
            conn.close()

if __name__ == "__main__":
    # Binds to 0.0.0.0 to allow external access from Docker host or Agent network
    mcp.run(transport='sse', host='0.0.0.0', port=8000)

🐳 Docker Configuration

To run this in production (or on Railway/AWS), we use a Docker container. We expose port 8000 so the Agent can connect via SSE.

`Dockerfile`

# Use a slim Python image
FROM python:3.11-slim

# Set working directory
WORKDIR /app

# Install system dependencies if needed (e.g. for building wheels)
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies
# hdbcli: SAP HANA Driver
# llama-index-vector-stores-hana: The specific integration package
# mcp: Model Context Protocol SDK
RUN pip install --no-cache-dir \
    hdbcli \
    llama-index \
    llama-index-vector-stores-hana \
    llama-index-embeddings-openai \
    "mcp[cli]"

# Copy the server code
COPY server.py .

# Expose the SSE port
EXPOSE 8000

# Environment variables should be injected at runtime,
# but we can set defaults or placeholders here.
ENV PYTHONUNBUFFERED=1

# Run the FastMCP server
CMD ["python", "server.py"]

🔌 Connecting Your Agent

Once the Docker container is running, your Agent needs to connect to the MCP server.

Option A: CrewAI Configuration

If you are using CrewAI, you can connect directly to the SSE stream.

from crewai import Agent, Task, Crew
from crewai_tools import MCPTool

# Initialize the MCP Tool connecting to your Docker container
sap_vector_tool = MCPTool(
    name="SAP HANA Search",
    mcps=["http://localhost:8000/sse"]  # Connects to the server.py inside Docker
)

sap_agent = Agent(
    role='SAP Specialist',
    goal='Retrieve accurate technical data from legacy SAP systems',
    backstory='You are an AI interface to the corporate SAP HANA knowledge base.',
    tools=[sap_vector_tool],
    verbose=True
)

# Example Task
task = Task(
    description="Find out the procedure for creating a new Vendor in SAP via RFC.",
    agent=sap_agent,
    expected_output="A summary of the vendor creation steps found in the vector DB."
)

crew = Crew(agents=[sap_agent], tasks=[task])
result = crew.kickoff()
print(result)

Option B: Generic Python Client

For custom agent implementations, you can inspect the tools using httpx and mcp.

import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.sse import sse_client

async def main():
    # Connect to the SSE endpoint
    async with sse_client("http://localhost:8000/sse") as (read, write):
        async with ClientSession(read, write) as session:
            # Initialize connection
            await session.initialize()

            # List available tools
            tools = await session.list_tools()
            print(f"Connected to: {tools}")

            # Call the tool
            result = await session.call_tool(
                "search_sap_knowledge_base",
                arguments={"query": "How to configure RFC connections?"}
            )
            print(result)

if __name__ == "__main__":
    asyncio.run(main())

🛠️ Troubleshooting

HANA Connectivity:
- Error: -10709: Connection failed (RTE:[89006] System call 'connect' failed)
- Fix: This usually means the Docker container cannot reach the SAP HANA host. If HANA is on a corporate VPN, ensure your Docker network is configured correctly or use a sidecar container (like NordLayer or Tailscale) to bridge the network.
LlamaIndex Version:
- Ensure you are using llama-index-vector-stores-hana. The core llama-index package no longer contains all vector stores by default; they are namespaced.

Docker Networking:

If running locally, use --network host (Linux) or ensure you map ports -p 8000:8000.

Example run command:

docker run -p 8000:8000 \
  -e SAP_HANA_HOST=zeus.hana.prod.eu-central-1.whitney.dbaas.ondemand.com \
  -e SAP_HANA_PORT=443 \
  -e SAP_HANA_USER=DBADMIN \
  -e SAP_HANA_PASSWORD=YourPassword123 \
  -e OPENAI_API_KEY=sk-... \
  sap-hana-mcp

🛡️ Quality Assurance

Status: ✅ Verified
Environment: Python 3.11
Auditor: AgentRetrofit CI/CD

Transparency: This page may contain affiliate links.

Querying SAP HANA Vectors with LlamaIndex and hdbcli

Querying SAP HANA Vectors with LlamaIndex and hdbcli

🏗️ Architecture Blueprint

Prerequisites

💻 The Bridge Code (server.py)

🐳 Docker Configuration

Dockerfile

🔌 Connecting Your Agent

Option A: CrewAI Configuration

Option B: Generic Python Client

🛠️ Troubleshooting

🛡️ Quality Assurance

💻 The Bridge Code (`server.py`)

`Dockerfile`