The system consists of multiple microservices working together to provide AI-powered parts search through RAG (Retrieval-Augmented Generation).

System Architecture - Complete Guide

🎯 Overview

The system consists of multiple microservices working together to provide AI-powered parts search through RAG (Retrieval-Augmented Generation).

📊 System Components

1. Frontend (React)

Port: 3000 (locally)
Features:
- 💬 Chat with AI agent
- 📄 PDF viewer with part coordinates
- 📊 Parts tables
- 🔗 Interactive linking (click part → search)

2. AI Service (LangChain/LangGraph Agents)

Port: 8001
Functions:
- Process user queries through agents
- RAG search through Weaviate
- Generate contextual responses

3. PDF Parser Service

Port: 8000
Functions:
- Parse PDF files
- Extract tables, schemas, images
- Prepare data for RAG

4. Data Preparation Service (Pipeline)

Port: 8003
Functions:
- D1: Data collection (GCS)
- D2: Data processing (parsing, embeddings)
- D3: Storage in Weaviate

5. Weaviate Service (Vector Database)

Port: 8002
Functions:
- Store documents with embeddings
- Vector search
- RAG retrieval

6. CLIP Service (Image Embeddings)

Port: 8002 (separate service on GPU)
Functions:
- Generate embeddings for images
- Visual search

7. Barcode Service (Optional)

Port: 8004
Functions:
- Detect and recognize barcodes/QR codes

🔄 Data Flows

Scenario 1: Adding New Data via Frontend Admin Panel

Frontend (Admin - upload PDF or part photo)
    ↓
POST /api/upload-pdf (or via Data Preparation Service)
    ↓
Data Preparation Service (Port 8003)
    ↓
    1. Load PDF from GCS or accept from frontend
    2. Call PDF Parser Service
    ↓
PDF Parser Service (Port 8000)
    ↓
    Parse PDF, extract:
    - Part tables
    - Schemas/diagrams
    - Images
    ↓
Data Preparation Service (continues processing)
    ↓
    3. Generate embeddings:
       - Text → Weaviate Service (BGE embeddings)
       - Images → CLIP Service (CLIP embeddings)
    ↓
Weaviate Service + CLIP Service
    ↓
    4. Store documents with embeddings
       - Text documents → PartDocument class
       - Images → PartImage class
    ↓
Weaviate DB ✅ Data ready for RAG search

API Endpoints for adding data:

# Option 1: Via Data Preparation Service (recommended)
POST http://data-prep-service:8003/process-pdf
{
    "pdf_path": "path/to/pdf.pdf",  # Path in GCS or URL
    "document_number": "SPD00805"    # Optional
}

# Option 2: Direct parsing via PDF Parser Service
POST http://pdf-parser-service:8000/parse
# multipart/form-data: file=PDF

# Option 3: For new part with photo
POST http://clip-service:8002/embed
# multipart/form-data: 
#   file=photo.jpg
#   metadata={"part_number": "ABC-123", "category": "engine", ...}

Scenario 2: Search via AI Agents (Chat)

Frontend - User: "Find oil filter for Toyota Camry"
    ↓
POST /query
    ↓
AI Service (Port 8001) - LangChain Agent
    ↓
    1. Agent analyzes query
    2. Generates embeddings for search
       (uses BGE-Large in Weaviate Service)
    ↓
Weaviate Service (Port 8002)
    ↓
    3. Vector search (RAG retrieval)
       - Find similar documents
       - Filter by metadata (Toyota, Camry)
       - Return top-10 results
    ↓
AI Service (continues processing)
    ↓
    4. Agent forms response with context
       - Uses LLaMA 3.1 8B
       - Generates natural response
       - Adds links to parts
    ↓
Frontend - Displays response + parts list

API Endpoints for chat:

# Main chat endpoint
POST http://ai-service:8001/query
{
    "query": "Find oil filter for Toyota Camry",
    "include_store_link": true  # Optional
}

# Response:
{
    "answer": "I found several oil filters for Toyota Camry...",
    "parts": [
        {
            "part_number": "ABC-12345",
            "description": "Engine oil filter",
            "pdf_link": "http://...",
            "page": 42,
            "coordinates": [10, 20, 50, 30],
            "store_link": "https://clintontractor.net/parts/..."
        }
    ],
    "confidence": 0.8
}

Scenario 3: Direct Search Without AI Agents (Frontend → Weaviate/CLIP)

Frontend can use services directly, without AI agents:

Frontend - User: Search by photo or text
    ↓
Option A: Text search
POST /search (Weaviate Service)
{ "query": "oil filter", "filters": {...} }

Option B: Image search
POST /embed (CLIP Service) → get embedding
POST /search (Weaviate Service) with embedding
    ↓
Weaviate Service + CLIP Service
    ↓
Returns search results
    ↓
Frontend - Displays results (table, cards)

Direct API endpoints (without AI agents):

# 1. Text search (via Weaviate Service)
POST http://weaviate-service:8002/search
{
    "query": "oil filter",
    "limit": 10,
    "where": {
        "path": ["manufacturer"],
        "operator": "Equal",
        "valueString": "Toyota"
    }
}

# 2. Visual search (via CLIP + Weaviate)
# Step 1: Convert photo to embedding
POST http://clip-service:8002/embed
# multipart/form-data: file=photo.jpg

# Step 2: Search in Weaviate with embedding
POST http://weaviate-service:8002/search
{
    "vector": [0.123, -0.456, ...],  # From CLIP Service
    "limit": 10
}

🏗️ Detailed API Architecture

Frontend → AI Service (Chat)

Frontend (React)
    ↓
POST /query
    ├─ QueryRequest { query, include_store_link }
    ↓
AI Service (LangChain Agent)
    ├─ Creates embedding via Weaviate Service
    ├─ Searches in Weaviate (RAG retrieval)
    ├─ Generates response via LLaMA
    ↓
QueryResponse
    ├─ answer: str
    ├─ parts: List[PartResponse]
    ├─ confidence: float

Frontend → Data Preparation Service (Adding PDF)

Frontend (Admin Panel)
    ↓
POST /process-pdf
    ├─ ProcessPDFRequest { pdf_path, document_number }
    ↓
Data Preparation Service
    ├─ Loads PDF from GCS or accepts file
    ├─ Calls PDF Parser Service
    │   ├─ POST /parse → parses PDF
    │   ├─ POST /extract-tables → extracts tables
    │   └─ POST /extract-images → extracts images
    ├─ Generates embeddings:
    │   ├─ Text → Weaviate Service (BGE)
    │   └─ Images → CLIP Service
    ├─ Stores in Weaviate:
    │   ├─ POST /store (Weaviate Service)
    │   └─ POST /store-images (Weaviate Service)
    ↓
PipelineRunResponse
    ├─ run_id: str (for MLflow tracking)
    ├─ status: str

Frontend → CLIP Service (Uploading Part Photo)

Frontend (Admin Panel)
    ↓
POST /embed
    ├─ multipart/form-data:
    │   ├─ file: photo.jpg
    │   └─ metadata: JSON string
    ↓
CLIP Service
    ├─ Generates CLIP embedding (768 numbers)
    ├─ Returns embedding + metadata
    ↓
Frontend or Backend
    ├─ Saves to Weaviate:
    │   POST /store (Weaviate Service)
    │   ├─ data_object: { ...metadata, photo_url }
    │   └─ vector: embedding (from CLIP)

Frontend → Weaviate Service (Direct Search)

Frontend
    ↓
POST /search
    ├─ SearchRequest
    │   ├─ query: str (text)
    │   ├─ vector: List[float] (optional, if embedding already exists)
    │   ├─ limit: int
    │   └─ where: Dict (metadata filters)
    ↓
Weaviate Service
    ├─ Generates embedding (if query is text)
    ├─ Vector search in Weaviate DB
    ├─ Applies filters (where)
    ├─ Returns top-K results
    ↓
SearchResponse
    ├─ results: List[Document]
    ├─ scores: List[float]

📋 Recommended Routes for Different Cases

1. Adding New PDF via Admin Panel

# Frontend → Data Preparation Service
POST http://data-prep-service:8003/process-pdf
{
    "pdf_path": "pdfs/manual.pdf",  # Path in GCS
    "document_number": "SPD00805"    # Optional
}

2. Adding New Part with Photo

# Option A: Via CLIP Service + Weaviate Service
# Step 1: Get embedding
POST http://clip-service:8002/embed
files: {"file": photo.jpg}
data: {"metadata": '{"part_number": "ABC-123", ...}'}

# Step 2: Save to Weaviate
POST http://weaviate-service:8002/store
{
    "documents": [{
        "content": "...",
        "embedding": [...],  # From CLIP
        "metadata": {...}
    }]
}

3. Search via Chat (with AI Agent)

# Frontend → AI Service
POST http://ai-service:8001/query
{
    "query": "Find oil filter for Toyota",
    "include_store_link": true
}

4. Direct Search (without AI Agent)

# Option A: Text search
POST http://weaviate-service:8002/search
{
    "query": "oil filter",
    "limit": 10,
    "where": {
        "path": ["manufacturer"],
        "operator": "Equal",
        "valueString": "Toyota"
    }
}

# Option B: Visual search
# 1. Get embedding from photo
POST http://clip-service:8002/embed
files: {"file": photo.jpg}

# 2. Search in Weaviate
POST http://weaviate-service:8002/search
{
    "vector": [0.123, -0.456, ...],  # From step 1
    "limit": 10
}

🔐 Integration Recommendations

For Frontend Developers:

Chat with AI agent: Use AI Service (/query)
Upload PDF: Use Data Preparation Service (/process-pdf)
Upload part photo: Use CLIP Service (/embed) → Weaviate Service (/store)
Direct search: Use Weaviate Service (/search) or CLIP Service + Weaviate Service

Order for Adding New Data:

1. Upload PDF/photo → GCS (optional)
2. Call Data Preparation Service (/process-pdf) or CLIP Service (/embed)
3. Data automatically enters Weaviate through pipeline
4. Data ready for RAG search via AI Service or directly via Weaviate Service

Monitoring and Tracking:

MLflow: Track pipeline runs via Data Preparation Service
Event Sourcing: Weaviate Service stores all change events
Health checks: All services have /health endpoints

🎯 Conclusion

Main workflow:

Adding data → Data Preparation Service or CLIP Service → Weaviate
AI search → AI Service → Weaviate → AI Service → Frontend
Direct search → Weaviate Service (or CLIP + Weaviate) → Frontend

Frontend can:

✅ Use AI Service for chat
✅ Use Weaviate Service for direct search
✅ Add data via Data Preparation Service or CLIP Service

System Architecture - Complete Guide

On this page