RAG System

User asks: "How do I configure authentication?"
    |
    v
Step 1: Query Enhancement
    |--- If conversation history exists:
    |    LLM rewrites vague follow-up → self-contained query
    |    Example: "What else?" → "What other Clerk auth features are available?"
    |--- If standalone question: use as-is
    |
    v
Step 2: Embedding Generation
    |--- OpenAI text-embedding-3-small
    |--- Query → 1536-dimensional vector
    |
    v
Step 3: pgvector Similarity Search
    |--- Cosine similarity: 1 - (embedding <=> query_vector)
    |--- Filter by similarity threshold (default: 0.35)
    |--- Optional category filter
    |--- Returns top 5 chunks (configurable)
    |
    v
Step 4: Context Assembly
    |--- Format chunks: [Source 1: Category]\nQ: ...\nA: ...
    |--- Estimated ~3-5K tokens (vs ~90K for full FAQ)
    |
    v
Step 5: LLM Generation
    |--- System prompt (tier-customized)
    |--- Context from relevant chunks
    |--- Conversation history
    |--- Answer with source references
    |
    v
Step 6: Response with Attribution
    |--- answer: "To configure authentication, ..."
    |--- sources: [{ question, category, similarity }]
    |--- chunksUsed: 3
    |--- tokensEstimated: 4200
    |--- usage: { promptTokens, completionTokens }

async answerQuestionWithRAG(
    question: string,
    options: AnswerWithRAGOptions = {}
  ): Promise<RAGAnswer> {
    const {
      userId: _userId,
      userTier,
      categories,
      limit = 5,
      similarityThreshold = 0.35, // Adjusted for cross-lingual queries and semantic variation
      conversationHistory = [],
      model: _model,
      temperature: _temperature,
    } = options

interface RAGAnswer {
  answer: string           // Generated answer text
  sources: Array<{
    question: string       // Original FAQ question
    category: string       // Knowledge base category
    similarity: number     // Cosine similarity score (0-1)
  }>
  chunksUsed: number       // Number of chunks used
  tokensEstimated: number  // Estimated context tokens
  usage?: {                // Actual token usage (if available)
    promptTokens: number
    completionTokens: number
    totalTokens: number
  }
  model?: string           // Model used for generation
}

export async function searchRAG(
  query: string,
  options: SearchOptions = {}
): Promise<SearchResult[]> {
  const {
    limit = 5,
    similarityThreshold = 0.35, // Adjusted for cross-lingual queries and semantic variation
    categories,
    includeMetadata = true,
  } = options

  // 1. Generate embedding for user query
  const apiKey = getEmbeddingApiKey()
  const embeddingModelId = getEmbeddingModel()
  const openai = createOpenAI({ apiKey })

  const { embedding: queryEmbedding } = await embed({
    model: openai.embedding(embeddingModelId),
    value: query,
  })

  // 2. Format embedding as PostgreSQL array string
  // Prisma doesn't automatically convert arrays to pgvector format
  const embeddingStr = `[${queryEmbedding.join(',')}]`

  // 3. Build SQL query with pgvector similarity
  // Cosine similarity: 1 - (embedding <=> query_embedding)
  // Higher score = more similar (1.0 = identical, 0.0 = unrelated)

  // Build category filter SQL
  const categoryFilterSQL =
    categories && categories.length > 0
      ? `AND category = ANY(ARRAY[${categories.map((c) => `'${c}'`).join(',')}])`
      : ''

  const metadataSQL = includeMetadata ? 'metadata,' : ''

  // Use $queryRawUnsafe to avoid template literal interpolation limits
  const sql = `
    SELECT
      id,
      content,
      question,
      answer,
      category,
      ${metadataSQL}
      1 - (embedding <=> '${embeddingStr}'::vector) as similarity
    FROM faq_chunks
    WHERE 1 - (embedding <=> '${embeddingStr}'::vector) > ${similarityThreshold}
    ${categoryFilterSQL}
    ORDER BY embedding <=> '${embeddingStr}'::vector
    LIMIT ${limit}
  `

  const results = await prisma.$queryRawUnsafe<SearchResult[]>(sql)

  return results
}

/**
   * Rewrite vague or follow-up questions using conversation context
   * This makes RAG searches work better for multi-turn conversations
   *
   * @param question - Current user question (may be vague or a follow-up)
   * @param history - Previous conversation messages
   * @returns Enhanced, self-contained question for RAG search
   *
   * @example
   * // Original: "Und was noch?"
   * // With context about Clerk auth
   * // Enhanced: "What other Clerk authentication features are available in the boilerplate?"
   */
  private async rewriteQueryWithContext(
    question: string,
    history: Message[]
  ): Promise<string> {
    // Take last 3 messages for context (avoid token bloat)
    const recentHistory = history.slice(-3)

    // Build prompt for query rewriting
    const contextPrompt = `You are a query rewriting assistant for a technical FAQ system about a Next.js SaaS Boilerplate.

Rewrite the user's follow-up question to be self-contained and specific for vector database search.

Conversation History:
${recentHistory.map((m) => `${m.role}: ${m.content}`).join('\n')}

Current Question: ${question}

Instructions:
1. If the question is already clear and specific, return it unchanged
2. If it's a follow-up (e.g., "Und was noch?", "What about X?"), incorporate context from history
3. Make it technical and specific to the Next.js boilerplate
4. Keep it concise (1-2 sentences max)
5. Output ONLY the rewritten question, nothing else

Rewritten Question:`

    try {
      const response = await this.aiService.answerQuestion(question, {
        context: contextPrompt,
        systemPrompt:
          'You are a query rewriting assistant. Output ONLY the rewritten question, nothing else.',
        stream: false,
      })

      const rewrittenQuery =
        response?.choices[0]?.message?.content?.trim() || question

      // Log for debugging (helps tune the system)
      console.log('[RAG Service] Query Rewriting:', {
        original: question,
        enhanced: rewrittenQuery,
        historyLength: recentHistory.length,
        changed: rewrittenQuery !== question,
      })

      return rewrittenQuery
    } catch (error) {
      // If rewriting fails, fall back to original question
      console.warn(
        '[RAG Service] Query rewriting failed, using original:',
        error
      )
      return question
    }
  }

No chunks found for: "How do I deploy to AWS?"
    |
    v
Fallback prompt injected:
    |--- Acknowledge limitation: "I don't have specific FAQ content..."
    |--- Provide boilerplate guidance based on tech stack
    |--- Reference relevant files/folders
    |--- Suggest rephrasing: "Could you ask more specifically?"
    |
    v
LLM generates helpful response WITHOUT hallucinating

{
  "question": "How do I configure Clerk authentication?",
  "conversationHistory": [
    { "role": "user", "content": "Tell me about auth" },
    { "role": "assistant", "content": "Kit uses Clerk for..." }
  ],
  "categories": ["Authentication"],
  "stream": true
}

{
  "answer": "To configure Clerk authentication...",
  "sources": [
    {
      "question": "How is Clerk integrated?",
      "category": "Authentication",
      "similarity": 0.87
    }
  ],
  "chunksUsed": 3,
  "tokensEstimated": 4200
}

CREATE TABLE faq_chunks (
  id          TEXT PRIMARY KEY DEFAULT gen_random_uuid(),
  question    TEXT NOT NULL,
  answer      TEXT NOT NULL,
  content     TEXT NOT NULL,        -- Combined searchable content
  category    TEXT NOT NULL,
  embedding   vector(1536),         -- OpenAI text-embedding-3-small
  metadata    JSONB,                -- Tags, related topics, keywords
  token_count INTEGER DEFAULT 0,
  created_at  TIMESTAMP DEFAULT NOW(),
  updated_at  TIMESTAMP DEFAULT NOW()
);

-- pgvector index for fast similarity search
CREATE INDEX idx_faq_chunks_embedding
  ON faq_chunks
  USING ivfflat (embedding vector_cosine_ops)
  WITH (lists = 100);

# Generate embeddings and seed the knowledge base
cd apps/boilerplate && npx prisma db seed

# Or run the FAQ seeder directly
cd apps/boilerplate && npx tsx prisma/seed-faq.ts