Skip to main content

Answer AI - RAG

Answer AI is a RAG (Retrieval-Augmented Generation) system responsible for generating contextual responses within the ai12z platform. ReAct treats this as the default tool (i.e., default integration). Being the default means that even if the ReAct LLM is unsure whether RAG can answer the question and no other integration can provide an answer, ReAct will still call Answer AI, which will then decide whether to respond or return "I don't know."

This section provides comprehensive guidance on configuring system prompts, managing model settings, and utilizing version control features to optimize your AI's performance.

Answer AI Configuration Interface​

The Answer AI configuration page provides a comprehensive interface for customizing your AI's behavior and settings:

  • Breadcrumb Navigation: Home > AI Settings > Answer AI
  • Purpose Statement: "Instruct the AI on how to respond to queries after retrieving relevant content from the vector database. This includes guidance on behavior and tone of voice."

Main Configuration Tabs​

  • Answer AI Prompt: Primary system prompt configuration and editing
  • History: Version control and comparison tools for prompt changes

System Prompt Editor​

The main editing interface includes:

  • Line-numbered Code Editor: Professional code editor with syntax highlighting
  • Real-time Editing: Direct editing of system prompts with immediate feedback
  • Current Content Display: Shows active prompt configuration
  • Save Functionality: Preserve changes and apply to the AI system

Configuration Options​

  • Temperature Slider: Control response creativity and randomness (0-1 scale)
  • Top P Slider: Nucleus sampling parameter for response variety (0-1 scale)
  • No Of Documents: Set the number of context documents to retrieve (adjustable slider)
  • Model Selection: Choose from available AI models via dropdown menu

Preview Section​

  • Live Preview: See how your prompt changes affect AI responses
  • Context Display: View current prompt content and formatting
  • Reference Information: Shows current time and date context available to the AI

Answer AI Overview​

Answer AI is a Retrieval-Augmented Generation (RAG) integration within the ai12z platform, designed to deliver accurate, context-based answers to user queries. It works by combining advanced reasoning (ReAct), a vector database for deep semantic search, and robust guardrails to ensure answers stay grounded in verified content.

Answer AI Block diagram

How It Works​

  1. User Query: The user submits a question via the AI search or chatbot interface.
  2. ReAct Engine: The ReAct (Reasoning & Action with Context, History, and Vision) module acts as the intelligent brain, orchestrating the retrieval and reasoning steps, it creates the vector query.
  3. Vector DB Search: The system performs a semantic search against the vector database, which stores content and metadata (including page text, tags, images, and source URLs) ingested from multiple sources—CMS, web scrapers, document repositories, knowledge bases, and cloud platforms.
  4. Context Filtering: Only the most relevant documents (typically the top 20) are selected and supplied as context to the LLM.
  5. System Prompt: The system prompt defines strict behavior for the LLM, ensuring that answers are generated strictly based on the provided context and with all required safety and compliance guardrails in place.
  6. Image AI Match: If relevant, Answer AI can also match images from the source content to further enrich answers.
  7. Streaming Results: The final answer—potentially including relevant images—is streamed back to the user in real time.

Key Features​

  • Contextual RAG: Answers are always grounded in the retrieved content—no hallucinations.
  • Guardrails: Strict controls are enforced through the system prompt to ensure safety, compliance, and factual accuracy.
  • Integrated with ReAct: Leverages contextual history, user actions, and multimodal content.
  • Default Integration: Answer AI is the default integration, ReAct will evaluate which integratin it will use, if the answer can not come from another integration it will default to the Answer AI

Comparison Feature in Answer AI​

When comparing multiple items such as products, ReAct enables a parameter called requiresReasoning. In this mode, ReAct sends separate vector queries for each product, calling Answer AI N times in parallel. The results are then aggregated by ReAct to generate a comprehensive comparison, delivering fast and accurate responses.

If requiresReasoning is false, Answer AI streams the response directly to the user without waiting for additional reasoning steps, resulting in faster output for standard questions.

Use Cases​

  • Knowledge base Q&A for websites and internal portals
  • Document retrieval and compliance
  • Product or support search, with live content grounding
  • Any scenario where you need trustworthy, explainable answers based on your own data

Answer AI

Dynamic Tokens​

Dynamic tokens are placeholders within a prompt that are replaced with dynamic content when a prompt is processed. Here is a list of Dynamic tokens you can use:

  • {query}: Inserts the user's question.
  • {vector-query}: Inserts the user's question.
  • {history}: Inserts the conversation history.
  • {title}: Inserts the title of the page where the search is conducted.
  • {origin}: Inserts the URL of the page.
  • {language}: Inserts the language of the page.
  • {referrer}: Inserts the referring URL that directed the user to the current page.
  • {attributes}: Allows insertion of additional; javascript would add it to the search control.
  • {org_name}: Inserts the name of the organization for which the Agent is configured.
  • {purpose}: Inserts the stated purpose of the AI bot.
  • {org_url}: Inserts the domain URL of the organization the bot is configured for.
  • {context_data} : Used by Answer AI output from the vector db
  • {tz=America/New_York} : Timezone creates the text of what is the time and date for the LLM to use
  • {image_upload_description} : Image descriptions of images uploaded by the site visitor into the search box processed by vision AI

Models that ai12z support for RAG​

Updated model selection interface showing expanded model options

OpenAI Models:

  • GPT-4.1-mini: Latest compact version with improved efficiency
  • GPT-4.1: Enhanced reasoning and language understanding capabilities
  • GPT-4o: Advanced multimodal model with vision capabilities
  • GPT-4o-mini: Default - Balanced performance and speed for most use cases

Google Models:

  • Gemini 2.5 Pro: Professional-grade model with advanced reasoning
  • Gemini 2.5 Flash: Fast response model optimized for speed
  • Gemini 2.5 Flash Lite: Lightweight version for basic tasks

Meta Models:

  • Llama 4 Maverick: Advanced reasoning with improved context handling
  • Llama 4 Scout: Optimized for exploration and discovery tasks

Anthropic Models:

  • Claude Opus 4: Latest generation with enhanced context understanding
  • Claude Sonnet 4: Balanced model for creative and analytical tasks

Amazon Models:

  • Amazon Nova Micro: Compact model for basic operations
  • Amazon Nova Lite: Lightweight option with good performance
  • Amazon Nova Pro: Professional-grade model for complex tasks
  • Amazon Nova Premier: Premium model with advanced capabilities

Defining System and User Prompts​

Auto Generate

You can re-edit the Agent information and check the check box, "Recreate Prompts" AI will use the information you entered when creating the Agent to recreate the system prompt. You can always go back and re-edit those properties. Your existing system prompt will be stored in history, so you can go back and compare.

Editing System Prompts​

When editing the System prompt, consider the following guidelines:

  • Keep the instructions clear and concise so the AI can understand them.

Best Practices​

  • Test the System prompts to see how the AI interprets them with different types of user queries.
  • Update prompts regularly to align with Agent goals or organization strategies changes.
  • Monitor the AI's responses to ensure they meet the expected quality and adjust the prompts accordingly.

Saving Changes​

After making your changes:

  1. Review the prompt to ensure accuracy and completeness.
  2. Click the "Save" button to apply the changes.
  3. Test the updated prompt with a few queries to ensure it functions as expected.

History and Version Control​

The History tab provides comprehensive version control and comparison tools for tracking prompt changes over time.

Version Management​

Every time you update the Answer AI System Prompt, a version is automatically saved to History, allowing you to:

  • Track Changes: Monitor how prompts evolve over time
  • Revert Changes: Return to previous versions if needed
  • Compare Versions: Side-by-side comparison of different prompt versions

History Interface Features​

  • Dropdown Selection: Choose specific saved versions from the history dropdown
  • Timestamp Display: Each version shows save date and time (e.g., "Sep 01, 2025, 07:14:03 PM")
  • Source Identification: Shows what triggered the save (manual edit or "Recreate Prompts" option)

Version Comparison Tool​

The History tab includes a powerful diff comparison feature:

Current Content vs History Content:

  • Left Panel: Shows the currently active prompt
  • Right Panel: Shows the selected historical version
  • Line-by-Line Comparison: Highlights specific changes between versions
  • Color-Coded Differences:
    • Blue highlighting for modified sections
    • Clear visual indicators of additions, deletions, and modifications

Comparison Features:

  • Syntax Highlighting: Both panels maintain code formatting and highlighting
  • Line Numbers: Easy reference for specific changes
  • Side-by-Side View: Simultaneous viewing of both versions
  • Detailed Diff Analysis: See exactly what changed between versions

Auto-Generated Versions​

In addition to manual saves, new versions are automatically created when you:

  • Edit the Agent and select the "Recreate Prompts" option
  • Make significant changes to agent configuration
  • Update system-level settings that affect prompt generation

This ensures comprehensive tracking of all prompt modifications, whether manual or system-generated.

History comparison interface showing side-by-side diff view

Conclusion​

Managing prompts effectively is crucial for the optimal performance of your AI interface. By following the guidelines outlined in this document, you can ensure that your AI provides relevant, accurate responses and in line with your organization's objectives.