LLM Provider Configuration

This guide explains how to configure Large Language Model (LLM) providers in Simba Intelligence. LLM providers power the AI capabilities that enable natural language querying, data source creation, and intelligent analysis features.

Overview

What are LLM Providers?

LLM providers are external AI services that power Simba Intelligence’s natural language capabilities. Think of them as the “brains” behind the system that understand your questions, analyze your data requirements, and generate intelligent responses. When you ask “What were our top-selling products last quarter?” in the Playground, an LLM provider:

Understands your natural language question
Analyzes your available data sources
Generates appropriate SQL queries
Interprets the results into meaningful insights

Supported LLM Providers

Simba Intelligence requires LLMs with structured output support, since JSON schema is used throughout. Older models like GPT-3.5 do not support this and cannot be used.

📝 Note: Tiers appear under two names depending on where you’re looking: the /llm-configuration UI labels them Lite and Power, while code, configuration, and logs use the underlying values LOW and HIGH respectively. These are the same two tiers, not four — see Understanding Tiers below.

Provider	Model	Tested	Tier Recommendation
🔵 Azure OpenAI	GPT-5.4+	✅ Tested	Lite or Power
🔵 Azure OpenAI	GPT-5.4+ Nano	✅ Tested	Lite
🟢 Vertex AI	Gemini 2.5+ Flash	✅ Tested	Lite
🟢 Vertex AI	Gemini 2.5+ Pro	✅ Tested	Power
🟠 AWS Bedrock	Claude Haiku 4.6+	✅ Tested	Lite
🟠 AWS Bedrock	Claude Sonnet 4.6+	✅ Tested	Lite or Power
🟠 AWS Bedrock	Claude Opus 4.6+	✅ Tested	Power

📝 Note: A Power (HIGH) model is required for more complex agents, such as the data source agent.

Gemini Configuration Notes

Gemini Flash: Set thinking_budget to 0 for maximum speed, or 128 for better quality
Gemini Pro: Set thinking_budget between 128 and 32768

Learn more about thinking

AWS Bedrock Configuration Notes

Claude Sonnet: Set max_tokens to 10000 to avoid failures. Leaving this parameter blank may cause unexpected errors.

Provider Capabilities

💬 Chat Capability

Natural language understanding and generation
Query interpretation and response creation
Business insight generation

🔍 Embeddings Capability

Semantic search and similarity matching
Intelligent caching of similar queries
Content understanding for better results

👁️ Vision Capability

Dashboard image analysis
Screenshot-to-data-source conversion
Visual mockup interpretation

LLM Tiers and Inheritance

The Chat capability can be configured at a tier, letting you match model cost/speed to the task. This section explains what tiers are and how Simba Intelligence resolves which configuration actually serves a request.

Understanding Tiers

Chat requests specify one of two tiers:

Tier (UI label)	Value (code / logs)	Purpose	Typical use
Power	`HIGH`	Powerful model for complex reasoning	Chat orchestration, data source/schema analysis
Lite	`LOW`	Fast, high-throughput model	Data retrieval, suggestions, token counting

The /llm-configuration UI shows the friendlier Lite / Power names, but the value actually stored, sent through the API, and printed in logs is LOW / HIGH. The rest of this guide uses HIGH/LOW since that’s what you’ll see in configuration, log output, and error messages. There is also Unspecified — leaving a capability’s tier field blank in the configuration UI. An unspecified configuration acts as a catch-all that can serve either a HIGH or LOW request, which is useful for simple tenants that only need a single model instead of separate HIGH/LOW configurations.

📝 Note: Only CHAT capability configurations use tiers. EMBEDDINGS and VISION configurations are not tiered — a tenant has at most one active configuration per non-chat capability.

Tier Inheritance (Fallback Chain)

Every tenant is not required to configure both tiers. If a tenant only configures HIGH, requests for LOW still need to resolve to something. Simba Intelligence resolves this using an ordered fallback chain, checking the following in order and stopping at the first match:

Own tenant, exact requested tier
Own tenant, Unspecified tier
Master tenant (VDD), exact requested tier
Master tenant (VDD), Unspecified tier
Own tenant, opposite tier (e.g. requested LOW, only HIGH is configured)
Master tenant (VDD), opposite tier — last resort

Key behaviors to understand:

“Master tenant” refers to the Composer/VDD (Visual Data Discovery) admin tenant, which acts as an org-wide fallback pool for tenants that don’t have their own configuration for a given tier.
Own-tenant options are always exhausted first. Steps 1–2 are checked before the master tenant is even looked up. This means a tenant with a valid Unspecified-tier configuration will never fall through to the master tenant for that capability.
If the current tenant is the master/VDD tenant, the master-specific steps (3, 4, 6) are skipped since they’d be duplicates of the own-tenant checks.
Non-chat capabilities (EMBEDDINGS, VISION) skip tier resolution entirely — they use a simpler own-tenant-then-master fallback.
Vision additionally falls back to the CHAT capability at HIGH tier if no vision capability is configured for the tenant at all.

Example: A tenant configures only a HIGH-tier Chat capability (no LOW, no Unspecified). A request for the LOW tier will check the tenant’s own LOW and Unspecified configs (steps 1–2, no match), then the master tenant’s HIGH and Unspecified configs (steps 3–4, no match), and finally fall back to the tenant’s own HIGH configuration (step 5) — meaning the “fast” tier silently gets served by the “powerful” model instead of failing. Use the Logging Configuration Guide to observe which step of this chain served any given request.

Prerequisites

Required Permissions

To configure LLM providers, you need:

Supervisor role in Simba Intelligence
Access to LLM Configuration interface (/llm-configuration)
Administrative privileges for system-wide AI configuration

AI Provider Account Requirements

For each provider you want to use: Google Vertex AI:

Google Cloud Platform account with billing enabled
Vertex AI API enabled in your GCP project
Service account with Vertex AI permissions
Sufficient API quota for your expected usage

Azure OpenAI:

Azure OpenAI resource
API key with appropriate usage limits
Sufficient credits/quota for your organization’s needs

AWS Bedrock:

AWS account with Bedrock access enabled
IAM user or role with Bedrock permissions
Model access granted for Claude or other desired models
Appropriate usage quotas configured

Accessing LLM Configuration

Log into Simba Intelligence with supervisor credentials
Navigate to LLM Configuration:
- Option 1: Visit directly at http://your-domain/llm-configuration
- Option 2: User menu → “LLM Configuration”
Verify access: You should see a tabbed interface with available providers

Configuration Interface Overview

The LLM Configuration interface provides:

Provider tabs: Separate tabs for each supported AI provider
Configuration forms: Provider-specific credential and parameter forms
Capability management: Enable/disable different AI capabilities per provider
Testing tools: Built-in connection testing and validation
Status indicators: Real-time status of provider configurations

Google Vertex AI Configuration

Google Vertex AI provides comprehensive AI capabilities, including advanced vision analysis for dashboard image processing.

Prerequisites for Vertex AI

Google Cloud Platform setup:

Create or select a GCP project with billing enabled

Enable Vertex AI API:

gcloud services enable aiplatform.googleapis.com

Create a service account:

gcloud iam service-accounts create simba-intelligence-ai \
  --display-name="Simba Intelligence AI Service"

Grant necessary permissions:

gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:simba-intelligence-ai@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

Generate service account key:

gcloud iam service-accounts keys create simba-intelligence-key.json \
  --iam-account=simba-intelligence-ai@YOUR_PROJECT_ID.iam.gserviceaccount.com

Vertex AI Configuration Process

Access Vertex AI tab in LLM Configuration interface

Enter service account JSON:

Copy the complete contents of your service account JSON file
Paste into the credentials text area
The JSON should include all required fields:

{
  "type": "service_account",
  "project_id": "your-gcp-project-id",
  "private_key_id": "your-private-key-id",
  "private_key": "-----BEGIN PRIVATE KEY-----\nYour-Private-Key-Content-Here\n-----END PRIVATE KEY-----\n",
  "client_email": "simba-intelligence-ai@your-project.iam.gserviceaccount.com",
  "client_id": "your-client-id",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/simba-intelligence-ai%40your-project.iam.gserviceaccount.com"
}

Configure global (shared) parameters — these apply across all capabilities for this provider:

location:        us-central1   (required, e.g. your preferred region)
temperature:     0.7           (optional, 0.0–1.0)
max_tokens:      8192          (optional)
thinking_budget: 128           (optional, 0 for max speed; 128–32768 for Pro)

Enable and configure capabilities — each capability only requires its own model_name: Chat Capability:
model_name: gemini-2.5-flash
Embeddings Capability:
model_name: gemini-embedding-001
Vision Capability:
model_name: gemini-2.5-flash
Save configuration:
- Click “Save” to store the configuration
- Verify all capabilities show as “Active”

Vertex AI Cost Management

Understanding costs:

Chat usage: Charged per input/output token
Embeddings: Charged per text embedding generated
Vision: Charged per image analyzed
Model selection: Different models have different pricing

Cost optimization tips:

Monitor usage in Google Cloud Console
Set up billing alerts for unexpected usage
Use semantic caching to reduce redundant API calls
Choose appropriate models for different use cases

Azure OpenAI Configuration

Azure OpenAI offers the proven GPT models with enterprise-grade security and compliance.

Azure OpenAI Configuration

Prerequisites:

Azure subscription with OpenAI resource created
Deployed models in your Azure OpenAI resource
API key and endpoint details

Configuration steps:

Access Azure OpenAI tab in LLM Configuration interface

Enter credentials (all required):

api_key:        your-azure-openai-api-key
azure_endpoint: https://your-resource-name.openai.azure.com/
api_version:    2025-01-01-preview

Configure global (shared) parameters:

temperature: 0.7   (optional, 0.0–1.0)
max_tokens:  4000  (optional)

Enable and configure capabilities — each capability has its own deployment_name: Chat Capability:
deployment_name: your-gpt5-chat-deployment
Embeddings Capability (optional):
deployment_name: your-embeddings-deployment

AWS Bedrock Configuration

AWS Bedrock provides access to various foundation models including Anthropic’s Claude, with enterprise-grade AWS integration.

Prerequisites for AWS Bedrock

AWS account setup:

AWS account with Bedrock access enabled in your region
IAM credentials with appropriate Bedrock permissions
Model access granted for desired models (Claude, etc.)
Sufficient service quotas for your expected usage

Required IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:GetModel",
        "bedrock:ListFoundationModels"
      ],
      "Resource": "*"
    }
  ]
}

Bedrock Configuration Process

Access AWS Bedrock tab in LLM Configuration interface

Enter credentials — region is required, plus either an api_key (bearer token) or an IAM key pair:

region:            us-east-1
api_key:           your-bedrock-api-key            (optional — use this OR the IAM keys below)
access_key_id:     AKIA...your-access-key          (optional)
secret_access_key: your-secret-access-key          (optional)
session_token:     temporary-session-token         (optional, for STS)

Configure global (shared) parameters:

temperature: 0.7    (optional)
max_tokens:  10000  (optional — set explicitly for Claude Sonnet to avoid failures)
provider:    anthropic   (optional — override Bedrock provider auto-detection)

Enable chat capability:

model_name: us.anthropic.claude-sonnet-4-6

Configure embeddings (if available):
model_name: amazon.titan-embed-text-v1

Bedrock Model Selection

Available model families:

Anthropic Claude: Excellent for reasoning and analysis
AI21 Labs Jurassic: Strong language understanding
Cohere Command: Multilingual capabilities
Amazon Titan: AWS-native models with competitive performance

Model selection criteria:

Performance requirements: Response quality and speed
Cost considerations: Different models have different pricing
Regional availability: Not all models available in all regions
Compliance requirements: Some models may have specific compliance certifications

Managing Capabilities

Understanding Capability Types

Chat Capability:

Purpose: Natural language understanding and generation
Used for: Query interpretation, response generation, insight creation
Configuration: Model selection, temperature, token limits

Embeddings Capability:

Purpose: Converting text to numerical representations for similarity matching
Used for: Semantic caching, query similarity detection, content understanding
Configuration: Model selection, dimension settings

Vision Capability:

Purpose: Image analysis and understanding
Used for: Dashboard mockup analysis, screenshot interpretation
Configuration: Model selection, image processing limits

Capability Configuration Best Practices

Chat capability optimization:

Temperature settings:
- 0.0-0.3: Deterministic, factual responses
- 0.4-0.7: Balanced creativity and accuracy (recommended)
- 0.8-1.0: Creative but potentially less accurate
Token limits: Set based on expected query/response complexity
Model selection: Balance performance, cost, and feature requirements

Embeddings optimization:

Dimension selection: Higher dimensions = better accuracy but higher cost
Model compatibility: Ensure embedding model matches your use case
Caching strategy: Configure appropriate cache retention for embeddings

Multi-provider strategy:

Primary provider: Choose most reliable provider for critical capabilities
Fallback providers: Configure secondary providers for redundancy
Capability specialization: Use different providers for different capabilities

Testing and Validation

Manual Validation Procedures

After configuring providers:

Test basic chat capability:
- Go to Playground interface
- Ask a simple question about your data
- Verify natural language response is generated
Test embeddings (if configured):
- Ask similar questions multiple times
- Verify responses improve with semantic caching
- Check cache hit rates in monitoring
Test vision capability (Vertex AI):
- Go to Data Agent interface
- Upload a dashboard screenshot
- Verify image analysis produces relevant insights
Performance validation:
- Monitor response times during testing
- Check API usage in provider dashboards
- Verify no rate limiting or quota issues

Troubleshooting Common Issues

Authentication failures:

Error: "Invalid API key" or "Authentication failed"
Solutions:
- Verify API key is correctly copied and not truncated
- Check API key permissions and scope
- Ensure billing is enabled and account is in good standing
- For service accounts, verify JSON formatting is correct

Model access issues:

Error: "Model not found" or "Access denied"
Solutions:
- Verify model name/ID is correct for your provider
- Check that model access is granted in provider console
- Ensure your region supports the requested model
- For AWS Bedrock, confirm model access is explicitly granted

Rate limiting or quota issues:

Error: "Rate limit exceeded" or "Quota exceeded"
Solutions:
- Check usage in provider dashboard
- Implement request throttling or retries
- Upgrade service tier if needed
- Distribute load across multiple providers

Security Best Practices

Credential Management

Secure storage:

Never store credentials in code or configuration files
Use environment variables or secure secret management systems
Encrypt credentials at rest using appropriate encryption
Limit credential access to authorized personnel only

Regular rotation:

API keys: Rotate every 90 days or according to security policy
Service accounts: Regenerate keys quarterly
Access review: Regularly audit who has access to credentials
Emergency procedures: Have procedures for immediate credential revocation

Network Security

Data privacy:

Understand data flow: Know what data is sent to each provider
Regional compliance: Use appropriate regions for data sovereignty
Data retention: Understand provider data retention policies

Access Control

Role-based access:

Limit configuration access: Only supervisors can configure LLM providers
Separate development/production: Use different credentials for different environments
Audit configuration changes: Log all changes to LLM provider configurations
Emergency access: Maintain emergency procedures for credential issues

Cost Management and Optimization

Understanding AI Provider Costs

Cost factors:

Token usage: Both input (prompts) and output (responses) tokens are charged
Model selection: Premium models cost more than basic models
Capability type: Chat, embeddings, and vision have different pricing
Request frequency: High-volume usage may trigger different pricing tiers

Cost Optimization Strategies

Provider selection:

Cost comparison: Compare pricing across providers for your use cases
Feature optimization: Use expensive features (like vision) only when necessary
Load balancing: Distribute load to take advantage of different pricing models

Ready to configure your first LLM provider? Start with the provider that best matches your organization’s requirements and follow the step-by-step instructions above. Remember: a well-configured AI foundation is essential for optimal Simba Intelligence performance.

​LLM Provider Configuration

​Overview

​What are LLM Providers?

​Supported LLM Providers

​Provider Capabilities

​LLM Tiers and Inheritance

​Understanding Tiers

​Tier Inheritance (Fallback Chain)

​Prerequisites

​Required Permissions

​AI Provider Account Requirements

​Accessing LLM Configuration

​Navigation to Configuration Interface

​Configuration Interface Overview

​Google Vertex AI Configuration

​Prerequisites for Vertex AI

​Vertex AI Configuration Process

​Vertex AI Cost Management

​Azure OpenAI Configuration

​Azure OpenAI Configuration

​AWS Bedrock Configuration

​Prerequisites for AWS Bedrock

​Bedrock Configuration Process

​Bedrock Model Selection

​Managing Capabilities

​Understanding Capability Types

​Capability Configuration Best Practices

​Testing and Validation

​Manual Validation Procedures

​Troubleshooting Common Issues

​Security Best Practices

​Credential Management

​Network Security

​Access Control

​Cost Management and Optimization

​Understanding AI Provider Costs

​Cost Optimization Strategies

LLM Provider Configuration

Overview

What are LLM Providers?

Supported LLM Providers

Provider Capabilities

LLM Tiers and Inheritance

Understanding Tiers

Tier Inheritance (Fallback Chain)

Prerequisites

Required Permissions

AI Provider Account Requirements

Accessing LLM Configuration

Navigation to Configuration Interface

Configuration Interface Overview

Google Vertex AI Configuration

Prerequisites for Vertex AI

Vertex AI Configuration Process

Vertex AI Cost Management

Azure OpenAI Configuration

Azure OpenAI Configuration

AWS Bedrock Configuration

Prerequisites for AWS Bedrock

Bedrock Configuration Process

Bedrock Model Selection

Managing Capabilities

Understanding Capability Types

Capability Configuration Best Practices

Testing and Validation

Manual Validation Procedures

Troubleshooting Common Issues

Security Best Practices

Credential Management

Network Security

Access Control

Cost Management and Optimization

Understanding AI Provider Costs

Cost Optimization Strategies