LLM Provider Configuration
This guide explains how to configure Large Language Model (LLM) providers in Simba Intelligence. LLM providers power the AI capabilities that enable natural language querying, data source creation, and intelligent analysis features.Overview
What are LLM Providers?
LLM providers are external AI services that power Simba Intelligence’s natural language capabilities. Think of them as the “brains” behind the system that understand your questions, analyze your data requirements, and generate intelligent responses. When you ask “What were our top-selling products last quarter?” in the Playground, an LLM provider:- Understands your natural language question
- Analyzes your available data sources
- Generates appropriate SQL queries
- Interprets the results into meaningful insights
Supported LLM Providers
Simba Intelligence requires LLMs with structured output support, since JSON schema is used throughout. Older models like GPT-3.5 do not support this and cannot be used.📝 Note: Tiers appear under two names depending on where you’re looking: the/llm-configurationUI labels them Lite and Power, while code, configuration, and logs use the underlying valuesLOWandHIGHrespectively. These are the same two tiers, not four — see Understanding Tiers below.
📝 Note: A Power (HIGH) model is required for more complex agents, such as the data source agent.
📝 Note: The Gemini provider is labeled Gemini Enterprise Agent Platform in the/llm-configurationUI. Its underlying provider ID is stillVERTEX_AI, which is what you will see in API responses and log output.
Gemini Configuration Notes
Gemini Configuration Notes
- Gemini 2.5 Flash: Set
thinking_budgetto0for maximum speed, or128for better quality - Gemini 2.5 Pro: Set
thinking_budgetbetween128and32768 - Gemini 3 or later: Set
thinking_leveltominimal,low,medium, orhigh, depending on the levels supported by the selected model
model_name containing a major version of 3 or higher uses thinking_level and ignores thinking_budget; earlier versions use thinking_budget. If the model name carries no recognizable version, whichever of the two you set is used.Learn more about thinkingAWS Bedrock Configuration Notes
AWS Bedrock Configuration Notes
- Claude Sonnet: Set
max_tokensto10000to avoid failures. Leaving this parameter blank may cause unexpected errors. - OpenAI GPT: Bedrock serves these through inference profiles only, so
model_nameneeds the geo prefix (for example,us.openai.gpt-5.6-sol).temperatureis not supported and is dropped if set. Optionalreasoning_effortacceptslow,medium,high,xhigh,max, and — on GPT-5.6 only —none. A value the chosen model does not accept fails immediately, listing the ones it does.
Microsoft Foundry Configuration Notes
Microsoft Foundry Configuration Notes
- Model family is required: one Foundry resource serves several wire protocols, so every capability must set
model_familyto eitheranthropic(Anthropic Messages API) oropenai_v1(OpenAI-compatible route). - Anthropic family: does not accept
temperature— current Claude models reject sampling parameters outright. It also cannot serve the Embeddings capability, since Anthropic publishes no embedding models; use theopenai_v1family on the same resource for embeddings. - Anthropic
max_tokens: left unset, Simba Intelligence sends 32,000 output tokens. Raise it per capability on models that allow more. - Endpoint: paste whichever form the Azure portal gives you — the bare resource host, the family base URL, or a full request URL. Simba Intelligence normalizes all three.
Provider Capabilities
💬 Chat Capability- Natural language understanding and generation
- Query interpretation and response creation
- Business insight generation
- Semantic search and similarity matching
- Intelligent caching of similar queries
- Content understanding for better results
- Dashboard image analysis
- Screenshot-to-data-source conversion
- Visual mockup interpretation
LLM Tiers and Inheritance
The Chat capability can be configured at a tier, letting you match model cost/speed to the task. This section explains what tiers are and how Simba Intelligence resolves which configuration actually serves a request.Understanding Tiers
Chat requests specify one of two tiers:
The
/llm-configuration UI shows the friendlier Lite / Power names, but the value actually stored, sent through the API, and printed in logs is LOW / HIGH. The rest of this guide uses HIGH/LOW since that’s what you’ll see in configuration, log output, and error messages.
There is also Unspecified — leaving a capability’s tier field blank in the configuration UI. An unspecified configuration acts as a catch-all that can serve either a HIGH or LOW request, which is useful for simple tenants that only need a single model instead of separate HIGH/LOW configurations.
📝 Note: OnlyCHATcapability configurations use tiers.EMBEDDINGSandVISIONconfigurations are not tiered — a tenant has at most one active configuration per non-chat capability.
Tier Inheritance (Fallback Chain)
Every tenant is not required to configure both tiers. If a tenant only configuresHIGH, requests for LOW still need to resolve to something. Simba Intelligence resolves this using an ordered fallback chain, checking the following in order and stopping at the first match:
- Own tenant, exact requested tier
- Own tenant, Unspecified tier
- Master tenant (VDD), exact requested tier
- Master tenant (VDD), Unspecified tier
- Own tenant, opposite tier (e.g. requested
LOW, onlyHIGHis configured) - Master tenant (VDD), opposite tier — last resort
- “Master tenant” refers to the Composer/VDD (Visual Data Discovery) admin tenant, which acts as an org-wide fallback pool for tenants that don’t have their own configuration for a given tier.
- Own-tenant options are always exhausted first. Steps 1–2 are checked before the master tenant is even looked up. This means a tenant with a valid Unspecified-tier configuration will never fall through to the master tenant for that capability.
- If the current tenant is the master/VDD tenant, the master-specific steps (3, 4, 6) are skipped since they’d be duplicates of the own-tenant checks.
- Non-chat capabilities (
EMBEDDINGS,VISION) skip tier resolution entirely — they use a simpler own-tenant-then-master fallback. - Vision additionally falls back to the
CHATcapability atHIGHtier if no vision capability is configured for the tenant at all.
HIGH-tier Chat capability (no LOW, no Unspecified). A request for the LOW tier will check the tenant’s own LOW and Unspecified configs (steps 1–2, no match), then the master tenant’s HIGH and Unspecified configs (steps 3–4, no match), and finally fall back to the tenant’s own HIGH configuration (step 5) — meaning the “fast” tier silently gets served by the “powerful” model instead of failing.
Use the Logging Configuration Guide to observe which step of this chain served any given request.
Prerequisites
Required Permissions
To configure LLM providers, you need:- Supervisor role in Simba Intelligence
- Access to LLM Configuration interface (
/llm-configuration) - Administrative privileges for system-wide AI configuration
AI Provider Account Requirements
For each provider you want to use: Gemini Enterprise Agent Platform (Google Vertex AI):- Google Cloud Platform account with billing enabled
- Vertex AI API enabled in your GCP project
- Service account with Vertex AI permissions
- Sufficient API quota for your expected usage
- Azure OpenAI resource
- API key with appropriate usage limits
- Sufficient credits/quota for your organization’s needs
- AWS account with Bedrock access enabled
- IAM user or role with Bedrock permissions
- Model access granted for Claude or other desired models
- Appropriate usage quotas configured
- An Azure AI Foundry resource with the models you intend to use deployed
- The resource endpoint and an API key
- Model access approved for the Anthropic and/or OpenAI models you plan to configure
Accessing LLM Configuration
Navigation to Configuration Interface
- Log into Simba Intelligence with supervisor credentials
- Navigate to LLM Configuration:
- Option 1: Visit directly at
http://your-domain/llm-configuration - Option 2: User menu → “LLM Configuration”
- Option 1: Visit directly at
- Verify access: You should see a tabbed interface with available providers
Configuration Interface Overview
The LLM Configuration interface provides:- Provider tabs: Separate tabs for each supported AI provider
- Configuration forms: Provider-specific credential and parameter forms
- Capability management: Enable/disable different AI capabilities per provider
- Testing tools: Built-in connection testing and validation
- Status indicators: Real-time status of provider configurations
Gemini Enterprise Agent Platform Configuration
The Gemini Enterprise Agent Platform (Google Vertex AI) provides comprehensive AI capabilities, including advanced vision analysis for dashboard image processing.Prerequisites for the Gemini Platform
Google Cloud Platform setup:- Create or select a GCP project with billing enabled
- Enable Vertex AI API:
- Create a service account:
- Grant necessary permissions:
- Generate service account key:
Gemini Platform Configuration Process
- Access the Gemini Enterprise Agent Platform tab in LLM Configuration interface
-
Enter service account JSON:
- Copy the complete contents of your service account JSON file
- Paste into the credentials text area
- The JSON should include all required fields:
-
Enable and configure capabilities. Parameters are set per capability, not shared across the provider — each capability you enable gets its own copy of the fields below.
Chat Capability:Embeddings Capability:Vision Capability:
📝 Note: Parameters that only affect generation —
temperature,max_tokens,thinking_budget,thinking_level— are hidden on Embeddings capabilities, where they have no meaning. -
Save configuration:
- Click “Save” to store the configuration
- Verify all capabilities show as “Active”
Gemini Platform Cost Management
Understanding costs:- Chat usage: Charged per input/output token
- Embeddings: Charged per text embedding generated
- Vision: Charged per image analyzed
- Model selection: Different models have different pricing
- Monitor usage in Google Cloud Console
- Set up billing alerts for unexpected usage
- Use semantic caching to reduce redundant API calls
- Choose appropriate models for different use cases
Azure OpenAI Configuration
Azure OpenAI offers the proven GPT models with enterprise-grade security and compliance.Azure OpenAI Configuration
Prerequisites:- Azure subscription with OpenAI resource created
- Deployed models in your Azure OpenAI resource
- API key and endpoint details
- Access Azure OpenAI tab in LLM Configuration interface
-
Enter credentials (all required):
-
Enable and configure capabilities. Each capability carries its own parameters:
Chat Capability:Embeddings Capability (optional):
AWS Bedrock Configuration
AWS Bedrock provides access to various foundation models including Anthropic’s Claude, with enterprise-grade AWS integration.Prerequisites for AWS Bedrock
AWS account setup:- AWS account with Bedrock access enabled in your region
- IAM credentials with appropriate Bedrock permissions
- Model access granted for desired models (Claude, etc.)
- Sufficient service quotas for your expected usage
Bedrock Configuration Process
- Access AWS Bedrock tab in LLM Configuration interface
-
Enter credentials —
regionis required, plus either anapi_key(bearer token) or an IAM key pair: -
Enable and configure capabilities. Each capability carries its own parameters:
-
Example — chat capability:
-
Example — embeddings capability (if available):
📝 Note — OpenAI GPT on Bedrock: Bedrock serves these models through inference profiles only, somodel_namemust include the geo prefix (for example,us.openai.gpt-5.6-sol).temperatureis not supported and is dropped if set. The optionalreasoning_effortparameter acceptslow,medium,high,xhigh,max, and — on GPT-5.6 only —none; a value the selected model does not accept fails immediately and the error lists the accepted values. If the model ID does not name the family (a bare profile ID), setprovider: openaiso the correct handling is applied.
Bedrock Model Selection
Available model families:- Anthropic Claude: Excellent for reasoning and analysis — the tested and recommended choice
- OpenAI GPT: Tested. Served through inference profiles only, so the model ID needs its geo prefix
- Amazon Titan / Nova: AWS-native models, used mainly for embeddings
- Other Bedrock families (AI21, Cohere, and so on) are reachable but untested with Simba Intelligence
- Performance requirements: Response quality and speed
- Cost considerations: Different models have different pricing
- Regional availability: Not all models available in all regions
- Compliance requirements: Some models may have specific compliance certifications
Microsoft Foundry Configuration
Microsoft Foundry serves several model vendors from a single Azure resource. One endpoint and API key reach them all; which wire protocol is used is chosen per capability with themodel_family parameter.
Prerequisites for Microsoft Foundry
- An Azure AI Foundry resource with the models you want deployed
- The resource endpoint and an API key from the Azure portal
- Model access approved for the Anthropic and/or OpenAI models you intend to configure
Foundry Configuration Process
- Access the Microsoft Foundry tab in LLM Configuration interface
-
Enter credentials (both required):
💡 Pro Tip: Azure shows this endpoint in several shapes — the bare resource host, a family base URL (
.../anthropic/,.../openai/v1), or a full request URL (.../openai/v1/chat/completions). Paste any of them. Simba Intelligence strips the trailing path segments and appends the correct route for the selected model family, so you will not end up with a doubled path and a 404 that looks like an authentication failure. -
Choose a model family per capability. This is the required
model_familyparameter: -
Configure each capability’s parameters:
Anthropic family:
temperatureis not offered: current Claude models reject sampling parameters with a 400. OpenAI Compatible family:Example — Chat on Claude, Embeddings on the OpenAI route:
⚠️ Important: The Anthropic family cannot serve the Embeddings capability — Anthropic publishes no embedding models. Configure embeddings with the openai_v1 family on the same Foundry resource, as in the example above.
📝 Note: Why max_tokens defaults to 32,000 on the Anthropic family: left entirely unset, the client looks up the model’s maximum output tokens by exact model ID, and a custom Foundry deployment name misses that lookup and silently falls back to 4,096 — truncating long generations. 32,000 is the floor across the Claude models Foundry serves, so it is safe for any deployment name.
Managing Capabilities
Understanding Capability Types
Chat Capability:- Purpose: Natural language understanding and generation
- Used for: Query interpretation, response generation, insight creation
- Configuration: Model selection, temperature, token limits
- Purpose: Converting text to numerical representations for similarity matching
- Used for: Semantic caching, query similarity detection, content understanding
- Configuration: Model selection, dimension settings
- Purpose: Image analysis and understanding
- Used for: Dashboard mockup analysis, screenshot interpretation
- Configuration: Model selection, image processing limits
Capability Configuration Best Practices
Chat capability optimization:- Temperature settings:
0.0-0.3: Deterministic, factual responses0.4-0.7: Balanced creativity and accuracy (recommended)0.8-1.0: Creative but potentially less accurate
- Token limits: Set based on expected query/response complexity
- Model selection: Balance performance, cost, and feature requirements
- Dimension selection: Higher dimensions = better accuracy but higher cost
- Model compatibility: Ensure embedding model matches your use case
- Caching strategy: Configure appropriate cache retention for embeddings
- Primary provider: Choose most reliable provider for critical capabilities
- Fallback providers: Configure secondary providers for redundancy
- Capability specialization: Use different providers for different capabilities
Testing and Validation
Manual Validation Procedures
After configuring providers:-
Test basic chat capability:
- Go to Playground interface
- Ask a simple question about your data
- Verify natural language response is generated
-
Test embeddings (if configured):
- Ask similar questions multiple times
- Verify responses improve with semantic caching
- Check cache hit rates in monitoring
-
Test vision capability (Gemini platform, Microsoft Foundry, or any multimodal chat model):
- Go to Data Agent interface
- Upload a dashboard screenshot
- Verify image analysis produces relevant insights
-
Performance validation:
- Monitor response times during testing
- Check API usage in provider dashboards
- Verify no rate limiting or quota issues
Troubleshooting Common Issues
Authentication failures:Security Best Practices
Credential Management
Secure storage:- Never store credentials in code or configuration files
- Use environment variables or secure secret management systems
- Encrypt credentials at rest using appropriate encryption
- Limit credential access to authorized personnel only
- API keys: Rotate every 90 days or according to security policy
- Service accounts: Regenerate keys quarterly
- Access review: Regularly audit who has access to credentials
- Emergency procedures: Have procedures for immediate credential revocation
Network Security
Data privacy:- Understand data flow: Know what data is sent to each provider
- Regional compliance: Use appropriate regions for data sovereignty
- Data retention: Understand provider data retention policies
Access Control
Role-based access:- Limit configuration access: Only supervisors can configure LLM providers
- Separate development/production: Use different credentials for different environments
- Audit configuration changes: Log all changes to LLM provider configurations
- Emergency access: Maintain emergency procedures for credential issues
Cost Management and Optimization
Understanding AI Provider Costs
Cost factors:- Token usage: Both input (prompts) and output (responses) tokens are charged
- Model selection: Premium models cost more than basic models
- Capability type: Chat, embeddings, and vision have different pricing
- Request frequency: High-volume usage may trigger different pricing tiers
Cost Optimization Strategies
Provider selection:- Cost comparison: Compare pricing across providers for your use cases
- Feature optimization: Use expensive features (like vision) only when necessary
- Load balancing: Distribute load to take advantage of different pricing models
Ready to configure your first LLM provider? Start with the provider that best matches your organization’s requirements and follow the step-by-step instructions above. Remember: a well-configured AI foundation is essential for optimal Simba Intelligence performance.