# AI LLM

An **LLM (Large Language Model)** is an AI model trained to understand and generate text (and sometimes code, images, etc.). In practice, your application sends a prompt to an LLM provider (e.g., OpenAI, Google, Anthropic), and receives a completion/response.

Flashback’s **Cloud and AI Gateway** lets you connect these providers **once**, then reuse them across your workspaces and repositories with:

* centralized credential management,
* usage monitoring (tokens, requests),
* and governance through AI Policies.

***

## How AI works in Flashback

Flashback’s AI layer combines:

* **AI LLM management** (provider connections),
* **AI Policy** (governance rules),
* **AI API Keys** (scoped keys your apps use to call AI through Flashback).\
  These policies can be scoped at **organization, workspace, or repository** level.

> In other words: you plug providers in, you govern usage, and your applications call Flashback with scoped keys.\
> (See Platform API Reference → AI for the full API details.)

***

## 1) AI LLM configurations (connect providers)

An **AI LLM configuration** is a secure connection to an external AI provider (credentials + endpoint + provider type), created **per workspace**.

Typical supported providers include:

* OpenAI-compatible providers
* Google (Gemini)
* Anthropic (Claude)

Flashback is designed to be **OpenAI-compatible**, which also makes it possible to connect on-prem or decentralized providers that expose an OpenAI-compatible API.

### Key properties and guarantees

* **Centralized configuration**: store and manage provider credentials in one place.
* **Multi-provider**: you can configure multiple providers and switch over time.
* **Security**: credentials are **encrypted at rest** and **never returned in API responses**.
* **Validation**: you can test a configuration to ensure credentials + endpoint work.
* **Monitoring**: usage stats (requests, tokens, policy violations) are available.

***

## 2) Repositories: where AI LLMs become usable by apps

A **Repository (Repo)** is the workspace-level container that groups resources (storage and AI) under a single API interface. From a client perspective, a Repo behaves like **one logical endpoint**:

* you attach one or more AI LLM configurations as resources,
* you choose an API surface to expose,
* and you generate repo-scoped API keys for your applications.

For AI, the Repo exposes an **OpenAI-compatible** endpoint type, and the keys you generate for AI are meant to be used with the AI resources attached to that Repo.

> Important: repo keys are shown only once at creation time; if you lose the secret, you must generate a new key.

***

## 3) Governing AI usage with AI Policies

AI Policies let you define natural-language governance rules such as:

* PII handling,
* security constraints,
* content boundaries,
* and other guardrails.

Policies can be scoped at:

* **Organization**
* **Workspace**
* **Repository**

Actions can include logging, alerting, or blocking (depending on your policy configuration).

***

## 4) Observability & operations

Once your AI LLMs are configured and attached to repos, you can monitor:

* total requests,
* tokens in / tokens out,
* and policy enforcement signals.

Operational best practices:

1. **Validate** configurations after creation or credential updates.
2. **Rotate** provider keys periodically.
3. **Prefer scoped repo keys** for applications (least privilege).
4. **Review stats** regularly to detect spikes or policy violations.
5. Clean up unused configurations to keep workspaces tidy.

***

## Where to go next

* If you need the exact endpoints and payloads, open **Platform API Reference → AI**:
  * AI LLMs (CRUD + validate + stats)
  * AI Policy
  * AI API Keys
* For storage + repo mechanics, see **Cloud Storage** and **Repositories** under Cloud and AI Gateway.

## Security and secret encryption

Secret encryption is a platform-wide security principle in Flashback, not only a Cloud Storage behavior.

For the full model (how secrets are encrypted, when decryption happens, and why this applies to buckets, AI providers, and other connected resources), see:

* [Security and Secret Encryption](/support-reference/security-and-secret-encryption.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.flashback.tech/flashback-platform/cloud-and-ai-gateway/ai-llm.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.