AI LLM

An LLM (Large Language Model) is an AI model trained to understand and generate text (and sometimes code, images, etc.). In practice, your application sends a prompt to an LLM provider (e.g., OpenAI, Google, Anthropic), and receives a completion/response.

Flashback’s Cloud and AI Gateway lets you connect these providers once, then reuse them across your workspaces and repositories with:

• centralized credential management,

• usage monitoring (tokens, requests),

• and governance through AI Policies.

---

How AI works in Flashback

Flashback’s AI layer combines:

• AI LLM management (provider connections),

• AI Policy (governance rules),

• AI API Keys (scoped keys your apps use to call AI through Flashback). These policies can be scoped at organization, workspace, or repository level.

In other words: you plug providers in, you govern usage, and your applications call Flashback with scoped keys. (See Platform API Reference → AI for the full API details.)

---

1) AI LLM configurations (connect providers)

An AI LLM configuration is a secure connection to an external AI provider (credentials + endpoint + provider type), created per workspace.

Typical supported providers include:

• OpenAI-compatible providers

• Google (Gemini)

• Anthropic (Claude)

Flashback is designed to be OpenAI-compatible, which also makes it possible to connect on-prem or decentralized providers that expose an OpenAI-compatible API.

Key properties and guarantees

• Centralized configuration: store and manage provider credentials in one place.

• Multi-provider: you can configure multiple providers and switch over time.

• Security: credentials are encrypted at rest and never returned in API responses.

• Validation: you can test a configuration to ensure credentials + endpoint work.

• Monitoring: usage stats (requests, tokens, policy violations) are available.

---

2) Repositories: where AI LLMs become usable by apps

A Repository (Repo) is the workspace-level container that groups resources (storage and AI) under a single API interface. From a client perspective, a Repo behaves like one logical endpoint:

• you attach one or more AI LLM configurations as resources,

• you choose an API surface to expose,

• and you generate repo-scoped API keys for your applications.

For AI, the Repo exposes an OpenAI-compatible endpoint type, and the keys you generate for AI are meant to be used with the AI resources attached to that Repo.

Important: repo keys are shown only once at creation time; if you lose the secret, you must generate a new key.

---

3) Governing AI usage with AI Policies

AI Policies let you define natural-language governance rules such as:

• PII handling,

• security constraints,

• content boundaries,

• and other guardrails.

Policies can be scoped at:

• Organization

• Workspace

• Repository

Actions can include logging, alerting, or blocking (depending on your policy configuration).

---

4) Observability & operations

Once your AI LLMs are configured and attached to repos, you can monitor:

• total requests,

• tokens in / tokens out,

• and policy enforcement signals.

Operational best practices:

1. Validate configurations after creation or credential updates.

2. Rotate provider keys periodically.

3. Prefer scoped repo keys for applications (least privilege).

4. Review stats regularly to detect spikes or policy violations.

5. Clean up unused configurations to keep workspaces tidy.

---

Where to go next

• If you need the exact endpoints and payloads, open Platform API Reference → AI:

  • AI LLMs (CRUD + validate + stats)

  • AI Policy

  • AI API Keys

• For storage + repo mechanics, see Cloud Storage and Repositories under Cloud and AI Gateway.