Latency-aware routing for edge and AI pipelines

Expiremental Guide: May contain errors as our technology continues to evolve. If you encounter any problems, please do not hesitate to contact us in Discord and give us your feedback.

The Problem

Today, developers push latency-sensitive workloads (media delivery, feature stores, embeddings, and model shards) to users around the world, but each application typically pins to a single storage endpoint per environment. That means guessing a “best” region, maintaining per-cloud SDK configs, and hard-coding endpoints into services and CDNs, which is brittle and costly. When traffic shifts or a region degrades, failover is manual and risky; keeping regional copies consistent is tedious. On top of that, observability is fragmented across vendor consoles, and there’s no easy way to enforce hard latency SLOs like “prefer sub-50 ms reads and automatically fail over if a node slows down.”

Flashback changes this by offering a single integration that works across many regions and providers. Bridge Node endpoints speak S3, GCS, or Azure Blob so your app only needs one protocol and one credential set, while Flashback exposes per-node status and measured latency so you can pick the fastest endpoint at runtime. With its repository abstraction, you can attach multiple vendor buckets to one repository and generate reusable READ/WRITE keys that work across all SDKs. The platform also gives you built-in guardrails and observability: minute-level stats and node-level latency you can poll to enforce SLOs, plus quotas that can automatically disable writes once thresholds are met. And because it’s future-proof, you can extend your footprint with additional regions or DePIN edges for cheaper distribution without ever having to re-plumb your app.

---

Step-by-Step Deployment Recommendations

1

Model your routing policy

Decide how your client should pick endpoints:

• Reads: choose the fastest Online Bridge Node for the target bucket/region. Use recent latency; keep a short cache (e.g., 60–120s). If latency spikes or timeouts occur, fail over to the next best node. Use node status/latency that Flashback measures via HeadBucket.

• Writes: prefer nodes geographically close and on the same provider as the bucket to reduce egress and tail latency; keep the same fallback logic. Bridge Node URLs follow https://<api>-<region>-<provider>.flashback.tech with public examples you can start from.

2

Create Buckets & Repository in Flashback

• Create/Link Cloud Buckets in Flashback

  • For each provider, add a Bucket in the dashboard: “Add Bucket” → select provider → provide credentials/role → validate.

  • Select DePin providers for cold storage. Be aware of latency increase.

• Create a Repository

  • “New Repository” → name it (e.g., app-data) → attach all created Buckets.

• Generate Repository API Keys

  • Create a WRITE key for your application and a READ key for services that only read.

  • Save the secret in your vault; you can’t retrieve it later from it.

3

Wire your backend client to Flashback

Create clients pointing at a Bridge Node endpoint; keep a small cache keyed by endpoint so you can swap quickly.

Python (boto3 / S3)

# fb_s3_client.py
import boto3
from botocore.client import Config

def s3_client_for(endpoint, key_id, key_secret):
    session = boto3.session.Session(
        aws_access_key_id=key_id, aws_secret_access_key=key_secret
    )
    return session.client("s3", endpoint_url=endpoint, config=Config(signature_version="s3v4"))

Node (aws-sdk v3 / S3)

// fbS3Client.ts
import { S3Client } from "@aws-sdk/client-s3";

const clients = new Map<string, S3Client>();
export function s3ClientFor(endpoint: string, keyId: string, secret: string) {
  if (!clients.has(endpoint)) {
    clients.set(endpoint, new S3Client({
      endpoint, region: "us-east-1",
      credentials: { accessKeyId: keyId, secretAccessKey: secret },
      forcePathStyle: true
    }));
  }
  return clients.get(endpoint)!;
}

(Use analogous clients for GCS/Azure if that’s your app’s native protocol.) Flashback S3 endpoints are compatible with standard S3 SDKs.

4

Discover nodes and pull live latency

List active nodes (bootstrap) and then poll node minute stats (optionally filtered by bucketId) to get availability and recent latency.

// fbNodeStats.js
const H = { Accept: "application/json", Authorization: `Bearer ${process.env.FB_JWT}` };

export async function listNodes() {
  const r = await fetch("https://backend.flashback.tech/node", { headers: H });
  return r.json(); // includes node metadata for routing tables
}

export async function nodeMinuteStats(bucketId) {
  const url = new URL("https://backend.flashback.tech/stats/nodes/minute");
  if (bucketId) url.searchParams.set("bucketId", bucketId);
  const r = await fetch(url, { headers: H });
  return r.json(); // includes per-node minute latency/ops
}

5

Select the fastest Online node (with TTL + circuit breaker)

// pickEndpoint.ts
import { listNodes, nodeMinuteStats } from "./fbNodeStats";

const CACHE_TTL_MS = 60_000;
let cache = { endpoint: "", expires: 0 };

export async function pickEndpointFor(bucketId?: string) {
  const now = Date.now();
  if (cache.endpoint && cache.expires > now) return cache.endpoint;

  const nodes = await listNodes();                      // seed candidates
  const stats = await nodeMinuteStats(bucketId);        // recent latency/availability

  // Rank Online nodes by recent latency; fall back to others if needed
  const online = rankByLatency(merge(nodes, stats)).filter(n => n.status === "Online");
  const chosen = (online[0] ?? rankByLatency(merge(nodes, stats))[0]);
  cache = { endpoint: chosen.endpoint, expires: now + CACHE_TTL_MS };
  return cache.endpoint;
}

// If a request times out or 5xx, invalidate cache so next call re-picks:
export function reportFailure() { cache.expires = 0; }

(Your “merge” and “rank” functions use the per-node latency fields from the stats response.)

6

Use the chosen endpoint for reads/writes

// fbStorage.ts
import { PutObjectCommand, GetObjectCommand } from "@aws-sdk/client-s3";
import { s3ClientFor } from "./fbS3Client";
import { pickEndpointFor, reportFailure } from "./pickEndpoint";

const KEY_ID = process.env.FB_KEY_ID!, SECRET = process.env.FB_KEY_SECRET!;

export async function putObject(bucket: string, key: string, body: Buffer) {
  const endpoint = await pickEndpointFor(bucket);
  const s3 = s3ClientFor(endpoint, KEY_ID, SECRET);
  try {
    await s3.send(new PutObjectCommand({ Bucket: bucket, Key: key, Body: body }));
  } catch (e) { reportFailure(); throw e; }
}

export async function getObject(bucket: string, key: string) {
  const endpoint = await pickEndpointFor(bucket);
  const s3 = s3ClientFor(endpoint, KEY_ID, SECRET);
  try {
    return await s3.send(new GetObjectCommand({ Bucket: bucket, Key: key }));
  } catch (e) { reportFailure(); throw e; }
}

(Set your endpoint strings using the URL pattern and public examples.)

7

(Optional) Pre-signed URLs for clients/CDNs

If browsers or edge workers fetch directly, generate a pre-signed URL against the chosen endpoint so traffic hits the fastest node without proxying through your backend. (The SDK list includes @aws-sdk/s3-request-presigner.)

8

Observe & alert on latency/SLOs

Build basic dashboards and alerts from minute and daily stats; keep an eye on tail latency and error rates.

• GET /stats/nodes/minute?bucketId=... for per-node latency/availability.

• GET /stats/minute and GET /stats/daily for repo/bucket ops and latency trends.

9

Validate & roll out

• Smoke test: list buckets and transfer a small object via your chosen nodes/endpoints.

• Failover drill: temporarily block the top endpoint to ensure your circuit breaker selects the next best node and retries.

• Tune TTL: start with 60–120s; shorten if your workloads see rapid regional swings.

10

Operations playbook

• Keys: rotate repo keys; secrets are encrypted, decrypted only in Bridge Node memory; not recoverable later.

• Health signals: bucket detail view surfaces Online/Disconnected/Offline and HeadBucket latency—use these to debug routing.

• Endpoints: keep at least two Bridge Node URLs per protocol handy (e.g., s3-us-east-1-aws.flashback.tech and one in the EU).

• Monitoring: alert when node latency breaches your SLO or when read/write error rates spike; pull from /stats/nodes/minute plus minute/daily stats.