Rate limits

The Siftsy Analyst API is built for scheduled BI refreshes — Power BI, Tableau, Looker, Metabase, or a warehouse loader that walks the workspace on a timer — not for a realtime feed. Those clients share a shape: a sequential, paginated loop that runs as fast as we answer, and a scheduler that fails the whole refresh if it sees a 429. The limits are designed around that:

• Per-minute caps are sized so one sequential refresh loop never trips them. Our response latency is the throttle. Burst caps only bite clients that fan out many parallel requests.
• Per-day budgets do the real governing. They fit eight full refreshes a day of a 500-post workspace — the Power BI Pro scheduler maximum — or one full refresh plus hourly incrementals, with headroom.
• Comment routes sit outside the refresh loop. They return a sample of evidence behind an analysis, not the thread, and their budgets are sized for sampling rather than export.

Defaults

Every request counts against the global budget. Requests to expensive routes also count against a class budget. Both have a burst cap (per UTC minute) and a daily budget (UTC day).

Plus:

• 500 MB response bytes / day / key (hard cap)
• 5 MB max single response (413 if larger)
• 30 failed auth attempts / minute / IP
• 3 active keys per team

Limits are per key. Siftsy can raise them for a given team or environment; ask if your workspace is larger than the worked budget below.

Why these numbers

Each class maps to what a request actually costs on our side:

• Comments (GET /posts/{id}/analysis/comments) loads one stored comment blob for that post and returns a sample. 1,000 a day is enough to pull source=notables for every post in a 500-post workspace once a day and still drill into specific posts; paging a full thread at 100 per page would spend most of that budget on one post, which is the point.
• Analysis comments (GET /analyses/{id}/comments) re-reads the comment blob for every post on the analysis — up to 1,000 — on every page, then filters and paginates in memory. It is the heaviest read path by an order of magnitude. One source=notables sample per campaign analysis per refresh, plus a few filtered variants, fits in 300.
• Collection posts walk several index pages per request. Syncing 20 collections at 5 pages each, eight times a day, is 800.
• Analysis reads are cheap lookups, but clients poll them in a loop. 60/minute fits three concurrent jobs polled at the 3-second pollAfterMs we return.
• Analysis create loads up to 100 comment blobs and aggregates them inline — one rollup per collection or tag per refresh is the intended cadence. If you recreate 20 rollups on all eight refreshes you will need a raise; ask.
• Analysis search does all of that and then runs a language model over every selected comment. It has real dollar cost and takes minutes; it is not a per-refresh operation.
• Adding content spends your team’s credits, so the credit balance is the real limit; these classes only stop a runaway loop. 200 post batches a day is 20,000 links; 20 account pulls a day is 2,000 discovered posts. Follow jobs at the pollAfterMs we return.

A worked BI budget

Reference workspace: 500 completed posts in 20 collections. This is the BI sync load order.

• Eight full refreshes a day ≈ 8,600 of 10,000 global, 160 of 100 analysis_create (so recreate rollups on one or two of the eight, not all), ~120 MB of 500 MB.
• One full refresh plus hourly incrementals (GET /posts?updated_since=…, ~5% of posts changed → 5 list pages + 50 per-post calls) ≈ 1,070 + 23 × 55 ≈ 2,400 global, ~35 MB. This is the cheaper pattern and leaves room for a 2,000-post workspace.
• Evidence samples on top: source=notables for every post once a day is 500 of 1,000 comments; one sample per campaign analysis per refresh is well inside 300 analysis_comments.

A sequential loop at 200–400 ms per request runs at 150–300 requests / minute, under the global cap. If your BI tool evaluates queries in parallel (Power BI does by default), keep it to two concurrent queries or the burst caps will trip.

429 responses

Honor Retry-After. It is seconds until the next UTC minute for a burst cap, or until UTC midnight for a daily budget or the byte cap. The body’s error.message says which:

Most BI schedulers do not retry on 429, so treat a 429 in a refresh log as a sizing problem: check X-RateLimit-Class to see which class tripped, then lower parallelism (burst) or reduce refresh frequency / switch to updated_since (daily).

Headers

GET /me returns the same information as JSON (rateLimit.remainingRequests, remainingDailyRequests, remainingBytes, and routeClasses with every class’s caps), so a refresh job can check its budget before starting and skip a full pull when the day’s budget is mostly spent.

Usage history

Team Settings → API in the Siftsy app shows requests and response bytes per key, per UTC day, for the last 30 days, plus the team’s totals for today, the last 7 days, and the window. It reads the same daily counters the limiter enforces, so it is the place to look when a refresh log shows a 429: you can see which key spent the budget and on which day. Counts include requests that were denied by a daily cap.

Working within the budget

• Do one full refresh a day (overnight) and make the rest incremental: GET /posts?updated_since=<last run> and only fetch metrics and analysis for the posts that come back.
• Treat comment routes as evidence, not a table: sample source=notables for the posts or analyses you are showing, and skip a per-post comments crawl. Totals and mix bars are already on GET /posts/{id}/analysis and the analysis itself.
• Prefer one selection analysis per collection or tag over many per-post calls when you want an aggregate — one POST /analyses replaces hundreds of per-post reads. Recreate rollups on one refresh a day, not every refresh.
• Poll analyses at the pollAfterMs we return; polling faster only spends analysis_read budget.
• Keep parallelism at two queries or fewer, and spread scheduled refreshes so they do not all start at the same UTC minute.