Lokalise Performance Tuning
Overview
Optimize Lokalise API throughput for translation pipelines by implementing cursor pagination, local caching, batch key operations (500/request), request throttling under the 6 req/s rate limit, and selective language downloads.
Prerequisites
@lokalise/node-api SDK v9+ (ESM) or REST API accessLOKALISE_API_TOKEN environment variable set- Understanding of project size (key count, language count) to calibrate batch sizes
- Optional: Redis or LRU cache library for persistent caching
Instructions
Step 1: Use Cursor Pagination for Large Datasets
Cursor pagination is significantly faster than offset pagination for projects with 5K+ keys. Offset pagination degrades as page numbers increase because the server must skip rows; cursor pagination uses a pointer.
import { LokaliseApi } from '@lokalise/node-api';
const lok = new LokaliseApi({ apiKey: process.env.LOKALISE_API_TOKEN! });
async function* getAllKeys(projectId: string) {
let cursor: string | undefined;
do {
const result = await lok.keys().list({
project_id: projectId,
limit: 500,
pagination: 'cursor',
cursor,
});
for (const key of result.items) yield key;
cursor = result.hasNextCursor() ? result.nextCursor : undefined;
} while (cursor);
}
let count = 0;
for await (const key of getAllKeys('PROJECT_ID')) {
count++;
}
console.log(`Fetched ${count} keys`);
Offset pagination comparison (avoid for large projects):
| Keys | Offset (limit=100) | Cursor (limit=500) | Time saved |
|---|
| 1,000 | 10 requests | 2 requests | 80% |
| 10,000 | 100 requests | 20 requests | 80% |
| 50,000 | 500 requests (~84s) | 100 requests (~17s) | 80% |
Step 2: Cache Translation Downloads Locally
Translation file downloads are the most expensive Lokalise operation. Cache them locally and use project last_activity timestamps to invalidate.
import { LokaliseApi } from '@lokalise/node-api';
import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'fs';
const lok = new LokaliseApi({ apiKey: process.env.LOKALISE_API_TOKEN! });
const CACHE_DIR = '.lokalise-cache';
interface CacheEntry {
url: string;
timestamp: string;
languages: string[];
}
function getCachePath(projectId: string, langIso: string): string {
return `${CACHE_DIR}/${projectId}/${langIso}.json`;
}
function getMetaPath(projectId: string): string {
return `${CACHE_DIR}/${projectId}/meta.json`;
}
async function downloadWithCache(projectId: string, langIso: , format = ) {
(, { : });
cachePath = (projectId, langIso);
metaPath = (projectId);
project = lok.().(projectId);
lastActivity = project.?. ?? project.;
((metaPath)) {
: = .((metaPath, ));
(meta. === lastActivity && (cachePath)) {
.();
.((cachePath, ));
}
}
bundle = lok.().(projectId, {
format,
: [langIso],
: ,
});
response = (bundle.);
data = response.();
(cachePath, .(data));
(metaPath, .({
: bundle.,
: lastActivity,
: [langIso],
}));
.();
data;
}
Step 3: Batch Key Operations
Lokalise supports creating, updating, and deleting up to 500 keys per request. Always batch instead of making individual requests.
async function createKeysBatched(projectId: string, keys: any[]) {
const BATCH_SIZE = 500;
const results = [];
for (let i = 0; i < keys.length; i += BATCH_SIZE) {
const batch = keys.slice(i, i + BATCH_SIZE);
const result = await lok.keys().create({
project_id: projectId,
keys: batch,
});
results.push(...result.items);
console.log(`Batch ${Math.floor(i / BATCH_SIZE) + 1}: created ${result.items.length} keys`);
await new Promise(r => setTimeout(r, 200));
}
return results;
}
async function updateKeysBatched(: , : <{key_id: ; [k: ]: }>) {
= ;
( i = ; i < updates.; i += ) {
batch = updates.(i, i + );
lok.().({
: projectId,
: batch,
});
( (r, ));
}
}
() {
= ;
( i = ; i < keyIds.; i += ) {
batch = keyIds.(i, i + );
lok.().({
: projectId,
: batch,
});
( (r, ));
}
}
Step 4: Implement Request Throttling
A proper request queue prevents 429 Too Many Requests errors and makes your integration resilient under load.
import PQueue from 'p-queue';
const queue = new PQueue({
concurrency: 5,
interval: 1000,
intervalCap: 5,
});
async function throttledRequest<T>(fn: () => Promise<T>): Promise<T> {
return queue.add(fn) as Promise<T>;
}
const project = await throttledRequest(() => lok.projects().get(projectId));
const keys = await throttledRequest(() => lok.keys().list({
project_id: projectId,
limit: 500,
pagination: 'cursor',
}));
const projectIds = ['PROJ_1', 'PROJ_2', 'PROJ_3', 'PROJ_4', 'PROJ_5'];
allProjects = .(
projectIds.( ( lok.().(id)))
);
Step 5: Async File Operations with Webhooks
File uploads and downloads are processed asynchronously by Lokalise. Instead of polling the process status endpoint, use webhooks to get notified when processing completes.
set -euo pipefail
curl -s -X POST "https://api.lokalise.com/api2/projects/${PROJECT_ID}/webhooks" \
-H "X-Api-Token: ${LOKALISE_API_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"url": "https://hooks.company.com/lokalise",
"events": [
"project.imported",
"project.exported",
"project.keys_added"
]
}' | jq '{webhook_id: .webhook.webhook_id, url: .webhook.url, events: .webhook.events}'
If you must poll (no webhook endpoint available):
async function waitForProcess(projectId: string, processId: string, timeoutMs = 120_000) {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const proc = await throttledRequest(() =>
lok.queuedProcesses().get(projectId, processId)
);
if (proc.status === 'finished') return proc;
if (proc.status === 'cancelled' || proc.status === 'failed') {
throw new Error(`Process ${processId} ${proc.status}: ${proc.message}`);
}
await new Promise(r => setTimeout(r, 2000));
}
throw new Error(`Process ${processId} timed out after ${timeoutMs}ms`);
}
Step 6: Selective Language Downloads (Delta Exports)
Downloading all languages when you only need one wastes bandwidth and API time. Always filter by language and, when possible, by modification timestamp.
async function downloadDelta(projectId: string, langIso: string, sinceTimestamp: string) {
const keys = await lok.keys().list({
project_id: projectId,
limit: 500,
pagination: 'cursor',
filter_translation_lang_ids: langIso,
});
return keys.items;
}
async function downloadSingleLanguage(projectId: string, langIso: string) {
const result = await lok.files().download(projectId, {
format: 'json',
filter_langs: [langIso],
original_filenames: false,
bundle_structure: '%LANG_ISO%.%FORMAT%',
: ,
: [],
});
result.;
}
Step 7: Measure and Benchmark
set -euo pipefail
echo "=== Lokalise API Benchmarks ==="
echo -n "Projects list: "
curl -s -o /dev/null -w "%{time_total}s" \
-H "X-Api-Token: ${LOKALISE_API_TOKEN}" \
"https://api.lokalise.com/api2/projects?limit=10"
echo ""
echo -n "Keys list (limit=500): "
curl -s -o /dev/null -w "%{time_total}s" \
-H "X-Api-Token: ${LOKALISE_API_TOKEN}" \
"https://api.lokalise.com/api2/projects/${PROJECT_ID}/keys?limit=500"
echo ""
echo -n "File download trigger: "
curl -s -o /dev/null -w "%{time_total}s" \
-X POST -H "X-Api-Token: ${LOKALISE_API_TOKEN}" \
-H "Content-Type: application/json" \
"https://api.lokalise.com/api2/projects/${PROJECT_ID}/files/download" \
-d '{"format":"json","filter_langs":["en"],"original_filenames":false}'
echo ""
echo -n "Rate limit headers: "
curl -s -D - -o /dev/null \
-H "X-Api-Token: ${LOKALISE_API_TOKEN}" \
"https://api.lokalise.com/api2/projects?limit=1" \
| grep -i x-ratelimit
Output
- Cursor pagination implemented for all list operations (80% fewer API calls)
- Translation download cache with timestamp-based invalidation
- Batch key operations (create/update/delete) using 500-key batches
- Request queue with PQueue throttling at 5 req/s (safety margin under 6 req/s limit)
- Webhooks configured for async file operations (replacing polling)
- Selective language downloads reducing bandwidth and processing time
- Benchmark results for baseline performance measurement
Error Handling
| Issue | Cause | Solution |
|---|
429 Too Many Requests | Exceeded 6 req/s global rate limit | Use PQueue throttling (Step 4), retry with exponential backoff |
| Slow file downloads | Large project with 50+ languages | Filter by filter_langs to download one language at a time |
| Pagination timeout | Offset pagination on 50K+ key projects | Switch to cursor pagination (Step 1) |
| Bulk create partial failure | Network timeout on large batch | Reduce batch size from 500 to 200 and add retry logic per batch |
| Cache stale after team edits | last_activity not granular enough | Reduce cache TTL or use webhooks to invalidate on project.translation_updated |
bundle_url expired | Download URL only valid for ~30 minutes | Fetch the URL and download immediately; do not store URLs for later |
Examples
Quick Rate Limit Check
set -euo pipefail
curl -s -D - -o /dev/null \
-H "X-Api-Token: ${LOKALISE_API_TOKEN}" \
"https://api.lokalise.com/api2/system/languages?limit=1" 2>/dev/null \
| grep -iE 'x-ratelimit' | while read -r line; do echo " $line"; done
Resources
Next Steps
- For debugging performance issues, collect diagnostics with
lokalise-debug-bundle.
- For SDK upgrade to get latest pagination improvements, see
lokalise-upgrade-migration.
- For setting up CI pipelines with optimized API usage, see
lokalise-ci-integration.
