Skip to main content
Findable runs as a single-process Node.js server (via PM2). All chat and flow requests stream responses over SSE, holding one open HTTP connection per active session. There is no built-in request queue or concurrency limiter—every incoming request is forwarded immediately to the upstream AI provider.

​
Concurrency Guidelines

​
External Service Limits

​
Scaling Recommendations

  • PM2 cluster mode — Set instances: 'max' in ecosystem.config.js. The human-input broker persists request state in the humaninput Cosmos DB container (partition key /executionId, per-item TTL) so it works correctly across multiple processes. A polling fallback (2 s interval) detects cross-instance responses.
  • Concurrency limiter — Add a semaphore (e.g. p-limit) before AI API calls to prevent flooding your Azure OpenAI quota.
  • Multiple AI deployments — Load-balance across Azure OpenAI deployments with separate RPM/TPM quotas.
  • Cosmos DB auto-scale — Enable auto-scale (up to 4,000+ RU/s) for production workloads.
  • Azure AI Search tier — Upgrade to S1+ for workloads exceeding 15 QPS.
Rule of thumb: For a typical team deployment, 20–50 concurrent active chats and 5–15 concurrent flows is a comfortable operating range without rate-limit pressure or degraded responsiveness.