Self-hosted LangSmith on Kubernetes: sizing, multi-stage layout, CPU floor

Last updated: June 16, 2026

Overview

This article covers three things customers ask when moving a self-hosted LangSmith deployment to production on Kubernetes:

  • How to size the backend API and queue worker by throughput tier.

  • Whether to use one cluster with namespaces or separate clusters per stage, and which Helm values differ.

  • Why the CPU floor exists, what the "1 CPU" vs "2 CPU" numbers mean, and whether scale-to-zero is supported.

The cluster minimum is still 16 vCPU / 64 GB RAM. HPA CPU target is ~70%. Production should run at least 2 replicas per service.

Resource sizing by throughput tier

There is no published official sizing matrix. The numbers below are directional starting points based on the architecture and the ~70% HPA CPU target. Load-test and tune from here.

LangSmith backend API

Tier

Throughput

Replicas

CPU request

Memory request

Low

< 100 traces/min

1–2

500m–1

1–2 GB

Medium

100–1,000 traces/min

2–4

1–2

2–4 GB

High

> 1,000 traces/min

4+

2–4

4–8 GB

Queue worker

Tier

Active runs

Replicas

CPU request

Memory request

Low

< 50

1–2

1

2 GB

Medium

50–200

2–4

2

4 GB

High

> 200

4+

2–4

4–8 GB

Set the HPA scale-down stabilization window to 300s to avoid oscillation. Keep minReplicas: 2 per service in production for HA.

Multi-stage deployment strategy

Two patterns work:

  • Single cluster, namespace per stage. Lower cost. Shared blast radius and resource pool. Fine for dev and test.

  • Separate clusters per stage. Recommended for enterprise production. Full isolation, independent RBAC, prod is protected from dev/test incidents.

Helm values that typically differ per stage:

Value

Dev

Test / Staging

Production

Replica counts

1

1–2

2+ with HPA

External DB

Internal/shared

Dedicated test DB

Dedicated prod DB

Blob storage

Local / MinIO

S3-compatible bucket

S3 prod bucket

Resource requests/limits

Minimal

Moderate

Full production sizing

HPA

Off

Optional

On

PodDisruptionBudgets

Off

Optional

On

Ingress TLS

Self-signed

Valid cert

Valid cert

Log level

DEBUG

INFO

WARN / ERROR

CPU floor and scale-to-zero

Why the CPU floor exists

The queue worker runs 10 concurrent agent runs per instance by default. On a single CPU, those 10 execution threads contend for one core, which causes scheduling latency. Multiple CPUs let goroutines and I/O run in parallel.

The "1 CPU" and "2 CPU" numbers in the docs are not contradictory:

  • 1 CPU is the Kubernetes resource request, the scheduling guarantee.

  • 2 CPU is the recommended limit or production request, the minimum for steady production operation.

Scale-to-zero with KEDA

The LangSmith Helm chart does not ship KEDA ScaledObjects. Scale-to-zero is feasible for queue worker pods using queue depth as the scaling metric, but you have to write the ScaledObject yourself. Do not scale the backend API or frontend to zero. They need to stay up to receive traces and serve the UI.

Cost optimization for idle agents

In priority order:

  1. KEDA ScaledObject on the queue worker, scaled by queue depth, to zero when idle.

  2. HPA with aggressive scale-down (scaleDown.stabilizationWindowSeconds: 60–120) and minReplicas: 1 for near-zero idle.

  3. Cluster autoscaler so idle nodes get reclaimed at the infra level.

  4. For predictable workloads, scheduled scaling (CronJobs that patch replica counts overnight).

References