StreamChunkTimeoutError during async streaming with langchain-openai

Symptom

An async streaming call against an OpenAI-compatible model fails mid-stream with:

No streaming chunk received for 120.0s (model=<model>, chunks_received=<N>).
The connection may be alive at the TCP layer but is not producing content.
Tune or disable via the `stream_chunk_timeout` constructor kwarg (set to None or 0 to disable)
or the `LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S` env var.

A structured warning is logged alongside the error:

langchain_openai.stream_chunk_timeout fired
extra: {source: "stream_chunk_timeout", timeout_s: ..., model_name: ..., chunks_received: ...}

It typically hits long-running agent workflows where the model can go more than 120 seconds between content chunks. Only async paths (astream, and ainvoke with streaming) are affected; sync stream() is not.

Cause

ChatOpenAI (and other BaseChatOpenAI subclasses) runs a per-chunk wall-clock watchdog called stream_chunk_timeout. It wraps the async SSE iterator and fires when no new parsed content chunk arrives within the threshold. The default is 120 seconds, read from LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S at model construction, falling back to 120.0 s if the env var is unset.

This is a different knob from request_timeout (alias timeout), which governs the overall HTTP call lifetime and is passed to httpx. httpx's read timeout resets on any bytes on the socket, including SSE keepalive comments (: keepalive) that many providers send to hold the connection open during long generations. The OpenAI SDK swallows those keepalives and never surfaces them as chunks, so a stream that is alive at the TCP/HTTP layer but producing no content can hang indefinitely from httpx's point of view. stream_chunk_timeout measures the gap between parsed chunks, so keepalives do not reset it and it fires on genuine content silence.

Raising the overall request timeout (600 s, etc.) does not help, because that setting governs a different failure mode. In particular, the model configuration timeout field in the LangSmith UI maps to request_timeout, not to stream_chunk_timeout, so changing it has no effect on this error.

Negative values passed to the kwarg or env var are rejected and fall back to the 120.0 s default with a warning log, so a bad config cannot silently disable the watchdog.

Resolution

Raise stream_chunk_timeout so the watchdog tolerates longer gaps between content chunks.

Option 1: Environment variable (recommended for self-hosted / Helm)

Set LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S on the queue pod. For a LangGraph self-hosted deployment using the langgraph-cloud Helm chart, add to values.yaml:

queue:
  deployment:
    extraEnv:
      - name: LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S
        value: "600"

Apply with helm upgrade and redeploy. The value is read at model construction, so pods must restart to pick it up.

Option 2: Constructor kwarg

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="...", stream_chunk_timeout=600)

Disabling the watchdog

Not recommended, since it removes protection against indefinitely stalled streams. If you need it off:

llm = ChatOpenAI(model="...", stream_chunk_timeout=None)
# or
llm = ChatOpenAI(model="...", stream_chunk_timeout=0)
# or set LANGCHAIN_OPENAI_STREAM_CHUNK_TIMEOUT_S=0

References

• ChatOpenAI API reference