Capturing traces locally when LangSmith is unreachable and replaying them later

Overview

Agents running in disconnected, on-prem, or air-gapped environments sometimes can't reach LangSmith when it's time to send a trace. By default, a trace that fails to send is lost. The LangSmith SDK has a mechanism to persist those traces to disk instead, so you can send them later once connectivity to a LangSmith endpoint (cloud or self-hosted) is available.

Capturing failed traces to disk

Set the following environment variables where your agent runs:

• LANGSMITH_FAILED_TRACES_DIR: directory path where the SDK writes traces it couldn't send. When set, any trace that fails to reach LangSmith is written here as a JSON file instead of being dropped.

• LANGSMITH_FAILED_TRACES_MAX_MB (optional): caps the total size of the failed-traces directory. Once the cap is reached, the SDK stops writing new files, so an unreachable endpoint over a long period can't fill the disk.

With these set, an agent that can't reach LangSmith keeps running normally. Traces accumulate as JSON files in the configured directory instead of being sent.

Replaying captured traces

Once the environment can reach a LangSmith endpoint again, replay the saved files by posting each one to the LangSmith ingestion API, using the same LANGSMITH_ENDPOINT and LANGSMITH_API_KEY you'd use for normal tracing. This works against either LangSmith SaaS or a self-hosted instance, since both expose the same ingestion endpoint.

A minimal replay approach:

for file in $(ls $LANGSMITH_FAILED_TRACES_DIR); do
  curl -X POST "$LANGSMITH_ENDPOINT/runs" \
    -H "x-api-key: $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d @"$LANGSMITH_FAILED_TRACES_DIR/$file"
done

Confirm each POST succeeds before removing the corresponding file, so a failed replay attempt doesn't lose the trace.

When to use this

This pattern fits any setup where an agent runs disconnected from LangSmith for periods of time and you want tracing data to catch up asynchronously once connectivity returns, rather than losing it. It's a capture-and-replay workaround built on SDK behavior, not a built-in bulk-import feature: LangSmith does not support re-importing traces through any other mechanism, so this only works for traces the SDK itself captured via LANGSMITH_FAILED_TRACES_DIR at the time they failed to send.

References

Check the LangSmith SDK and tracing documentation for the current list of supported environment variables, since LANGSMITH_FAILED_TRACES_DIR and LANGSMITH_FAILED_TRACES_MAX_MB are being formalized in official docs.