We monitor time to first token every minute. The server running this probe is in Ashburn, Virginia, and sends short prompts through the Kottos AI gateway to record how long the first visible token takes to arrive. The charts show the median (p50) and the slowest tenth (p90) of the previous hour for the last 24 hours. We send a short prompt so that time to first token is a proxy for RTT + queue.
The charts get the data from https://data.kottos.ai/probe.json, updated every minute. The data holds a p50, p90, sample count and error count for every minute of the last 24 hours. Times are in milliseconds.
curl -s https://data.kottos.ai/probe.json | python3 -c '
import json, sys
d = json.load(sys.stdin)
for leg in d["legs"]:
s = d["series"][leg["step"]]
print(leg["step"], s["p50"][-1], s["p90"][-1], s["n"][-1])'