How to Measure TTFT and Inter-Token Latency With Recorded Traffic
This guide builds a check that fails when time to first token (TTFT) or inter-token latency (ITL) regresses on recorded LLM traffic. It uses the LLM streaming lab in mock-lab: a small model server whose timing you control, a client that sends it streamed chat requests, and make targets for every step. You record traffic, replay it against the fast server, slow the server down and watch the same replay fail. Then you apply the same commands to your own model server.