TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling.

Wed, 01 Oct 2025 01:00:00 +0200

Junyi Chen, Chuheng Du, Renyuan Liu, Shuochao Yao, Dingtian Yan, Jiang Liao, Shengzhong Liu, Fan Wu, Guihai Chen:
TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling. CoRR abs/2510.02758 (2025)