Fast Tokens, Slow Agents: Budgeting the Entire AI Workflow

An agent can receive fast model responses and still miss its task deadline. Tool pauses, sandbox startup, repeated reasoning, cache restoration, and verification all consume the same end-to-end budget, but are often measured by different systems. Recent work on workflow-aware serving, agentic workload characterization, and elastic prefill-decode capacity makes this a timely control problem. This white paper proposes a workflow budget contract that carries a deadline, quality floor, resource envelope, and recovery reserve across orchestration and inference boundaries. Its contribution is an implementable control specification and evaluation protocol, not a new serving engine or measured performance result. A worked example shows how locally acceptable calls can exhaust a workflow budget. The central recommendation is to optimize correct, policy-compliant tasks completed on time while retaining engine-level latency and tenant-fairness measurements.

Artificial Intelligence Cloud Computing Software Engineering