A little flexibility is all you need: On the asymptotic value of flexible capacity in parallel queuing systems

We analytically study optimal capacity and flexible technology selection in parallel queuing systems. We consider N stochastic arrival streams that may wait in N queues before being processed by one of many resources (technologies) that differ in their flexibility. A resource's ability to process k different arrival types or classes is referred to as level-k flexibility. We determine the capacity portfolio (consisting of all resources at all levels of flexibility) that minimizes linear capacity and linear holding costs in high-volume systems where the arrival rate λ → ∞. We prove that "a little flexibility is all you need": the optimal portfolio invests O(λ) in specialized resources and only O(√λ) in flexible resources and these optimal capacity choices bring the system into heavy traffic. Further, considering symmetric systems (with type-independent parameters), a novel "folding" methodology allows the specification of the asymptotic queue count process for any capacity portfolio under longest-queue scheduling in closed form that is amenable to optimization. This allows us to sharpen "a little flexibility is all you need": the asymptotically optimal flexibility configuration for symmetric systems with mild economies of scope invests a lot in specialized resources but only a little in flexible resources and only in level-2 flexibility, but effectively nothing (o(√λ)) in level-k < 2 flexibility. We characterize "tailored pairing" as the theoretical benchmark configuration that maximizes the value of flexibility when demand and service uncertainty are the main concerns.