Concurrency & Thread Pools

Segments can be searched in parallel, but unbounded concurrency buys queueing, not throughput.

The question

How many workers is the right number?

The structure responsible

Bounded pool, bounded queue, timeouts, backpressure

In this lab: Run a request stream through a simulated pool and watch p99 react to every knob.

building the index…