Skip to content

GuideLLM benchmark options

A GuideLLM run accepts name, request_format, profile, constraints, and exactly one data item.

Profiles

Use one profile per named run:

profile: {kind: synchronous}

profile: {kind: throughput, max_concurrency: 32, rampup_duration: 10}

profile: {kind: concurrent, streams: [1, 8, 32], max_concurrency: 32}

profile: {kind: constant, rate: 10}

profile: {kind: poisson, rate: 10}

profile: {kind: sweep, sweep_size: 10, strategy_type: constant}

profile: {kind: replay, time_scale: 1.0}

GuideLLM may add profiles without requiring a vLLM Optimizer release because profile fields are passed through. Consult its benchmark guide for version-specific fields.

Constraints

Stop after a request count:

constraints:
  - kind: max_requests
    count: 1000

Stop after a duration for each profile strategy:

constraints:
  - kind: max_duration
    seconds: 2m

Constraints can be combined and are passed through to GuideLLM.

A GuideLLM run constrained only by max_requests uses a one-hour hard cap when timeouts.benchmark is omitted. Set it explicitly for longer workloads. A request count does not provide a safe workload-duration estimate; duration-constrained runs can use the derived timeout instead.

max_requests is a stopping condition, not a request-serialization setting. Throughput, constant, and poisson profiles may issue requests concurrently; GuideLLM drains in-flight requests before it finishes. Use profile: {kind: synchronous} when each request must wait for the previous response, or set max_concurrency: 1 where the selected profile supports it. The vllm-opt CLI preserves GuideLLM's normal console and request-draining lifecycle.

Request formats

Choose the vLLM-compatible route required by the dataset and model:

request_format: /v1/completions

request_format: /v1/chat/completions

request_format: /v1/responses

request_format: /v1/embeddings

The default is /v1/completions.