Skip to content

MVP configuration contract

Configuration

experiment:
  name: qwen-h100
  output_dir: runs
  seed: 42

server:
  model: /models/Qwen3-32B
  tensor-parallel-size: 4
  dtype: bfloat16

tune:
  max-num-seqs:
    values: [64, 128, 256]
  gpu-memory-utilization:
    min: 0.85
    max: 0.95
    step: 0.05

env:
  CUDA_VISIBLE_DEVICES: "0,1,2,3"

tune_env:
  VLLM_LOGGING_LEVEL:
    values: ["INFO", "WARNING"]

benchmark:
  repeats: 4
  runs:
    - name: throughput
      request_format: /v1/completions
      profile:
        kind: throughput
        max_concurrency: 16
      constraints:
        - kind: max_duration
          seconds: 2m
      data:
        - kind: synthetic_text
          prompt_tokens: 256
          output_tokens: 128

baseline:
  enabled: true

optimization:
  maximize: output_tokens_per_second
  sampler: tpe
  trials: 18

timeouts:
  startup: 900
  benchmark: 20m

execution:
  host: 127.0.0.1
  health_path: /health
  shutdown_grace: 15
  retry:
    max_attempts: 2

logging:
  level: INFO

Required sections

experiment and server are required. A runnable experiment also requires at least one benchmark.runs entry and a non-empty optimization.maximize metric.

server.model (inline or inherited from server.config) must be an existing local directory. Inline paths are relative to the experiment YAML; paths in a native server config are relative to that native file. The vllm-opt CLI never downloads a model.

Server values

Use server.config for a native vLLM YAML file; explicit server and tuned flags override its values. Other entries under server are fixed vLLM arguments. Top-level tune and tune_env accept either:

values: [a, b, c]

or an inclusive stepped range:

min: 1
max: 8
step: 1

vLLM flags are rendered deterministically. true emits a presence flag, false emits its --no- form, and null omits it; a fixed list emits one flag followed by every item. All processes are launched with argument arrays, never interpolated shell text. Fixed environment variables use top-level env. Unless server.host is set explicitly, the vllm-opt CLI supplies --host 127.0.0.1. execution.host selects the address used for health checks and defaults to the same loopback address.

Benchmark runs

GuideLLM runs require a unique filesystem-safe name, one profile mapping, and exactly one item in data. With benchmark.engine: vllm, use name and args instead; do not include GuideLLM-only fields. constraints may contain any GuideLLM constraint mapping. Nested GuideLLM values are flattened to GuideLLM's CLI format.

GuideLLM always writes results.json; its console output is preserved in benchmark.log. Profiles such as throughput, concurrent, and sweep are passed through without a vLLM Optimizer allowlist.

  • grid evaluates the complete Cartesian product and rejects trials.
  • random and tpe require a positive trials value.
  • Requested trials are capped with a warning at the number of unique configurations.
  • A repeated Optuna suggestion is marked skipped and replaced before any vLLM process starts.
  • experiment.seed controls Random and TPE sampling when supplied.

Scoring

optimization.maximize names one metric supplied by the selected backend. For each benchmark repeat, the vllm-opt CLI averages that metric across the workloads returned by the profile. It then takes the arithmetic mean across eligible repeats for each named benchmark run and the arithmetic mean across benchmark runs for the overall trial score.

Every named benchmark must meet the minimum eligible repeat count and request failure policy. Legacy stored runs without an aggregation policy retain median scoring during regeneration. Only completed tuned trials enter the ranking. The baseline is reported separately and can still be the best observed recommendation in HTML.

Timeouts and retries

timeouts.startup and timeouts.benchmark are positive durations. Durations accept seconds or strings such as 30s, 2m, and 1h.

When timeouts.benchmark is omitted, the vllm-opt CLI uses the GuideLLM duration constraint, workload strategy count, and a safety margin. A GuideLLM run constrained only by max_requests uses a one-hour hard cap when no explicit timeout is provided; set timeouts.benchmark for longer workloads. vLLM Bench Serve retains its 180-second default. The literal value auto is invalid.

execution.retry.max_attempts defaults to one. Only failures marked retryable are attempted again. Startup and benchmark timeouts and recognized connection failures are transient; CUDA OOM, invalid arguments, and unsupported configurations are not.

Logging

logging.level accepts DEBUG, INFO, WARNING, ERROR, or CRITICAL and defaults to INFO. DEBUG streams labeled server and benchmark output while still writing complete log files. --verbose overrides the YAML level with DEBUG for one invocation.