Benchmark configuration

Set benchmark.engine to guidellm (the default) or vllm. Each run invokes the selected engine against the same vLLM trial. vTune saves raw JSON and benchmark.log, then normalizes the result for scoring and reporting.

vLLM Bench Serve

Use args exactly as flags following vllm bench serve:

benchmark:
  engine: vllm
  repeats: 2
  runs:
    - name: random-throughput
      args:
        dataset-name: random
        random-input-len: 512
        random-output-len: 128
        random-prefix-len: 0
        num-prompts: 1000
        request-rate: inf
        max-concurrency: 32
        percentile-metrics: ttft,tpot,itl,e2el
        metric-percentiles: 50,90,95,99
        ignore-eos: true

vTune automatically uses --backend vllm and also owns model, host, port, base-url, save-result, append-result, result-dir, and result-filename; do not put them under args. Underscores and hyphens are both accepted in keys. true adds a flag and false omits it. A list repeats its flag for every item. Other scalar values are passed as strings, so new vLLM options do not require a vTune release.

Common dataset forms:

# ShareGPT JSON
args:
  dataset-name: sharegpt
  dataset-path: /benchmarks/ShareGPT_V3.json
  num-prompts: 500

# Hugging Face dataset
args:
  dataset-name: hf
  dataset-path: organization/dataset
  hf-split: test
  num-prompts: 500

# Custom JSON or JSONL supported by the installed vLLM
args:
  dataset-name: custom
  dataset-path: /benchmarks/requests.jsonl
  custom-output-len: 128
  num-prompts: 500

# Prefix-repetition workload
args:
  dataset-name: prefix_repetition
  prefix-repetition-prefix-len: 1024
  prefix-repetition-suffix-len: 128
  prefix-repetition-num-prefixes: 16
  prefix-repetition-output-len: 64
  num-prompts: 512

Other upstream datasets include burstgpt, sonnet, random-mm, random-rerank, custom_audio, custom_image, spec_bench, speed_bench, and timed_trace. Their arguments can change with vLLM; use the official reference and place its flags under args.

vTune aliases output_throughput, request_throughput, and total_token_throughput to output_tokens_per_second, requests_per_second, and total_tokens_per_second. It retains raw fields and latency percentiles such as p99_ttft_ms. Completed, failed, and missing requests use the same error-aware ranking as GuideLLM.

GuideLLM

A GuideLLM run accepts name, request_format, profile, constraints, and exactly one data item.

Profiles

Use one profile per named run:

profile: {kind: synchronous}

profile: {kind: throughput, max_concurrency: 32, rampup_duration: 10}

profile: {kind: concurrent, streams: [1, 8, 32], max_concurrency: 32}

profile: {kind: constant, rate: 10}

profile: {kind: poisson, rate: 10}

profile: {kind: sweep, sweep_size: 10, strategy_type: constant}

profile: {kind: replay, time_scale: 1.0}

GuideLLM may add profiles without requiring a vTune release because profile fields are passed through. Consult its benchmark guide for version-specific fields.

Constraints

Stop after a request count:

constraints:
  - kind: max_requests
    count: 1000

Stop after a duration for each profile strategy:

constraints:
  - kind: max_duration
    seconds: 2m

Constraints can be combined and are passed through to GuideLLM.

max_requests is a stopping condition, not a request-serialization setting. Throughput, constant, and poisson profiles may issue requests concurrently; GuideLLM drains in-flight requests before it finishes. Use profile: {kind: synchronous} when each request must wait for the previous response, or set max_concurrency: 1 where the selected profile supports it. vTune preserves GuideLLM's normal console and request-draining lifecycle.

Request formats

Choose the vLLM-compatible route required by the dataset and model:

request_format: /v1/completions

request_format: /v1/chat/completions

request_format: /v1/responses

request_format: /v1/embeddings

The default is /v1/completions.

Synthetic data

data:
  - kind: synthetic_text
    prompt_tokens: 256
    prompt_tokens_stdev: 32
    prompt_tokens_min: 128
    prompt_tokens_max: 384
    output_tokens: 128
    output_tokens_stdev: 16
    output_tokens_min: 64
    output_tokens_max: 192
    turns: 1

GuideLLM also exposes synthetic_image and synthetic_video; use their version-specific fields from its dataset guide.

Hugging Face data

data:
  - kind: huggingface
    source: garage-bAInd/Open-Platypus
    load_kwargs:
      split: train

kind: hf is an alias. source may also be a local dataset directory.

Local files

JSON and JSONL use the same kind:

data: [{kind: json_file, path: /data/prompts.jsonl}]

Other supported file loaders follow the same shape:

data: [{kind: csv_file, path: /data/prompts.csv}]

data: [{kind: text_file, path: /data/prompts.txt}]

data: [{kind: parquet_file, path: /data/prompts.parquet}]

data: [{kind: arrow_file, path: /data/prompts.arrow}]

data: [{kind: hdf5_file, path: /data/prompts.hdf5}]

data: [{kind: db_file, path: /data/prompts.db}]

data: [{kind: tar_file, path: /data/prompts.tar}]

Files must use columns GuideLLM recognizes automatically. vTune does not yet expose GuideLLM's custom column-mapper option.

Trace replay

profile: {kind: replay, time_scale: 1.0}
data:
  - kind: trace_synthetic
    path: /data/trace.jsonl
    timestamp_column: timestamp
    prompt_tokens_column: input_length
    output_tokens_column: output_length

mooncake and weka are alternative GuideLLM trace kinds with their own default column names.

Repeats and errors

benchmark:
  repeats: 3

vTune takes the median score across repeats. It records successful, errored, and incomplete request counts. A workload with more than 50% errored or incomplete requests is excluded; a trial with no eligible workload is not ranked. Remaining trials are ordered by lowest error percentage, lowest error count, then highest configured metric.

See the official GuideLLM dataset guide for file schemas and fields that vary by GuideLLM release.