Skip to content

v0.1.0a20 — vLLM 0.30 sampling and trustworthy benchmark feedback

Highlights

  • The runtime extra now targets vLLM 0.30.x and GuideLLM 0.7.x.
  • Generation benchmarks default to temperature zero; explicit nonzero sampling is preserved and produces a startup warning for each affected named run.
  • Benchmark validation, progress tracking, diagnostics, and report confidence now agree with the effective command and recorded measurements.

Package versions and release artifacts

The package metadata and runtime version both change from 0.1.0a19 to 0.1.0a20. This remains an alpha release.

Platform Install
PyPI pip install "vllm-optimizer==0.1.0a20"
Linux/WSL runtime pip install "vllm-optimizer[runtime]==0.1.0a20"

The GitHub release attaches vllm_optimizer-0.1.0a20-py3-none-any.whl, vllm_optimizer-0.1.0a20.tar.gz, and sbom.json. The protected PyPI workflow produces build provenance for the distributions.

Sampling and benchmark execution

  • GuideLLM generation routes receive an explicit temperature, while supported non-generation routes receive no sampling setting.
  • Temperatures must be finite and in [0, 2]. Multiple completions (n > 1) require an explicit nonzero effective temperature.
  • vLLM argument normalization accepts optional -- prefixes, inline values, and dotted JSON fields. JSON field names retain their underscores, and duplicate aliases fail preflight.
  • Extra-body temperature overrides follow vLLM parser precedence. Disabled extra-body flags are omitted before JSON validation.
  • Failed benchmarks include their run name, log path, and available diagnostics. Request-completion failure codes remain recoverable by offline reclassification.
  • Progress uses the same argument spellings as command construction and excludes warmups and enabled readiness requests. Readiness is disabled by default.

Report confidence

Drift compares repeat halves within the same trial and workload. The report labels changes within the configured threshold separately from detected drift. Retained baselines still warn about insufficient repeats or drift, and drift thresholds must be finite and non-negative. Temperature zero reduces sampling variation; repeated timings can still vary.

Validation

Release gates cover Python 3.11/3.12 CI, public and private regression tests, lint, typing, coverage, strict documentation, clean wheel installation, dependency audit, SBOM generation, and GPU smoke evidence for the exact release commit. Private fixtures and GPU output remain outside the repository.

Full changelog: v0.1.0a19...v0.1.0a20.