Skip to content

v0.1.0a21 — Performance targets and simpler guides

Highlights

  • Maximize throughput while requiring latency/throughput limits for every workload.
  • Write limits as "200 ms", "1 s", "50 req/s", or numbers in canonical units.
  • Follow one complete quick start; find advanced options through shorter guides.

Package versions and release artifacts

Package metadata and runtime version change from 0.1.0a20 to 0.1.0a21. This remains an alpha release. After publication:

Use Install
Core package pip install "vllm-optimizer==0.1.0a21"
Linux/WSL GPU runtime pip install "vllm-optimizer[runtime]==0.1.0a21"

The GitHub release attaches vllm_optimizer-0.1.0a21-py3-none-any.whl, vllm_optimizer-0.1.0a21.tar.gz, and sbom.json. The PyPI workflow produces build provenance for the distributions.

Performance targets

Use optimization.constraints alongside the maximize objective. Latency supports average/P50/P95/P99 and us/ms/s; throughput supports average and req/s or tok/s. Each constraint has one inclusive min/max bound. See the configuration example.

Search continues through its configured budget. TPE receives the original objective plus separate violations. Only configurations meeting every target can win, including baselines and fresh finalists. Missing evidence fails a target. Measurements and target decisions appear in terminal, JSON, CSV, and HTML and survive retries, regeneration, and failure-policy reclassification.

Repeated GuideLLM workloads now ignore invocation IDs and environment object representations when matching repeats; semantic workload changes still fail coverage. Numeric canonical latency metrics survive secret redaction.

Release automation and validation

Merging a package-version bump starts the release pipeline. After software and exact-commit private-regression gates pass, it creates the matching tag and GitHub release, attaches the three artifacts, and publishes to PyPI. Unchanged versions do not create another release; existing tags are never moved. The private evidence must cover the actual merged commit, so publication may remain blocked until its protected evidence marker is updated. See the release checklist.

Feature validation covers both benchmark backends and Grid/Random/TPE on a real GPU, plus private regression and public package checks. Tests, model data, and GPU artifacts remain outside the repository. Release gates additionally require fresh GPU evidence for the exact release commit.

Full changelog: v0.1.0a20...v0.1.0a21.