Skip to content

v0.1.0a15

Highlights

  • Added opt-in fresh finalist validation after the search stage.
  • Made winner decisions conservative and evidence-based, with an explicit No clear winner outcome when measurements do not separate candidates.
  • Preserved the boundary between search observations and validation evidence.
  • Added offline regeneration and reclassification support for validated runs.

Release artifacts

Platform Install
PyPI pip install "vllm-optimizer==0.1.0a15"
Linux/WSL runtime pip install "vllm-optimizer[runtime]==0.1.0a15"

The universal wheel, source distribution, and SBOM are attached to the release.

Finalist validation

The new analysis.finalist_validation section enables a fixed validation budget:

analysis:
  finalist_validation:
    top_k: 2
    repeats: 6

After search, the baseline and selected candidates run sequentially with fresh servers and fresh measurements. The configured repeat budget is required in full for every selected candidate. Validation scores and artifacts are kept separate from search scores, and a failed or incomplete finalist is not promoted.

The report declares a winner only when the validated objective favors one candidate on every matched workload, observed ranges are separated, and the 95% Student's t intervals of workload means do not overlap. Otherwise it shows No clear winner and presents the best observed candidate without calling it the winner. This is a descriptive uncertainty guard, not a significance test or proof of a global optimum.

Omitting analysis.finalist_validation preserves the existing drift-triggered validation behavior.

Reporting and reproducibility

  • Persisted validation selection, status, repeat budget, search ranking, and selection decisions in run JSON.
  • Added validation-aware confidence sections, recommendation labels, accepted ranking behavior, offline report regeneration, and reclassification.
  • Updated LLM report prompts to avoid causal parameter claims, invented ablations, and treating ranked observations as proof of a winner.

Documentation

  • Documented fresh finalist validation, its configuration, evidence rules, runtime cost, and known limitations.
  • Clarified that adaptive-search associations are observations, not causal effects.

Validation

The public and available private suites, package checks, lint, types, coverage, strict documentation build, audit, and release workflow are run for this release. Hardware smoke and real-backend validation remain dependent on the manual GPU workflow.

Full changelog: v0.1.0a14...v0.1.0a15.