Skip to content

v0.1.0a6 — Trustworthy progress and GuideLLM completion

August 31, 2026

This alpha makes long experiments easier to follow and makes incomplete GuideLLM results impossible to mistake for valid measurements.

Highlights

  • Live timestamped terminal progress with a spinner, active stage, elapsed time, trial bar, per-repeat scores, and final session duration.
  • More transparent HTML reports: labelled charts, ordered benchmark runs, standard metric tables, and explicit scoring and importance methodology.
  • GuideLLM now runs without vTune's console-interactive override, preserving its normal in-flight request draining behavior.

GuideLLM completion safety

  • Removed --disable-console-interactive from the GuideLLM command vTune owns. GuideLLM now receives the same normal execution lifecycle as a direct invocation.
  • A zero-exit benchmark with no completed requests now becomes benchmark_no_completed_requests, pointing directly to benchmark.log.
  • Throughput, constant, and poisson profiles may schedule requests concurrently, but GuideLLM must drain in-flight requests before completion.

Progress and artifacts

  • Added timestamped stages and total session elapsed time.
  • Added a compact TTY spinner for active stages and a trial progress bar.
  • Show each repeat's benchmark score and its aggregate score.
  • Avoid the redundant repeats/001 artifact directory when repeats: 1.
  • Warn clearly when all benchmark requests fail or are ineligible.

Reports and configuration

  • Made schema_version optional while remaining compatible with schema 1.
  • Added ordered benchmark rankings, labelled graph axes, and available throughput, TTFT, end-to-end, and total-time summaries.
  • Explained workload eligibility, repeat aggregation, trial scoring, and the exploratory parameter-importance calculation in the HTML report.
  • Added an optional OpenAI-compatible report summary:
analysis:
  llm_summary:
    base_url: https://api.example.com/v1
    model: your-model
    api_key_env: VTUNE_LLM_API_KEY

The key is read only from the environment and is never written to YAML, artifacts, logs, or reports.

CLI and documentation

  • Made vtune --help list validate, retry, reproduce, export, and report with direct examples.
  • Updated the full YAML reference, benchmark guide, scoring guide, reports, quick start, and changelog.

Validation

  • 156 private tests and 28 subtests passed.
  • Python compilation and whitespace/diff checks passed.
  • The test suite covers GuideLLM command construction, zero-completion detection, report generation, configuration compatibility, and CLI help.

Install

pip install vtune==0.1.0a6

Use pip install "vtune[runtime]==0.1.0a6" to install vLLM and GuideLLM with vTune. The pure-Python wheel supports inspection on Linux, Windows, and macOS; running vLLM experiments remains Linux-only.