402 lines
19 KiB
Markdown
402 lines
19 KiB
Markdown
# Boss and party balance lab
|
|
|
|
`scripts/balance_lab.py` runs a chosen World of Warcraft, Conquest of Azeroth,
|
|
or Runes of Magic five-player party against the authored boss catalog. It is an
|
|
interactive Python controller and report generator, not a second combat model.
|
|
The actual fights execute through `scripts/balance-lab/runtime.ts`, which imports
|
|
the game's live TypeScript combat store, party AI, abilities, talents, gear,
|
|
threat, mitigation, healing, boss scaling, and enemy attacks.
|
|
|
|
This split is intentional. Balance formulas stay in the game runtime. Python
|
|
owns selection, repeatable run configuration, aggregation, CSV/Markdown/HTML
|
|
output, and outlier flags. Do not copy combat coefficients into the Python
|
|
script.
|
|
|
|
## Quick start
|
|
|
|
Install the normal project dependencies, then start the guided prompt:
|
|
|
|
```powershell
|
|
npm run balance:lab
|
|
```
|
|
|
|
The prompt selects:
|
|
|
|
1. one random party, five manual members, or an automatic multi-party sweep;
|
|
2. a WoW, CoA, or RoM party catalog; automatic mode can select all three;
|
|
3. a level from 1 through 80;
|
|
4. random parties, guaranteed coverage of all class/spec builds, or a
|
|
searchable selection of specific class/spec builds;
|
|
5. how many unique 1-tank/1-healer/3-damage configurations to generate per
|
|
source when random selection is used;
|
|
6. all bosses, WoW bosses, RoM bosses, or a searchable multi-boss selection;
|
|
7. how many deterministic simulations to run for every party/boss pairing; and
|
|
8. the maximum duration of each fight.
|
|
|
|
Automatic class/spec selection has three strategies:
|
|
|
|
- **Random parties** preserves the original seeded behavior and generates the
|
|
requested number of unique parties per source.
|
|
- **All specs** creates the minimum deterministic set of valid parties needed
|
|
to include every available class/spec build for the selected WoW, CoA, or
|
|
RoM source at least once.
|
|
- **Specific specs** treats each selected build as required coverage and fills
|
|
the remaining tank, healer, and damage slots with distinct compatible builds.
|
|
When alternatives exist, required builds are not reused as fillers.
|
|
|
|
Coverage parties always contain one tank, one healer, and three different
|
|
damage builds. Their composition is deterministic for the party seed, and no
|
|
build is duplicated inside one party.
|
|
|
|
The active member is driven by the shared benchmark policy while the other four
|
|
use the live companion AI. The healer is the default active member. Use
|
|
`--active-slot` to change it.
|
|
|
|
## GM app and server execution
|
|
|
|
A server-authorized GM sees **Balance Lab** directly underneath **GM Tools** on
|
|
the main menu. That screen exposes the automatic sweep controls: party source,
|
|
level, class/spec selection strategy, a searchable spec picker, simulations per
|
|
party/boss, duration, random seed, boss category, and a searchable multi-boss
|
|
picker. Random mode exposes the requested parties per source; coverage modes
|
|
show the derived minimum party count before submission.
|
|
|
|
**Run on server** sends only that allow-listed configuration to the authenticated
|
|
GM API. The server places it in a one-at-a-time queue, launches the same Python
|
|
controller and live TypeScript runtime used by the command line, and writes
|
|
artifacts under:
|
|
|
|
```text
|
|
DATA_DIR/balance-lab/jobs/<job-id>/
|
|
```
|
|
|
|
The browser polls lightweight status records; it does not execute combat ticks.
|
|
Closing the browser does not stop a running server job. Completed jobs and their
|
|
metadata remain in **Run history** across server restarts, with the newest run
|
|
selected automatically. The history shows the most recent 100 jobs and can be
|
|
searched or filtered by status. A job that was still queued or running when the
|
|
server stopped is restored as failed (it cannot safely resume a half-finished
|
|
deterministic sweep) and can be rerun.
|
|
|
|
Each job directory contains a private `job.json` status record alongside its
|
|
report artifacts. On startup, `server/balance-lab-jobs.mjs` rebuilds history from
|
|
those records and treats a directory containing `suite-report.json` as complete.
|
|
Completed server-job directories from the pre-history implementation are
|
|
imported from `suite-report.json` plus `suite-config.json`, then receive a new
|
|
`job.json`; their requester is shown as `unknown (legacy run)` because that was
|
|
not previously recorded.
|
|
Do not add `job.json` to the artifact allow-list: it is server state, not a GM
|
|
download. The standalone `npm run balance:lab` path remains the way to spend the
|
|
local development device's CPU.
|
|
|
|
## In-app analysis and previous runs
|
|
|
|
Select any completed entry in **Run history**, then choose **View graphs and
|
|
filterable data**. The app reads that run's `suite-report.json` through the
|
|
authenticated artifact endpoint and caches it for the current screen session.
|
|
The dashboard includes:
|
|
|
|
- boss damage to the whole party (incoming DTPS);
|
|
- party damage applied directly to each boss;
|
|
- average successful kill time;
|
|
- boss DPS and damage taken for every class/specialization;
|
|
- effective healing done and effective healing received per second for every
|
|
class/specialization;
|
|
- complete boss tables with pressure versus median, total incoming damage,
|
|
defeat/wipe rates, deaths, and health;
|
|
- complete class/spec tables with total and boss DPS, healing done/received per
|
|
second, DTPS, absolute damage taken, healing done/received and absorb totals,
|
|
deaths, source/role-median tuning flags, and a direct ability-damage
|
|
drill-down for every build; and
|
|
- a complete Abilities tab with learned/available abilities (including zero-use
|
|
rows), exact runtime IDs, AI categories, active-run coverage, casts, hits,
|
|
casts per minute, effective/raw damage, DPS, damage share, damage per hit,
|
|
healing, overheal, absorbs, and cooldown; and
|
|
- complete party-composition tables with output, incoming pressure, outcomes,
|
|
and deaths.
|
|
|
|
The overview graphs intentionally show at most the 15 highest visible values so
|
|
labels remain readable. They never discard report data: the Bosses, Class/spec,
|
|
and Parties tabs retain every matching row. Search, party-source, boss-source,
|
|
role, metric, and sort-direction controls filter the display without modifying
|
|
the saved report. **View ability damage** on a class/spec row opens the
|
|
Abilities tab with that exact build selected, damaging abilities ranked by
|
|
effective damage, and its casts, hits, DPS, damage share, and damage-per-hit
|
|
details visible in the table. **Show all specs** clears that drill-down.
|
|
|
|
Server requirements:
|
|
|
|
- the normal checkout and development dependencies must be present because the
|
|
combat bridge uses `vite-node`;
|
|
- Python 3 must be installed, or `BALANCE_LAB_PYTHON` must name its executable;
|
|
and
|
|
- the signed-in account must have the server-issued `gm` role.
|
|
|
|
The GM screen calls `/api/gm/balance-lab/runtime` when it opens and shows the
|
|
resolved Python version. If Python 3 is unavailable, **Run on server** is
|
|
disabled and no job is queued. On the canonical TrueNAS deployment, the service
|
|
command installs `python3`, prints `python3 --version` during startup, and sets
|
|
`BALANCE_LAB_PYTHON=python3`; use the exact YAML in `DEPLOYMENT.md` rather than
|
|
the older Node-only command.
|
|
|
|
`BALANCE_LAB_MAX_CONCURRENT` defaults to `1` and is capped at four.
|
|
`BALANCE_LAB_MAX_QUEUED` defaults to eight and is capped at 50. Raising
|
|
concurrency permits parallel server processes but can make both the game server
|
|
and simulations less responsive. Requests are capped before launch, subprocess
|
|
arguments are passed as an array without a shell, artifact names are
|
|
allow-listed, and every job and artifact endpoint authenticates the account's
|
|
server-issued GM role.
|
|
|
|
## Repeatable command-line runs
|
|
|
|
List exact party build IDs at a level:
|
|
|
|
```powershell
|
|
python scripts/balance_lab.py --source wow --level 60 --list-members
|
|
```
|
|
|
|
List boss keys:
|
|
|
|
```powershell
|
|
python scripts/balance_lab.py --source wow --level 60 --list-bosses
|
|
```
|
|
|
|
Run a random party against every authored boss:
|
|
|
|
```powershell
|
|
python scripts/balance_lab.py --source coa --level 60 --party-mode random --party-seed 42 --seeds 1,2,3 --boss-category all
|
|
```
|
|
|
|
Generate 25 unique WoW parties and run five simulations per party against all
|
|
bosses:
|
|
|
|
```powershell
|
|
python scripts/balance_lab.py `
|
|
--source wow `
|
|
--level 60 `
|
|
--party-mode automatic `
|
|
--party-configurations 25 `
|
|
--simulations-per-party 5 `
|
|
--boss-category all
|
|
```
|
|
|
|
Generate ten parties for each of WoW, CoA, and RoM, but test only RoM bosses:
|
|
|
|
```powershell
|
|
python scripts/balance_lab.py `
|
|
--source all `
|
|
--level 60 `
|
|
--party-mode automatic `
|
|
--party-configurations 10 `
|
|
--simulations-per-party 3 `
|
|
--boss-category rom
|
|
```
|
|
|
|
Automatically test a selected group of bosses:
|
|
|
|
```powershell
|
|
python scripts/balance_lab.py `
|
|
--source coa `
|
|
--party-mode automatic `
|
|
--party-configurations 20 `
|
|
--simulations-per-party 5 `
|
|
--boss "gundrak/entry-29932" `
|
|
--boss "utgarde-keep/entry-23954"
|
|
```
|
|
|
|
Run an exact party against Eck only:
|
|
|
|
```powershell
|
|
python scripts/balance_lab.py `
|
|
--source wow `
|
|
--level 40 `
|
|
--party-mode manual `
|
|
--member "warrior:Protection" `
|
|
--member "priest:Holy" `
|
|
--member "mage:Fire" `
|
|
--member "rogue:Combat" `
|
|
--member "hunter:Marksmanship" `
|
|
--active-slot 2 `
|
|
--boss "gundrak/entry-29932" `
|
|
--seeds 1,2,3 `
|
|
--duration 60
|
|
```
|
|
|
|
Manual selectors accept an exact generated build ID or a unique
|
|
`class-id:specialization`/`class-name:specialization` value. Repeat `--boss` to
|
|
run any selected subset. Omit it to run every boss in the selected boss
|
|
category. `--seeds 4,8,15` is an advanced exact-seed override for
|
|
`--simulations-per-party`.
|
|
|
|
## Output
|
|
|
|
Each run creates `artifacts/balance-lab/<timestamp>/` unless `--output-dir` is
|
|
provided.
|
|
|
|
| File | Contents |
|
|
| --- | --- |
|
|
| `report.html` | Searchable and sortable tables containing every selected member and every boss. |
|
|
| `report.md` | Human-readable complete party and boss tables. |
|
|
| `report.json` | Aggregated machine-readable report. |
|
|
| `raw-runs.json` | Lossless per-seed, per-boss, per-member, and per-ability runtime results. |
|
|
| `bosses.csv` | One aggregate row for every boss, ordered by party damage taken per second. |
|
|
| `party_members.csv` | One aggregate row for every selected party member. |
|
|
| `member_by_boss.csv` | Every party member against every boss. |
|
|
| `member_abilities.csv` | Every observed player/companion ability for every run. |
|
|
| `boss_abilities.csv` | Every observed boss ability for every run. |
|
|
| `runs.csv` | One row per boss and deterministic seed. |
|
|
| `config.json` | Exact rerunnable selection. |
|
|
| `catalog.json` | Party and boss choices shown to the controller. |
|
|
|
|
No report is truncated to a top-N list.
|
|
|
|
Automatic mode creates a complete report directory for every generated party
|
|
under `parties/<party-id>/` and adds these suite-level files:
|
|
|
|
| File | Contents |
|
|
| --- | --- |
|
|
| `suite-report.html` | Searchable/sortable tables for every tested class/spec, party configuration, and boss. |
|
|
| `suite-report.md` / `suite-report.json` | Complete human- and machine-readable suite summaries. |
|
|
| `party_configurations.csv` | Every generated party, its full composition, party DPS, DTPS, deaths, and outcomes. |
|
|
| `class_specs_across_parties.csv` | Every class/spec observed across every configuration, compared with its source/role median. |
|
|
| `class_abilities_across_parties.csv` | Every learned/available and observed class/spec ability, including unused rows, usage frequency, damage contribution, healing, and absorbs. |
|
|
| `class_spec_by_boss.csv` | Every observed class/spec broken down against every boss it fought. |
|
|
| `bosses_across_parties.csv` | Every boss aggregated across all generated parties. |
|
|
| `boss_by_party.csv` | Every boss and party-configuration pairing. |
|
|
| `suite_runs.csv` | Every individual seed run with its party configuration ID. |
|
|
| `suite-config.json` | The exact automatic-generation seed, counts, filters, and generated party build IDs. |
|
|
|
|
## Metrics and interpretation
|
|
|
|
- Boss `incomingDtps` is effective damage received by all five members divided
|
|
by active fight time. Damage from summoned hostile actors is included.
|
|
- `averageIncomingDamage` is total healer workload and therefore includes fight
|
|
length. Read it beside DTPS, wipe rate, deaths, and time to defeat.
|
|
- Member `dps` includes damage to the boss and adds. `bossDps` includes only
|
|
direct boss damage. Suite class/spec `dtps` is damage taken per active second;
|
|
`totalDamageTaken` is the absolute healer/survivability burden across every
|
|
included run.
|
|
- Member `hps` is effective healing done per active second and excludes
|
|
overhealing. `hpsReceived` is effective healing received per active second,
|
|
attributed from each healing event's target (including self-heals).
|
|
`totalHealing` and `totalHealingReceived` are the corresponding absolute
|
|
values across the included runs. Reports created before target attribution
|
|
was added remain readable but require a rerun to populate healing received.
|
|
- Ability `activeRuns` counts simulations with at least one cast, hit, damage,
|
|
healing, overheal, or absorb event. `usageRate` divides that by every run for
|
|
that build; `damageShare` divides the ability's effective damage by the
|
|
class/spec's total effective damage. `available · unused` means the ability
|
|
was learned and executable at the tested level/talent setup but emitted no
|
|
combat event. This is a review signal, not proof of an AI defect: resurrection,
|
|
execute, interrupt, dispel, emergency, and long-cooldown abilities can be
|
|
legitimately unused when their condition never occurs.
|
|
- Companion AI currently applies several instant attacks directly and therefore
|
|
may emit hits/damage without a separate cast event. Use **Active runs** and
|
|
**Hits** as the authoritative usage indicators; use **Casts** and **Casts/min**
|
|
as additional detail, especially for the active-player benchmark controller.
|
|
- Boss pressure flags in a party report compare DTPS with the non-zero median
|
|
boss for that exact party. Suite boss flags compare across all generated
|
|
parties. Suite class/spec flags compare DPS with every observed spec from the
|
|
same source and role. `high` begins at 1.25 times the median and `very
|
|
high` at 1.5; `low` begins at 0.75 and `very low` at 0.5.
|
|
- Flags are triage signals, not automatic tuning decisions. A boss can have high
|
|
DTPS but low total damage because it dies quickly. A member can have low boss
|
|
DPS because it spends time healing, tanking, or damaging adds.
|
|
|
|
The isolated boss room deliberately removes campaign-final health promotion so
|
|
all bosses use the same testing-room durability rule at the chosen level.
|
|
Runtime IDs include the simulation seed, which deterministically varies the
|
|
game's stable target and cooldown rolls.
|
|
|
|
## Coverage and limitations
|
|
|
|
The selectable roster includes every authored WoW and CoA specialization plus
|
|
every supported RoM class-role pair. Existing generated party talent plans are
|
|
used when available. Missing WoW plans are legally allocated through their
|
|
matching live talent tree; CoA builds allocate the chosen specialization tree
|
|
and class tree through the live essence/prerequisite rules.
|
|
|
|
Headless simulation executes combat-store boss attacks and their authored
|
|
damage, healing, control, aura, summon, threat, and timing effects. It does not
|
|
mount the Three.js scene, play visuals or sounds, or run scene-owned world
|
|
objects. Encounter rules that require movement through the original dungeon,
|
|
clicking an object, or a renderer-owned trigger require a live Boss Testing
|
|
playtest in addition to this numeric audit.
|
|
|
|
One selected member must occupy the active player slot because the production
|
|
runtime distinguishes the player from four companions. The report marks that
|
|
member. When a suspected class outlier could be policy-sensitive, rerun with a
|
|
different `--active-slot`; do not compare active and companion results without
|
|
noting that control-policy difference.
|
|
|
|
Companion builds initialize every resource pool required by their learned
|
|
abilities, not only the class's primary pool. Secondary Energy and Focus
|
|
regenerate, and any companion with a Rage pool gains Rage from landed weapon
|
|
attacks; this is what makes druid Cat and Bear rotations executable while Mana
|
|
remains the class's primary resource. A completed companion cast resolves the
|
|
member's currently learned rank before applying its effects, so cast-time spell
|
|
damage and healing cannot fall back to rank one.
|
|
|
|
`playerbots-inspired-v2` keeps level-10-and-up non-healer builds on their
|
|
specialization's authored damage priority whenever that build knows a profile
|
|
damage action. Reactive interrupts, taunts, control, and defenses still take
|
|
precedence. Bear and Cat actions are isolated to their matching Feral specs;
|
|
Overpower, Revenge, and Victory Rush require an explicitly modeled trigger;
|
|
and Drain Soul, Hammer of Wrath, and Kill Shot obey their target-health gates.
|
|
|
|
## Code map
|
|
|
|
- `scripts/balance_lab.py`: prompts, command-line parsing, random/manual/automatic
|
|
party selection, searchable boss selection, process orchestration, aggregates,
|
|
flags, and all party/suite report formats.
|
|
- `scripts/balance-lab/runtime.ts`: live headless fight setup, controlled clock,
|
|
exact party construction, boss execution, and lossless event attribution.
|
|
- `src/ui/BalanceLabScreen.tsx`: GM form, searchable boss selection, persistent
|
|
run-history selection, progress display, report loading, and authenticated
|
|
artifact opening/downloading.
|
|
- `src/ui/BalanceLabResults.tsx`: in-app graphs, report filters, sorting, summary
|
|
cards, and complete boss/class-spec/party tables. Add new dashboard metrics
|
|
here only after they exist in `suite-report.json`.
|
|
- `src/app/balanceLabClient.ts`: typed GM Balance Lab API client.
|
|
- `server/balance-lab-jobs.mjs`: validated job specifications, bounded
|
|
one-at-a-time process queue, progress parsing, `job.json` persistence/startup
|
|
restoration, report-summary extraction, and artifact allow-list.
|
|
- `server/server.mjs`: GM-authenticated runtime-preflight, job, status, and
|
|
artifact HTTP routes.
|
|
- `src/game/benchmark/benchmarkBuilds.ts`: selectable spec builds and legal
|
|
talent allocation. The older canonical cohort remains unchanged.
|
|
- `src/game/partyStore.ts`: `createConfiguredPartyMember`, the narrow factory
|
|
that preserves production stats, equipment, all learned-ability resource
|
|
pools, and appearance for an exact companion.
|
|
- `src/game/bossTesting.ts`: the authoritative all-boss catalog and isolated
|
|
room construction.
|
|
|
|
When extending the tool, add fields to `raw-runs.json` first, aggregate them in
|
|
Python second, and document the metric here. Keep formula ownership in `src/game`
|
|
and keep reports complete rather than introducing top-N filtering.
|
|
|
|
## Validation
|
|
|
|
For tool changes, run at minimum:
|
|
|
|
```powershell
|
|
python -m py_compile scripts/balance_lab.py
|
|
python -m unittest scripts.test_balance_lab
|
|
node --test server/balance-lab-jobs.test.mjs server/server.test.mjs
|
|
python scripts/balance_lab.py --source wow --level 20 --party-mode automatic --party-configurations 2 --boss "gundrak/entry-29932" --simulations-per-party 1 --duration 5 --output-dir artifacts/balance-lab-smoke
|
|
npm test -- --run src/game/benchmark src/game/partyStore.test.ts src/game/partyRuntime.test.ts
|
|
npm run build
|
|
npm run balance:lab:playtest
|
|
npm run release:check
|
|
```
|
|
|
|
The Playwright smoke test uses an isolated temporary database, logs in as a GM,
|
|
checks that Balance Lab is below GM Tools, runs one real Eck server simulation,
|
|
verifies all eleven suite artifacts, exercises the overview and ability graphs
|
|
plus every complete table, checks history, role, class/spec, and unused-ability
|
|
filtering, and captures desktop/mobile screenshots
|
|
under `playtest-artifacts/balance-lab/`. The server unit tests separately restart
|
|
the job service and prove that completed history is restored while interrupted
|
|
work is marked failed. Use a longer all-boss run before making production tuning
|
|
changes.
|