Replace bot tuning guesses with enforced calibration

This commit is contained in:
Jay
2026-07-26 05:13:54 -04:00
parent fffd60ad9d
commit 0ac69b101a
11 changed files with 690 additions and 162 deletions
+27 -10
View File
@@ -12,6 +12,9 @@ export JAVA_HOME="/Applications/Android Studio.app/Contents/jbr/Contents/Home"
./gradlew :engine:jvmTest # engine tests (JVM)
./gradlew :engine:testAndroidHostTest # same suite, Android variant
./gradlew :sim:run --args="50000" # simulate 50k hands, print bot stats
./gradlew :sim:test # fast calibration-policy tests
./gradlew :sim:run --args="styles" # enforced controlled style experiment
./gradlew :sim:run --args="calibrate" # enforced 4×100k fixed-pool skill + style calibration
./gradlew :app:assembleDebug # build the APK
```
@@ -44,12 +47,21 @@ export JAVA_HOME="/Applications/Android Studio.app/Contents/jbr/Contents/Home"
169 starting hands so `looseness` means "plays the top N%".
4. **The simulator is how bots get tuned.** Run it after any bot change, across
several seeds — a single seed will happily agree with a wrong conclusion.
The quick 50k run is diagnostic; only `calibrate` is an enforced result.
5. **A skill parameter must not smuggle in a style change.** Several bugs came
from exactly this: `positionAwareness` silently reduced hands played,
`potOddsRespect` systematically loosened weak players (which is a *winning*
adjustment, so it inverted the gradient), and error direction overwrote style
entirely. Skill should change how *well* a decision is made, not how loose or
tight the player is.
6. **Pot odds are the post-flop baseline.** Never apply a blanket implied-odds
discount: it is categorically wrong on the river, and future value on earlier
streets must account for future costs and reverse implied odds before it is
called an advantage.
7. **DecisionTrace is the coach contract.** It records raw pot odds, the actual
adjusted threshold, every adjustment, intended and chosen actions, and whether
a skill error changed the decision. The coach explains these values; it does
not reconstruct hidden bot logic.
## Testing notes
@@ -59,8 +71,9 @@ export JAVA_HOME="/Applications/Android Studio.app/Contents/jbr/Contents/Home"
share one: bots consume RNG proportional to their `equityIterations`, so a
shared stream means changing a profile silently changes the cards dealt.
- `./gradlew :sim:run --args="chart"` dumps the starting-hand ranking.
- Small samples lie. 1,000 hands is not enough to rank profiles — use 50,000+
before believing a gradient.
- Small samples lie. 1,000 hands is not enough to rank profiles. Use the paired
4×100k fixed-opponent calibration before accepting a skill change; it computes
confidence bounds and exits nonzero when the contract fails.
## Status
@@ -70,14 +83,18 @@ export JAVA_HOME="/Applications/Android Studio.app/Contents/jbr/Contents/Home"
action order, malformed agent output, **TDA Rule 47** (incomplete raises do not
reopen betting, but several that cumulatively reach a full raise do), and
**TDA Rule 20** (odd chip to the first winner left of the button).
- Bots: profiles play like their labels (Rock 14.5% VPIP against a 12% setting;
`ProfileBehaviourTest` asserts this). Win rates are in a plausible range —
roughly +20 bb/100 for a strong seat rather than the earlier +113.
- **Adjacent top tiers are not separable.** Advanced and Expert sit inside
seed-to-seed noise of each other over 100k hands. The simulator therefore
asserts each level beats the one *two* tiers below it, which holds on every
seed tried; claiming strict adjacent ordering from one seed would be reading
noise as signal.
- Bots: controlled style calibration holds style constant against the same five
opponents and deal seed. Rock is 10.3% VPIP, Maniac 67.5%; looseness ordering,
Calling Station passivity (9.8% PFR, 0.27 AF), Maniac aggression, and PFR
relationships all pass.
- Skill calibration pairs four 100k-hand seeds. Every candidate occupies the same
seat against the same fixed opponent pool and deal seed. Advanced and Expert
are allowed to overlap, but both must beat Intermediate and Intermediate must
beat Beginner with a positive 95% lower confidence bound. Current lower bounds
are +26.63, +16.19, and +24.71 bb/100 respectively.
- Expert differs by mechanism: it alone maintains opponent reads. The aggression
prior is measured by the controlled neutral TAG experiment (0.229 observed,
0.22 configured), not selected because it looks plausible.
### Rules invariants that are easy to get wrong