Ship PairwiseGP from Botorch into BoFire by Jimbo994 · Pull Request #768 · experimental-design/bofire

Jimbo994 · 2026-05-20T21:22:07Z

Motivation

This PR adds PairwiseGPSurrogate, a Gaussian process surrogate that infers a latent utility function from pairwise comparison data, wrapping BoTorch's PairwiseGP. It introduces a PairwiseTrainableSurrogate mixin with a fit(experiments, preferences) signature: experiments holds the candidate designs (tagged with a labcode), and preferences holds winner/loser pairs.

Supporting data-model additions were needed to mirror BoTorch's PairwiseGP defaults: a new SmoothedBoxPrior, a concrete instantiable Interval prior constraint, and an optional initial_value on the GreaterThan prior constraint (without it the kernel lengthscale initializes to 0 → non-PSD covariance).

Have you read the Contributing Guidelines on pull requests?

yes

Have you updated `CHANGELOG.md`?

yes

Test Plan

New tests/bofire/surrogates/test_pairwise_gp.py (10 tests): fit + predict, latent-utility ranking recovery (Kendall-Tau), sign-convention symmetry (swapping A/B slots + flipping sign yields the identical fit), dumps/loads round-trip, and all validation errors (missing/duplicate labcode, missing preference columns, unknown labcodes, dropped ties).
Data-model specs added for PairwiseGPSurrogate (valid + 2 invalid), SmoothedBoxPrior, Interval, the updated GreaterThan, and a ScaleKernel with a SmoothedBoxPrior outputscale — all exercised by the existing serialization / deserialization round-trip suite.
pytest tests/bofire/data_models + surrogate suite: 1373 passed, ruff clean.
The new tutorial docs/tutorials/advanced_examples/pairwise_gp.qmd renders successfully with quarto render.

…ogate now correctly influence GP fitting

jduerholt

Hi Jim,

looks already really nice!

I let a few comments. I try to get the other PR on the Priors merged in today, and then you can merge main in again and also go over the comments.

Best,

Johannes

jduerholt · 2026-05-21T10:38:37Z

+            )
+        return self
+
+    @model_validator(mode="after")


Why do we need this? We do not have it anywhere else, or?

I think, this is not needed and should also not be enforced here.

yes you are right, will remove it.

jduerholt · 2026-05-21T10:44:49Z

+                self.kernel,
+                batch_shape=torch.Size(),
+                active_dims=list(range(n_dim)),
+                features_to_idx_mapper=None,


Here we need to the stuff to the features_to_idx mapper to have feature dependent kernels. Have a look at the singletaskgp

ok, I adjusted this. We now bring this in BoTorchSurrogate, and have done some refactoring as a result of it, you will see it later. Could remove some duplicated code!

jduerholt · 2026-05-21T10:46:20Z

+        )
+        fit_gpytorch_mll(mll, options=self.training_specs, max_attempts=50)
+
+    def _predict(self, transformed_X: pd.DataFrame):


Can we not get this from somewhere via inheritnce?

Should be, or?

yes you are right. will do this.

jduerholt · 2026-05-21T10:50:13Z

+                "PairwiseGPSurrogate requires unique labcodes."
+            )
+
+        if not isinstance(preferences, pd.DataFrame):


I would remove this, we are having this from the typing

removed in next push

Resolved prior-constraint conflicts: adopted hotfix/noiseprior's generalized initial_value support across Positive/GreaterThan/LessThan, kept the concrete Interval type and PairwiseGP-related specs. Co-Authored-By: Claude Opus 4.7 <[email protected]>

jduerholt · 2026-05-22T06:25:08Z

@Jimbo994: the stuff regarding the priors is merged.

- #4: remove validate_scaler_features from PairwiseGPSurrogate -- it duplicated the validator already on TrainableBotorchSurrogate. - #6: add a `likelihood` field to PairwiseGPSurrogate ("probit" default, or "logit"). PairwiseGP has no noise prior to expose, but the probit/logit likelihood choice is now configurable. - #7: drop the PairwiseGPSurrogate._predict override and inherit it from BotorchSurrogate -- PairwiseGP.posterior ignores observation_noise, so the base implementation is equivalent. - #9: remove the runtime isinstance(preferences, pd.DataFrame) check; the fit() type hint already documents the contract. Comments #5 (features_to_idx_mapper) and #8 (labcode vs index) are parked for further discussion with the reviewer. Co-Authored-By: Claude Opus 4.7 <[email protected]>

jduerholt · 2026-05-22T10:04:57Z

I did some more changes yesterday evening, you find it in main.

Add `get_feature_indices` to the PairwiseTrainableSurrogate mixin and pass it as `features_to_idx_mapper` in `_fit_pairwise`, so feature-specific kernels (a collection of sub-kernels each acting on a subset of inputs) resolve their feature keys to datapoint-tensor columns. PairwiseGPSurrogate does not derive from TrainableBotorchSurrogate, so the method is defined on the mixin rather than inherited; it is structurally identical to TrainableBotorchSurrogate.get_feature_indices (noted in its docstring). Adds a mixed continuous/categorical feature-specific-kernel test. Co-Authored-By: Claude Opus 4.7 <[email protected]>

# Conflicts: # bofire/data_models/priors/interval.py # tests/bofire/data_models/specs/prior_constraints.py

…ces (#5) Per reviewer discussion, avoid duplicating `get_feature_indices`. It needs `inputs`, `categorical_encodings` and `engineered_features`; the first two are already on `BotorchSurrogate`, so `engineered_features` is hoisted to join them: - `engineered_features` field + `validate_aggregations` move from `TrainableSurrogate` to the `BotorchSurrogate` data model; the duplicate field declaration on `LinearDeterministicSurrogate` is removed. - functional `BotorchSurrogate.__init__` sets `engineered_features`; redundant assignments drop from `TrainableBotorchSurrogate`, `LinearDeterministicSurrogate` and `PairwiseGPSurrogate`. - `get_feature_indices` moves to functional `BotorchSurrogate`; the copies on `TrainableBotorchSurrogate` and the `PairwiseTrainableSurrogate` mixin are removed. `SingleTaskGP` and `PairwiseGP` now inherit one shared copy. This supersedes the mixin-local copy from commit bb4d1fd. The `CategoricalDeterministicSurrogate` spec gains the inherited `engineered_features` field. Co-Authored-By: Claude Opus 4.7 <[email protected]>

Jimbo994 · 2026-05-22T14:29:45Z

@jduerholt this is ready for another round of review after the tests complete.

jduerholt

Looks almost done, only thing missing besides my comments is to do proper registration in data_models/priors/_register.py

jduerholt · 2026-05-26T06:25:11Z

        return v
+
+    @model_validator(mode="after")
+    def validate_aggregations(self):


Can you rename this to validate_engineered_features? validate_aggregations is legacy.

had to rename in linear_deterministic_surrogate too.

jduerholt · 2026-05-26T06:35:11Z

+    inputs: Inputs
+    outputs: Outputs
+    predict: Callable[..., pd.DataFrame]
+    input_preprocessing_specs: InputTransformSpecs


Are the input_preprocessing_specs also provided from there? I do not think so. Have a look.

Also check for categorical_encodings and the rest here.

input_preprocessing_specs, and categorical_encodings, engineered_features are inherited from BotorchSurrogate and Surrogate. The scalar is now set in the PairwiseGPSurrogate.

Jimbo994 · 2026-05-26T09:45:00Z

Looks almost done, only thing missing besides my comments is to do proper registration in data_models/priors/_register.py

So we place priors and constraints on the lengthscale and outputscale, which are related to the RBFKernel and ScaleKernel respectively, they should already be handled in the data_models/priors/_register.py. We don't place any constraints or priors on the likelihood so we need no further changes here? Or am I missing something?

# Conflicts: # CHANGELOG.md

jduerholt and others added 11 commits May 20, 2026 09:06

vanilla claude stuff

457ab02

fix(surrogates): noise_prior and noise_constraint on SingleTaskGPSurr…

6821f42

…ogate now correctly influence GP fitting

roundtrip for bug

af1404c

generalize

7e7b003

add priors, some refactor

2aad969

fixes and addittions

f9a01e1

add tests

2f6541e

update mutltiask gp to botorch defaults

65b7152

Merge remote-tracking branch 'origin/main' into feature/pairwise

45fb402

added docs

73481ac

add to changelog

efa3a6c

Jimbo994 requested a review from jduerholt May 20, 2026 21:22

fix hypeopt

45d5278

jduerholt requested changes May 21, 2026

View reviewed changes

jduerholt and others added 6 commits May 21, 2026 13:06

some changes

27a5538

cleaning up the api

92c648d

make test less flaky

bdad654

fix seruialization

7d81545

changelog updated

6bcab72

Jimbo994 and others added 3 commits May 22, 2026 14:15

Merge remote-tracking branch 'origin/main' into feature/pairwise

5485c0c

# Conflicts: # bofire/data_models/priors/interval.py # tests/bofire/data_models/specs/prior_constraints.py

jduerholt reviewed May 26, 2026

View reviewed changes

rename validators

897abd0

Merge remote-tracking branch 'origin/main' into feature/pairwise

e912190

# Conflicts: # CHANGELOG.md

jduerholt merged commit c418a59 into main May 26, 2026
10 of 11 checks passed

Jimbo994 deleted the feature/pairwise branch May 26, 2026 10:18

Conversation

Jimbo994 commented May 20, 2026

Motivation

Have you read the Contributing Guidelines on pull requests?

Have you updated CHANGELOG.md?

Test Plan

Uh oh!

jduerholt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jduerholt commented May 22, 2026

Uh oh!

jduerholt commented May 22, 2026

Uh oh!

Jimbo994 commented May 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jduerholt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jimbo994 commented May 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Have you updated `CHANGELOG.md`?

Jimbo994 commented May 22, 2026 •

edited

Loading