Skip to content

Commit 222494f

Browse files
DOC: V1 improve migration guide
1 parent c3b70fc commit 222494f

File tree

1 file changed

+121
-19
lines changed

1 file changed

+121
-19
lines changed

doc/v1_migration_guide.rst

+121-19
Original file line numberDiff line numberDiff line change
@@ -53,12 +53,19 @@ Specifies the approach for calculating prediction intervals, especially in advan
5353
- **v0.9**: Part of ``MapieRegressor``. Configured for the main prediction process.
5454
- **v1**: Specific to ``CrossConformalRegressor`` and ``JackknifeAfterBootstrapRegressor``, indicating the interval calculation approach (``"base"``, ``"plus"``, or ``"minmax"``).
5555

56-
``cv`` (includes ``groups``)
57-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
56+
``cv``
57+
~~~~~~~
5858
The ``cv`` parameter manages the cross-validation configuration, accepting either an integer to indicate the number of data splits or a ``BaseCrossValidator`` object for custom data splitting.
5959

6060
- **v0.9**: The ``cv`` parameter was included in ``MapieRegressor``, where it handled cross-validation. The option ``cv="prefit"`` was available for models that were already pre-trained.
61-
- **v1**: The ``cv`` parameter is now only present in ``CrossConformalRegressor``, with the ``prefit`` option removed. Additionally, the ``groups`` parameter was removed from the ``fit`` method, allowing groups to be directly passed to ``cv`` for processing.
61+
- **v1**: The ``cv`` parameter is now only present in ``CrossConformalRegressor``, with the ``prefit`` option removed.
62+
63+
``groups``
64+
~~~~~~~~~~~
65+
The ``groups`` parameter is used to specify group labels for cross-validation, ensuring that the same group is not present in both training and calibration sets.
66+
67+
- **v0.9**: Passed as a parameter to the ``fit`` method.
68+
- **v1**: The ``groups`` present is now only present in ``CrossConformalRegressor``. It is passed in the ``.conformalize()`` method instead of the ``.fit()`` method.
6269

6370
``prefit``
6471
~~~~~~~~~~
@@ -81,12 +88,12 @@ Defines additional parameters exclusively for prediction.
8188
- **v0.9**: Passed additional parameters in a flexible but less explicit manner, sometimes mixed within training configurations.
8289
- **v1**: Now structured as a dedicated dictionary, ``predict_params``, to be used during calibration (``conformalize`` method) and prediction stages, ensuring no overlap with training parameters.
8390

84-
``aggregation_method``
85-
~~~~~~~~~~~~~~~~~~~~~~
86-
The ``aggregation_method`` parameter defines how predictions from multiple conformal regressors are aggregated when making point predictions.
91+
``aggregation_method`` & ``aggregate_predictions``
92+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
93+
The aggregation method and technique for combining predictions in ensemble methods.
8794

8895
- **v0.9**: Previously, the ``agg_function`` parameter specified the aggregation method, allowing options such as the mean or median of predictions. This was applicable only when using ensemble methods by setting ``ensemble=True`` in the ``predict`` method.
89-
- **v1**: The ``agg_function`` parameter has been renamed to ``aggregation_method`` for clarity. It now serves the same purpose in selecting an aggregation technique but is specified at prediction time rather than during class initialization. Additionally, the ``ensemble`` parameter has been removed, as ``aggregation_method`` is relevant only to the ``CrossConformalRegressor`` and ``JackknifeAfterBootstrapRegressor`` classes.
96+
- **v1**: The ``agg_function`` parameter has been devided into two separate parameters: ``aggregate_predictions`` and ``aggregation_method``. ``aggregate_predictions`` is specific to ``CrossConformalRegressor``, and it specifies how predictions from multiple conformal regressors are aggregated when making point predictions. ``aggregation_method`` is specific to ``JackknifeAfterBootstrapRegressor``, and it specifies the aggregation technique for combining predictions across different bootstrap samples during calibration.
9097

9198
``Other parameters``
9299
~~~~~~~~~~~~~~~~~~~~
@@ -142,15 +149,28 @@ MAPIE v1 introduces two distinct methods for prediction:
142149

143150
Below is a side-by-side example of code in MAPIE v0.9 and its equivalent in MAPIE v1 using the new modular classes and methods.
144151

145-
MAPIE v0.9 code
146-
~~~~~~~~~~~~~~~
152+
Example 1: Split Conformal Prediction
153+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
154+
155+
Description
156+
############
157+
Split conformal prediction is a widely used method for generating prediction intervals, it splits the data into training, calibration, and test sets. The model is trained on the training set, calibrated on the calibration set, and then used to make predictions on the test set. In `MAPIE v1`, the `SplitConformalRegressor` replaces the older `MapieRegressor` with a more modular design and simplified API.
158+
159+
MAPIE v0.9 Code
160+
###############
161+
162+
Below is a MAPIE v0.9 code for split conformal prediction in case of pre-fitted models:
147163

148164
.. code-block:: python
149165
150166
from sklearn.linear_model import LinearRegression
151167
from mapie.estimator import MapieRegressor
152168
from mapie.conformity_scores import GammaConformityScore
153169
from sklearn.model_selection import train_test_split
170+
from sklearn.datasets import make_regression
171+
172+
# Generate synthetic data
173+
X, y = make_regression(n_samples=100, n_features=2, noise=0.1)
154174
155175
# Step 1: Split data
156176
X_train, X_conf_test, y_train, y_conf_test = train_test_split(X, y, test_size=0.4)
@@ -174,13 +194,19 @@ MAPIE v0.9 code
174194
prediction_points_v0 = v0.predict(X_test)
175195
176196
Equivalent MAPIE v1 code
177-
~~~~~~~~~~~~~~~~~~~~~~~~
197+
########################
198+
199+
Below is the equivalent MAPIE v1 code for split conformal prediction:
178200

179201
.. code-block:: python
180202
181203
from sklearn.linear_model import LinearRegression
182204
from mapie.estimator import SplitConformalRegressor
183205
from mapie.utils import conf_split
206+
from sklearn.datasets import make_regression
207+
208+
# Generate synthetic data
209+
X, y = make_regression(n_samples=100, n_features=2, noise=0.1)
184210
185211
# Step 1: Split data with conf_split (returns X_train, y_train, X_conf, y_conf, X_test, y_test)
186212
X_train, y_train, X_conf, y_conf, X_test, y_test = conf_split(X, y)
@@ -204,15 +230,91 @@ Equivalent MAPIE v1 code
204230
prediction_points_v1 = v1.predict(X_test)
205231
206232
207-
5. Additional migration examples
208-
--------------------------------
233+
Example 2: Cross-Conformal Prediction
234+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
235+
236+
Description
237+
############
238+
239+
Cross-conformal prediction extends split conformal prediction by using multiple cross-validation folds to improve the efficiency of the prediction intervals. In MAPIE v1, `CrossConformalRegressor`` replaces the older `MapieRegressor`` for this purpose.
240+
241+
MAPIE v0.9 code
242+
###############
243+
244+
Below is a MAPIE v0.9 code for cross-conformal prediction:
245+
246+
.. code-block:: python
247+
248+
from sklearn.ensemble import RandomForestRegressor
249+
from mapie.estimator import MapieRegressor
250+
from mapie.conformity_scores import CrossConformalConformityScore
251+
from sklearn.model_selection import train_test_split
252+
from sklearn.datasets import make_regression
253+
254+
# Generate synthetic data
255+
X, y = make_regression(n_samples=100, n_features=2, noise=0.1)
256+
257+
# Step 1: Split data
258+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4)
259+
260+
# Step 2: Train the model on the training set
261+
regression_model = RandomForestRegressor(
262+
n_estimators=100,
263+
max_depth=5
264+
)
265+
266+
# Step 3: Initialize MapieRegressor with the prefit model and cross-conformal conformity score
267+
v0 = MapieRegressor(
268+
estimator=regression_model,
269+
cv=3,
270+
conformity_score=CrossConformalConformityScore()
271+
)
272+
273+
# Step 4: Fit MAPIE on the calibration set
274+
v0.fit(X_conf, y_conf)
275+
276+
# Step 5: Make predictions with confidence intervals
277+
prediction_intervals_v0 = v0.predict(X_test, alpha=0.1)[1][:, :, 0]
278+
prediction_points_v0 = v0.predict(X_test)
279+
280+
Equivalent MAPIE v1 code
281+
########################
282+
283+
Below is the equivalent MAPIE v1 code for cross-conformal prediction:
284+
285+
.. code-block:: python
209286
210-
We will provide further migration examples :
287+
from sklearn.ensemble import RandomForestRegressor
288+
from mapie.estimator import CrossConformalRegressor
289+
from mapie.utils import conf_split
290+
from sklearn.datasets import make_regression
211291
212-
- **Prefit Models**: Using ``SplitConformalRegressor`` with ``prefit=True``
213-
- **Non-Prefit Models**:
292+
# Generate synthetic data
293+
X, y = make_regression(n_samples=100, n_features=2, noise=0.1)
294+
295+
# Step 1: Split data with conf_split (returns X_train, y_train, X_conf, y_conf, X_test, y_test)
296+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4)
297+
298+
# Step 2: Train the model on the training set
299+
regression_model = RandomForestRegressor(
300+
n_estimators=100,
301+
max_depth=5
302+
)
214303
215-
- ``SplitConformalRegressor`` without ``prefit``
216-
- ``CrossConformalRegressor`` with ``fit_params`` (e.g., ``sample_weight``) and ``predict_params``
217-
- ``ConformalizedQuantileRegressor`` with ``symmetric_intervals=False``
218-
- ``JackknifeAfterBootstrapRegressor`` with custom configurations
304+
# Step 3: Initialize CrossConformalRegressor with the prefit model, cross-validation folds, and conformity score
305+
v1 = CrossConformalRegressor(
306+
estimator=regression_model,
307+
confidence_level=0.9, # equivalent to alpha=0.1 in v0.9
308+
cv=3, # cross-validation folds
309+
conformity_score="absolute"
310+
)
311+
312+
# Step 4: fit the model on the training set
313+
v1.fit(X_train, y_train)
314+
315+
# Step 5: Conformalize the model on the calibration set. In CrossConformalRegressor, the calibration set should be equal to the training set.
316+
v1.conformalize(X_train, y_train)
317+
318+
# Step 1: Make predictions with confidence intervals
319+
prediction_intervals_v1 = v1.predict_set(X_test)
320+
prediction_points_v1 = v1.predict(X_test)

0 commit comments

Comments
 (0)