Quantiles not monotonically increasing on test set

[ejhlego]

Hi,

I was keen to use this package since when I've tried to do conformal prediction using quantile regression in the past I've encountered the common issue of quantile forecast results not increasing monotonically.

I see this package aimed to solve that but I am still seeing some NOT monotonic results in my quantiles! It seemed to work before this change feat: support pre-fitted estimators (https://github.com/radix-ai/conformal-tights/pull/19).

I think it could be because I'm passing a fitted model lgbm model into ConformalCoherentQuantileRegressor and so the part that uses XGBoost to model the quantiles doesn't happen possibly.

[lsorber]

Hi @ejhlego, apologies for the delay in responding to this question!

Your analysis of the problem is correct: the fact that you supply a fitted model in combination with PR #19 could break monotonicity.

However, PR #19 is only available on our main branch, it's not yet released to PyPI. This is intentional, as that PR is part of a new version that is still in development.

That means that the solution to your problem is to not install Conformal Tights from GitHub directly, but to use the latest version on PyPI with pip install --upgrade conformal-tights.

Could you check if that solves the issue? If so I'll go ahead and close this one. Thank you!

[lsorber]

Hi @ejhlego, we just released v0.4.0 which brings a number of improvements. Specifically, #28 may help address the issue you reported. If that solves it for you, please let me know so that I can close this issue, thank you!

[lsorber]

For the record: no release on PyPI suffered from this issue, only a transient version on main. Will close this issue as completed, but let me know if you do encounter something like this again!