ConfOpt is a flexible hyperparameter optimization library, blending the strenghts of quantile regression with the calibration of conformal prediction.

Find out how to include it in your ML workflow below! 👇

User Guide

• Installation
• Getting Started
• Advanced Usage

📈 Benchmarks

ConfOpt is significantly better than plain old random search, but it also beats established tools like Optuna or traditional Gaussian Processes!

The above benchmark considers neural architecture search on complex image recognition datasets (JAHS-201) and neural network tuning on tabular classification datasets (LCBench-L).

For a fuller analysis of caveats and benchmarking results, refer to the latest methodological paper.

🔬 Theory

ConfOpt implements surrogate models and acquisition functions from the following papers:

• Adaptive Conformal Hyperparameter Optimization: arXiv, 2022

• Optimizing Hyperparameters with Conformal Quantile Regression: PMLR, 2023

• Enhancing Performance and Calibration in Quantile Hyperparameter Optimization: arXiv, 2025