SimonBlanke/Gradient-Free-Optimizers

✨ Added

• `boundary` optimizer parameter for out-of-bounds candidate handling, with `clip`, `reflect`, `periodic`, `random`, and `intermediate` strategies
• Documentation and runnable example for boundary strategies
• SciPy stats continuous distributions can now be used as search-space dimensions; optimizers operate on quantiles internally and pass `ppf` values to objective functions
• Example for SciPy distribution-backed search spaces
• `SearchParams` dict subclass carrying optimization metadata as private attributes for tooling integration

📋 Changed

• SciPy moved from core dependency to optional extra (`pip install gradient-free-optimizers[scipy]`)
• All optimizers ported from direct numpy imports to the internal array/math backends, enabling numpy-free operation
• All pandas imports made lazy, reducing startup cost
• Bayesian Optimization normalizes inputs to [0, 1] before GP fitting for better length-scale stability
• GPR surrogate switched from Matern ν=0.5 to ν=2.5 with hyperparameter optimization (`n_restarts_optimizer=3`)
• TPE bandwidth selection changed from hardcoded 1.0 to Silverman's rule
• Performance improvements in GPR kernel and KDE score computation via vectorized distance matrices
• License metadata migrated to PEP 639 format (`license = "MIT"` under `[project]`), fixing `SetuptoolsDeprecationWarning`

🗑️ Removed

• Legacy optimizer implementations (`optimizers_legacy/`)

🐛 Fixed

• Warm-start rows with distribution values outside the search space are now dropped instead of being silently clipped to boundary quantiles
• NaN distribution quantile positions now fall back to the midpoint quantile instead of propagating NaN to objective functions
• KDE bandwidth computation on degenerate data (all identical points) causing division-by-zero
• KDE bandwidth not recomputed on refit with new data
• `min`/`max` broadcasting in the pure-Python array backend
• `norm_cdf`/`norm_pdf` recursion when checking for iterability

🧪 Tests

• Coverage for SciPy distribution-backed search spaces across optimizers, memory warm starts, SMBO warm starts, constraints, and ask/tell
• Unit tests for all six internal estimators
• Pure-Python backend integration tests (no numpy, no scipy)
• Expanded coverage for distributed module and ask/tell interface
• CI: no-scipy and no-numpy isolation jobs, coverage collection
• CI: strict mode (`GFO_CI_STRICT`) prevents dependency-isolation tests from silently skipping when the wrong packages are installed
• Full Changelog: https://github.com/SimonBlanke/Gradient-Free-Optimizers/compare/v1.12.0...v1.13.0

📋 What's Changed

• Bump pytest from 9.0.2 to 9.0.3 by @dependabot[bot] in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/92
• Full Changelog: https://github.com/SimonBlanke/Gradient-Free-Optimizers/compare/v1.11.1...v1.12.0

🐛 Fixed

• `optimum="minimum"` parameter in `search()` had no effect on the actual optimization. The objective adapter received the raw function instead of the negated one, causing the optimizer to maximize regardless of the `optimum` setting. The negation was only applied to the progress bar display.
• Full Changelog: https://github.com/SimonBlanke/Gradient-Free-Optimizers/compare/v1.10.0...v1.10.1

✨ Added

• New optimizer module (`optimizers/`) using the Template Method Pattern with explicit hook methods (`_iterate_continuous_batch`, `_iterate_categorical_batch`, `_iterate_discrete_batch`)
• Extended search space dimension types: continuous `(min, max)` tuples, categorical `["a", "b"]` lists, and discrete numerical NumPy arrays
• `DimensionType` enum, `DimensionInfo` dataclass, and `DimensionMasks` for dimension-aware vectorized operations
• Automatic vectorization for search spaces with 1000+ dimensions via `DimensionIteratorMixin`
• `resolution` parameter for `GridSearchOptimizer` and `DirectAlgorithm` to handle continuous dimensions
• Mixed-type distance metric (Gower-like) for the DIRECT algorithm across heterogeneous dimensions
• Lazy search data construction in `ResultsManager` for reduced memory footprint during optimization
• State management via property setters with automatic history tracking in `CoreOptimizer`
• + 3 more

📋 Changed

• All optimizers reimplemented to comply with the new Template Method architecture
• Legacy optimizer implementations preserved in `optimizers_legacy/` (not part of public API)
• SciPy restored as a core dependency
• Wall clipping algorithm reworked
• Optimizer initialization refactored (`finish_initialization`, `_generate_position` pattern)
• Converter enhanced with dimension type analysis (`_analyze_dimension_types`)
• Updated CI workflow configuration

🐛 Fixed

• `finish_initialization` in Downhill Simplex and other optimizers
• `_move_random` in sequential model-based optimizers
• Init position and `evaluate_init` override issues in optimizer subclasses
• Empty scores edge case in evaluation
• Full Changelog: https://github.com/SimonBlanke/Gradient-Free-Optimizers/compare/v1.9.0...v1.10.0

✨ Added

• Private array backend (`_array_backend`) for pure Python array operations without NumPy
• Private math backend (`_math_backend`) for mathematical operations without SciPy
• Native `DecisionTreeRegressor` implementation
• Native `ExtraTreesRegressor` implementation
• Native `RandomForestRegressor` implementation
• Native `GradientBoostingRegressor` implementation
• `SimpleProgressBar` class as fallback when tqdm is unavailable
• Sigma self-adaptation for `EvolutionStrategyOptimizer`
• + 5 more

📋 Changed

• scikit-learn is now an optional dependency (native estimators used by default)
• SciPy is now an optional dependency
• tqdm is now an optional dependency
• Complete reimplementation of Powell's Method with improved line search algorithms
• Reworked README with new 3D optimization animation
• Consolidated CI workflows into single `ci.yml`
• Restructured test directory (`tests/test_main/`, `tests/test_internal/`, etc.)
• Improved error messages with actionable suggestions

🗑️ Removed

• `BayesianRidge` estimator
• Linear GP option from Gaussian Process regressor

🐛 Fixed

• Golden section search algorithm in Powell's Method
• Mutable default argument anti-pattern (`constraints=[]` changed to `constraints=None`)
• Missing `@functools.wraps` on internal decorators
• Division by zero edge case in print-times
• Bug in evaluate method

📦 Build

• fix error during twine upload: ["unrecognized or malformed field: 'license-file'](https://github.com/pypa/twine/issues/1216)

📦 Python Version Support

• Removed support for Python 3.9
• Added support for Python 3.14
• Updated CI workflows for macOS, Ubuntu, and Windows

📦 Package Metadata

• Added keywords and classifiers to pyproject.toml for better PyPI discoverability

🧪 Testing Improvements

• Optimized test suite for faster execution
• Reduced test iterations in multiple test files
• Added `optimizers_representative` parametrization for subset testing
• Fixed sporadic test failures:
• `test_inf_nan_1`: Relaxed lower bound from 10 to 5 for probabilistic validation
• `test_start_temp_1`: Added deterministic random seeds and relaxed assertion to allow equality

📋 What's Changed

• Bump pytest from 8.3.4 to 8.3.5 by @dependabot[bot] in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/64
• ⚡️ Speed up function `normalize` by 79% by @misrasaurabh1 in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/69
• ⚡️ Speed up function `normalize` by 90% by @misrasaurabh1 in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/68
• Fix for issue "add support to maximize and minimize objective-function" by @smilingprogrammer in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/71
• ⚡️ Speed up method `LipschitzFunction.find_best_slope` by 81% by @misrasaurabh1 in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/67
• Resolve warnings of NumPy library by @emmanuel-ferdman in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/79
• Bump pytest from 8.3.5 to 8.4.0 by @dependabot[bot] in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/81
• Bump pytest from 8.4.0 to 8.4.1 by @dependabot[bot] in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/83
• + 1 more

✨ New Contributors

• @misrasaurabh1 made their first contribution in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/69
• @smilingprogrammer made their first contribution in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/71
• @emmanuel-ferdman made their first contribution in https://github.com/SimonBlanke/Gradient-Free-Optimizers/pull/79
• Full Changelog: https://github.com/SimonBlanke/Gradient-Free-Optimizers/compare/v1.7.1...v1.7.2

📋 Changes

• add docstring to public API classes
• add type hints
• drop support for python 3.8

📋 Changes

• add support for numpy v2
• add support for pandas v2
• add support for python 3.12
• transfer setup.py to pyproject.toml
• change project structure to src-layout

📋 Changes

• add Genetic algorithm optimizer
• add Differential evolution optimizer

📋 Changes

• add Grid search parameter that changes direction of search
• add SMBO parameter that enables to avoid replacement of the sampling
• fix bug in transition probability of stochastic-hill-climbing and simulated-annealing

📋 Changes

• add DIRECT algorithm
• automatically add random initial positions if necessary (often requested)