enarjord/passivbot

📦 Highlights

• Live authoritative state now always uses the staged account-state pipeline. The legacy live refresh selector was removed; use the `v7.10` branch if you need to compare old behavior.
• Staged live planning now requires explicit freshness for account state, completed candles, and market snapshots before Rust order calculation.
• Live market prices now come from a dedicated market snapshot provider instead of incomplete candle paths, with strict retry/fail behavior for missing ticker data.
• Live fill events now distinguish detected fills from realized-PnL enrichment. Close fills can log `pnl=pending`, PnL-dependent logic waits for enrichment, and an enrichment log is emitted when authoritative PnL arrives.
• Rust order orchestration now emits only the next most-likely flat entry order for live symbols without a position, while preserving full grid expansion once a position exists.
• Live initial-entry posting has an optional executor-side distance gate to reduce far-from-market EMA-driven order churn.
• Forager selection now supports score hysteresis and improved INFO/DEBUG diagnostics.
• Candle refreshes now prioritize active symbols, defer broad background warmup by default, cap forager refresh wall time, and tolerate bounded open-ended 1m tail gaps.
• + 2 more

✨ New Tools

• `passivbot tool ticker-probe`
• `passivbot tool ticker-endpoint-probe`
• `passivbot tool hyperliquid-abstraction-probe`

📦 Configuration Notes

• `live.authoritative_refresh_mode` was removed.
• `live.price_distance_threshold` was removed.
• `live.initial_entry_exec_max_market_dist_pct` controls the narrower live-only initial-entry posting economy gate.
• `live.forager_score_hysteresis_pct` defaults to `0.02`.
• `live.max_active_candle_tail_gap_minutes` defaults to `10`.
• `live.max_forager_candle_refresh_seconds` defaults to `45`.
• `live.max_ohlcv_fetches_per_minute` defaults to `24`.
• `live.recv_window_ms` defaults to `10000`.
• + 1 more

📦 Upgrade Notes

• Reinstall after pulling so the Python package and Rust extension are rebuilt for `7.11.0`.
• Operators running live bots should expect the first startup after update to rebuild the Rust extension if stale.
• Review configs for removed `live.authoritative_refresh_mode` and `live.price_distance_threshold` keys; config normalization strips stale `price_distance_threshold`, but the legacy refresh selector is no longer supported.
• Watch early live logs after upgrade for exchange-specific ticker, candle, and account-state diagnostics, especially on KuCoin and Hyperliquid.
• See `CHANGELOG.md` for the full change list.

📦 Highlights

• Added the OHLCV v2 foundation under `caches/ohlcvs/`, with monthly chunk storage, SQLite catalog metadata, legacy cache import, persistent gap tracking, and v2-aware backtest preparation.
• Added `passivbot tool inspect-ohlcvs` for inspecting v2 OHLCV cache coverage, chunk validity, persistent gaps, and recent fetch attempts.
• Updated the canonical schema and mirrored example profile to `configs/examples/default_trailing_grid_long_npos7.json`, with `config_version = v7.10.0`.
• Removed inflated grid re-entry behavior. Grid re-entries are now always normal-or-cropped, while historical inflated order-type ids remain decodable for old fills and restart compatibility.
• Renamed collateral-agnostic strategy-equity metrics to canonical `*_strategy_eq` names, while keeping old `*_strategy_pnl_rebased` and `*_hsl` names as input/result aliases.
• Added day-denominated duration metrics alongside existing hour metrics for high exposure, peak recovery, position held, and position unchanged.
• Added richer backtest artifacts: `dataset.json`, `strategy_equity` in `balance_and_equity.csv.gz`, artifact loading helpers, and single-coin fill plotting for notebooks.

📦 Upgrade Notes

• Reinstall after upgrading:
• `python3 -m pip install -e .`
• or
• `python3 -m pip install -e ".[full]"`
• If the Rust extension looks stale, rebuild it in the active environment:
• `maturin develop --release`
• New configs should use `config_version = "v7.10.0"`.
• `configs/examples/default_trailing_grid_long_npos10.json` has been replaced by `configs/examples/default_trailing_grid_long_npos7.json`.
• + 2 more

📦 OHLCV v2 Foundation

• Added v2 OHLCV chunk storage and catalog modules for backtest-oriented candle preparation.
• Backtests can prepare HLCV payloads from the v2 local store when coverage is already available, while keeping `caches/hlcvs_data/` as the fast reusable prepared bundle cache.
• Legacy daily OHLCV shards, including compressed `.npz` shards, can be imported into the v2 store.
• Persistent exchange-side gaps are tracked so repeated fetch attempts do not endlessly retry known unavailable ranges.
• Combined-exchange backtests now have a v2-aware load path and clearer progress logging while preparing candles.

🧪 Backtest Artifacts and Analysis

• Backtest runs now persist `dataset.json` with the HLCV cache files used by the run.
• Added `src/backtest_artifacts.py` helpers for loading config, analysis, fills, balance/equity, HLCVs, timestamps, BTC/USD prices, and market settings from an artifact directory.
• Notebook workflows can call `load_backtest_artifact_workspace(...)`, `candles_for_coin(...)`, and `plot_fills_for_coin(...)`.
• `balance_and_equity.csv.gz` now includes collateral-agnostic `strategy_equity`.
• Backtest BTC collateral is initialized at the first active trading step instead of during EMA warmup.
• `drawdown_worst_mean_1pct` metrics now derive drawdowns from the full-resolution equity curve before averaging the worst 1% of daily worst drawdowns.

📦 Metrics and Optimization

• Canonical collateral-agnostic strategy-equity metrics now use `*_strategy_eq`.
• Deprecated `*_strategy_pnl_rebased` and `*_hsl` metric names are accepted as aliases for configs, limits, visibility filters, Pareto tools, and older stored result files.
• `peak_recovery_hours_pnl` now uses net realized PnL (`pnl + fee_paid`) and includes the open tail from the last realized-PnL peak to the end of the backtest.
• Day-duration metric variants were added for high exposure, peak recovery, position held, and position unchanged.
• Suite-mode limit semantics are centralized so `passivbot optimize` and `passivbot tool pareto` resolve omitted `stat=` consistently from `backtest.aggregate`.

📦 Config and Runtime Defaults

• The hardcoded schema defaults and mirrored example config now use a trailing-grid `n_positions = 7` profile.
• Default approved coins, scenarios, optimizer bounds, scoring, and limits were refreshed.
• Shorts are disabled by default through zero short total wallet exposure.
• Example configs now prefer canonical day-duration metrics where appropriate.

📦 Order Behavior

• Inflated grid re-entry behavior was removed from current live, backtest, and runtime paths.
• Grid entries remain normal-or-cropped so effective wallet exposure limits are observed without pulling future size forward.
• Historical inflated order-type ids remain readable for old fills and restart compatibility.

📦 Highlights

• Backtest and live now share a cleaner `pnls_max_lookback_days` contract, including `"all"` support, correct rolling-window behavior, and restored `backtest.visible_metrics` filtering.
• Hyperliquid HIP-3 live support is much more robust: startup and steady-state state discovery are fixed, sizing balance is reconciled against hidden reserve/margin, and dedicated probe tools are now shipped.
• Config and CLI behavior are stricter and clearer: schema-tagged configs, live-owned shared execution settings, inherited live runtime flags exposed on backtest/optimize CLIs, dotted override fixes, and fail-loud validation for invalid `unstuck_ema_dist`.
• Live runtime behavior is more operationally stable: Bybit fill-history storms are reduced, Binance trade-history pagination is fixed, exchange-aware EMA pacing and hourly jitter reduce API bursts, and live runs archive logs by default.
• Container/runtime and tooling support improved substantially: canonical live container contract, env-driven config rendering, better HLCV cache naming, richer Pareto tooling, downloader CLI cleanup, and new Hyperliquid diagnostics tools.

📦 Upgrade Notes

• Reinstall after upgrading:
• `python3 -m pip install -e .`
• or
• `python3 -m pip install -e ".[full]"`
• If the Rust extension looks stale, rebuild it in the active environment:
• `maturin develop --release`
• `config.backtest.market_orders_allowed`, `config.backtest.market_order_near_touch_threshold`, and `config.backtest.pnls_max_lookback_days` are no longer accepted. Set them under `config.live`.
• `live.pnls_max_lookback_days` now uses one shared contract across live, HSL, plotting, and backtests:
• + 7 more

📦 Config, CLI, and Runtime Contracts

• Added formal top-level `config_version` schema tagging starting at `v7.9.0`, with canonical defaults and the mirrored example config carrying the schema version and older configs migrating during load.
• Shared live/backtest execution settings now live only under `config.live`, avoiding silent divergence between runtime and backtest behavior.
• CLI/runtime ownership for inherited `live.*` fields is now explicit, so `backtest` and `optimize` help/projected configs expose the correct inherited runtime flags.
• Dotted CLI overrides now create missing intermediate config sections instead of silently failing when a raw config omits them.
• `live.approved_coins` / `live.ignored_coins` now use a canonical shape with explicit per-side support for `"all"`, and legacy `empty_means_all_approved` inputs migrate with warnings instead of remaining part of canonical config.
• `passivbot live -u/--user` and `-pmld/--pnls-max-lookback-days` are restored as curated default-help shorthands.
• `passivbot optimize --help-all` now exposes fixed per-side bot runtime overrides such as `entry_grid_inflation_enabled` and selected HSL runtime flags without turning them into optimizer dimensions.
• Invalid `unstuck_ema_dist` values that would silently disable unstuck now fail loudly in config loading and in optimize-bounds validation.

🧪 Backtest, Optimize, and Metrics

• Backtests now obey the live-owned `pnls_max_lookback_days` setting correctly, and market-order behavior is treated as a correctness fix instead of preserving legacy bug-compatibility.
• Backtest rolling realized-PnL state now actually expires by time for `pnls_max_lookback_days > 0`, so auto-unstuck and realized-loss gating use the true in-window peak/current pair instead of a stale all-time maximum of the rolling series.
• `backtest.visible_metrics` filtering is restored for standalone terminal output: `null` shows optimize-derived metrics, `[]` shows all, and explicit lists add extras without affecting saved `analysis.json`.
• Added a short-term `entry_grid_inflation_enabled` compatibility flag for inflated grid re-entries, with warnings that cropped-only behavior is the forward path.
• Zero-fill backtests no longer crash during analysis/plotting when balance/equity samples exist but no fills were produced.
• Added trade-level metrics such as `win_rate`, `win_rate_w`, and `trade_loss_{max,mean,median}` plus optimizer-facing ratio metrics like `paper_loss_ratio`, `exposure_ratio`, and weighted variants.
• Liquidation reporting now uses an explicit Rust-provided liquidation flag instead of inferring liquidation from drawdown metrics.
• HLCV dataset caches under `caches/hlcvs_data/` now use descriptive directory names with exchange, coin label/count, effective date range, and cache-hash suffix.
• + 1 more

🐛 Live Runtime Stability and Exchange Fixes

• Fixed `CCXTBot.create_ccxt_sessions()` using generic exchange names instead of futures-specific CCXT ids, which could fetch wrong market sets and cascade-fail updates.
• Fixed Binance trade-history pagination sending future `endTime` and too-tight 7-day windows, eliminating `-4181 "Invalid start time"` issues on sparse symbols.
• Fixed Bybit closed-pnl pagination storms by deriving fill-lookback coverage from durable cache metadata instead of a session-local flag.
• EMA bundle refresh now uses exchange-aware pacing: strict exchanges honor per-symbol delays while zero-delay exchanges keep concurrent refresh behavior.
• Added random hourly `init_markets` jitter so colocated bots do not fire heavy refresh bursts simultaneously.
• `passivbot live` now archives each run to a timestamped log file under `logs/` and keeps `logs/{user}.log` as a stable alias for tooling.

📦 Hyperliquid and HIP-3

• Hyperliquid stock-perp state sync now uses dex-aware discovery for startup and steady-state reconciliation, including unapproved-symbol live state on the same dex.
• Isolated HIP-3 live trading remains explicitly unsupported; unsupported live state now fails loudly instead of running partially.
• Hyperliquid live sizing now reconciles hidden reserve/margin by restoring:
• HIP-3 cross-position `marginUsed`
• Passivbot-managed resting non-reduce-only entry-order reserve
• while still ignoring external/manual orders.
• Added supported Hyperliquid probe tools to the CLI and docs for balance, order-margin, and position-balance diagnostics.

📦 Tools, Container Runtime, and Operational UX

• Added a canonical live-container runtime contract around `Dockerfile_live`, `container/entrypoint.sh`, env-generated `api-keys.json`, env-driven config rendering, and documented Compose/Railway deployment flow.
• Removed the legacy standalone downloader entrypoint and replaced it with the unified `passivbot download ...` path plus a dedicated `src/ohlcv_download.py`.
• Pareto tooling improved further: `passivbot tool pareto` can default to the newest local Pareto output, accept run or `pareto/` dirs, use stored metrics outside the original objective list when direction is known, and now defaults to the `ideal` selector.
• Added and polished Hyperliquid probe tools as first-class supported diagnostics instead of ad hoc investigation scripts.

📦 Short Release Summary

• Passivbot `v7.9.1` is a runtime-correctness and operator-UX release. The main themes are:
• correct rolling PnL lookback semantics in backtests
• stronger live/backtest config ownership and CLI exposure
• better Hyperliquid HIP-3 state and balance handling
• more stable live exchange refresh behavior
• clearer release/runtime tooling around logs, containers, downloads, and diagnostics

📦 Highlights

• Equity hard-stop loss is now a full live + backtest feature with Rust-owned runtime handling, richer metrics, and deterministic fake-live replay coverage.
• Optimization now supports `pymoo` as a first-class backend and uses it by default, with explicit objective goals and live NSGA-II / NSGA-III configuration.
• A new `passivbot tool pareto` explorer makes it easier to inspect and select Pareto candidates from local optimize results.
• Config loading now runs through a canonical staged pipeline with in-code schema defaults, compatibility migrations for older config keys, and a new canonical example config path.
• Passivbot now ships a unified `passivbot` CLI with clearer default help, install-profile separation, stronger environment mismatch detection, and stricter Rust freshness checks.
• A built-in monitor stack is now available: publisher, relay, browser dashboard, TUI, and helper wrappers.

📦 Upgrade Notes

• Reinstall after upgrading.
• `python3 -m pip install -e .`
• or
• `python3 -m pip install -e ".[full]"`
• If Rust looks stale or the wrong binary is being used, run:
• `maturin develop --release`
• `optimize.backend` now defaults to `pymoo`, so optimize users need the full install profile with the new dependency set.
• `configs/template.json` is no longer the canonical starting point. Use `configs/examples/default_trailing_grid_long_npos10.json` or omit the config path to start from schema defaults.
• + 2 more

📦 Equity Hard Stop Loss

• Added the account-level equity hard-stop framework to live and backtest, with Rust-managed drawdown state, tier tracking, RED latching, cooldown restart logic, and no-restart handling.
• Added richer HSL metrics and plots, including per-year trigger and restart counts, time-share metrics, halt and restart metrics, panic-close metrics, and `hard_stop_drawdown.png`.
• Added deterministic fake-live replay tooling and scenarios so RED halt, cooldown behavior, manual intervention, and restart policies can be tested locally.
• Clarified HSL semantics around no-restart threshold clamping, sub-interval EMA fallback, and market panic execution.

📦 Optimization and Pareto Tooling

• Added the `pymoo` backend alongside DEAP and made it the default optimizer backend.
• Activated real runtime support for nested `optimize.pymoo.*` settings, including `algorithm`, shared crossover and mutation settings, and NSGA-III reference-direction configuration.
• Changed optimizer scoring to explicit `{metric, goal}` specs instead of implicit sign-based weights, while keeping legacy string-list configs readable.
• Added auto-sized NSGA-III population defaults when `optimize.population_size` is `null`.
• Added `passivbot tool pareto`, a CLI Pareto explorer for filtering local Pareto candidates with optimizer-style limit expressions and selecting a single candidate using methods such as knee, reference-point, ideal-point, weighted utility, lexicographic, or outranking selection.
• The Pareto explorer can now default to the newest local `optimize_results/.../pareto`, accept either a run directory or a `pareto/` directory, show the retained front's ideal point, and use stored metrics outside the original `optimize.scoring` list when their direction is known.

📦 Configs, CLI, and Install Flow

• Moved canonical defaults to `src/config/schema.py` and made schema defaults the no-config-path behavior for `live`, `backtest`, and `optimize`.
• Added staged config normalization, runtime compilation helpers, migration logging, and backward-compatibility renames for older config keys.
• Replaced the old implicit template workflow with `configs/examples/default_trailing_grid_long_npos10.json`.
• Added the unified `passivbot` CLI, curated default help, `--help-all`, and install-profile guidance for `live`, `full`, and `dev`.
• Restored `passivbot live --user` / `-u` as the curated shorthand for `live.user`, and added a curated shorthand for `live.pnls_max_lookback_days` as `--pnls-max-lookback-days` / `-pmld` in the default live help.
• Added environment mismatch detection so stale shell shims and wrong-entrypoint installs fail loudly or re-exec into the active environment.
• Tightened Rust extension freshness checks to reduce silent stale-binary runs.

📦 Monitoring and Diagnostics

• Added the local monitor publisher for bot snapshots, event streams, and retained fill, price-tick, and candle history.
• Added the read-only monitor relay with websocket streaming and recent-message replay.
• Added the browser dashboard, terminal TUI, `monitor-web`, and `monitor-dev`.
• Added standalone trailing diagnostics tooling for recomputing trailing-entry and trailing-close behavior from saved snapshots or manual input.
• Added repro and sync sidecar tools for investigation and deployment workflows.

🧪 Backtest and Runtime Behavior

• Rust now owns more of the market-vs-limit execution intent, and live plus backtest now consume the same shared logic.
• Market fills in backtests now use taker fees, optional taker-fee override support, and explicit maker/taker liquidity labeling in `fills.csv`.
• Backtest BTC metrics now always use BTC equity instead of mirroring USD analysis when `btc_collateral_cap = 0`.
• ADG terminal smoothing now uses the last up to three daily samples instead of an EMA over the full run.
• Executable min-cost filtering now uses actual rounded executable size rather than raw market metadata, improving forager tradability filtering.
• First-timestamp cache handling for newly listed coins is more robust, avoiding fetches from invalid early dates.

📋 Exchange and Market Coverage

• Hyperliquid HIP-3 handling is more robust across margin-mode detection, state sync, and source-dir resolution for stock-perp backtests.
• Isolated HIP-3 live trading is explicitly blocked for now instead of partially supported.

📦 Discord Forum Announcement Draft

• Passivbot v7.9.0 is out.
• This release covers the full user-facing diff from the current `master` branch to v7.9.0, with major changes across live trading, backtesting, optimization, config handling, CLI UX, and monitoring.
• Important upgrade note first:
• After pulling, reinstall Passivbot in your active environment.
• Live-only: `python3 -m pip install -e .`
• Backtest / optimize / research: `python3 -m pip install -e ".[full]"`
• If Rust looks stale, run `maturin develop --release`
• Main changes in v7.9.0:
• + 15 more

📦 Telegram Announcement Draft

• Passivbot v7.9.0 is out.
• Key changes:
• full live + backtest equity hard-stop framework with richer metrics and fake-live replay tooling
• `pymoo` optimizer backend added and now the default
• new `passivbot tool pareto` explorer for filtering and selecting Pareto candidates
• canonical staged config pipeline with schema defaults and new example config path
• unified `passivbot` CLI with better help and stricter environment / Rust freshness checks
• new monitor stack: publisher, relay, web dashboard, and TUI
• + 10 more

📋 Changed

• Dual balance routing (raw vs hysteresis-snapped) - Live and orchestrator flows now carry both `balance_raw` (raw wallet balance) and `balance` (hysteresis-snapped balance). Sizing/order-shaping paths use snapped balance, while risk/accounting paths use raw balance (including realized-loss gate peak/floor checks, TWEL entry/auto-reduce gating, and auto-unstuck allowance calculations). This applies consistently across live and backtest via Rust orchestrator input.
• WEL denominator behavior split by mode - Live now uses a hard fixed denominator for per-symbol WEL (`total_wallet_exposure_limit / config.bot.{pside}.n_positions`), removing runtime denominator drift from open-position count. Backtests now expose `backtest.dynamic_wel_by_tradability` (default `true`): when enabled, WEL uses tradability-aware denominator growth (`min(n_positions, n_tradable_max)`) based on coins with real candles, and does not shrink after delistings; when disabled, backtests use the same fixed denominator as live.
• Bulk price fetch for Hyperliquid - `calc_ideal_orders` now uses a single `allMids` API call to get prices for all symbols instead of individual `get_current_close` calls per symbol (1 call vs ~70). Falls back to per-symbol fetches for non-Hyperliquid exchanges or on error.
• Sequential margin mode setting for Hyperliquid - Margin mode and leverage API calls are now sequential with a small delay instead of being fired in parallel, reducing API burst on coin changes.

🐛 Fixed

• Bybit fill-event qty inflation on duplicate pages - `BybitFetcher` now deduplicates `fetch_my_trades` rows by exec id before canonicalization/coalescing, preventing duplicate pagination rows from inflating canonical `qty`, `fees`, and close PnL.
• Balance peak drift in wrong direction under hysteresis - Peak reconstruction (`balance + (pnl_cumsum_max - pnl_cumsum_last)`) previously used hysteresis-snapped balance in some paths. Since snapped balance can stay stale while `pnl_cumsum_last` changes fill-by-fill, this made reconstructed peak drift down after profits and up after losses. Peak/PnL-accuracy-sensitive paths now use raw balance (`balance_raw`) consistently.
• Pytest Rust-module bootstrap fallback - Test bootstrap now tries the project venv `passivbot_rust` package before falling back to the lightweight stub when tests are launched outside the venv, reducing false failures from missing/incorrect Rust module resolution.
• `max_ohlcv_fetches_per_minute` ignored when forager slots open - The rate limit config was only applied when all position slots were full. With open slots (the common case), all candidate symbols were fetched without rate limiting, causing 429 errors on Hyperliquid.
• Hyperliquid positions+balance double fetch - `fetch_positions` and `fetch_balance` now share a single API call via a dedup lock instead of making two identical `clearinghouseState` requests per execution cycle.
• Thundering herd on minute boundary - `get_candles` no longer force-refreshes all symbols simultaneously when a new minute boundary crosses. A 1-candle staleness tolerance prevents the TTL override that caused all symbols to fetch at once.
• Candle refresh TTLs aligned to 1-minute finalization - Active candle refresh TTL raised from 10s to 60s and EMA close TTL from 30s to 60s, matching the actual 1-minute candle finalization interval.
• Boot stagger for multi-bot setups - Added `boot_stagger_seconds` config (default 30s for Hyperliquid) to randomize startup delay, preventing simultaneous API bursts when multiple bots share the same IP.
• + 7 more

✨ Added

• Fill events doctor tool - Added `src/tools/fill_events_doctor.py` to audit cached fill events and auto-repair known Bybit duplicate-fill anomalies without requiring exchange API calls. Bybit startup now runs doctor by default (can be disabled with `PASSIVBOT_FILL_EVENTS_DOCTOR=off`).
• ---
• Full changelog: https://github.com/enarjord/passivbot/compare/v7.8.3...v7.8.4

✨ Added

• Added `live.max_realized_loss_pct` (default `0.05`) to block close orders that would realize losses beyond a peak-balance-relative threshold. This applies to normal closes, WEL/TWEL auto-reduce, and unstuck closes; panic closes remain exempt.

🐛 Fixed

• Fixed false-positive stale Rust-extension detection after identical rebuilds.
• Fixed suite base scenario coin fallback to use base approved coins.
• Fixed aggregate-method handling in optimizer scoring and Pareto analysis so configured aggregate modes are respected without double-correcting objectives.

📦 Highlights

• Added configurable backtest candle aggregation via `backtest.candle_interval_minutes` (default `1`).
• Added high-exposure duration metrics for long/short (`high_exposure_hours_{mean,max}_{long,short}`).
• Added `total_wallet_exposure.png` output for backtests.

🐛 Fixed

• Corrected total wallet exposure metrics for short-only configs by using absolute exposure magnitude.
• Fixed timestamp day bucketing in analysis to avoid phantom first-day samples with aggregated intervals.
• Fixed forager `fills_plots` alignment when using candle intervals > 1m by plotting against the effective backtest candle stream.
• Fixed candle-interval test robustness and aggregation alignment behavior across suite/optimizer paths.

📋 Changed

• Updated `configs/template.json` defaults/bounds/scenarios (including `btc_collateral_cap`, `maker_fee_override`, and optimize limits).

📦 Notes

• Candle aggregation improves backtest/optimizer speed but loses intra-interval fill ordering granularity.
• No user migration steps required.
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.8.1...v7.8.2

📦 FillEventsManager Production Transition

• PnL tracking now uses FillEventsManager exclusively - Legacy `update_pnls` path removed
• Fill events include psize/pprice - Each fill annotated with position size and VWAP entry price
• Support for all exchanges: Binance, Bybit, Bitget, GateIO, Hyperliquid, KuCoin, OKX

📦 Comprehensive Logging Improvements (7 rounds of refinement)

• Tag Standardization:
• `[memory]`, `[warmup]`, `[hourly]`, `[fills]`, `[mapping]`, `[candle]`, `[ranking]`, `[mode]`
• Level Adjustments:
• Routine API/cache messages: INFO → DEBUG
• CCXT API payloads: DEBUG → TRACE
• Strict mode gaps: WARNING → DEBUG
• Persistent gaps: WARNING → INFO
• Throttling:
• + 6 more

🐛 Bug Fixes

• Bybit: Fixed missing PnL on some close fills - Pagination bug in `_fetch_positions_history()` caused records to be skipped when >100 existed

📝 Documentation

• New `docs/ai/log_analysis_prompt.md` - comprehensive logging guidelines
• New `docs/ai/exchange_api_quirks.md` - exchange-specific limitations
• New `docs/ai/debugging_case_studies.md` - debugging reference

🗑️ Removed

• `--shadow-mode` CLI flag
• `live.pnls_manager_shadow_mode` config option
• Legacy pnls methods

📦 Migration Notes

• No action required - FillEventsManager automatically fetches and caches fill data
• Old `{user}_pnls.json` cache files can be safely deleted after upgrading
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.6.2...v7.7.0

🐛 🔧 Critical Bug Fixes

• Zero-candle warmup fix: Large warmup spans (>2 days) now properly trigger gap-filling via CCXT even when end_ts touches present. Previously, bots would synthesize thousands of zero-candles for historical gaps instead of fetching real data. First warmup may take longer (one-time backfill), subsequent restarts are fast (~6s vs ~156s).
• One-way mode entry fix: Disabled sides no longer block entries when using one-way mode. The bot now correctly respects side configuration when choosing initial entry direction.
• Bybit leverage errors: "Leverage not modified" (110043) and "margin mode not modified" (110026) errors are now handled gracefully instead of logging full tracebacks.

📦 📊 Logging Improvements

• Structured log prefixes: All log messages now use bracketed category tags for easy filtering:
• `[config]` - Configuration changes
• `[fills]` - Fill events and PnL
• `[candle]` - Candle synthesis/replacement
• `[bybit]`, `[binance]`, etc. - Exchange-specific logs
• Health summary enhancements: Now includes realized PnL sum when fills > 0 (e.g., `fills=3 (pnl=+12.50)`)
• Reduced log spam:
• Zero-candle logs rate-limited to once per minute per symbol
• + 3 more

✨ ⚙️ New Configuration Options

• `live.enable_archive_candle_fetch` (default: false): Opt-in to use exchange archive data for candle fetching. Disabled by default to avoid timeout issues. Backtester always uses archives regardless of this setting.
• `live.warmup_jitter_seconds` (default: 30): Random delay before warmup to prevent API rate limit storms when multiple bots start simultaneously.
• `live.max_concurrent_api_requests` (default: null): Optional global concurrency limit for CCXT API calls.
• `backtest.maker_fee_override` (default: null): Override maker fees in backtest/optimizer (useful for testing different fee tiers).

📦 🖥️ UI Improvements

• TWEL clarification: Startup banner now shows "TWEL" (Total Wallet Exposure Limit) instead of "Exposure" to clarify it's a limit, not current exposure. Long+short mode shows both limits (e.g., `TWEL: L:125% S:85%`).
• Dynamic banner width: Startup banner now calculates width dynamically to prevent misaligned borders.

🐛 🐛 Other Fixes

• Windows cache compatibility: folder names now replace `:` with `_` on Windows or when `WINDOWS_COMPATIBILITY=1` env var is set (#547, thanks @FelixJongleur42)
• Pareto dashboard: Fixed JavaScript callback errors when switching between tabs (#550, thanks @646826)

📦 📦 Dependencies

• Added `openpyxl` to `requirements-live.txt` (required for Bitget archive XLSX parsing)

📦 ⚠️ Upgrade Notes

• 1. First startup may be slower: The zero-candle fix causes a one-time backfill of historical data. Subsequent restarts will be fast.
• 2. Windows users: Existing caches will be orphaned and re-downloaded due to the folder naming change.
• 3. Archive fetching disabled by default: If you relied on archive data for live bots, set `live.enable_archive_candle_fetch: true` in your config.

📦 📈 Statistics

• 66 commits since v7.6.1
• 62 files changed
• +9,196 / -2,529 lines
• Full changelog: https://github.com/enarjord/passivbot/blob/master/changelog.md

📦 Highlights

• CCXT upgrade to 4.5.22 for better exchange compatibility, along with KuCoin hedge-mode enforcement and order payload fixes.
• Canonical suite metrics & Pareto revamp ensuring optimizer/backtester share identical outputs, with centralized Pareto logic.
• Cleaner balance/position orchestration: bots fetch balance and positions concurrently, logging once both are fresh.

📋 Detailed Changes

• Upgraded CCXT to v4.5.22 and added regression tests to detect future CCXT-breaking changes.
• KuCoin bot now enforces hedge mode (when supported), handles exchange refusal gracefully, and sends correct `marginMode`/`positionSide` order params.
• All exchange wrappers now keep `fetch_positions()` and `fetch_balance()` separate; callers can refresh each independently.
• `update_positions_and_balance()` gathers balance/positions concurrently, logs position deltas after both complete, and emits a single balance-change event so equity logging always uses fresh positions/uPnL.
• Added `pareto_core.py` consolidating Pareto dominance/crowding logic with extreme-preserving pruning.
• Suite runner and optimizer now emit the same canonical suite metrics payload (aggregate stats + per-scenario means).
• ParetoStore delegates to the shared helper so dominance/clustering rules are consistent across CLI tools.
• Added forward/backward TP grid overrides (PR #532) plus template/normalize fixes when loading live or suite configs.
• + 5 more

📋 What's Changed

• Bring back forward/backward TP grid logic by @Couillos in https://github.com/enarjord/passivbot/pull/532
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.5.4...v7.5.7

📦 Highlights

• Added collateral-agnostic performance metrics (`adg_pnl`, `mdg_pnl`, PnL-based Sharpe/Sortino, with weighted variants) and documented them.
• Improved coin overrides UX and safety with new docs, tests, and debug logging.
• Refactored position/balance handling to separate updates and optimize Hyperliquid’s combined fetch.
• Robustified data-sync, fill-events cache layout, and Kucoin open-order pagination.
• Expanded optimizer limit handling with a new `limit_utils` module and tighter limit/scoring integration.

📦 Metrics & Analysis

• New metrics: `adg_pnl`, `mdg_pnl` (mean/median daily PnL over end-of-day balance), and PnL-based `sharpe_ratio_pnl`/`sortino_ratio_pnl`, plus weighted versions.
• Added `docs/metrics.md` entries clarifying equity vs PnL Sharpe/Sortino semantics.
• Types/analysis wiring updated across Rust and Python consumers; optimization weights and config canonicalization recognize the new metrics.
• Added optimizer-side limit utilities (`limit_utils`) and integration tests to enforce limit normalization/penalties consistently.

📦 Coin Overrides

• Documented full override behavior (`docs/coin_overrides.md`) with inline/file-based examples, allowed fields, path resolution, and pitfalls.
• Added debug logs when overrides are initialized and when override values are used.
• Expanded tests to cover path resolution, missing override files, and retention through config formatting.

🐛 Exchange & Sync Fixes

• Kucoin: `fetch_open_orders` now paginates (no more 50-order cap).
• Hyperliquid: `fill_events_manager` now supports the exchange; fetch positions/balance combined once per cycle.
• General: position/balance fetching split for all exchanges (dedicated `fetch_balance`/`fetch_positions`), with balance caching to avoid double-hits on combined endpoints.
• `sync_tar.py`: pulling a single file no longer nests an extra directory.
• Optimizer: new limit handling pipeline (`limit_utils`), enhanced scoring/limits parsing, and additional tests for optimizer limits.

📦 Fill Events

• Cache root now `caches/{exchange}/{user}` (was `caches/{exchange}_{user}`).
• Hyperliquid fetcher path registered; CLI no longer rejects hyperliquid users.

♻️ Core Refactors & Safeguards

• `update_positions` and new `update_balance` separated; main loops call both.
• Balance caching reuse and rate-limit/network guards added.
• Tests added for balance split and Hyperliquid combined fetch reuse.
• Added optimizer limit integration tests and helper coverage for limit utilities.

📝 Docs

• Added coin override guide and metrics reference updates.

🧪 QA/Tests

• New tests: override path handling, balance split behavior, Hyperliquid fetch reuse, Hyperliquid fetcher wiring.
• Optimizer limit integration and limit_utils tests added.
• Existing suite continues to pass on targeted runs (spot-checked new tests).
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.5.0...v7.5.4

📋 Changes

• Risk controls & TWEL enforcer
• Added `risk_we_excess_allowance_pct`, `risk_wel_enforcer_threshold`, and companion Rust helpers that auto-reduce positions when wallet exposure drifts beyond the buffer.
• Implemented TWEL (Total Wallet Exposure Limit) gating in Rust with Python wiring, per-position prioritisation, and optimizer bounds/CLI flags.
• Refined unstuck logic so both live bot and backtester use the same Rust helpers; unstuck can now be disabled by setting a negative threshold.
• Optimizer overhaul
• Shared-memory HLCV bundles and per-scenario slicing remove per-worker copies, cutting RAM growth as CPU count increases.
• Streaming evaluator outputs: each candidate logs Pareto updates immediately, duplicate detection got stricter, and starting-config logging/immediate termination handling improved.
• Suite support landed: optimisations can now evaluate multiple scenarios per candidate (with config cleaners, override helpers, and scenario metrics aggregation).
• + 15 more

📋 What's Changed

• Fix res.get() trying to expanding the objectives in case of seen hash by @TayyabTalha in https://github.com/enarjord/passivbot/pull/526
• V7.5.0 twel enforcer by @TayyabTalha in https://github.com/enarjord/passivbot/pull/527
• V7.5.0 twel enforcer by @enarjord in https://github.com/enarjord/passivbot/pull/531
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.4.4...v7.5.0

📋 What's Changed

• add metrics total_wallet_exposure_max, mean, median by @backstab in https://github.com/enarjord/passivbot/pull/528

✨ New Contributors

• @backstab made their first contribution in https://github.com/enarjord/passivbot/pull/528
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.4.1...v7.4.4

📋 Changes

• Introduced configurable REST endpoint overrides via configs/custom_endpoints.json, including CLI/config opt‑in and automatic websocket disabling.
• Added loader intelligence for hostname placeholders and detailed startup logging.
• Documented the setup flow and added pytest coverage to ensure ccxt rewrites and headers behave as intended.

📦 Market Data & Candles

• Introduced an asynchronous CandlestickManager with on-disk caching, metadata, and gap repair.
• Added EMA/log-range series APIs, optional pagination limits, and higher-timeframe fetching (_tf_to_ms, get_latest_ema_*, etc.).
• Align warmup windows between backtester and live; new config keys (inactive_coin_candle_ttl_minutes, entry_grid_spacing_log_*, filter_log_range_ema_span) replace legacy rolling-window settings.
• procedures.get_first_timestamps_unified() now defers Bitget lookups until cheaper exchanges fail and iteratively discovers earliest timestamps.

📦 Risk & Execution Controls

• Expanded noise → log-range filters for coin selection and grid spacing, including hourly EMA modulation.
• Panic-close and auto-unstuck pipelines hardened; existing unstuck safeguards extended with exchange overrides.
• Live bot now prevents disabled sides from refilling approved lists and only logs the disable event once.
• Added hourly memory snapshots to Passivbot telemetry.

🧪 Backtester & Optimizer

• Refined warmup/backfill handling (first_timestamp_ms, 1h alignment, improved gap repair).
• Optimizer gains fine-tune mode (--fine_tune_params), CLI-friendly bounds and limits, and updated docs.
• Various Rust backtest enhancements (analysis.rs, extended metrics, timestamp clamping).

📦 Configs & Examples

• Template and example configs refreshed to use new log-range EMA spans, trailing weights, and log spacing fields.
• Added max_memory_candles_per_symbol defaults and refined per-exchange warmup ratio handling.
• Removed stale all_approved.json example; hyperliquid guide/docs updated accordingly.

📦 Tooling & Requirements

• Rust toolchain pinned to 1.76; requirements.txt now includes maturin/portalocker/portalocker.
• README & docs clarified around installation, Windows venv activation, optimizer fine-tuning, and backtest data (downloader-based archives).

📦 Miscellaneous

• Logging tidy-ups (throttled EMA progress, reduced noisy lines).
• Several .gitignore additions, dependency updates, and formatting runs (black).
• Tests expanded for CandlestickManager and Passivbot’s unstuck flow, ensuring new behaviour is well-covered.

📋 What's Changed

• Merge v7.4.0_noisiness into master: noise-aware filters, smarter panic exits, and Rust backtest upgrades by @enarjord in https://github.com/enarjord/passivbot/pull/523
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.3.21...v7.4.0

📋 Changes

• Introduced an asynchronous CandlestickManager with disk-backed caching, automatic gap repair, and earliest-ts metadata. Requests now clamp to discovered history, and a new pytest suite guards the cache behaviour.
• Reworked symbol screening: filter_markets is now a shared utility, and UTC handling in ts_to_date is consistent across the bot.
• Order execution pipeline gained “panic close” support and refined order-type handling; a follow-up patch fixed an edge case where panic close could fail.
• Rust backtest tooling grew substantially (new analysis module, ADX-driven trailing flip logic, richer metrics, error handling, and max-flip tracking). Supporting dependencies were added in the Rust workspace.
• Documentation refresh: README now covers Windows activation paths and references the Hyperliquid vault; outdated Passivbot imagery was removed.
• Miscellaneous housekeeping: expanded .gitignore, removal of stale notebooks/configs, and other formatting cleanups.

📋 What's Changed

• Update README for Windows users by @ozymotv in https://github.com/enarjord/passivbot/pull/517
• Window crashes fix 7.3.15 by @ozymotv in https://github.com/enarjord/passivbot/pull/516
• Zone recovery improvements by @Belgianwafflecorp in https://github.com/enarjord/passivbot/pull/520
• Fix import for generate mcap list by @TayyabTalha in https://github.com/enarjord/passivbot/pull/521

✨ New Contributors

• @Belgianwafflecorp made their first contribution in https://github.com/enarjord/passivbot/pull/520
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.3.20...v7.3.21

📋 Changes

• officially added Kucoin exchange

📋 Changes

• backtester will now update effective n_positions and wallet exposure limits for each coin dynamically, like live bot does
• increase grid/trailing allocations by 1% to overshoot grid_trailing_ratio_threshold

📋 Changes

• rollback ccxt from v4.4.96 to v4.4.90 due to breaking changes

📋 Changes

• replaces coin flags with coin overrides
• adds coin overrides to backtest
• adds safeguards to prevent bot making duplicate orders
• up ccxt version to 4.4.96
• allow selecting subset of objectives for pareto front when using src/pareto.py
• remove mimic_backtest_1m_delay feature from live bot
• other fixes and improvements

📋 What's Changed

• Master by @msei99 in https://github.com/enarjord/passivbot/pull/510
• New requirements-live.txt and passivbot-live service added to docker-compose.yml by @Couillos in https://github.com/enarjord/passivbot/pull/508
• skipping get_first_timestamp exception by @JohnyX in https://github.com/enarjord/passivbot/pull/511
• Fixes for trailing entries and closes by @TayyabTalha in https://github.com/enarjord/passivbot/pull/512
• V7.3.14 coin overrides by @enarjord in https://github.com/enarjord/passivbot/pull/515

✨ New Contributors

• @Couillos made their first contribution in https://github.com/enarjord/passivbot/pull/508
• @JohnyX made their first contribution in https://github.com/enarjord/passivbot/pull/511
• @TayyabTalha made their first contribution in https://github.com/enarjord/passivbot/pull/512
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.3.13...v7.3.14

📋 Changes

• Replace bot parameters
• close_grid_min_markup and close_grid_markup_range
• with
• close_grid_markup_start and close_grid_markup_end
• see docs/configuration.md for more details
• while logic has changed, backwards compatibility with old param names/values is maintained by automatically adjusting param values
• Fixes for binance fetch pnl
• Add param `config.live.mimic_backtest_1m_delay` which when true adds delays of live order execution of up to 1m for better backtest conformity
• + 1 more

📋 What's Changed

• V7.3.12 bounds by @enarjord in https://github.com/enarjord/passivbot/pull/504
• Windows bugs fix by @ozymotv in https://github.com/enarjord/passivbot/pull/503
• pandas: 1.4.0 -> 1.4.3 workaround for numpy/pdb bug by @eliottness in https://github.com/enarjord/passivbot/pull/506
• V7.3.13 markup by @enarjord in https://github.com/enarjord/passivbot/pull/507

✨ New Contributors

• @ozymotv made their first contribution in https://github.com/enarjord/passivbot/pull/503
• @eliottness made their first contribution in https://github.com/enarjord/passivbot/pull/506
• Full Changelog: https://github.com/enarjord/passivbot/compare/v7.3.10...v7.3.13

📋 Changes

• fixes a bug where older config formats had wrong optimizer bounds

📋 Changes

• pareto front is stored in memory with periodic flushes to disk
• fix for init pnls when account has no fills

📋 Changes

• add time delay of at least 60 seconds between each computing of dist to ideal for pareto front during opt
• new pareto visualizer for 4+ objectives
• live bot: apply min effective cost filtering on non forager configs

📋 Changes

• split parameter filter_rolling_window into
• filter_noisiness_rolling_window
• filter_volume_rolling_window
• rename parameter filter_relative_volume_clip_pct -> filter_volume_drop_pct
• further WIP on exchange Defx

📋 Changes

• separate bot parameter double_down_factor into trailing and grid
• small improvements and fixes

📋 Changes

• attempt multiple perturbation schemes if duplicate is caught
• round opt starting configs to sig digits
• return actual score if duplicate fails to perturb
• use top 15 mcap, not top 20