epi052/feroxbuster

📋 What's Changed

• fixed multiple rate limiting bugs by @epi052 in https://github.com/epi052/feroxbuster/pull/1277
• added new heuristics for identifying directory listing by @epi052 in https://github.com/epi052/feroxbuster/pull/1277
• Added web fonts to ignored extensions list by @auk0x01 in https://github.com/epi052/feroxbuster/pull/1274
• Added android/termux install instructions to docs by @pg9051

✨ New Contributors

• @auk0x01 made their first contribution in https://github.com/epi052/feroxbuster/pull/1274
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.13.0...v2.13.1

📋 What's Changed

• add --scope option by @epi052 in https://github.com/epi052/feroxbuster/pull/1271
• add STATE_FILENAME environment variable to control feroxbuster state file name/location by @epi052 in https://github.com/epi052/feroxbuster/pull/1271
• Special thanks to @lidorelias3 and @0x7274 for their ideas, bug reporting, and documentation contributions :partying_face:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.12.0...v2.13.0

📦 **Response Size Limiting** (`--response-size-limit`)

• Issue #1260: Added new `--response-size-limit` flag to limit the size of response bodies read during scanning
• Helps prevent memory exhaustion when scanning applications with very large responses
• Default limit set to 4MB, configurable via command line and config file
• Responses that exceed the limit are marked as truncated but still processed for status codes and headers

📦 **Unique Response Filtering** (`--unique`)

• Issue #1240/635: Added new `--unique` flag to filter out duplicate responses using SimHash with Hamming distance analysis
• Helps reduce noise by showing only unique content, especially useful when scanning large applications with similar pages
• Uses advanced similarity detection to identify duplicate responses even when they're not identical

📦 **Auto Content-Type Headers** (`--data-json` and `--data-urlencoded`)

• PR #1234: Added new convenience flags for automatically setting Content-Type headers and POST method by @zar3bski
• `--data-json`: Automatically sets `Content-Type: application/json`, configures data payload, and sets method to POST
• `--data-urlencoded`: Automatically sets `Content-Type: application/x-www-form-urlencoded`, URL-encodes the payload, and sets method to POST
• Both flags support reading data from files using `@filename` syntax (e.g., `--data-json @payload.json`)
• Simplifies common POST request scenarios by eliminating need to manually set headers and methods

📦 **Dynamic Scan Limit Management**

• Issue #817: Enhanced scan management menu with ability to view and modify scan limits in real-time
• Issue #1254: Added capability to increase scan limits through the interactive scan management menu
• Added "Waiting" status visibility for scans that are queued due to limits

📦 **Shell Completion Updates**

• PR #1229: Fixed Fish shell completion generation in build script by @zer0x64
• Corrected duplicate Zsh completion generation that was preventing Fish completions from being properly generated

📦 **Enhanced SSL Error Messaging**

• Issue #1258: Improved SSL error messages to provide more helpful debugging information
• Better error context and clearer explanations when SSL/TLS issues occur

📦 **Updated Link Discovery**

• Issue #1077: Fixed bug in link extractor functionality related to force recursion handling
• Updated LinkFinder regex patterns to latest version from upstream project

📦 **Dependencies**

• Updated various dependencies to their latest versions for security and performance improvements

📦 **Code Quality**

• PR #1247: Fixed clippy linting warnings to enable compilation with `--deny warnings` by @karanabe

📦 **Configuration**

• Added `unique` option to configuration file example
• Added `response_size_limit` option to configuration file example
• Enhanced banner display to show unique filtering and response size limit status when enabled

✨ New Contributors

• @zer0x64 made their first contribution in https://github.com/epi052/feroxbuster/pull/1229
• @zar3bski made their first contribution in https://github.com/epi052/feroxbuster/pull/1234
• @karanabe made their first contribution in https://github.com/epi052/feroxbuster/pull/1245
• ---
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.11.0...v2.12.0

📋 What's Changed

• add `--scan-dir-listings` by @epi052 in https://github.com/epi052/feroxbuster/pull/1192
• add `--request-file` by @epi052 in https://github.com/epi052/feroxbuster/pull/1192
• add `--protocol` by @epi052 in https://github.com/epi052/feroxbuster/pull/1192
• add `--limit-bars` by @epi052 in https://github.com/epi052/feroxbuster/pull/1192

📦 Brief descriptions

• `--request-file` and `--protocol` - read in a raw http request file, from burp or similar
• `--scan-dir-listings` - force recursion into folders where directory listing is enabled
• `--limit-bars` - cap the number of progress bars displayed

📝 Documentation

• [`--request-file` and `--protocol`](https://epi052.github.io/feroxbuster-docs/docs/examples/raw-request/)
• [`--scan-dir-listings`](https://epi052.github.io/feroxbuster-docs/docs/examples/scan-dir-listings/)
• [`--limit-bars`](https://epi052.github.io/feroxbuster-docs/docs/examples/limit-bars/)
• Special thanks to @Raymond-JV, @Tib3rius, @libklein, and @L1-0 for their suggestions and support :tada:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.10.4...v2.11.0

📋 What's Changed

• `--filter-regex` now looks at headers in addition to response body by @epi052 in https://github.com/epi052/feroxbuster/pull/1142
• clarified wording for headers in `ferox-config.toml` by @JulianGR in https://github.com/epi052/feroxbuster/pull/1152
• added winget releaser workflow by @sitiom in https://github.com/epi052/feroxbuster/pull/1155
• scan management menu now shows the estimated time left to scan by @epi052 in https://github.com/epi052/feroxbuster/pull/1142
• made `--cookies` parsing more robust by @epi052 in https://github.com/epi052/feroxbuster/pull/1142
• added ARM build for mac (`aarch64-macos`) to releases by @epi052 in https://github.com/epi052/feroxbuster/pull/1142
• fixed an issue where estimated time to complete would show `0s` before the scan was finished by @epi052 in https://github.com/epi052/feroxbuster/pull/1142

✨ New Contributors

• @JulianGR made their first contribution in https://github.com/epi052/feroxbuster/pull/1152
• @sitiom made their first contribution in https://github.com/epi052/feroxbuster/pull/1155

📦 Special thanks :pray:

• The following folks submitted bugs, PRs, and feature requests (in no particular order). They're the real MVPs.
• @JulianGR
• @L1-0
• @sitiom
• @wikamp-collaborator
• @sa7mon
• @swordfish0x0
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.10.3...v2.10.4

📋 What's Changed

• 1105 - improve json logs for post processing by @epi052 in https://github.com/epi052/feroxbuster/pull/1114
• 1097 - included configuration field in state file by @epi052 in https://github.com/epi052/feroxbuster/pull/1114
• 1118 - using --data implies POST HTTP verb by @epi052 in https://github.com/epi052/feroxbuster/pull/1114
• 1122 - fixed mishandling of whitespace for supplied request headers by @epi052 in https://github.com/epi052/feroxbuster/pull/1114
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.10.2...v2.10.3

📋 What's Changed

• removed scan target headers from feroxbuster-update check by @epi052 in https://github.com/epi052/feroxbuster/pull/1033
• `--collect-backups` accepts custom backup extension list by @epi052 in https://github.com/epi052/feroxbuster/pull/1035
• fixed issue where `--silent` included too much info on found dir by @epi052 in https://github.com/epi052/feroxbuster/pull/1067
• `--parallel` time limit enforced on individual directories instead of main thread by @epi052 in https://github.com/epi052/feroxbuster/pull/1072
• query fontconfig to determine if Noto Color Emoji is installed by @tritoke in https://github.com/epi052/feroxbuster/pull/1083
• updated upstream library that was causing `tokio-runtime-worker` panics; they're correctly reported as Err from the library now
• re-added `.deb` install method to releases
• fixed issue where early redirect loop would cause ferox to hang indefinitely

✨ New Contributors

• @tritoke made their first contribution in https://github.com/epi052/feroxbuster/pull/1083
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.10.1...v2.10.2

📋 What's Changed

• fixed scan menu range issue by @epi052 in https://github.com/epi052/feroxbuster/pull/936
• enable reading extensions from file by @andreademurtas in https://github.com/epi052/feroxbuster/pull/976
• fixed collect backups filtering by @epi052 in https://github.com/epi052/feroxbuster/pull/1016
• added http/2 support by @epi052 in https://github.com/epi052/feroxbuster/pull/1020
• allowed --json in conjunction with --silent by @epi052 in https://github.com/epi052/feroxbuster/pull/1022

✨ New Contributors

• @andreademurtas made their first contribution in https://github.com/epi052/feroxbuster/pull/976
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.10.0...v2.10.1

📋 What's Changed

• Adds server and client certificate management; enables mTLS by @lavafroth in https://github.com/epi052/feroxbuster/pull/892
• `--server-certs`
• `--client-cert`
• `--client-key`

✨ New Contributors

• @lavafroth made their first contribution in https://github.com/epi052/feroxbuster/pull/892 :tada:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.9.5...v2.10.0

📋 What's Changed

• 878 support raw urls by @epi052 in https://github.com/epi052/feroxbuster/pull/884
• special thanks to @aroly for reporting the issue, @lavafroth for the workaround, and @aancw for furthering the discussion! :tada:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.9.4...v2.9.5

📋 Changes

• reduced memory usage overall
• improved speed during link extraction
• updated dependencies. most significantly, indicatif, which hasn't been updated for well over a year (purposely postponed)
• added optional install path to `install-nix.sh`

📋 What's Changed

• random improvements by @epi052 in https://github.com/epi052/feroxbuster/pull/877
• docs: add DrorDvash as a contributor for bug by @allcontributors in https://github.com/epi052/feroxbuster/pull/879
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.9.3...v2.9.4

📋 What's Changed

• added extensions and status codes into auto filtering decision calculus by @epi052 in https://github.com/epi052/feroxbuster/pull/869
• Special thanks to @0xdf223 for letting me know about the shortcoming :partying_face:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.9.2...v2.9.3

📋 What's Changed

• changed default value for `--extract-links` to `true` => added `--dont-extract-links` to turn off the new default behavior by @epi052 in https://github.com/epi052/feroxbuster/pull/834
• can load a wordlist from its url over http/https by @epi052 in https://github.com/epi052/feroxbuster/pull/834
• updated README with alternative installation methods for brew and chocolatey by @aancw in https://github.com/epi052/feroxbuster/pull/824
• fixed divide by zero error by @epi052 in https://github.com/epi052/feroxbuster/pull/834
• added check for forced recursion when directory listing detected by @epi052 in https://github.com/epi052/feroxbuster/pull/834
• Special thanks to @aancw for the code, @acut3 for the bug report, and @xaeroborg for the nice quality of life idea :tada:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.9.1...v2.9.2

📋 What's Changed

• Implement auto update feature by @aancw in https://github.com/epi052/feroxbuster/pull/813
• scan management can now canx scans started with `-u` | `--stdin` | the menu itself by @epi052 in https://github.com/epi052/feroxbuster/pull/821
• feroxbuster can be installed via chocolatey by @aancw in https://github.com/epi052/feroxbuster/pull/807
• fix resume with offset when `--methods` | `--extensions` are used by @epi052 in https://github.com/epi052/feroxbuster/pull/823
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.9.0...v2.9.1

📋 What's Changed

• banner is shown again after exiting scan management menu by @aancw in https://github.com/epi052/feroxbuster/pull/804
• improved auto-filtering accuracy
• Fixed issue where a wildcard redirect caused every request to recurse into that directory by @epi052 in https://github.com/epi052/feroxbuster/pull/808; id'd by @0xdf223
• fixed bug where `--auto-tune` and `--rate-limit` could be set in the same scan via `--smart`/`--thorough` composite settings; id'd by @GenericUser123

✨ New Contributors

• @aancw made their first contribution in https://github.com/epi052/feroxbuster/pull/804 :partying_face:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.8.0...v2.9.0

📋 What's Changed

• Fixes #761 | Updated Dockerfile and CONTRIBUTING docs by @aidanhall34 in https://github.com/epi052/feroxbuster/pull/762
• fixed bug in extractor that wasn't correctly comparing extracted domains
• fixed bug in Makefile
• fixed auto-bail icon; wasn't displaying properly on some terminals
• added visual cues for auto-tune's rate adjustments
• added visual cue when auto-bail is triggered
• when Content-Length header is missing or 0, now check the body length as well in case that differs
• fixed issue where auto-tune wasn't adjusting upward as often as it should have been
• + 12 more

✨ New Contributors

• @aidanhall34 made their first contribution in https://github.com/epi052/feroxbuster/pull/762
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.7.3...v2.8.0

📋 What's Changed

• FIX 732 ensure --no-state is respected even through --time-limit by @kmanc in https://github.com/epi052/feroxbuster/pull/733
• Fix incorrect username in Contributors by @n0kovo in https://github.com/epi052/feroxbuster/pull/749
• fixed #716; wordlist entries with leading slash are trimmed by @epi052 in https://github.com/epi052/feroxbuster/pull/750
• fixed #743; redirects always show full url as Location by @epi052 in https://github.com/epi052/feroxbuster/pull/750
• fixed #748; cancelled scans persist across ctrl+c by @epi052 in https://github.com/epi052/feroxbuster/pull/750

✨ New Contributors

• @kmanc made their first contribution in https://github.com/epi052/feroxbuster/pull/733
• @n0kovo made their first contribution in https://github.com/epi052/feroxbuster/pull/749
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.7.2...v2.7.3

📋 What's Changed

• removed superfluous if statement by @herrcykel in https://github.com/epi052/feroxbuster/pull/580
• upgraded leaky-bucket to 0.12.1 by @udoprog in https://github.com/epi052/feroxbuster/pull/604
• updated dependencies by @epi052 in https://github.com/epi052/feroxbuster/pull/670
• upgraded clap from 3.x to 4.x by @epi052 in https://github.com/epi052/feroxbuster/pull/671
• 661 fix double dir scan by @epi052 in https://github.com/epi052/feroxbuster/pull/672
• fixed invalid uri exception during extraction by @epi052 in https://github.com/epi052/feroxbuster/pull/706

✨ New Contributors

• @herrcykel made their first contribution in https://github.com/epi052/feroxbuster/pull/580
• @udoprog made their first contribution in https://github.com/epi052/feroxbuster/pull/604
• Full Changelog: https://github.com/epi052/feroxbuster/compare/2.7.1...v2.7.2

📋 What's Changed

• fixed bug in auto-tune
• extensions now accept values with leading period, i.e. `-x .php` and `-x php` behave identically (leading period gets stripped)
• if no url scheme is given, `https` is prepended to the target (`-u hackerone.com` becomes `https://hackerone.com`)
• support for secondary default wordlist location added (`/usr/local/share/seclists...`)
• :tada: Special thanks to @jhaddix, @ippsec, @postmodern, and @DonatoReis for their reports / ideas :tada:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/2.7.0...2.7.1

📋 What's Changed

• 535 status code filter overhaul by @epi052 in https://github.com/epi052/feroxbuster/pull/536
• For a more in-depth explanation of how status code filtering has changed, please see [the docs](https://epi052.github.io/feroxbuster-docs/docs/examples/filter-status/). Here are the cliff notes:
• `--filter-status` and `--status-codes` are now mutually exclusive options
• `--status-codes` works the same way it always has: by providing an allow-list for status codes. Any status code not included in `--status-codes` will be filtered out
• If a value is given to `--filter-status`, that status code will be filtered out, while all other status codes are allowed to proceed
• Finally, the default path to the wordlist on windows has been updated to look in the current directory: `.\SecLists\Discovery\Web-Content\raft-medium-directories.txt`
• :tada: Special thanks to @0xdf223 and @ThisLimn0 :tada:
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.6.4...2.7.0

📋 What's Changed

• bugfix where if multiple http methods were used, and returned responses that should have been shown to the user, only the first method was shown. This was an error in how responses were identified, but is now fixed.
• Thanks to @godylockz for spotting another issue!
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.6.3...v2.6.4

📋 What's Changed

• bugfix related to #501, `--proxy` was handled fine, but logic for `--replay-proxy` was flawed. Both exhibit the same behavior now.
• Thanks to @godylockz for spotting the issue!
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.6.2...v2.6.3

📋 Changes

• The Scan Management Menu now has the ability to manage filters as well! More details and demonstrations are available in the [docs](https://epi052.github.io/feroxbuster-docs/docs/examples/cancel-scan/#cancel-a-scan).

📋 What's Changed

• add and remove filters via scan management menu by @epi052 in https://github.com/epi052/feroxbuster/pull/533 & https://github.com/epi052/feroxbuster/pull/528
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.6.1...v2.6.2

📋 Changes

• fixed a bug where `--collect-backups` wasn't requesting backups from the same directory where the original was found

📋 What's Changed

• Add `--no-state` option, filter queries from links, title-case headers by @godylockz in https://github.com/epi052/feroxbuster/pull/474
• added `--collect-extensions` and `--dont-collect`
• added `--collect-words`
• added `--collect-backups`
• added `--burp`
• added `--burp-replay`
• added `--smart`
• added `--thorough`
• + 12 more

📋 What's Changed

• docs: add narkopolo as a contributor for ideas by @allcontributors in https://github.com/epi052/feroxbuster/pull/437
• docs: add justinsteven as a contributor for ideas by @allcontributors in https://github.com/epi052/feroxbuster/pull/436
• Added support for specifying cookies with the `-b` flag by @7047payloads in https://github.com/epi052/feroxbuster/pull/444
• docs: add 7047payloads as a contributor for code by @allcontributors in https://github.com/epi052/feroxbuster/pull/455
• docs: add unkn0wnsyst3m as a contributor for ideas by @allcontributors in https://github.com/epi052/feroxbuster/pull/456
• Add support of multiple methods during scan #440 by @MD-Levitan in https://github.com/epi052/feroxbuster/pull/441
• added option groups to help output and updated dependencies by @epi052 in https://github.com/epi052/feroxbuster/pull/463
• Directory Listing & Web Scraping Links by @godylockz in https://github.com/epi052/feroxbuster/pull/464
• + 4 more

✨ New Contributors

• @7047payloads made their first contribution in https://github.com/epi052/feroxbuster/pull/444
• @MD-Levitan made their first contribution in https://github.com/epi052/feroxbuster/pull/441
• @godylockz made their first contribution in https://github.com/epi052/feroxbuster/pull/464
• Full Changelog: https://github.com/epi052/feroxbuster/compare/v2.4.1...v2.5.0

📋 Changes

• Revised Scan ~~cancel~~ Management Menu to allow for adding a new scan ([docs](https://epi052.github.io/feroxbuster-docs/docs/examples/cancel-scan/)); thanks to [@narkopolo](https://github.com/narkopolo) for the suggestion! :partying_face:
• Added `original_url` entry to json output, suggestion courtesy of [@justinsteven](https://github.com/justinsteven) :tada:
• updated dependencies to current versions

📋 Changes

• integrated bugfix; submitted by @cortantief
• added `--random-agent` feature; submitted by @dsaxton
• added regex support for `--dont-scan`; idea form @mzpqnxow, implemented by me :upside_down_face:

📋 Changes

• fixed wildcard filtering when wildcard response is 0
• fixed total # of urls reported to be scanned in the 'overall' progress bar (was increasing out of sync with reality)

📋 Changes

• Default status codes now include `500`.
• logging while using `--parallel` now uses the value of `-o|--output` as a seed to create a directory named `OUTPUT_VALUE-TIMESTAMP.logs/`. Within the directory, an individual log file is created for each target passed over stdin.