hasherezade/pe-sieve

BUGFIX + Fixed dumping reflectively loaded PEs (automatically find suitable base) - Issue https://github.com/hasherezade/pe-sieve/issues/134 + Protected against DLL hijacking + Fixed building resources with MinGW + Other small fixes *See also: [HollowsHunter v0.4.1.1](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.4.1.1) with the latest PE-sieve & [MalUnpack v1.0](https://github.com/hasherezade/mal_unpack/releases/tag/1.0) with the latest PE-sieve*

BUGFIX + In Thread Scan (`/threads`): fixed incompatibility with Windows 11 >= 23H2 which caused many threads to be flagged unnecessarily + Overall reduction of false positives during threads scan + Other small fixes FEATURE + Added detailed report about the indications found in each detected thread (complete call-stack with symbols, etc) *See also: [HollowsHunter v0.4.1](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.4.1) with the latest PE-sieve*

BUGFIX + Fixed error in Import Table reconstruction (Issue #131) *See also: [HollowsHunter v0.4.0.2](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.4.0.2) with the latest PE-sieve*

FEATURE + Improved thread scan. Rewritten scanning callstack for anomalies. + New parameter: [`/rebase`](https://github.com/hasherezade/pe-sieve/wiki/4.12.-Rebase-dump-to-the-default-base-(rebase)) - allows to rebase the dumped module to its original base (or to the default one, if the original is not known) + New parameter: `/report` - allowing to define what type of results from the scan should result in filing a report (possibility to obtain a detailed report about all the scans that were performed, not only which of them detected suspicious indicators) + More details about the detected patches: identify breakpoints, padding, etc. + Updated [Python bindings](https://github.com/hasherezade/pe-sieve/tree/master/bindings) REFACT + Dumping optimization: dump the same memory region only once, even if it was detected by multiple scanning filters + Other improvements, and bugfixes *See also: [HollowsHunter v0.4.0](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.4.0) with the latest PE-sieve* 

REFACT + Refactored to use a new pattern matching engine ([SigFinder](https://github.com/hasherezade/sig_finder)) for shellcode detection. Improved performance. FEATURE + Added new parameter [`/pattern <file>`](https://github.com/hasherezade/pe-sieve/wiki/4.11.-Detect-shellcode-by-custom-patterns-(pattern)) allowing to supply custom signatures to be searched in memory. The format is defined by SigFinder and described in the relevant [README](https://github.com/hasherezade/sig_finder/blob/master/README.md). If `pattern` file was defined, a .tag file for the found patterns will be generated, with the extension `.pattern.tag` + New fields in the `scan_report.json`: + Save the PE-sieve version with which the scan was performed (`scanner_version`) + In `workingset_scan` section: added `patterns` section with information about found patterns: + `total_matched` (count of all patterns matched, including the hardcoded ones) + `custom_matched` (count of patterns matched from the set defined by the user in `pattern` file) + New fields in the `dump_report.json`: + If `pattern.tag` file was generated, the name of this file will be added in the `pattern_tags_file` field of the relevant module. *See also: [HollowsHunter v0.3.9](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.9) & [MalUnpack v0.9.9](https://github.com/hasherezade/mal_unpack/releases/tag/0.9.9) with the latest PE-sieve* 

FEATURE + New options for shellcode detection ( [`/shellc`](https://github.com/hasherezade/pe-sieve/wiki/4.1.-Detect-shellcodes-(shellc)#option-shellc) ): + detect by patterns (previous mode): available with `/shellc P` + detect by stats (new mode): available with `/shellc S` + possibility to mix both modes: options `/shellc B` (both) or `/shellc A` (any) + Detection of obfuscated memory regions ( [`/obfusc`](https://github.com/hasherezade/pe-sieve/wiki/4.10.-Detect-obfuscated-areas-(obfusc)) ) + Caching of detected shellcodes (guarantee that the dumped shellcode is the same as detected shellcode - no second read) *See also: [HollowsHunter v0.3.8](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.8.1) & [MalUnpack v0.9.8](https://github.com/hasherezade/mal_unpack/releases/tag/0.9.8) with the latest PE-sieve* 

FEATURE + improved integration with other languages (Python, Golang: https://github.com/hasherezade/pe-sieve/discussions/112), improvements in the API + in ThreadScan: calculate the entropy of the detected area: + decreased number of false positives (filtering by entropy) + added a new section to the ThreadScanReport: `stats` + in dump report: set `"is_shellcode" : 1` only if the code pattern was matched (to distinguish cases when i.e. the shellcode was encrypted and detected by thread scan) BUGFIX + fixed a bug in libPEconv (buffer boundary check: https://github.com/hasherezade/pe-sieve/commit/cabdd4691b37b81889fc29f437e990b8e6cf47a8) + fixed crash if the output filter was set ( https://github.com/hasherezade/pe-sieve/discussions/113 - missing check if the dump report was generated ) + fixed backward compatibility with Windows XP ( #42 ) + decreased number of false positives when searching for PE files in non-executable memory REFACT + faster search for code signatures: skip padding from the scan *See also: [HollowsHunter v0.3.6](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.6) & [MalUnpack v0.9.7](https://github.com/hasherezade/mal_unpack/releases/tag/0.9.7) with the latest PE-sieve* 

📖 [README.md](https://github.com/hasherezade/pe-sieve/blob/master/README.md) FEATURE + Added API function: [`PESieve_scan_ex`](https://github.com/hasherezade/pe-sieve/wiki/5.-API#pesieve_scan_ex) - allowing to retrieve scan and dump JSON reports directly into the supplied memory buffer ( Issue #105 ) + Allow to scan own workingset ( Issue #104 ) + Added one more shellcode pattern ( Issue #108 ) + Added version information to resources BUGFIX + Fixed getting stuck on scanning for PE artifacts (in some rare cases) + Fixed checking mapped modules against the image on disk (fixed issue with the remote module not being copied) + Fixed false positive - MUI files detected as implanted, when using 32bit scanner on 64bit system (FS redirection issue) + Other small fixes *See also: [HollowsHunter v0.3.5](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.5) & [MalUnpack v0.9.6](https://github.com/hasherezade/mal_unpack/releases/tag/0.9.6) with the latest PE-sieve* 

📖 [README.md](https://github.com/hasherezade/pe-sieve/blob/master/README.md) FEATURE + In `/mignore` - removed buffer limit (Details: https://github.com/hasherezade/pe-sieve/pull/99). WARNING: [API change](https://github.com/hasherezade/pe-sieve/blob/master/include/pe_sieve_types.h#L108)) + New param: `/threads`, enabling scan of the threads' callstack . This is another layer of shellcode detection, allowing to capture "sleeping beacons", and others, decrypted just before the execution. (Read more [here](https://github.com/hasherezade/pe-sieve/wiki/4.9.-Scan-threads-callstack-(threads)))  *See also: [HollowsHunter v0.3.4](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.4) with the latest PE-sieve*

📖 [README.md](https://github.com/hasherezade/pe-sieve/blob/master/README.md) BUGFIX + Fully redesigned IAT scan, providing much better precision (Issues: #98, #92, #77) + Fixed processing PEs with relocation table containing empty records (https://github.com/hasherezade/libpeconv/issues/30) + Fixed false positives in some of the code scans FEATURE + Added optional caching (Issue #94) + Improved auto-detection of import reconstruction mode (`/imp A`) : set [`R0`, `R1` modes](https://github.com/hasherezade/pe-sieve/wiki/4.3.-Import-table-reconstruction-(imp)) depending on the sizes of found IATs of particular types + Do not exclude .NET modules from code scan. Improved filtering of the changes typical for .NET. + Changed reporting of IAT hooks to the format consistent with inline hooks reports ( more details [here](https://github.com/hasherezade/pe-sieve/wiki/4.7.-Scan-for-IAT-Hooks-(iat)) ) *See also: [HollowsHunter v0.3.3](https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.3) & [MalUnpack v0.9.1](https://github.com/hasherezade/mal_unpack/releases/tag/0.9.1) with the latest PE-sieve* 

📖 [README.md](https://github.com/hasherezade/pe-sieve/blob/master/README.md) BUGFIX + Fixed memory leak ( Issue #95 ) + Trim invalid sections while reconstructing the payload ( Issue #96 ) + Fixed overeager imports reconstruction ( Issue #97 ) + Improved auto-detection of import reconstruction mode FEATURE + Added new modes of import reconstruction (`/imp`) : `R0`-`R2` : from restrictive to aggressive ( more info [here](https://github.com/hasherezade/pe-sieve/wiki/4.3.-Import-table-reconstruction-(imp)) ) + Report if the process reflection was used in a scan + Automatically turn on `/refl` mode if scan of inaccessible data requested ( `/data 4`, `/data 5`) *See also HollowsHunter: https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.2* 

📖 [README.md](https://github.com/hasherezade/pe-sieve/blob/master/README.md) BUGFIX + Fixed invalid condition check on scanning data (Issue #93) + In imp rec auto mode (`/imp 1`): do not overwrite import table of .NET modules (it was destroying imports) (Issue #89) + Improved detection if a PE is in a virtual or raw mode - fixed an issue in dumping of some PEs + Improvements in code scan (Issue #15) + Improved reporting of unreachable modules *See also HollowsHunter: https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.1.3*

FEATURE + Changes in presenting application parameters. Refactored to use [ParamKit library](https://github.com/hasherezade/paramkit/) + Recognize Virtual Table hooks ( Issue #88 ) BUGFIX + Improve recognizing when to rebuild import table from scratch ( Issue #89 ) + Improve detecting when to realign the payload ( Issue #90 ) + Do not include calls to own exports in the Import Table reconstruction ( Issue #91 ) *See also HollowsHunter: https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.1* 

FEATURE + supported force-read of inaccessible pages (`PAGE_NOACCESS`) when running in [the reflection mode (`/refl`)](https://github.com/hasherezade/pe-sieve/wiki/4.8.-Create-Process-Reflection-(refl)): + automatic if the inaccessible page is within the PE module + on-demand if the inaccessible page is somewhere else in the workingset (depending on the selected `/data` mode) + added more options for scanning non-executable pages [(`/data`)](https://github.com/hasherezade/pe-sieve/wiki/4.4.-Scan-non-executable-memory-(data)) + added one more mode of [IAT hooks scan (`/iat`)](https://github.com/hasherezade/pe-sieve/wiki/4.7.-Scan-for-IAT-Hooks-(iat)), allowing to filter out hooks that lead to any system DLL + in hook resolving function: recognize and parse one more jump type + in shellcode detection: added one more pattern BUGFIX + Fixed error in printing JSON reports of some of the scan types (missing headers) REFACT + refactoring and optimization of the function resolving hooks + removed not needed flags for process reflection creation (optimization) *See also HollowsHunter: https://github.com/hasherezade/hollows_hunter/releases/tag/v0.3.0*

 FEATURE + added a new pattern for detecting 64-bit shellcodes + added [return codes](https://github.com/hasherezade/pe-sieve/blob/master/include/pe_sieve_return_codes.h) informing about the result of the run + removed unused parameter : `/mfilter` + in JSON: added an indicator if the replaced module was linked to the PEB BUGFIX + Fixed error in dumping some of the PEs (issue caused by the invalid ImageSize calculated: Issue #85)

FEATURE + in JSON: report the size of the patch, status, as decimal (rather than hexadecimal) BUGFIX + Fixed crash on processing a malformed export table (Issue #84)

FEATURE + Improved parameters accessibility: grouped into more categories, sorted. + Display hints for misspelled parameters + Added parameter `jlvl` allowing to regulate the level of details included in the JSON report. Allow to list hooks/patches in the `scan_report`. + Improved hook parsing: identify hooks created by replacing the target of existing JMP/CALL BUGFIX + Improved reading remote memory (fixed a bug that was causing PE-sieve to stuck in some cases of reading inaccessible memory) + Do not include initial protection in the check of memory access rights

FEATURE + In DLL: use `__cdecl` calling convention (instead of `__stdcall`) + In case if scanning data is enabled (`/data` parameter) scan for hooks also the sections that are marked as non-executable (if they contain code patterns) + Added a count of sections scanned for patches to the report BUGFIX + Fixed a bug in detecting a section with Entry Point (affecting unpacking of some packers, such as ASPack - Issue #73 ) + Fixed bug in libPEconv: do not treat empty relocation blocks as invalid REFACT + Some internal cleanup and refactoring

BUGFIX: + Fixed error in scanning workingset of some applications (Issue #68)

BUGFIX + Fixed broken detection of ASProtect ( Issue #66 ) + Fixed broken parsing of a hexadecimal PID ( Issue #65 ) + Fixed errors on code scan (caused by invalid relocation table check) + Do not assume that the section 0 is always executable + Fixed bug in scanning 64bit modules by a 32bit scanner FEATURE + Added one more pattern to detect 64 bit code REFACT + Refactored identifying executable sections

BUGFIX + Fixed hanging during the IAT scan of some PEs + Fixed error in converting paths from the `/Device/` format + Fixed not listing results of the mapping scan in the summary FEATURE + added one more .NET policy (in `/dnet` parameter) + In the summary: changed `detached` to `unreachable_file`

FEATURE + Detailed info about a single parameter can be requested by: `/<parameter> ?` + New modes in the `/data` parameter + New parameter `/dnet` allowing to enable treating .NET modules differently than native ones + Report about PE implants and shellcode implants separately + Added information in the report: process bitness, is process managed (vs native) + Minor changes in the API: `PEsieve_version` implemented as a constant + Allow for partial scanning of 64 bit processes by a 32 bit scanner BUGFIX + If the parameter `/refl` chosen, the process reflection should be used for both scan and dump + Fixed switching back to the original console color after printing in color (improved look on Powershell console) + Fixed recognizing if the PE in the memory is in raw or virtual mode (it was giving invalid results for some payloads) + Fixed broken parameter `/mfilter` REFACT + Refactored parsing of the parameters + Internal refactoring and cleanup of the scanner

FEATURE + Scan virtual caves BUGFIX + Fixed `/mginore` option (filtering out selected modules from the scan) + Fixed wrong calculation of a patch size

FEATURE + New parameter: `/refl` allowing to make a process reflection before scanning + Support scanning PEs with no sections (i.e. packed with Crinkler) (Issue #46) + Allow to build as static library + Dump report about hooks even if dumping the modules failed + Show the time of scanning workingset BUGFIX + Fixed JSON in code scan report (in case of reporting errors) + Fixed problems with dumping PEs with bogus Image Size (prevent crashes, try to autodetect the correct size) REFACT + Internal refactoring (i.e. in scanning workingset) + Set debug privilege unconditionally

BUGFIX + Fixed crashes on printing json report (when run with `/json` parameter ) + Fixed bug in coping input arguments

 FEATURE + New parameter: `/iat` allowing to scan for IAT Hooking (Issue #57) + Report about functions that could not be recovered (in case of import fixing) BUGFIX + Silence out (more) logs in the `quiet` mode + Updated [libPeConv](https://github.com/hasherezade/libpeconv) with bugfixes + Do not start scanning if info requested (params: `/version` or `/help`) + Fixed JSON format in `dump_report.json` (removed redundant comma)

 FEATURE + Added dump reports (`dump_report.json`) + Renamed scan report (from `report.json` to `scan_report.json`) + Added parameter: `/mignore <modules>` - to exclude defined list of modules form the scan BUGFIX + Fixed bug in libPeConv causing incomplete import recovery + Added more patterns to recognize shellcodes + Fixed false positive in path comparison (expand relative paths before comparing) + Silence out logs in the `quiet` mode Internal refactoring.

 FEATURE + Detect Module Overloading (Issue #47 ) + Allow for supplying PID in a hexadecimal form (Issue #49) + In a report: present the allocation type in form of a string (i.e. "MEM_IMAGE") instead of number BUGFIX + Added fixing Entry Points of .NET modules (Issue #48 ) + Fixed a bug causing false positives during patches detection (invalid identification of non-executable sections as executable) + Fixed a bug causing not dumping of some of the detected modules (invalid offset calculation during dump: Issue #45) + Improved detection of PEs embedded in a shellcode (Issue #44 ) + More precise validation of found PE artefacts

 FEATURE + Create a MiniDump for a process detected as suspicious (option `/minidmp`) (Issue #43) + Support Linux-style parameter switch ( i.e. `-shellc` as an equivalent of `/shellc`) (Issue #40) BUGFIX + Restored broken backward compatibility with Windows XP (Issue #42)

 FEATURE + Report about PEs with modified headers separately (do not treat them as replaced). Show details about what part of the PE header was modified. BUGFIX + Fixed: imports for remapped modules were not rebuilded. + Fixed: imports for 64bit shellcodes were not recognized. (The shellcode bitness should be recognized before searching its imports.) + Improved accuracy of searching beginning of the implanted module + Fixed: invalid limits for workingset scan (causing the highest pages remaining unscanned) + Fixed: unneccessery changes in the alignments of the implanted PE (Issue #39)