unexpectedpanda/retool

📋 Changes

• _Feature_: You can now exclude titles with the following classification tags by excluding add-ons:
• `(Avatar)`
• `(Theme)`
• _Feature_: Titles with `(Soundtrack)` in their names now get assigned to the `Audio` category.
• _Change_: During the cross-region comparison phase, titles from the _World_ region are now treated as equivalent to the _USA_, _Europe_, and _Japan_ regions when checking for modern editions, promote editions, and demote editions as defined in `internal-config.ini`.
• _QuackShot Starring Donald Duck (USA) (En,Ja) (Sega Channel)_
• _QuackShot Starring Donald Duck (World) (En,Ja) (Rev A)_
• _Fix_: Modified the PlayStation Vita ID regex.
• + 1 more

📋 Changes

• _Fix_: Added yet another No-Intro date format: `(MM-DD)`. This should help auto-detect versions in the Satellaview DAT file.
• _Fix_: If you prefer RetroAchievements titles, and a compilation is the only variant of a RetroAchievements title, then that compilation is now selected in all scenarios.
• _Change_: Retool used to append `(Demo)` to the short names of demos it found that were missing a demo tag -- it did this to prevent demos from getting confused with full versions of titles during title comparison. Unfortunately, it also made adding these titles to clone lists confusing for contributors, as you had to know what Retool was doing behind the scenes, and when to add the `(Demo)` string to a title in the clone list to match that behavior.

📋 Changes

• _Feature_: Retool now reports the uncompressed space you've saved after filtering (issue #361). Thanks for getting things started on this one, [steven-sheehy](https://github.com/steven-sheehy)!
• _Fix_: Retool's icon is now shown on the macOS dock instead of the Python Launcher icon when running Retool GUI. Unfortunately it doesn't seem possible to change the name of the app in the dock or menu without providing an app bundle.

📋 Changes

• _Change_: Excluding Add-ons now also excludes titles with `(DLC)`, `(Addon)`, and `(Addon for XBLA)` tags (issue #359). Thanks [steven-sheehy](https://github.com/steven-sheehy)!
• _Fix_: Running tests through Hatch doesn't rely on there being a `.dev` file in Retool's root folder any more.
• _Fix_: Fixed missing grip handle graphic between the DAT files list and the rest of the app.
• _Fix_: If a user cancels out of adding DAT files, and there's no DAT files in the list, Retool now returns the placeholder text to "No DAT files added yet".
• _Fix_: If you select Override global settings in the system Options tab, but nothing else in the options tab, Retool no longer outputs an empty options string (`(-)`) in the filename.
• _Fix_: I bought myself a Mac, and so the following things have been fixed in the GUI on macOS:
• The Settings and About options are now in the proper menus, and not moved by macOS heuristics into different places.
• The delete key now works for removing items from list boxes.
• + 7 more

📋 Changes

• _Feature_: Added some Nintendo mastering codes.
• _Change_: Category matching is no longer case-sensitive (issue #357).

📋 Changes

• _Fix_: Prevents stacking of RetroAchievements tags if a DAT file is processed more than once.
• _Fix_: Fixed a type warning when dragging and dropping files using Retool with PySide6 6.10.0.0.

📋 Changes

• _Change_: Titles with the `(Video)` tag are now categorized as Videos.
• _Fix_: Updated QT version bundled with the compiled Windows version to correct issue #354.

📋 Changes

• _Change_: The `<rom>` elements in output DAT files are now listed in alphabetical order based on file name. This makes it easier to find problems when analyzing DAT files.
• _Fix_: Removed video regular expressions from the tags Retool ignores when grouping titles together. These were added accidentally in 2.4.0.
• _Fix_: Original header output now works again for custom No-Intro DAT files (for example, RVZ and WUX versions of Redump DAT files). Turns out reading the file in as bytes required extra work to deal with LF line endings.
• _Fix_: The data source that provided RetroAchievements hashes has switched from providing ISO, BIN, and CHD hashes to CHD and RVZ hashes. As such, Retool can now add RetroAchievements labels to [MAME Redump DAT files](https://github.com/MetalSlug/MAMERedump).

📋 Changes

• Fix: Turns out pathing in QT is case sensitive, and a path rename broke wherever there were arrow icons on buttons. This has now been fixed.
• Fix: The `<retool>` element is now added to output DAT file headers again in all circumstances. This enables the skipping of files that have already been processed.
• Fix: Added a Benesse ID regex to version detection.

📋 Changes

• CPU: Intel Core i7 14700K (28 threads across 8 P-Cores and 12 E-cores)
• RAM: 64GB DDR DD5-6400
• Disk: Samsung SSD 990 Pro 2TB
• _Feature_: Retool can now assign titles as RetroAchievements compatible by adding a `retroachievements="yes"` attribute on `game` or `machine` tags. You can also set your 1G1R title selection to prefer RetroAchievement titles. RetroAchievements data is retrieved from an external source. If the source stops updating or becomes unavailable, using RetroAchievements features won't be effective.
• _Feature_: Unrecognized attributes in `game` and `machine` elements are now passed through to the output DAT file.
• _Feature_: Unrecognized child elements in the `game` and `machine` elements are now passed through to the output DAT file.
• _Change_: There's now a `versionIgnore` array in `internal-config.json`, which details the titles that shouldn't be picked up by automatic version detection. Retool's version detection originally caused confusion in creating clone lists, where you'd have to get tricky with workarounds for titles like _Pokemon - Black Version 2_, as Retool would see it as version 2.0 of _Pokemon - Black_ &mdash; not its own game.
• _Change_: Windows no longer uses the maximum amount of CPU cores available to it in all scenarios. The cost of spinning up a process in Python under Windows is very high, meaning that using more cores can mean less performance than fewer cores in many cases. Instead, Retool makes a ballpark guess at the best number of processes to use for best performance. While there is a penalty for adding processes in Linux as well, the total processing time is still so small you may as well just use all cores anyway. MacOS likely suffers the same fate as Windows as it also uses the `spawn` method instead of `fork` to create a process, but I don't have the hardware to test, so no changes have been made there.
• + 42 more

📋 Changes

• _Fix_: Retool now handles empty description fields.

📋 Changes

• _Fix_ (#338): Fixed post filters not working.

📋 Changes

• _Fix_: Fixed clone list and `internal-config.json` minimum version detection.

📋 Changes

• _Feature_ (#278): You can now choose to prefer the oldest production version of a title instead of the newest.
• _Fix_ (#308): Budget rereleases are now promoted above original titles with high revisions. For example, `Example Title (USA) (PlayStation the Best)` is now chosen over `Example Title (USA) (Rev 3)`. This is on the assumption that budget rereleases most likely contain the latest revisions. This was also necessary to implement the oldest version feature properly.
• _Fix_: Fixed the flags in the output DAT file filename to reflect system settings when system settings are in use, instead of reflecting global settings.

📋 Changes

• _Fix_ (#329): Fixed Retool crashing when Prefer titles ripped from modern rereleases was enabled.
• _Change_: Because case can change frequently between DAT file revisions, overrides and post filters are no longer case sensitive. This increases convenience, but also means you're more likely to shoot your own foot off with partial and regex matches, so caveat utilitor.
• _Change_: The way the version is reported has changed to accommodate dynamic versioning in Hatch. Instead of 2.03.5, Retool now shows 2.3.5.
• _Chore_: Lately Windows Defender has been finding false positives on the compiled Windows version of Retool. While there's not much that can be done about this other than marking an exception in Defender on your machine, Python, PyInstaller, and UPX have all been updated in the hope that enough changes have occured to prevent the incorrect alert from happening.
• _Chore_: Cleaned up how Retool outputs to screen. Made more use of code page 437 characters to freshen up the look.
• _Chore_: Started reorganizing where functions live for future clean up and optimizations. While I don't really expect anyone will go code diving, those who do will find things are stored in an inconsistent fashion until this task is complete.

📋 Changes

• _Change_ (#324): When you specify `--output` in a non-interactive terminal, the output is now written to a file, not `STDOUT`.
• _Fix_ ([#321](https://github.com/unexpectedpanda/retool/issues/321#issuecomment-2041463439)): If you didn't populate your region priority list with all the regions in a clone list filter, the condition would fail to `true`, causing odd title selection. This has now been fixed.

📋 Changes

• _Feature_: Clone list and metadata updates now use threaded downloads to reduce wait times.

📋 Changes

• _Fix_: Fixed Retool falsely thinking CLRMAMEPro DAT files have already been processed.
• _Chore_: Updated dependencies. Locked down PySide6 version due to a [bug](https://bugreports.qt.io/projects/PYSIDE/issues/PYSIDE-2665?filter=allopenissues) that interferes with testing.

📋 Changes

• _Change_: The Prefer regions over languages setting now also overrides superset selection.
• _Fix_: DTD file detection wasn't updated to take into account the new entry point paths. This didn't matter on Windows, but did affect Linux and MacOS.

📋 Changes

• _Feature_: Some changes aimed at ROMVault and DATVault users:
• You can now choose not to add MIA attributes to titles and ROMs from clone lists. This is mainly useful if you're a DATVault subscriber.
• You can now add a quick import folder through File > Settings. When you click the Add DAT files recursively from your quick import folder button, all DAT files in that folder and its subfolders are loaded into Retool.
• You can now replace your input DAT files with the Retool version instead of creating new files. Make sure you've backed up your original DAT files first.
• By default, Retool no longer processes files it has already processed. You can bypass this by going to the Options tab and enabling Allow processing of already processed files.
• _Feature_: Thanks to a rewrite of the compilations code and Retool's new testing framework, you can now choose how to handle compilations. There are four modes:
• Default: Chooses individual titles most of the time. Only chooses compilations when they have a higher region, language, or clone list priority, or contain unique titles. When choosing a compilation for unique titles, if other titles in the compilation have individual equivalents, the individual titles are also included, leading to some title duplication.
• Prefer individual titles: Chooses individual titles regardless of region, language, and clone list priorities, and discards compilations unless they contain unique games. You're likely to prefer this mode if you use ROM hacks or Retro Achievements. When choosing a compilation for unique titles, if other titles in the compilation have individual equivalents, the individual titles are also included, leading to some title duplication.
• + 18 more

📋 Changes

• _Fix_: Ensured MAME Redump DAT files have unique config settings, but load Redump clone lists and metadata (#305).
• _Fix_: Fixed titles without regions being assigned a blank set of regions instead of being set to `(Unknown)`. This was most obvious when converting Redump BIOS DAT files.
• _Fix_: When outputting a DAT file of removed titles while using legacy parent/clone format, the removes DAT file no longer contains clones found in the legacy parent/clone DAT file.
• _Fix_: More reliable sorting of exclusions and user option tags in filenames and descriptions.
• _Fix_: If a title is in a `<game>` or `<machine>` node, that node is used in the output DAT file (#305).
• _Chore_: Style fixes and reduction of unnecessary work across the code base in preparation for building out more comprehensive tests.

📋 Changes

• _Feature_: Retool now supports [MAME Redump](https://github.com/MetalSlug/MAMERedump/tree/main) sets. These DATs match against Redump clone lists.
• _Feature_: You can now choose to use `<machine>` for your title nodes in the output DAT file instead of `<game>`.
• _Change_ (#299): `<rom>` and `<disk>` nodes in an input DAT file no longer need a size assigned for Retool to process them. This is because CHDs are often defined with only a hash.
• _Fix_ (#300): If you run Retool CLI on a folder with only one DAT file, and no titles are in that DAT file that match your preferences, Retool no longer ends the task with a crash.

📋 Changes

• _Breaking change_: The clone list format has changed. Update to 2.02.0 to avoid issues.
• _Feature_: You can now output a DAT using the local name of a title in unicode, if it's available in the metadata scraped from Redump and No-Intro, or included in a clone list. For example, instead of `Shining Force II - Inishie no Fuuin (Japan)`, you can output `シャイニング●フォースII 『古の封印』 (Japan)`. Tags like `(Japan)` and `(Disc 1)` remain in English.
• _Feature_: You can now add `filters` to a title entry in clone lists. Think of the `filters` parameter as an easy way to isolate specific titles in a search result, and apply changes to them based on conditions.
• `matchRegions`: `array[string]`. Regions a title must have for the condition to be true.
• `matchLanguages`: `array[string]`. Languages a title must have for the condition to be true.
• `matchString`: `regex string`. A regex string that must find a match in a title's full name for the condition to be true.
• `regionOrder`: `object[string, array[string]]`. The user region order that must be fulfilled for the condition to be true.
• `categories`: `array[string]`. A list of categories to assign to the title.
• + 27 more

📋 Changes

• Added a few extra No-Intro DAT formats to take into account recent Lynx changes.

📋 Changes

• Retool now makes better selection choices when a multi-region title is involved. Mostly this affects users that place Europe higher than USA in their region priority, however USA users might notice improvements in some places too, particularly when selecting single titles in preference of compilations.
• If there are duplicate `game` nodes in an input DAT, they are now removed.

📋 Changes

• Fixed a bug that would crash the clone list updater when it found unicode characters.
• Fixed a bug that would crash Retool if you used an override to include some compilations.

📋 Changes

• Added an extra code to the PlayStation 1 / 2 disc ID regex.
• Fixed a bug where Retool wouldn't choose between two competing supersets, and clones wouldn't be assigned.
• Fixed an uncommon bug that crashed Retool when the user preferred regions over languages.
• Fixed some button tooltips in the GUI that incorrectly mentioned "regions" instead of "languages".

📋 Changes

• You can now press the `Del` key on your keyboard in the DAT, region, and language lists to remove the selected entry.
• The title tool now decodes HTML entities like `&amp;` in title names.
• The Fujitsu version code regex was modified to take into account a new variation.