assemblerflow/flowcraft

✨ New features

• Added support for the report system to:
• `maxbin2`
• Added new `manifest.config` with the pipeline metadata

✨ New components

• `Kraken2`: Taxonomic identification on FastQ files

🐛 Bug fixes

• Fix bug in `momps`component related to added in the introduction of the clear input parameter
• Fixed bug with the `-ft` parameters not retrieving the dockerhub tags for
• all the components.
• Fixed bug in the `megahit` process where the fastg mode would break the process
• Fix inspect and report mode to fetch the nextflow file independently of its
• position in the `nextflow run` command inside the .nextflow.log file.
• Fix parsing of .nextflow.log file when searching for `nextflow run` command.
• Fixed bug between mash_sketch_fasta and mash_dist.

📋 Minor/Other changes

• Added option to `dengue_typing` to retrieve closest reference sequence and link it
• with a secondary channel into `mafft`
• New version of DEN-IM recipe
• Now prints an ordered list of components
• Moved taxonomy results from `results/annotation/` to `results/taxonomy/
• Now prints an ordered list of components
• Moved taxonomy results from results/annotation/ to results/taxonomy/

✨ New features

• Added new `recipe` system to flowcraft along with 6 starting recipes.
• Recipes are pre-made and curated pipelines that address specific questions.
• To create a recipe, the `-r <recipe_name>` can be used. To list available
• recipes, the `--recipe-list` and `--recipe-list-short` options were added.
• Added `-ft` or `--fetch-tags` which allows to retrieve all DockerHub
• container tags.
• Added function to collect all the components from the components classes,
• replacing the current process_map dictionary implementation. Now, it will be
• + 1 more

📋 Components changes

• Added new `disableRR` param in the `spades` component that disables repeat
• resolution
• The `abyss` and `spades` components emit GFA in a secondary channel.
• The new `bandage` component can accept either FASTA from a primary channel
• or GFA from a secondary channel.
• Updated skesa to version 2.3.0.
• Updated mash based components for the latest version - 1.6.0-1.

✨ New components

• Added component `abyss`.
• Added component `bandage`.
• Added component `unicycler`.
• Added component `prokka`.
• Added component `bcalm`.
• Added component `diamond`.

📋 Minor/Other changes

• Added removal of duplicate IDs from `reads_download` component input.
• Added seed parameter to `downsample_fastq` component.
• Added bacmet database to `abricate` component.
• Added default docker option to avoid docker permission errors.
• Changed the default URL generated by inspect and report commands.
• Added directives to `-L` parameter of build module.

🐛 Bug fixes

• Fixed forks with same source process name.
• Fixed `inspect` issue when tasks took more than a day in duration.
• Added hardware address to `inpsect` and `report` hash.

✨ Features

• Added a new `clearInput` parameter to components that change their input.
• The aim of this option is to allow the controlled removal of temporary files,
• which is particularly useful in very large workflows.

📋 Components changes

• Updated images for components `mash_dist`, `mash_screen` and
• `mapping_patlas`.

✨ New components

• Added component `fast_ani`.

📋 Minor/Other changes

• Added `--export-directives` option to `build` mode to export component's
• directives in JSON format to standard output.
• Added more date information in `inspect` mode, including the year and the
• locale of the executing system.

✨ Features

• Added `report` run mode to Flowcraft that displays the report of any given
• pipeline in the Flowcraft's web application. The `report` mode can be executed
• after a pipeline ended or during the pipeline execution using the `--watch`
• option.
• Added standalone report HTML at the end of the pipeline execution.
• Components with support for the new report system:
• `abricate`
• `assembly_mapping`
• + 17 more

📋 Minor/Other changes

• Refactored report json for components `mash_dist`, `mash_screen` and
• `mapping_patlas`

🐛 Bug fixes

• Fixed issue where `seq_typing` and `patho_typing` processes were not feeding
• report data to report compiler.
• Fixed fail messages for `process_assembly` and `process_viral_assembly`
• components

📋 Components changes

• `mapping_patlas`: refactored to remove temporary files used to create
• sam and bam files and added data to .report.json. Updated databases to pATLAS
• version 1.5.2.
• `mash_screen` and `mash_dist`: added data to .report.json. Updated databases
• to pATLAS version 1.5.2.

✨ New components

• Added component `fasterq_dump`
• Added component `mash_sketch_fasta`
• Added component `mash_sketch_fastq`
• Added component `downsample_fastq` for FastQ read sub sampling using seqtk
• Added component `momps` for typing of Legionella pneumophila
• Added component `split_assembly`
• Added component `mafft`
• Added component `raxml`
• + 3 more

📋 Minor/Other changes

• Added check for `params.accessions` that enables to report a proper
• error when it is set to `null`.
• Added `build` option to export component parameters information in JSON format.
• Fixed minor issue preventing the `maxbin2` and `split_assembly` components
• from being used multiples times in a pipeline
• Added a catch to the `filter_poly` process for cases where the input file is empty.
• spades template now reports the exit code of spades' execution

🐛 Bug fixes

• Removed the need for the nf process templates to have an empty line
• at the beginning of the template files.
• Fixed issue when the `inspect` mode was executed on a pipeline directory
• with failed processes but with the work directory removed (the log files
• where no longer available).
• Fixed issue when the `inspect` mode was executed on a pipeline without the
• memory directory defined.
• Fixed issue in the `inspect` mode, where there is a rare race condition between
• + 5 more

📦 Improvements

• The parameter system has been revamped, and parameters are now component-specific
• and independent by default. This allows a better fine-tuning of the parameters
• and also the execution of the same component multiple times (for instance in a fork)
• with different parameters. The old parameter system that merged identical parameters
• is still available by using the `--merge-params` flag when building the pipeline.
• Added a global `--clearAtCheckpoint` parameter that, when set to true, will remove
• temporary files that are no longer necessary for downstream steps of the pipeline
• from the work directory. This option is currently supported for the `trimmomatic`,
• + 1 more

✨ New components

• `maxbin2`: An automatic tool for binning metagenomic sequences.
• `bowtie2`: Align short paired-end sequencing reads to long reference
• sequences.
• `retrieve_mapped`: Retrieves the mapped reads of a previous bowtie2 mapping process.

✨ New recipes

• `plasmids`: A recipe to perform mapping, mash screen on reads
• and also mash dist for assembly based approaches (all to detect
• plasmids). This also includes annotation with abricate for the assembly.
• `plasmids_mapping`: A recipe to perform mapping for plasmids.
• `plasmids_mash`: A recipe to perform mash screen for plasmids.
• `plasmids_assembly`: A recipe to perform mash dist for plasmid
• assemblies.

📋 Minor/Other changes

• Added "smart" check when the user provides a typo in pipeline string
• for a given process, outputting some "educated" guesses to the
• terminal.http://192.92.149.169:80/inspect/5b585070ed1cb0dbb9a696c724b7f9ed7844e1afe5188ef3aa44c7c3cc5b539d
• Added "-cr" option to show current recipe `pipeline_string`.
• Changed the way recipes were being parsed by `proc_collector` for the
• usage of `-l` and `-L` options.
• Added check for non-ascii characters in colored_print.
• Fixed log when a file with the pipeline is provided to -t option
• + 1 more

🐛 Bug fixes

• Fixed pipeline names that contain new line characters.
• Fixed pipeline generation when automatic dependencies were added right after a fork
• Template: sistr.nf: Fixed comparison that determined process status.
• Fixed issue with `--version` option.

✨ New components

• `card_rgi`: Anti-microbial gene screening for assemblies
• `filter_poly`: Runs PrinSeq on paired-end FastQ files to remove low complexity sequences
• `kraken`: Taxonomical identification of FastQ files
• `megahit`: Metagenomic assembler for paired-end FastQ files
• `metaprob`: Preforms read binning on metagenomic FastQ files
• `metamlst`: Checks the Sequence Type of metagenomic FastQ reads using Multilocus Sequence Typing
• `metaspades`: Metagenomic assembler for paired-end FastQ files
• `midas_species`: Taxonomical identification of FastQ files on the species level
• + 2 more

✨ Features

• Added `inspect` run mode to flowcraft for displaying the progress overview
• during a nextflow run. This run mode has an `overview` and `broadcast` options
• for viewing the progress of a pipeline.

📋 Minor/Other changes

• Changed `mapping_patlas` docker container tag and variable
• (PR [#76](https://github.com/assemblerflow/assemblerflow/pull/76)).
• The `env` scope of nextflow.config now extends the `PYTHONPATH`
• environmental variable.
• Updated indexes for both `mapping_patlas` and `mash` based processes.
• New logo!

🐛 Bug Fixes

• Template: fastqc_report.py: Added fix to trim range evaluation.
• Script: merge_json.py: Fixed chewbbaca JSON merge function.