In the strict syntax, variables must be declared with def and must not specify a type:

each row is simply a list of columns.

splitCsv() reads each line into an array, and each comma-separated value in the line becomes an element in the array

output: path "UPPER-${input_file}" script: """ cat '$input_file' | tr '[a-z]' '[A-Z]' > 'UPPER-${input_file}'

You know how to collect outputs from a batch of process calls and feed them into a joint analysis or summation step.

things could get a little tricky, because we need to be able to handle an arbitrary number of input files. Specifically, we can't write the command up front, so we need to tell Nextflow how to compose it at runtime based on what inputs flow into the process.

we're not adding the operator in the context of a channel factory, but to an output channel.

A named workflow is a workflow that can be called by other workflows:

As a best practice, params should be used only in the entry workflow and passed to workflows and processes as explicit inputs.

MHG is formed by identifying and grouping all homologous sequences

evolutionary events

are encapsulated within the same MHG

write another pipeline that calls on one of those processes, you just need to type one short import statement to use the relevant module. This is better than just copy-pasting the code, because if later you decide to improve the module, all your pipelines will inherit the improvements.

They encapsulate applications and dependencies in portable, self-contained packages that can be easily distributed. Containers are also key to enabling predictable and reproducible results.

Nextflow was one of the first workflow technologies to fully embrace containers for data analysis pipelines.

Today, workflows may comprise dozens of distinct container images. Pipeline developers must manage and maintain these containers and ensure that their functionality precisely aligns with the requirements of every pipeline task.

Wave — a container provisioning and augmentation service that is fully integrated with the Nextflow and Nextflow Tower ecosystems.

Wave allows developers to manage containers as part of the pipeline itself

Our platform combines novel hardware with AI-enabled bioinformatics to unlock the personalized medicine potential of the gut microbiome

Specifying the Conda environments in a separate configuration profile is therefore recommended to allow the execution via a command line option and to enhance the workflow portability

process.conda = 'samtools'

You can also download Conda lock files from Wave build pages. These files list every package and its dependencies, so Conda doesn’t need to resolve the environment. This makes environment setup faster and more reproducible.

conda '/some/path/my-env.yaml'

launch directory

profiles are applied in the order in which they are specified on the command line.

The standard profile is used by default when no profile is specified.

Config scopes are used to group related config options

The only difference when compared with legacy syntax is that the process is not bound with specific input and output channels, as was previously required using the from and into keywords respectively

Another exciting feature of Nextflow DSL 2 is the ability to compose built-in operators, pipeline processes and sub-workflows with the pipe (|) operator

mimics the expressiveness of the Unix pipe model

due to their inherent complexity and the limited availability of decontamination pipelines compared with those for marker gene datasets

multiple profiles can be loaded

later profiles can overwrite earlier profiles.

Please only use Conda as a last resort i.e. when it’s not possible to run the pipeline with Docker, Singularity, Podman, Shifter, Charliecloud, or Apptainer.

Development and prototyping: When you need flexibility to update packages
Custom/proprietary software: When containers aren't available
Resource-constrained environments: Where container overhead is problematic
Legacy systems: Where container runtimes aren't available

Primary: Use Docker/Singularity containers
Development: Use Wave to generate containers from conda specs
Fallback: Use conda only when containers aren't feasible
Future: Leverage conda lock files for maximum reproducibility

Process directives allow the specification of settings for the task execution such as cpus, memory, container, and other resources in the workflow script.

it’s strongly suggested to define the process settings in the workflow configuration file instead of the workflow script

quick alternative to building Conda packages in the local computer

Wave allows the provisioning of containers based on the conda directive used by the processes in your pipeline

Seqera Platform access token is not mandatory, but it is recommended in order to access private container repositories and pull public containers without being affected by service rate limits

FastGA finds alignments between two genome sequences more than an order of magnitude faster

stores millions of alignments in a fraction of the space of a conventional CIGAR-string

using a trace-point encoding

We carefully separate the problems of genome alignment and genome homology

by not conflating these two steps, FastGA can be used for other downstream tasks such as finding recurrent insertions due to transposable elements

The key idea is to reduce the number of k-mers inspected for seed matches by using only those that are minimizers in a window of some small size

it is a way to perform a dry-run

provide a dummy script that mimics the execution

ou can define a command stub, which replaces the actual process command when the -stub-run or -stub command-line option is enabled:

In this Review, we examine the main machine learning concepts, tasks and applications that are relevant for experimental and clinical microbiologists

provide the minimal toolbox

understand, interpret and use machine learning

investigated the concordance between the species and genus level calls across the tools

we built a new marker gene database with 43 markers for bacteria+archaea and 48 markers for fungi

We then built the database using recent versions of NCBI RefSeq: version 221 for both bacteria (329,194 assemblies) and archaea (1,911) and version 222 for fungi (564)

The final database was 4.1 GB, containing 3,335,783 sequences.

Lemur, a marker-gene-based long-read taxonomic profiler

Magnet, a genome-based validation tool for confirming the presence and absence of microbial genomes present in a sample

we introduce Lemur and Magnet, a pair of tools optimized for lightweight and accurate taxonomic profiling for long-read shotgun metagenomic datasets

BugBuster is a fully containerized, modular, and reproducible workflow implemented in Nextflow. The pipeline streamlines analysis at level of reads, contigs, and metagenome-assembled genomes (MAGs), offering dedicated modules for taxonomic profiling and resistome characterization.

Thanks to the use of containers, BugBuster can be deployed with minimal configuration on workstations, high-performance clusters, or cloud platforms

an update to GTDB-Tk that uses a divide-and-conquer approach where user genomes are initially placed into a bacterial reference tree with family-level representatives followed by placement into an appropriate class-level subtree comprising species representatives

Genome Taxonomy Database (GTDB; https://gtdb.ecogenomic.org) provides a phylogenetically consistent and rank normalized genome-based taxonomy for prokaryotic genomes sourced from the NCBI Assembly database.

GTDB uses relative evolutionary divergence (RED) to delineate higher-rank taxa and average nucleotide identity (ANI) to delineate species clusters

Groovy-style type annotations should be used instead:

The take: section is used to declare the inputs of a named workflow:

Statements and script declarations can not be mixed at the same level.

Abstract

unlike the linear number of k-mers in a sequence, the number of subsequences grows exponentially

best practices guidelines enforced by the project further ensure that the pipelines are robust, well-documented, and validated against real-world datasets.

Convert basic Nextflow modules to nf-core compatible modules

potential limitation of shotgun sequencing is the complexity of bioinformatics pipelines required for its analysis

scripts were made available for researchers

KBase platform [37] offers a user-friendly interface that allows for the analysis of data using most of the tools described in this publication

Unexpectedly, k-mer approaches resulted in rather high false positive rates, which may lead to misinterpretations of microbial community composition.

course demonstrates how to implement a simple variant calling pipeline with GATK (Genome Analysis Toolkit)

linear workflow

accessory files

understand what it does and how it should be configured before attempting to run it

ln -s $NXF_HOME/assets pipelines

nf-core project enforces strong guidelines for how pipelines are structured, and how the code is organized, configured and documented.

subworkflows

reuse chunks of code across different pipelines

flexible while minimizing maintenance burden

'utility' or housekeeping subworkflows

accessory functions

Nextflow works best with an active internet connection, as it is able to fetch all pipeline requirements.

It may seem like a lot of work to accomplish the same result as the original pipeline, but you do get all those lovely reports generated automatically

features of nf-core, including input validation and some neat metadata handling capabilities

unstructured data does not have a predefined data model, it is not easily processed and analyzed through conventional data tools and methods. It is best managed in nonrelational or NoSQL databases or in data lakes, which are designed to handle massive amounts of raw data in any format.

NCBI taxonomy dump

highlights source code in red for errors and yellow for warnings

Problems tab. Here, you can search for diagnostics

language server parses scripts and config files according to the Nextflow language specification, which is more strict than the Nextflow CLI

Include declarations in scripts and config files act as links, and ctrl-clicking them opens the corresponding script or config file.

view the definition of a symbol (e.g., a workflow, process, function, or variable),

can format your scripts and config files based on a standard set of formatting rules

right-click the symbol, select Rename Symbol

Format Document command in the command palette

There are three most popular pipelines used for NGSanalyses: QIIME, mothur and MetAMOS

Using first-passage analysis validated by Monte Carlo simulations, we quantitatively characterize nucleotide-specific error rates during RNA polymerase II transcription

New taxa are added to the Taxonomy database as data are deposited for them.

Visualization were created with pgfplots

knowledge gap between microbes that are only studied en masse as communities and those select few species whose molecular, genetic, or physiological diversity is studied in detail.

apply these tools to the myriad species that live in the understudied corners of our world.

Statisticians have taught for decades that the most efficient and robust experimental designs vary multiple factors simultaneously and then deconvolve the effects and interactions with simple statistical models

counted the number of PubMed articles that refer to each species in their title or abstract

few sips can help lower your overall body temperature, mimicking its natural decline before you sleep

PCN was then calculated for each sample as the ratio between the mean coverage of plasmid contigs and the mean coverage of the chromosome

enables us to follow the relative abundances of these strains over time in the human donor

For each microbiome sample, its MNS was derived by searching its sequence against those of all samples produced by past studies

evaluate UMA models on a diverse set of applications

Pairs of short reads with small edit distances, along with their unique molecular identifier tags, have been exploited to correct sequencing errors in both reads and tags.

Sync Google Drive in linux using podman and rclone

let’s automate the command using systemd

First, we create a symbolic link to the unit file in the /etc/systemd/system directory.

It’s easier than teaching kids, and it’s more exciting in some ways,” said an AI trainer

More recently, Lior Pachter has argued that t-SNE and the related UMAP do not serve a meaningful purpose in data analysis and are only useful for producing art.

feature extraction attempts to reduce the dimensionality of a dataset by building a compressed representation of the input features

Methods like t-stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP) faithfully capture and reveal local and non-linear relationships in complex microbiome datasets, but their tuning is finicky

Whether taxonomic or functional profiles provide a better discriminatory power in downstream analysis is subject to debate [23,24,25].

Different channels can have the same package, so conda must handle these channel collisions.

many environments can still not be classified well at the species level11 because of database incompleteness. Our strategy for tackling this problem was to design sylph so that researchers can create customized databases from their novel genomes or MAGs, although this requires the generation of new genomes for researchers working in undercharacterized microbiomes.

make it too easy to build manufacturing sites that could cause more pollution

avoid rigorous environmental review

16S amplicon sequencing exhibited extreme bias toward the most abundant taxon

let AI surface which applicants match your given criteria.

especially interested incandidates with prior wet lab experience and a generalist quantitative mindset.

Minimap2 follows a typical seed-chain-align procedure as is used by most full-genome aligners

Myers-Briggs Type Indicator

Myers-Briggs Type Indicator® (MBTI®) assessment

We extracted 202 question-answer pairs from the KB and 39 questions generated by GPT-4 for training and testing purposes

David A. Relman,

capability to create mirror life is likely at least a decade away and would require large investments and major technical advances

$50 copay/visit

If you make a 1 byte change and push the file again, you'll use another 500 MB of storage and no bandwidth

KPop, a novel versatile method based on full k-mer spectra and dataset-specific transformations, through which thousands of assembled or unassembled microbial genomes can be quickly compared

simplified signatures (“sketches”) based on some dataset-independent choices

the most relevant methods to classify or compare microbial genomes based on k-mers can be broadly divided into the following categories:

KPop is able to accurately map sequences onto a low-dimensional space

we observed surprising cases of viruses targeted by microbes not expected to be viable hosts.

CRISPR spacers frequently matched multiple MGEs

Previous efforts to design enzymes have largely focused on finding geometric matches between model active sites and preexisting protein structures, an approach akin to buying a suit from a thrift store; it is unlikely the fit will be perfect.

Unlike hard links, which point directly to the file data on the disk, symlinks are independent files that contain a path to another file or directory

Git treats symbolic links as special files that store the path to the target file. When you add a symbolic link to a Git repository, Git records the link information rather than the contents of the target file. Here’s how Git handles symbolic links during various operations:

Their carriage often corresponds with changes to the host transcriptome

Macromolecular binding pockets, on the other hand, are located on the protein surface and are often shallower

paste it in Thunderbird as a quote block (Ctrl+Shift+o).

taking fully advantage of our algorithm might involve coordination between multiple colleagues in a lab who are constructing plasmids with different expected sequences.

it could be further reduced by executing time-consuming dynamic programming only for some query-reference pairs that necessitate high levels of accuracy and by introducing parallel computing

theoretical minimum number of reads that is required for the reliable consensus calculation is 30 reads per plasmid

Please only use Conda as a last resort, i.e., when it’s not possible to run the pipeline with Docker or Singularity.

To enable data augmentations and stitching of multiple contigs together, we introduce two special tokens. The ‘#’ token is used to join sequences from the same species with uncertain distance to each other, while the ‘@’ token is used for sequences that are from the same contig/strand and are near each other.

Evo 2 can also leverage its unique representation of biological complexity to generate new genomic sequences

previously demonstrated that machine learning models trained on prokaryotic genomic sequences can model the function of DNA, RNA, and proteins

VOGDB, which is a database of virus orthologous groups. VOGDB is a multi-layer database that progressively groups viral genes into groups connected by increasingly remote similarity

Specific gut species distinguish left-sided versus right-sided CRC (area under the curve = 0.66) with an enrichment of oral-typical microbes

totally AI-generated resumes have a sameness to them that recruiters can tell right away. They all use similar language, and they’re almost identical.

The Trump administration is preparing to cancel a large swath of federal funding for California

or methodological differences in screening algorithms.

can also be defined using structures

, I increasingly use the Nim programming language for data processing tasks. Nim is under-appreciated in computational science but it is a very capable Python replacement for non-numerical data processing. At a high level, Nim is as easy to write as Python and as fast as C

It’s really helpful to be able to dictate my academic papers using my phone when inspiration hits me, wherever that may be.

removed potential virulence genes and secretion systems (T3SS and T6SS) to ensure safety

BioReason reasons over unseen biological entities and articulates decision-making through interpretable, step-by-step biological traces, offering a transformative approach for AI in biology that enables deeper mechanistic insights

Extremophile Campaign: In Your Home (ECIYH)

really cold places like near your air conditioner drip tray

identified 90% more genus-level phage-host interactions than traditional assembly-based methods

While some tech giants neared or imposed widespread layoffs last year, compensation for their CEOs climbed as much as tens of millions of dollars, according to an ABC News analysis of data released by research firm Equilar in May and June.

Metabolome analyses identify 15 mediating metabolites in pregnancy that improve ADHD prediction

ISCB will grant remote presentation options for reasons associated to maternity/paternity leave, care for a family member, personal/medical disability, sickness, financial hardship, or potential visa problems.

Measuring the sequence divergence (eukANI) between 173 pairs of sister species representing 65 orders of eukaryotes, we find that the degree of sequence similarity between species varies considerably across taxonomic groups and is not consistent for species within a genus

67 eukaryotic “odb10” datasets from the Benchmarking Universal Single-Copy Orthologs (BUSCO) website (busco.ezlab.org), which specifies the SCOs common to members of selected taxonomic group

In prokaryotes, homologous recombination, the basis of gene flow, depends directly on the degree of genomic sequence divergence, whereas in sexually reproducing eukaryotes, reproductive incompatibility can stem from changes in very few genes.

even in cases in which organisms themselves are neither in hand nor witnessed