Ruffus Decorators

See also

Indicator objects

Core

Decorator	Examples
@originate (Summary / Manual) Creates (originates) a set of starting file from scratch (ex nihilo!) Only called to create files which do not exist. Each item in output is created by a separate job.	@originate ( output, [extras,…] )
@split (Summary / Manual) Splits a single input into multiple output Globs in output can specify an indeterminate number of files.	@split ( input, output, [extras,…] )
@transform (Summary / Manual) Applies the task function to transform input data to output.	@transform ( input, suffix() , output, [extras,…] ) @transform ( input, regex() , output, [extras,…] ) @transform ( input, formatter() , output, [extras,…] )
@merge (Summary / Manual) Merges multiple input files into a single output.	@merge (input, output, [extras,…] )

Combinatorics

Decorator	Examples
@product (Summary / Manual) Generates the product between sets of input, i.e. all vs all comparisons.	@product ( input, formatter() ,*[* input, formatter(), output, [extras,…] )
@permutations (Summary / Manual) Generates the permutations, between all elements of a set of Input Analogous to the python itertools.permutations permutations(‘ABCD’, 2) –> AB AC AD BA BC BD CA CB CD DA DB DC	@permutations ( input, formatter(), tuple_size, output, [extras,…] )
@combinations (Summary / Manual) Generates the permutations, between all elements of a set of Input Analogous to the python itertools.combinations combinations(‘ABCD’, 3) –> ABC ABD ACD BCD Generates the combinations, between all the elements of a set of Input: i.e. r-length tuples of input elements with no repeated elements (A A) and where order of the tuples is irrelevant (either A B or B A, not both).	@combinations ( input, formatter(), tuple_size, output, [extras,…] )
@combinations_with_replacement (Summary / Manual) Generates the permutations, between all the elements of a set of Input Analogous to the python itertools.permutations combinations(‘ABCD’, 3) –> ABC ABD ACD BCD Generates the combinations_with_replacement, between all the elements of a set of Input: i.e. r-length tuples of input elements with no repeated elements (A A) and where order of the tuples is irrelevant (either A B or B A, not both).	@combinations_with_replacement ( input, formatter(), tuple_size, output, [extras,…] )

Advanced

Decorator	Examples
@subdivide (Summary / Manual) Subdivides each input into multiple Outputs. The number of output can be determined at runtime using globs. Many to Even More operator. Do not use split as a synonym for subdivide.	@subdivide ( input, regex() , [ inputs | add_inputs(input_pattern), ] output, [extras,…] ) @subdivide ( input, formatter([] ), [ inputs | add_inputs(input_pattern), ] output, [extras,…] )
@transform (Summary / Manual) Generates both input & output from regular expressions Useful for adding additional file dependencies	@transform ( input, regex() , [ inputs | add_inputs(input_pattern), ] output, [extras,…] ) @transform ( input, formatter() , [ inputs | add_inputs(input_pattern), ] output, [extras,…] )
@collate (Summary / Manual) Groups multiple input using regular expression matching. Multiple input generating identical output are collated together.	@collate (input, regex() , output, [extras,…] ) @collate (input, regex() , inputs | add_inputs(input_pattern), output, [extras,…] ) @collate (input, formatter() , output, [extras,…] ) @collate (input, formatter() , inputs | add_inputs(input_pattern), output, [extras,…] )
@follows (Summary / Manual) Indicates task dependency optional mkdir prerequisite (see Manual)	@follows ( task1, 'task2' )) @follows ( task1, mkdir( 'my/directory/' ))
@posttask (Summary / Manual) Calls function after task completes Optional touch_file indicator (Manual)	@posttask ( completion_signal_func ) @posttask (touch_file( 'task1.done' ))
@active_if (Summary / Manual) Switches tasks on and off at run time Evaluated each time you call pipeline_run(…), pipeline_printout(…) or pipeline_printout_graph(…) Dormant tasks behave as if they are : up to date and have no output.	@active_if ( on_or_off1, [on_or_off2, ...] )
@jobs_limit (Summary / Manual) Limits the amount of multiprocessing for the specified task Ensures that fewer than N jobs are run in parallel for this task Overrides multiprocess parameter in pipeline_run(…)	@jobs_limit ( NUMBER_OF_JOBS_RUNNING_CONCURRENTLY )
@mkdir (Summary / Manual) Generates paths for os.makedirs	@mkdir ( input, suffix() , output ) @mkdir ( input, regex() , output ) @mkdir ( input, formatter() , output)
@graphviz (Summary / Manual) Customise the task graphics in flowcharts from pipeline_printout_graph(…)	@graphviz ( graphviz_parameter = XXX, [graphviz_parameter2 = YYY ...])

Esoteric!

Decorator	Examples
@files (Summary / Manual) I/O parameters skips up-to-date jobs Should use @transform etc instead	@files( parameter_list ) @files( parameter_generating_function ) @files ( input_file, output_file, other_params, … )
@parallel (Summary / Manual) By default, does not check if jobs are up to date Best used in conjuction with @check_if_uptodate	@parallel ( parameter_list ) @parallel ( parameter_generating_function )
@check_if_uptodate (Summary / Manual) Custom function to determine if jobs need to be run	@check_if_uptodate ( is_task_up_to_date_function )
Tip The use of this is discouraged. @files_re (Summary) I/O file names via regular expressions start from lists of file names or glob results skips up-to-date jobs	@files_re ( input, matching_regex, [input_pattern,] output, ... )