grape.recipe.pipeline Configure a Grape pipeline in one simple step
Download

grape.recipe.pipeline Ranking & Summary

grape.recipe.pipeline Tags

Python library Python pipeline

grape.recipe.pipeline Description

grape.recipe.pipeline is a Python module that prepares everything that is needed to run Grape pipelines so you don't have to write any command line options.MotivationHere at the CRG, we configure all our RNASeq pipeline runs in a central place before running the Grape pipelines. Once all the accessions and pipeline profiles have been defined and the buildout parts have been created, we start and execute them.When we receive Fastq or bam files for a project, we typically have to:1. Define the accessions and profiles in grape.buildout/accessions/MyProject/db.cfg grape.buildout/profiles/MyProject/db.cfg2. Create a pipeline project folder in grape.buildout/pipelines/MyProject3. Configure the buildout in grape.buildout/pipelines/MyProject/buildout.cfg4. Run the buildout in grape.buildout/pipelines/MyProject5. Run the first pipelines in grape.buildout/pipelines/MyProject/parts/*/Using this approach, we can batch run any number of pipelines and never have to think about the command line options.InstallationThe grape.recipe.pipeline package is already installed by grape.pipeline, so you don't have to do this.If you want to install it as part of a buildout, you would have to add this configuration to the buildout:: parts = grape.recipe.pipeline eggs = grape.recipe.pipeline recipe = hexagonit.recipe.download url = http://big.crg.cat/~mroder/grape/grape.recipe.pipeline-1.1.tar.gz md5sum = 5bd87d4d56a61b019ccc854f040f1d6d destination = src/grape.recipe.pipeline strip-top-level-dir = true hash-name = falseThen you can use this recipe like this:: parts = TestRun ... pipeline configuration omitted for brevity ... recipe=grape.recipe.pipeline accession = TestRun1ConfigurationHere's a complete example of how the pipelines are configured, taken from the Test project in grape.buildout.First we define an accession in accession/Test/db.cfg:: species = Homo sapiens readType = 2x76 cell=NHEK rnaExtract=LONGPOLYA localization=CELL replicate=1 qualities=solexa type=fastq file_location = ${buildout:directory}/src/testdata/testA.r2.fastq.gz ${buildout:directory}/src/testdata/testA.r1.fastq.gz ${buildout:directory}/src/testdata/testB.r2.fastq.gz ${buildout:directory}/src/testdata/testB.r1.fastq.gz sample = Test_TestLocaltrun_read_stats_sample_testA.2 Test_TestLocaltrun_read_stats_sample_testA.1 Test_TestLocaltrun_read_stats_sample_testB.2 Test_TestLocaltrun_read_stats_sample_testB.1 mate_id = testA.2 testA.1 testB.2 testB.1 pair_id = testA testA testB testB label = Test Test Test Test type = fastqThen we need to define pipeline run and in profiles/MyProject/db.cfg:: parts = TestRun TEMPLATE = ${buildout:directory}/src/pipeline/template3.0.txt PROJECTID = Test DB = Test_RNAseqPipeline COMMONDB = Test_RNAseqPipelineCommon THREADS = 8 MAPPER = GEM MISMATCHES = 2 CLUSTER = mem_6 ANNOTATION = ${buildout:directory}/src/testdata/H.sapiens.EnsEMBL.55.test.gtf GENOMESEQ = ${buildout:directory}/src/testdata/H.sapiens.genome.hg19.test.fa recipe=grape.recipe.pipeline accession = TestRunThe pipelines/Test/buildout.cfg looks like this:: extends = ../dependencies.cfg ../../accessions/Test/db.cfg ../../profiles/Test/db.cfgThere are pointers to the accession and profile. The dependencies file takes care of installing all the dependencies, like overlap, flux, gem, and the Grape pipeline. It also installs grape.recipe.pipeline, as describe in the above Installation section.Product's homepage

grape.recipe.pipeline Related Software