I recently pushed a WDL that had in the workflow section duplicate variable "pairName". In the method configs section, I kept entering a value for the first of the duplicate and saw that the value wouldn't "stick". It was upon noticing the lack of "stickiness" that I noticed that the variable was a "dup". Upon noticing the duplicate, I entered the same value for the other variable and then the value stuck.
Should WDL allow such duplicates? Should firecloud accept such WDL?
this doesn't seem to be a showstopper but could potentially cause confusion!
-eddie
wm8b1-75c:WDLs esalinas$ firecloud -m push -t Workflow -y ssyn qc.wdl
Succesfully pushed. Reponse:
{
"name": "qc",
"createDate": "2016-03-04T14:33:12Z",
"payload": "task QC_Task {\n\t\n\tFile tumorBam\n\tFile tumorBamIdx\n\tFile normalBam\n\tFile normalBamIdx\n\tFile regionFile\n\tFile readGroupBlackList\n\tFile captureNormalsDBRCLZip\n\tString caseId\n\tString controlId\n\n\tcommand <<<\n\n\t###############################################\n\t# Copy Number QC for Capture\n\t#Make lane lists for tumor and normal\n\t#MakeLaneList_12\n\tjava -jar /usr/local/bin/MakeLaneList.jar ${tumorBam} ${caseId}.lanelist.txt ;\n\tjava -jar /usr/local/bin/MakeLaneList.jar ${normalBam} ${controlId}.lanelist.txt ; \n\n\t#make region coverages per lane for tumor\n\t#RegionCovPerLane_18\n\tfor CHROM in seq 1 24 ; \n\t\tdo \n\t\techo \"Working on $CHROM for ${tumorBam}\"\n\t\tSCATTER_DIR=\"scatter.$CHROM\" ; \n\t\tmkdir -v $SCATTER_DIR\n\t\tOUTPATH=echo -ne \"$SCATTER_DIR/chr$CHROM.${caseId}.rcl\" ; \n\t\techo \"OUTPATH is $OUTPATH\"\n\t\techo \"chrom is $CHROM\"\n\t\tjava -jar /usr/local/bin/RegionCovPerLane.jar ${tumorBam} ${regionFile} $OUTPATH $CHROM \n\tdone ;\n\twait ; \n\t/usr/local/bin/rcl.gather.sh ${caseId}.rcl ; \n\trm -rf scatter.* ;\n\n\t#make region coverages per lane for control\n\t#RegionCovPerLane_18\n\tfor CHROM in seq 1 24 ; \n\t\tdo \n\t\techo \"Working on $CHROM for ${normalBam}\"\n\t\tSCATTER_DIR=\"scatter.$CHROM\" ; \n\t\tmkdir -v $SCATTER_DIR\n\t\tOUTPATH=echo -ne \"$SCATTER_DIR/chr$CHROM.${controlId}.rcl\" ; \n\t\techo \"OUTPATH is $OUTPATH\"\n\t\techo \"chrom is $CHROM\"\n\t\tjava -jar /usr/local/bin/RegionCovPerLane.jar ${normalBam} ${regionFile} $OUTPATH $CHROM \n\tdone ;\n\twait ; \n\t/usr/local/bin/rcl.gather.sh ${controlId}.rcl ; \n\trm -rf scatter.* ;\n\n\t#run the mat-lab based report\n\t#CopyNumberQCReport_27\n\tcp -vir /CopyNumberQCReport_27/unzip/* . \n\t#Make a file of files paths\n\t#This command aims to list the zip files contents, filtering out all but the file paths and then write the paths to a file\n\tunzip -l ${captureNormalsDBRCLZip} |sed -r \"s/^\s+//g\"|sed -r \"s/^[0-9]+//g\"|sed -r \"s/^\s//g\"|sed -r \"s/^\S//g\"|sed -r \"s/^\s[0-9:]\s//g\"|grep -Pv '^Date\s+Time\s+Name'|grep -Pv '^[\-\s]$'|grep -Pv '\.zip$'|grep -Pv '^files$' > capture_normals_db_wdl\n\t#this command actually performs the unzipping\n\tunzip ${captureNormalsDBRCLZip}\n\t./run_fh_CopyNumberQCReport.sh /opt/MATLAB/MATLAB_Compiler_Runtime/v710/ ${caseId}_${controlId} \\n\t ${caseId}.rcl ${controlId}.rcl ${caseId}.lanelist.txt ${controlId}.lanelist.txt \\n\t ${readGroupBlackList} ${regionFile} capture_normals_db_wdl NA NA\n\n\t>>>\n\n\truntime {\n\t\tdocker: \"broadinstitute/eddiesqcdocker\"\n\t\tmemory: \"24 GB\"\n\t\tdefaultDisks: \"local-disk 100 SSD\"\n\t\t}\n\n\toutput {\n\t\t#lane lists\n\t\tFile tumorBamLaneList=\"${caseId}.lanelist.txt\"\n\t\tFile normalBamLaneList=\"${controlId}.lanelist.txt\"\n\t\tFile tumorRCL=\"${caseId}.rcl\"\n\t\tFile normalRCL=\"${controlId}.rcl\"\n\t\t}\n\t}\n\ntask PicardMultipleMetricsPair {\n\n\tFile tumorBam\n\tFile tumorBamIdx\n\tFile normalBam\n\tFile normalBamIdx\n\tFile refFasta\n\n\tcommand <<<\n\n\tfor BAM in ${tumorBam} ${normalBam} ; \n\t\tdo\n\t\tBASE=basename $BAM ; \n\t\techo \"Processing $BAM ...\" ; \n\t\t/usr/local/jre1.8.0_73/bin/java -Xmx4g -jar /usr/local/bin/picardtools-2.1.0.jar CollectMultipleMetrics I=$BAM O=$BASE.multiple_metrics R=${refFasta} PROGRAM=CollectAlignmentSummaryMetrics PROGRAM=CollectInsertSizeMetrics PROGRAM=QualityScoreDistribution PROGRAM=MeanQualityByCycle PROGRAM=CollectBaseDistributionByCycle PROGRAM=CollectSequencingArtifactMetrics PROGRAM=CollectQualityYieldMetrics PROGRAM=CollectGcBiasMetrics ;\n\t\tdone ;\n\n\t#zip up tumor reports\n\tTUMOR_BASE=basename ${tumorBam} ; \n\tzip tumor_metrics.zip $TUMOR_BASE.multiple_metrics.*\n\n\t#zip up control reports\n\tNORMAL_BASE=basename ${normalBam}; \n\tzip normal_metrics.zip $NORMAL_BASE.multiple_metrics.*\n\n\t>>>\n\n\truntime {\n\t\tdocker: \"broadinstitute/eddiesqcdocker\"\n\t\tmemory: \"24 GB\"\n\t\tdefaultDisks: \"local-disk 100 SSD\"\n\t\t}\n\n\toutput {\n\t\tFile normalReportsZip=\"normal_metrics.zip\"\n\t\tFile tumorReportsZip=\"tumor_metrics.zip\"\n\t\t}\n\n\t}\n\ntask PicardMultipleMetrics {\n\n\tFile bam\n\tFile bamIndex\n\tFile refFasta\n\n\tcommand <<<\n\n\tfor BAM in ${bam} ; \n\t\tdo\n\t\tBASE=basename $BAM ; \n\t\techo \"Processing $BAM ...\" ; \n\t\t/usr/local/jre1.8.0_73/bin/java -Xmx4g -jar /usr/local/bin/picardtools-2.1.0.jar CollectMultipleMetrics I=$BAM O=$BASE.multiple_metrics R=${refFasta} PROGRAM=CollectAlignmentSummaryMetrics PROGRAM=CollectInsertSizeMetrics PROGRAM=QualityScoreDistribution PROGRAM=MeanQualityByCycle PROGRAM=CollectBaseDistributionByCycle PROGRAM=CollectSequencingArtifactMetrics PROGRAM=CollectQualityYieldMetrics PROGRAM=CollectGcBiasMetrics ;\n\t\tdone ;\n\n\t#zip up tumor reports\n\tBASE=basename ${bam} ; \n\tzip piccard_multiple_metrics.zip $BASE.multiple_metrics.*\n\n\t>>>\n\n\truntime {\n\t\tdocker: \"broadinstitute/eddiesqcdocker\"\n\t\tmemory: \"24 GB\"\n\t\tdefaultDisks: \"local-disk 100 SSD\"\n\t\t}\n\n\toutput {\n\t\tFile metricsReportsZip=\"piccard_multiple_metrics.zip\"\n\t\t}\n\n\t}\n\n\n\n\n\n\ntask ContESTTask {\n\tFile tumorBam\n\tFile tumorBamIdx\n\tFile normalBam\n\tFile normalBamIdx\n\tFile refFasta\n\tFile refFastaIdx\n\tFile refFastaDict\n\tFile exomeIntervals\n\tFile SNP6Bed\n\tFile HapMapVCF\n\n\tcommand <<<\n\t#run contest \n\tjava -Xmx1024m -Djava.io.tmpdir=/tmp -cp /usr/local/bin/Queue-1.4-437-g6b8a9e1-svn-35362.jar org.broadinstitute.sting.gatk.CommandLineGATK -T Contamination -I:eval ${tumorBam} -I:genotype ${normalBam} -et NO_ET -L ${exomeIntervals} -L ${SNP6Bed} -isr INTERSECTION -R ${refFasta} -l INFO -pf ${HapMapVCF} -o contamination.af.txt --trim_fraction 0.03 --beta_threshold 0.05 -br contamination.base_report.txt -mbc 100 --min_genotype_depth 30 --min_genotype_ratio 0.8 \n\n\t#Get contamination column and value\n\tCONTAM_COL=head -1 contamination.af.txt|tr \"\\t\" \"\\n\"|grep -Pin '^contamination$'|grep -Po '^\\d+:'|tr -d \":\" ; \n\tCONTAM_VAL=cut -f $CONTAM_COL contamination.af.txt|tail -1 ;\n\tCONTAM_NM=cut -f1 contamination.af.txt|tail -1; \n\n\t#write contam data for validation\n\techo -ne \"$CONTAM_NM\t$CONTAM_VAL\n\" > contamination_validation.array_free.txt\n\t\n\n\t#Contamination validation/consensus\n\tpython /usr/local/populateConsensusContamination_v26/contaminationConsensus.py --pass_snp_qc false\\n\t --output contest_validation.output.tsv \\n\t --annotation contamination_percentage_consensus_capture\\n\t --array contamination_validation.array_free.txt --noarray contamination_validation.array_free.txt\n\n\n\n\t>>>\n\n\truntime {\n\t\tdocker: \"broadinstitute/eddiesqcdocker\"\n\t\tmemory: \"24 GB\"\n\t\tdefaultDisks: \"local-disk 100 SSD\"\n\t\t}\n\n\toutput {\n\t\tFile contamDataFile=\"contamination_validation.array_free.txt\"\n\t\tFile contestAFFile=\"contamination.af.txt\"\n\t\tFile contestBaseReport=\"contamination.base_report.txt\"\n\t\tFile validationOutput=\"contest_validation.output.tsv\"\n\t\t}\n\n\n\n\n\t}\n\n\ntask CrossCheckLaneFingerprints {\n\n\tFile tumorBam\n\tFile normalBam\n\tFile tumorBamIdx\n\tFile normalBamIdx\n\tString pairName\n\tFile HaplotypeDBForCrossCheck\n\tString validationStringencyLevel\n\n\n\tcommand <<<\n\n\n\t\t#ConvertDependentsList[version=8] (custom!)\n\t\t#prepare input for the convert command\n\t\techo -ne \"${pairName}_Normal\t${normalBam}\n${pairName}_Tumor\t${tumorBam}\n\" > conv_file.input.tsv \n\t\t#make the options file\n\t\tOPTIONS_FILE=${pairName}_CCLFP.options\n\t\techo -ne \"I=${normalBam}\nI=${tumorBam}\n\" > $OPTIONS_FILE ; \n\t\t\n\t\t#Cross Check!\n\t\t#CrosscheckLaneFingerprints[version=9]\n\t\tmkdir -v tmp\n\t\tjava -Xmx3g -jar /usr/local/bin/CrosscheckReadGroupFingerprints.jar OPTIONS_FILE=$OPTIONS_FILE H=${HaplotypeDBForCrossCheck} TMP_DIR=pwd/tmp QUIET=true EXIT_CODE_WHEN_MISMATCH=0 OUTPUT=crosscheck.stats.txt VALIDATION_STRINGENCY=${validationStringencyLevel}\n\n\t\t#CrosscheckLaneFingerprintsReport[version=10]\n\t\t#Run report\n\t\t/usr/local/bin/crosscheck_report.pl --stats crosscheck.stats.txt ${normalBam} ${tumorBam} conv_file.input.tsv \n\n\t>>>\n\n\truntime{\n\t\tdocker: \"broadinstitute/eddiesqcdocker\"\n\t\tmemory: \"24 GB\"\n\t\tdefaultDisks: \"local-disk 100 SSD\"\t\t\n\t\t}\n\n\n\toutput {\n\t\tFile crossCheckStats=\"crosscheck.stats.txt\"\n\t\tFile crossCheckReport=\"report.html\"\n\t\t}\n\n\n\n\t}\n\n\n\nworkflow QC_Workflow {\n\n\tFile tumorBam\n\tFile normalBam\n\tFile tumorBamIdx\n\tFile normalBamIdx\n\tFile regionFile\n\tString caseId\n\tString controlId\n\tFile readGroupBlackList\n\tFile captureNormalsDBRCLZip\n\tString SampleName\n\tString pairName\n\tFile refFasta\n\tFile refFastaIdx\n\tFile refFastaDict\n\tFile exomeIntervals\n\tFile SNP6Bed\n\tFile HapMapVCF\n\tString pairName\n\tFile HaplotypeDBForCrossCheck\n\tString validationStringencyLevel\t\t\n\n\tcall QC_Task {\n\t\tinput : \n\t\t\ttumorBam=tumorBam,\n\t\t\tnormalBam=normalBam,\n\t\t\ttumorBamIdx=tumorBamIdx,\n\t\t\tnormalBamIdx=normalBamIdx,\n\t\t\tregionFile=regionFile,\n\t\t\tcaseId=caseId,\n\t\t\tcontrolId=controlId,\n\t\t\treadGroupBlackList=readGroupBlackList,\n\t\t\tcaptureNormalsDBRCLZip=captureNormalsDBRCLZip\n\t\t}\n\n\n\n\n\tcall ContESTTask {\n\t\tinput : \n\t\t\ttumorBam=tumorBam,\n\t\t\ttumorBamIdx=tumorBamIdx,\n\t\t\tnormalBam=normalBam,\n\t\t\tnormalBamIdx=normalBamIdx,\n\t\t\trefFasta=refFasta,\n\t\t\trefFastaIdx=refFastaIdx,\n\t\t\trefFastaDict=refFastaDict,\n\t\t\texomeIntervals=exomeIntervals,\n\t\t\tSNP6Bed=SNP6Bed,\n\t\t\tHapMapVCF=HapMapVCF\n\t\t}\n\n\n\tcall CrossCheckLaneFingerprints {\n\t\tinput:\n\t\t\ttumorBam=tumorBam,\n\t\t\tnormalBam=normalBam,\n\t\t\ttumorBamIdx=tumorBamIdx,\n\t\t\tnormalBamIdx=normalBamIdx,\n\t\t\tpairName=pairName,\n\t\t\tHaplotypeDBForCrossCheck=HaplotypeDBForCrossCheck,\n\t\t\tvalidationStringencyLevel=validationStringencyLevel\n\t\t}\n\n\n\t#Piccard\n\tcall PicardMultipleMetrics as tumorMM {\n\t\tinput:\n\t\t\tbam=tumorBam,\n\t\t\tbamIndex=tumorBamIdx,\n\t\t\trefFasta=refFasta\n\t\t}\n\tcall PicardMultipleMetrics as normalMM {\n\t\tinput:\n\t\t\tbam=normalBam,\n\t\t\tbamIndex=normalBamIdx,\n\t\t\trefFasta=refFasta\n\t\t}\n\n\n\t}",
"url": "http://agora.dsde-prod.broadinstitute.org/api/v1/methods/esalinas/qc/1",
"documentation": "",
"synopsis": "ssyn",
"entityType": "Workflow",
"snapshotId": 1,
"namespace": "esalinas"
}