Quantcast
Channel: Ask the FireCloud Team — GATK-Forum
Viewing all articles
Browse latest Browse all 1147

optional files not localized? bug?

$
0
0

I have an optional File input ("detailMetrics") in my WDL. (see WDL task opening with inputs including the optional input)

task generateMetrics_task {
    String ID
    File BAM
    File BAM_IDX
    File? detailMetrics
    File DBSNP
    File DBSNPIDX
    File REFERENCE
    Int diskGB

In the FireCloud UI under the submission for the task, the file is identified as an input. See the text copied/pasted from the UI here:

FilterWorkflow.generateMetrics_taskHide
Call #1:
ID:operations/ENni9pyiKxjRsY6Z2aKmhJcBIP3g3tG1AioPcHJvZHVjdGlvblF1ZXVl
Status:
warning
Failed
Started:02/09/2017 at 11:03:41 AM (2 hours ago)
Ended:02/09/2017 at 12:35:11 PM (an hour ago)
Inputs:Hide
BAM_IDX→gs://fc-4520e968-446a-4f9f-85e2-8fcc901269dc/REBC-ACB7-TTP1-A-1-1-D-A49V-36.bai
diskGB→750
ID→REBC-ACB7-TTP1-A-1-1-D-A49V-36
DBSNPIDX→gs://firecloud-tcga-open-access/tutorial/reference/dbsnp_134_b37.leftAligned.vcf.idx
detailMetrics→gs://fc-4520e968-446a-4f9f-85e2-8fcc901269dc/REBC-ACB7-TTP1-A-1-1-D-A49V-36.pre_adapter_detail_metrics
REFERENCE→gs://firecloud-tcga-open-access/tutorial/reference/Homo_sapiens_assembly19.fasta
DBSNP→gs://firecloud-tcga-open-access/tutorial/reference/dbsnp_134_b37.leftAligned.vcf
BAM→gs://fc-4520e968-446a-4f9f-85e2-8fcc901269dc/REBC-ACB7-TTP1-A-1-1-D-A49V-36.bam
Outputs:None

In the exec.sh script, the file is referenced (see how GREP picks it from the exec.sh):

esalina@gtx590:~/REBC/merge_work/docker$ gsutil cat gs://fc-7119068c-40f9-4960-9dee-1d73730e9c1c/e0faaf2a-a193-49b4-9ed0-17ddcc5dab86/FilterWorkflow/398c7660-97c6-4079-b98b-bbce037edda1/call-generateMetrics_task/exec.sh|grep -Pi 'metric'
#see if a metrics file was passed in and create one if not
METRICS_IN_LEN=`echo '/cromwell_root/fc-4520e968-446a-4f9f-85e2-8fcc901269dc/REBC-ACB7-TTP1-A-1-1-D-A49V-36.pre_adapter_detail_metrics'|tr -d "\n"|wc -c` ;
if [ "$METRICS_IN_LEN" -eq "0" ] ;
    /usr/local/bin/java  -Xmx3600M  -jar /usr/local/CollectSequencingArtifactMetrics/picard.1.895.jar CollectSequencingArtifactMetrics \
    echo "Passing the metrics file along!"
    mv -v /cromwell_root/fc-4520e968-446a-4f9f-85e2-8fcc901269dc/REBC-ACB7-TTP1-A-1-1-D-A49V-36.pre_adapter_detail_metrics REBC-ACB7-TTP1-A-1-1-D-A49V-36.pre_adapter_detail_metrics
echo $? > /cromwell_root/generateMetrics_task-rc.txt.tmp
mv /cromwell_root/generateMetrics_task-rc.txt.tmp /cromwell_root/generateMetrics_task-rc.txt
esalina@gtx590:~/REBC/merge_work/docker$ 

However, the file was apparently NOT localized, because when the script ran, a file-not-found error was observed. If I grep the JES log in text before the "switching to status running docker" for the file there is no output.

esalina@gtx590:~/REBC/merge_work/docker$ gsutil cat gs://fc-7119068c-40f9-4960-9dee-1d73730e9c1c/e0faaf2a-a193-49b4-9ed0-17ddcc5dab86/FilterWorkflow/398c7660-97c6-4079-b98b-bbce037edda1/call-generateMetrics_task/generateMetrics_task.log|grep -B 10000 -Pi 'status:.running'|grep -Pi 'pre_adapter_detail_metrics'

Is this is bug?

I note that in the method-config, I used a file literal. I enclosed it with double quotes. Also, I note that in the UI, the file was apparently picked up (see "detailMetrics→gs://fc-4520e968-446a-4f9f-85e2-8fcc901269dc/REBC-ACB7-TTP1-A-1-1-D-A49V-36.pre_adapter_detail_metrics" in the task inputs section above)

Attached if it may be helpful is a copy of the JES log.


Viewing all articles
Browse latest Browse all 1147

Trending Articles