I have a task that is the last task in the workflow. Its gcloud alpha genomics describe output is:
esalina@gtx590:~/benchmarking$ gcloud alpha genomics operations describe operations/EJzrrKinKxiohvSf7s_t6gIg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
done: true
metadata:
'@type': type.googleapis.com/google.genomics.v1.OperationMetadata
clientId: ''
createTime: '2017-02-25T11:32:33Z'
endTime: '2017-02-25T11:34:10Z'
events:
- description: start
startTime: '2017-02-25T11:33:08.421318432Z'
- description: pulling-image
startTime: '2017-02-25T11:33:10.946862092Z'
- description: localizing-files
startTime: '2017-02-25T11:33:45.985306414Z'
- description: running-docker
startTime: '2017-02-25T11:33:53.274120769Z'
- description: delocalizing-files
startTime: '2017-02-25T11:33:54.498750004Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/nozzle.html"
startTime: '2017-02-25T11:33:55.901959069Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/nozzle.RData"
startTime: '2017-02-25T11:33:57.060829757Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/variants.vcf"
startTime: '2017-02-25T11:33:58.718975202Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/variant_effect_output.txt"
startTime: '2017-02-25T11:34:00.694531228Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/vepReport.html"
startTime: '2017-02-25T11:34:02.030846479Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/force_called.call_stats.txt"
startTime: '2017-02-25T11:34:03.722244615Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/df.log.txt"
startTime: '2017-02-25T11:34:05.004386006Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/dstat.log.txt"
startTime: '2017-02-25T11:34:06.319586373Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/mutect_nozzle_task-rc.txt"
startTime: '2017-02-25T11:34:07.393543466Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/variant_nozzle_table.dat"
startTime: '2017-02-25T11:34:08.572725025Z'
- description: ok
startTime: '2017-02-25T11:34:09.729690266Z'
labels: {}
projectId: broad-firecloud-benchmark
request:
'@type': type.googleapis.com/google.genomics.v1alpha2.RunPipelineRequest
ephemeralPipeline:
docker:
cmd: /bin/bash /cromwell_root/exec.sh
imageName: broadinstitute/broadmutationcalling_beta:benchmark_nozzle_1
inputParameters:
- name: __extra_config_gcs_path
- localCopy:
disk: local-disk
path: exec.sh
name: exec
- localCopy:
disk: local-disk
path: fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-VEP_Task/variant_effect_output.txt
name: mutect_nozzle_task.vepVCFFile-0
- localCopy:
disk: local-disk
path: fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-MutectFC_Task/MuTectFC.call_stats.txt
name: mutect_nozzle_task.fcCallStats-0
- localCopy:
disk: local-disk
path: fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-GatherAndOncotate_Task/MuTect1.call_stats.vcf.filtered.vcf.annotated.vcf
name: mutect_nozzle_task.vcfFile-0
- localCopy:
disk: local-disk
path: fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-VEP_Task/variant_effect_output.txt_summary.html
name: mutect_nozzle_task.vepReportHTML-0
name: CallingGroup_Workflow
outputParameters:
- localCopy:
disk: local-disk
path: mutect_nozzle_task-rc.txt
name: mutect_nozzle_task-rc.txt
- localCopy:
disk: local-disk
path: df.log.txt
name: df.log.txt
- localCopy:
disk: local-disk
path: force_called.call_stats.txt
name: force_called.call_stats.txt
- localCopy:
disk: local-disk
path: vepReport.html
name: vepReport.html
- localCopy:
disk: local-disk
path: variant_nozzle_table.dat
name: variant_nozzle_table.dat
- localCopy:
disk: local-disk
path: variant_effect_output.txt
name: variant_effect_output.txt
- localCopy:
disk: local-disk
path: dstat.log.txt
name: dstat.log.txt
- localCopy:
disk: local-disk
path: nozzle.RData
name: nozzle.RData
- localCopy:
disk: local-disk
path: nozzle.html
name: nozzle.html
- localCopy:
disk: local-disk
path: variants.vcf
name: variants.vcf
projectId: broad-firecloud-benchmark
resources:
bootDiskSizeGb: 10
disks:
- mountPoint: /cromwell_root
name: local-disk
sizeGb: 78
type: PERSISTENT_HDD
minimumCpuCores: 1
minimumRamGb: 7
zones:
- us-central1-b
- us-central1-c
- us-central1-f
pipelineArgs:
clientId: ''
inputs:
__extra_config_gcs_path: gs://cromwell-auth-broad-firecloud-benchmark/00f5bfe7-c698-4460-aaf9-78e3b26159fc_auth.json
exec: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/exec.sh
mutect_nozzle_task.fcCallStats-0: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-MutectFC_Task/MuTectFC.call_stats.txt
mutect_nozzle_task.vcfFile-0: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-GatherAndOncotate_Task/MuTect1.call_stats.vcf.filtered.vcf.annotated.vcf
mutect_nozzle_task.vepReportHTML-0: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-VEP_Task/variant_effect_output.txt_summary.html
mutect_nozzle_task.vepVCFFile-0: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-VEP_Task/variant_effect_output.txt
labels: {}
logging:
gcsPath: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/mutect_nozzle_task.log
outputs:
df.log.txt: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/df.log.txt
dstat.log.txt: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/dstat.log.txt
force_called.call_stats.txt: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/force_called.call_stats.txt
mutect_nozzle_task-rc.txt: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/mutect_nozzle_task-rc.txt
nozzle.RData: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/nozzle.RData
nozzle.html: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/nozzle.html
variant_effect_output.txt: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/variant_effect_output.txt
variant_nozzle_table.dat: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/variant_nozzle_table.dat
variants.vcf: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/variants.vcf
vepReport.html: gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/89cdfe00-4a00-4780-a0df-fc2a4371ab12/CallingGroup_Workflow/00f5bfe7-c698-4460-aaf9-78e3b26159fc/call-mutect_nozzle_task/vepReport.html
projectId: broad-firecloud-benchmark
resources:
bootDiskSizeGb: 10
disks:
- autoDelete: false
mountPoint: ''
name: local-disk
sizeGb: 78
source: ''
type: PERSISTENT_HDD
minimumCpuCores: 1
minimumRamGb: 7
noAddress: false
preemptible: false
zones:
- us-central1-b
- us-central1-c
- us-central1-f
serviceAccount:
email: default
scopes:
- https://www.googleapis.com/auth/genomics
- https://www.googleapis.com/auth/compute
runtimeMetadata:
'@type': type.googleapis.com/google.genomics.v1alpha2.RuntimeMetadata
computeEngine:
diskNames:
- local-disk-204270014222369576
instanceName: ggp-204270014222369576
machineType: us-central1-b/n1-standard-2
zone: us-central1-b
startTime: '2017-02-25T11:32:37Z'
name: operations/EJzrrKinKxiohvSf7s_t6gIg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
esalina@gtx590:~/benchmarking$
esalina@gtx590:~/benchmarking$
As can be seen in it, it took about 2 minutes to run. In contrast, it is still "Running" in the UI
CallingGroup_Workflow.mutect_nozzle_taskHide
Call #1:
ID:operations/EJzrrKinKxiohvSf7s_t6gIg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:
Running
Started:February 25, 2017, 6:32 AM (10 hours ago)
Ended:Pending...
Inputs:Show
Outputs:None
stdout:mutect_nozzle_task-stdout.log
stderr:mutect_nozzle_task-stderr.log
JES log:mutect_nozzle_task.log
This is the last task of the last workflow in the submission that is causing the entire submission to be marked "Submitted" rather than "Done"