In the UI I see that a job started last night (around 5:19PM) is apparently still running this morning (~9:00AM) :
Call #15:
ID:operations/EJbDvOimKxiPoePtu4iSrLUBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:
Running
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:Pending...
Inputs:Show
Outputs:None
stdout:Mutect1_Task-14-stdout.log
stderr:Mutect1_Task-14-stderr.log
JES log:Mutect1_Task-14.log
But when I give the OperationsID to google I see that it is not running, but finished last night about 30 min or so after it started and that the last even is "ok":
wm8b1-75c:workflows esalinas$ gcloud alpha genomics operations describe operations/EJbDvOimKxiPoePtu4iSrLUBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
done: true
metadata:
'@type': type.googleapis.com/google.genomics.v1.OperationMetadata
clientId: ''
createTime: '2017-02-23T22:19:52Z'
endTime: '2017-02-23T22:41:00Z'
events:
- description: start
startTime: '2017-02-23T22:21:23.013200636Z'
- description: pulling-image
startTime: '2017-02-23T22:24:53.220856254Z'
- description: localizing-files
startTime: '2017-02-23T22:26:04.698573946Z'
- description: running-docker
startTime: '2017-02-23T22:33:37.684133510Z'
- description: delocalizing-files
startTime: '2017-02-23T22:40:46.430394214Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/52af88fa-61f0-4d7a-9f07-2daf8880f65e/CallingGroup_Workflow/a2f28101-f791-4477-b50c-78eee3eba380/call-Mutect1_Task/shard-14/MuTect1.call_stats.txt"
startTime: '2017-02-23T22:40:48.214386407Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/52af88fa-61f0-4d7a-9f07-2daf8880f65e/CallingGroup_Workflow/a2f28101-f791-4477-b50c-78eee3eba380/call-Mutect1_Task/shard-14/Mutect1_Task-14-rc.txt"
startTime: '2017-02-23T22:40:49.654216992Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/52af88fa-61f0-4d7a-9f07-2daf8880f65e/CallingGroup_Workflow/a2f28101-f791-4477-b50c-78eee3eba380/call-Mutect1_Task/shard-14/MuTect1.power.wig.txt"
startTime: '2017-02-23T22:40:51.580064108Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/52af88fa-61f0-4d7a-9f07-2daf8880f65e/CallingGroup_Workflow/a2f28101-f791-4477-b50c-78eee3eba380/call-Mutect1_Task/shard-14/dstat.log.txt"
startTime: '2017-02-23T22:40:52.854040039Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/52af88fa-61f0-4d7a-9f07-2daf8880f65e/CallingGroup_Workflow/a2f28101-f791-4477-b50c-78eee3eba380/call-Mutect1_Task/shard-14/MuTect1.coverage.wig.txt"
startTime: '2017-02-23T22:40:57.161349237Z'
- description: copied 1 file(s) to "gs://fc-6e92fceb-ef44-4648-a5b0-71cacc0248eb/52af88fa-61f0-4d7a-9f07-2daf8880f65e/CallingGroup_Workflow/a2f28101-f791-4477-b50c-78eee3eba380/call-Mutect1_Task/shard-14/df.log.txt"
startTime: '2017-02-23T22:40:58.624717699Z'
- description: ok
startTime: '2017-02-23T22:40:59.925851688Z'
labels: {}
Thus, there seems to be a job status inconsistency.
I want to mention that no task in the workflow is marked as failed. Moreover, the task marked "Running" is preventing a gather step from starting. Below is a copy/paste of the status from the UI.
Workflow ID:a2f28101-f791-4477-b50c-78eee3eba380
Status:
Running
Submitted:February 23, 2017, 5:10 PM (15 hours ago)
Started:February 23, 2017, 5:10 PM (15 hours ago)
Inputs:Show
Outputs:None
Workflow Log:workflow.a2f28101-f791-4477-b50c-78eee3eba380.log
Workflow Timing:Show
Calls:
CallingGroup_Workflow.MutectFC_TaskHide
Call #1:
ID:operations/EJnDvOimKxjZ05HIwsrxzw0g3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:50 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:MutectFC_Task-stdout.log
stderr:MutectFC_Task-stderr.log
JES log:MutectFC_Task.log
CallingGroup_Workflow.Mutect1_TaskHide
Call #1:
ID:operations/ELPEvOimKxiLnZHm5q7L42Ig3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-0-stdout.log
stderr:Mutect1_Task-0-stderr.log
JES log:Mutect1_Task-0.log
Call #2:
ID:operations/EMnDvOimKxjco8eCtreuoS0g3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-1-stdout.log
stderr:Mutect1_Task-1-stderr.log
JES log:Mutect1_Task-1.log
Call #3:
ID:operations/ENbDvOimKxignPHluL6G63kg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-2-stdout.log
stderr:Mutect1_Task-2-stderr.log
JES log:Mutect1_Task-2.log
Call #4:
ID:operations/ENDDvOimKxizytWQ-eyshocBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-3-stdout.log
stderr:Mutect1_Task-3-stderr.log
JES log:Mutect1_Task-3.log
Call #5:
ID:operations/ELTDvOimKxiz6JjZtdmmjWYg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-4-stdout.log
stderr:Mutect1_Task-4-stderr.log
JES log:Mutect1_Task-4.log
Call #6:
ID:operations/ENjDvOimKxjV3bq1jJ_6giwg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-5-stdout.log
stderr:Mutect1_Task-5-stderr.log
JES log:Mutect1_Task-5.log
Call #7:
ID:operations/EPXDvOimKxjwsdj28ZOLxqUBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:36 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-6-stdout.log
stderr:Mutect1_Task-6-stderr.log
JES log:Mutect1_Task-6.log
Call #8:
ID:operations/EOvDvOimKxj1ysW3voqF7I4BIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-7-stdout.log
stderr:Mutect1_Task-7-stderr.log
JES log:Mutect1_Task-7.log
Call #9:
ID:operations/ELXDvOimKxjfu-ve9NedyUYg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-8-stdout.log
stderr:Mutect1_Task-8-stderr.log
JES log:Mutect1_Task-8.log
Call #10:
ID:operations/EIzEvOimKxjP6b6GtJ_Y1dEBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-9-stdout.log
stderr:Mutect1_Task-9-stderr.log
JES log:Mutect1_Task-9.log
Call #11:
ID:operations/EKvEvOimKxiS5vauk66Ejyog3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-10-stdout.log
stderr:Mutect1_Task-10-stderr.log
JES log:Mutect1_Task-10.log
Call #12:
ID:operations/ENnDvOimKxiiqLWQgqr_87oBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-11-stdout.log
stderr:Mutect1_Task-11-stderr.log
JES log:Mutect1_Task-11.log
Call #13:
ID:operations/EOjDvOimKxiqqZ7jv7Sjqkog3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-12-stdout.log
stderr:Mutect1_Task-12-stderr.log
JES log:Mutect1_Task-12.log
Call #14:
ID:operations/EK_DvOimKxiogKnXn4q43QMg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-13-stdout.log
stderr:Mutect1_Task-13-stderr.log
JES log:Mutect1_Task-13.log
Call #15:
ID:operations/EJbDvOimKxiPoePtu4iSrLUBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:
Running
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:Pending...
Inputs:Show
Outputs:None
stdout:Mutect1_Task-14-stdout.log
stderr:Mutect1_Task-14-stderr.log
JES log:Mutect1_Task-14.log
Call #16:
ID:operations/EOrDvOimKxjdq4jgxv6YnJ0BIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-15-stdout.log
stderr:Mutect1_Task-15-stderr.log
JES log:Mutect1_Task-15.log
Call #17:
ID:operations/EMXDvOimKxiZiufezYq40K8BIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-16-stdout.log
stderr:Mutect1_Task-16-stderr.log
JES log:Mutect1_Task-16.log
Call #18:
ID:operations/ENXDvOimKxiho8Pv1LfZhJoBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-17-stdout.log
stderr:Mutect1_Task-17-stderr.log
JES log:Mutect1_Task-17.log
Call #19:
ID:operations/EJ7DvOimKxiama3VyPemvb4BIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:50 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-18-stdout.log
stderr:Mutect1_Task-18-stderr.log
JES log:Mutect1_Task-18.log
Call #20:
ID:operations/EILEvOimKxiRwM3ime6Ti-ABIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-19-stdout.log
stderr:Mutect1_Task-19-stderr.log
JES log:Mutect1_Task-19.log
Call #21:
ID:operations/EN_DvOimKxiFlu3gq6TStO4BIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-20-stdout.log
stderr:Mutect1_Task-20-stderr.log
JES log:Mutect1_Task-20.log
Call #22:
ID:operations/ELzDvOimKxjbxfbbn_-dmacBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-21-stdout.log
stderr:Mutect1_Task-21-stderr.log
JES log:Mutect1_Task-21.log
Call #23:
ID:operations/EJHEvOimKxjbwfa-pe-2uCIg3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:43 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-22-stdout.log
stderr:Mutect1_Task-22-stderr.log
JES log:Mutect1_Task-22.log
Call #24:
ID:operations/EJjDvOimKxjcpdS9ub66xk8g3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:50 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-23-stdout.log
stderr:Mutect1_Task-23-stderr.log
JES log:Mutect1_Task-23.log
Call #25:
ID:operations/EKnDvOimKxii59Gtgo3ex9cBIN6K28SDHCoPcHJvZHVjdGlvblF1ZXVl
Status:Done
Started:February 23, 2017, 5:19 PM (15 hours ago)
Ended:February 23, 2017, 5:50 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:Mutect1_Task-24-stdout.log
stderr:Mutect1_Task-24-stderr.log
JES log:Mutect1_Task-24.log
CallingGroup_Workflow.CallSomaticMutations_131_Prepare_TaskHide
Call #1:
ID:operations/EPn-meimKxjl3rfSku6d71og3orbxIMcKg9wcm9kdWN0aW9uUXVldWU
Status:Done
Started:February 23, 2017, 5:10 PM (15 hours ago)
Ended:February 23, 2017, 5:19 PM (15 hours ago)
Inputs:Show
Outputs:Show
stdout:CallSomaticMutations_131_Prepare_Task-stdout.log
stderr:CallSomaticMutations_131_Prepare_Task-stderr.log
JES log:CallSomaticMutations_131_Prepare_Task.log