I have two identical workflows (broad-firecloud-ibmwatson/CBB_20181127_CGA_WES_Characterization_TCGAControlledAccess_test
and broad-fc-getzlab-workflows/CBB_20181127_CGA_WES_Characterization_TCGAControlledAccess_test_2), with the only difference being that one was created under the FireCloud Billing project broad-firecloud-ibmwatson and the other was created under the FC billing project broad-fc-getzlab-workflows. I am running the getz lab WES characterization workflow on a single pair in each of these workspaces. The workflow fails in the former but runs through to completion in the later. The error is reproducible. The error occurs in the the two tasks (mutect1 and mutect2) that are scattered. Each scattered task call fails. Here is one of the failure messages:
message: Call input and runtime attributes evaluation failed for Mutect1_Task
causedBy:
message: Failed to evaluate input 'diskGB' (reason 1 of 1): [Attempted 1 time(s)] - StorageException: pet-103169287383783708406@broad-firecloud-ibmwatson.iam.gserviceaccount.com does not have storage.objects.get access to fc-secure-b14d3c32-f817-411c-8e75-3298c52bba97/1f5413f3-b31b-4040-971c-6f7571eef537/CGA_Production_Analysis_Workflow/60e2f010-c8b3-4187-8115-9f3acd799d3a/call-CallSomaticMutations_Prepare_Task/glob-fedfc53b55ca349744292cb186d61b54/gatk-scatter.0000000010.interval_list.
In a separate workspace, however, created under the broad-firecloud-ibmwatson billing project, the same workflow on the same entity pair passed. (This separate workspace has 179 pairs, while in my two test workspaces I only placed a single pair.) If the problem is not obvious, and can't grant your support team read access to the two workspaces. Note that the data is TCGA controlled access.