Yesterday I used FC, and I made two submissions. Each submission had a single workflow. EAch workflow had a scatter in it.
A number of the "scatter" calls end with something like the following in the JES log :
4ef75fcc34a4: Waiting
ba2a9f5e76ae: Waiting
89b43d834129: Waiting
fa3c6d254026: Waiting
error pulling image configuration: Get https://dseasb33srnrn.cloudfront.net/registry-v2/docker/registry/v2/blobs/sha256/5b/5bb8b784523ce1586c0f923de4d1cf3013c2126cae62fbb5b61beed2afb21d36/data?Expires=1487804016&Signature=EIjH0N1cbqvz~vEOV5bXeCFttj7lxv8CPHsa1LFGDhJB0aJCbKYNrhJ28wAVXNzj5c0zK2VnTsHlI4lIJ1VVAd4KHjH52iMcy3YmJnp52FXwTDDwYqnHqovdTGVeYda2ltwnupUPIgyfGII9h5viYEm2BdSrMtwrycmGjijKt7k_&Key-Pair-Id=APKAJECH5M7VWIS5YZ6Q: net/http: TLS handshake timeout
2017/02/22 22:34:16 I: Copying /var/log/google-genomics/*.log to gs://fc-ef9602f6-0cbc-448c-8fd9-de9d05b66f86/62829caa-7c9d-421e-a9f1-084a8c2e7d10/CallingGroup_Workflow/aa85173c-c8d2-4f64-b3e1-a176db5b49d2/call-Mutect1_Task/shard-10/
2017/02/22 22:34:16 I: Running command: sudo gsutil -h Content-type:text/plain -q -m cp /var/log/google-genomics/*.log gs://fc-ef9602f6-0cbc-448c-8fd9-de9d05b66f86/62829caa-7c9d-421e-a9f1-084a8c2e7d10/CallingGroup_Workflow/aa85173c-c8d2-4f64-b3e1-a176db5b49d2/call-Mutect1_Task/shard-10/
Note the
error pulling image configuration: Get https://dseasb33srnrn.cloudfront.net/registry-v2/docker/registry/v2/blobs/sha256/5b/5bb8b784523ce1586c0f923de4d1cf3013c2126cae62fbb5b61beed2afb21d36/data?Expires=1487804016&Signature=EIjH0N1cbqvz~vEOV5bXeCFttj7lxv8CPHsa1LFGDhJB0aJCbKYNrhJ28wAVXNzj5c0zK2VnTsHlI4lIJ1VVAd4KHjH52iMcy3YmJnp52FXwTDDwYqnHqovdTGVeYda2ltwnupUPIgyfGII9h5viYEm2BdSrMtwrycmGjijKt7k_&Key-Pair-Id=APKAJECH5M7VWIS5YZ6Q: net/http: TLS handshake timeout
For such "timeout" reasons, at least one scatter in each workflow was marked as "Failed" in the UI. For that reason each of the workflows failed.
