Quantcast
Channel: Ask the FireCloud Team — GATK-Forum
Viewing all articles
Browse latest Browse all 1147

Failure in firecloud despite success locally

$
0
0

Hi--

I have a method that succeeds locally yet fails when run in FireCloud (discussed this with @Ruchi at Fc office hours).

I wrote the docker file being used in this method (image available here: tmajarian/epacts-mkl-dstat:latest, see below for dockerfile) but have only been able to get it running locally. See the wdl below for a simple test case that succeeds locally.

task test_epacts {

    command {
        /tmp2/EPACTS-3.2.6/bin/test_run_epacts.sh
    }

    runtime {
        docker: "tmajarian/epacts-mkl-dstat@sha256:0533a755da58883de35beb801db382015167c122fa3eecf126d1073e986d1530"
        memory: "10 GB"
        disks: "local-disk 20 SSD"
        bootDiskSizeGb: 10
    }

  output {
        Array[File] out = glob("**/*.pdf")
    }
}

workflow w {
    call test_epacts
}

I am currently getting this failure from FC:

message: Task w.test_epacts:NA:1 failed. JES error code 5. Message: 10: Failed to delocalize files: failed to copy the following files: "/mnt/local-disk/glob-444572a90afc2dab3b2d14c9b0564dcd.list -> gs://fc-98b9dbc7-6dcf-493d-b4fc-cd3dcb844b1f/acd13331-7fb1-4e8a-a725-cc130569e24d/w/99f3ee14-9036-47f3-b0ce-cec301b841d2/call-test_epacts/glob-444572a90afc2dab3b2d14c9b0564dcd.list (cp failed: gsutil -q -m cp -L /var/log/google-genomics/out.log /mnt/local-disk/glob-444572a90afc2dab3b2d14c9b0564dcd.list gs://fc-98b9dbc7-6dcf-493d-b4fc-cd3dcb844b1f/acd13331-7fb1-4e8a-a725-cc130569e24d/w/99f3ee14-9036-47f3-b0ce-cec301b841d2/call-test_epacts/glob-444572a90afc2dab3b2d14c9b0564dcd.list, command failed: CommandException: No URLs matched: /mnt/local-disk/glob-444572a90afc2dab3b2d14c9b0564dcd.list\nCommandException: 1 file/object could not be transferred.\n); /mnt/local-disk/test_epacts-rc.txt -> gs://fc-98b9dbc7-6dcf-493d-b4fc-cd3dcb844b1f/acd13331-7fb1-4e8a-a725-cc130569e24d/w/99f3ee14-9036-47f3-b0ce-cec301b841d2/call-test_epacts/test_epacts-rc.txt (cp failed: gsutil -q -m cp -L /var/log/google-genomics/out.log /mnt/local-disk/test_epacts-rc.txt gs://fc-98b9dbc7-6dcf-493d-b4fc-cd3dcb844b1f/acd13331-7fb1-4e8a-a725-cc130569e24d/w/99f3ee14-9036-47f3-b0ce-cec301b841d2/call-test_epacts/test_epacts-rc.txt, command failed: CommandException: No URLs matched: /mnt/local-disk/test_epacts-rc.txt\nCommandException: 1 file/object could not be transferred.\n); /mnt/local-disk/glob-444572a90afc2dab3b2d14c9b0564dcd/* -> gs://fc-98b9dbc7-6dcf-493d-b4fc-cd3dcb844b1f/acd13331-7fb1-4e8a-a725-cc130569e24d/w/99f3ee14-9036-47f3-b0ce-cec301b841d2/call-test_epacts/glob-444572a90afc2dab3b2d14c9b0564dcd/ (cp failed: gsutil -q -m cp -L /var/log/google-genomics/out.log /mnt/local-disk/glob-444572a90afc2dab3b2d14c9b0564dcd/* gs://fc-98b9dbc7-6dcf-493d-b4fc-cd3dcb844b1f/acd13331-7fb1-4e8a-a725-cc130569e24d/w/99f3ee14-9036-47f3-b0ce-cec301b841d2/call-test_epacts/glob-444572a90afc2dab3b2d14c9b0564dcd/, command failed: CommandException: No URLs matched: /mnt/local-disk/glob-444572a90afc2dab3b2d14c9b0564dcd/*\nCommandException: 1 file/object could not be transferred.\n)"

Both stderr and stdout are empty from the firecloud job, yet they are present locally. See attached for the local stdout, full jes log, and dockerfile.

I have already gone through disk issues in a similar wdl but the docker image does include dstat. Any ideas would be awesome! Thanks.


Viewing all articles
Browse latest Browse all 1147

Trending Articles