Hi GATK Team,
I am trying to run th PreProcessing Best Practices on my own computer. What I have done are:
a) changed COMMENT1-COMMENT4 in the .inputs.json with the right absolute paths point to the files, and leave the rest COMMENT unchanged
b) run the command:
```
java -jar cromwell-36.jar run /PATH/TO/processing-for-variant-discovery-gatk4.wdl --inputs /PATH/TO/processing-for-variant-discovery-gatk4.b37.wgs.inputs.json
```
I got some questions/problems:
1) I installed docker and tried to pull genomes-in-cloud by:
```
docker pull broadintitute/genomes-in-cloud
```
but got:
```
Using default tag: latest
Error response from daemon: pull access denied for broadintitute/genomes-in-cloud, repository does not exist or may require 'docker login'
```
I tried to define the specific version for genomes-in-cloud but also got the same message, and tried to run the image, it would say the image was not found locally.
2) The pipeline created a lot of intermediate files, for example, unaligned.bam, unsort.bam etc. The uBAM file as the input is 14GB and the intermediate files can reach over 400GB, I know those files are the output of some tools and inputs for the other tools ,but is this size normal? The pipeline could not finish and I am still debugging, but I am wondering if all those intermediate files would be automatically removed if the pipeline was completed, since I found some script involved "rm" command in the execution folders.
3) The Best Practices is mainly for WGS analysis, but the description states it also suits for WES data with minor modification, do you have any suggestion about the "minor modification"
4) To fix the genomes-in-the-cloud issue, I am trying to get rid of docker (which is also due to that I need to run the pipeline on the cluster later), I found a post that state the pipeline should be able to run without docker but need to adapt some of the paths to executables based on your local setup. At this moment, I think I need download the contained tools in the image such as samtools and BWA, do you have any other suggestion for which variables in .wdl or .json should be changed, or do you have any existing script for local running without docker.
Thanks for your time,
Best
I am trying to run th PreProcessing Best Practices on my own computer. What I have done are:
a) changed COMMENT1-COMMENT4 in the .inputs.json with the right absolute paths point to the files, and leave the rest COMMENT unchanged
b) run the command:
```
java -jar cromwell-36.jar run /PATH/TO/processing-for-variant-discovery-gatk4.wdl --inputs /PATH/TO/processing-for-variant-discovery-gatk4.b37.wgs.inputs.json
```
I got some questions/problems:
1) I installed docker and tried to pull genomes-in-cloud by:
```
docker pull broadintitute/genomes-in-cloud
```
but got:
```
Using default tag: latest
Error response from daemon: pull access denied for broadintitute/genomes-in-cloud, repository does not exist or may require 'docker login'
```
I tried to define the specific version for genomes-in-cloud but also got the same message, and tried to run the image, it would say the image was not found locally.
2) The pipeline created a lot of intermediate files, for example, unaligned.bam, unsort.bam etc. The uBAM file as the input is 14GB and the intermediate files can reach over 400GB, I know those files are the output of some tools and inputs for the other tools ,but is this size normal? The pipeline could not finish and I am still debugging, but I am wondering if all those intermediate files would be automatically removed if the pipeline was completed, since I found some script involved "rm" command in the execution folders.
3) The Best Practices is mainly for WGS analysis, but the description states it also suits for WES data with minor modification, do you have any suggestion about the "minor modification"
4) To fix the genomes-in-the-cloud issue, I am trying to get rid of docker (which is also due to that I need to run the pipeline on the cluster later), I found a post that state the pipeline should be able to run without docker but need to adapt some of the paths to executables based on your local setup. At this moment, I think I need download the contained tools in the image such as samtools and BWA, do you have any other suggestion for which variables in .wdl or .json should be changed, or do you have any existing script for local running without docker.
Thanks for your time,
Best