WDL tasks localize during JES runs using WDL like so:
task myTask {
File myFileToLocalize
Array[File] otherFilesToLocalize
...
command <<<
#process ${myFileToLocalize} ....
#process ${otherFilesToLocalize sep="somesep"} ...
>>>
...
also, WDL tasks can specify disk size for attached disk to help ensure that there is sufficient disk-space for files like so (in the example a user-configurable setting):
task diskTask {
Int diskGB
.....
runtime {
disks: "local-disk ${diskGB} HDD"
...
It would seem like a good idea to have a special variable to automatic allocation of disk size of localized files. For example instead of the call being like
call diskTask {
input:
...
diskGB=100
}
or
call diskTask {
input:
...
diskGB=prepTask.calcDisk
}
or
call diskTask {
input:
...
diskGB=size(someInputFile)
}
that maybe the variable could be like :
call bigTask {
inputs:
fileOne=someFileOfUnknownSize,
fileTwo=anotherFileOfUnknownSize,
diskGB=autoSize()+autoSize()*0.2
}
where "autoSize" automatically calculates sizes for files localized and adds them and I did some "+*" to add some factor from that (to have space for output files from the run) (in this case 20% of the disk for the output)
I wonder about people's thoughts on this?