Nextflow: Unparseable Data format error on staging file

[apeltzer]

Bug report

Trying to execute this pipeline: https://github.com/nf-core/ampliseq/blob/dev on sample files.

Expected behavior and actual behavior

The reference data variable should force a download of the URL https://www.arb-silva.de/fileadmin/silva_databases/qiime/Silva_132_release.zip and stage that file to a process that creates a reference database out of it.

However, I get this error here:

N E X T F L O W  ~  version 18.10.1
Launching `nf-core/ampliseq` [boring_noether] - revision: a584a06840 [dev]
=======================================================
                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~'
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

nf-core/ampliseq v1.0dev"
=======================================================
Pipeline Name  : nf-core/ampliseq
Pipeline Version: 1.0dev
Run Name       : boring_noether
Reads          : data
Max Memory     : 128 GB
Max CPUs       : 28
Max Time       : 2d
Output dir     : ./results
Working dir    : /beegfs/work/iiipe01/2018-12-06_RRNA/work
Container Engine: singularity
Container      : nfcore/ampliseq:latest
Current home   : /home-link/iiipe01
Current user   : iiipe01
Current path   : /beegfs/work/iiipe01/2018-12-06_RRNA
Script dir     : /home-link/iiipe01/.nextflow/assets/nf-core/ampliseq
Config Profile : binac
=========================================
[warm up] executor > pbs
[d7/33ff2d] Submitted process > get_software_versions
[2f/d7d1ed] Submitted process > fastqc (N85_S282)
ERROR ~ Error executing process > 'make_SILVA_132_16S_classifier (1)'

Caused by:
  Unparseable date: "Tue, 10 Apr 2018 13:39:14 GMT"


 -- Check '.nextflow.log' file for details
[ea/e2ce74] Submitted process > fastqc (N83_S178)
[nf-core/ampliseq] Pipeline Complete
WARN: Killing pending tasks (3)

Steps to reproduce the problem

nextflow run nf-core/ampliseq -profile test,docker will unfortunately not reproduce this, as the DB is not built in this small case to limit execution times for Travis CI. I can send you a testcase if this is required.

Program output

See nextflow.log

nextflow.log.txt

Environment

• Nextflow version: 18.10.1
• Java version: module load devel/java_jdk/1.8.0u112
• Operating system: Linux

[pditommaso]

It looks that the error is raised when hashing the (remote) file

java.text.ParseException: Unparseable date: "Tue, 10 Apr 2018 13:39:14 GMT"
    at java.text.DateFormat.parse(DateFormat.java:366)
    at nextflow.file.http.XFileSystemProvider.readHttpAttributes(XFileSystemProvider.groovy:392)
    at nextflow.file.http.XFileSystemProvider.readHttpAttributes(XFileSystemProvider.groovy:383)
    at nextflow.file.http.XFileSystemProvider.readAttributes(XFileSystemProvider.groovy:357)
    at java.nio.file.Files.readAttributes(Files.java:1737)
    at nextflow.util.CacheHelper.hashFile(CacheHelper.java:203)
    at nextflow.util.CacheHelper.hasher(CacheHelper.java:158)

However the following snippet works fine:

def x = file('https://www.arb-silva.de/fileadmin/silva_databases/qiime/Silva_132_release.zip')
println nextflow.util.CacheHelper.hasher(x).hash()

Can you try it?

[apeltzer]

No same error (tried in the nextflow console shell):

 def x = file('https://www.arb-silva.de/fileadmin/silva_databases/qiime/Silva_132_release.zip') 
groovy> println nextflow.util.CacheHelper.hasher(x).hash() 

Exception thrown

java.text.ParseException: Unparseable date: "Tue, 10 Apr 2018 13:39:14 GMT"
    at nextflow.file.http.XFileSystemProvider.readHttpAttributes(XFileSystemProvider.groovy:392)
    at nextflow.file.http.XFileSystemProvider.readHttpAttributes(XFileSystemProvider.groovy:383)
    at nextflow.file.http.XFileSystemProvider.readAttributes(XFileSystemProvider.groovy:357)
    at nextflow.util.CacheHelper.hashFile(CacheHelper.java:203)
    at nextflow.util.CacheHelper.hasher(CacheHelper.java:158)
    at nextflow.util.CacheHelper.hasher(CacheHelper.java:90)
    at nextflow.util.CacheHelper.hasher(CacheHelper.java:86)
    at nextflow.util.CacheHelper.hasher(CacheHelper.java:82)
    at nextflow.util.CacheHelper$hasher.call(Unknown Source)
    at ConsoleScript0.run(ConsoleScript0:2)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

[pditommaso]

Umm, let me guess, do you have german locale setting, right? 😬

[apeltzer]

Yes, Sir!

[iiipe01@login03 2018-12-06_RRNA]$ locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=
[iiipe01@login03 2018-12-06_RRNA]$

[pditommaso]

The date parsing function is locale depend, therefore it does not recognise the weekday string 😭

Temporary workaround specify the english locale on the command line:

nextflow -Duser.country=UK -Duser.language=en run .. etc

[apeltzer]

I changed it in my .bashrc on the cluster and it works as well:

Pipeline Name  : nf-core/ampliseq
Pipeline Version: 1.0dev
Run Name       : distraught_wiles
Reads          : data
Max Memory     : 128 GB
Max CPUs       : 28
Max Time       : 2d
Output dir     : ./results
Working dir    : /beegfs/work/iiipe01/2018-12-06_RRNA/work
Container Engine: singularity
Container      : nfcore/ampliseq:latest
Current home   : /home-link/iiipe01
Current user   : iiipe01
Current path   : /beegfs/work/iiipe01/2018-12-06_RRNA
Script dir     : /home-link/iiipe01/.nextflow/assets/nf-core/ampliseq
Config Profile : binac
=========================================
[warm up] executor > pbs
[a2/c77519] Submitted process > fastqc (N84_S281)
[d9/f76f7f] Submitted process > fastqc (N85_S282)
[eb/9c5d59] Submitted process > fastqc (N82_S280)
[f9/96500b] Submitted process > get_software_versions
[2e/69b641] Submitted process > fastqc (N83_S178)
[a5/38b77d] Submitted process > trimming (N82_S280)
[c6/5ec608] Submitted process > trimming (N85_S282)
[4d/9df2bb] Submitted process > trimming (N84_S281)
[69/4e5a21] Submitted process > trimming (N83_S178)
[f2/3de1b4] Submitted process > metadata_category_pairwise (1)
[9a/f56cff] Submitted process > metadata_category_all (1)
[85/1ba541] Submitted process > output_documentation (1)

[apeltzer]

Ok but we found the issue and its something fixable at some point & nothing too serious - thanks a bunch :-)

[pditommaso]

Included in version 18.12.0-edge.