Dada2: Updating DADA2-formatted reference databases

[guidohooiveld]

Hi,
I noticed Patrick Schloss cq. the MOTHUR team recently made available the MOTHUR-compatible SILVA v128 and RDP v16 reference files. Both are 'newer' than the ones currently available for use with DADA2.

Would it be possible to reformat / make available these newer files for DADA2 on Zenodo as well?

Thanks,
Guido

[adityabandla]

+1 with the request above

Further, along similar lines, is it valid to assign taxonomies to DADA2 sequence variants, say using the 99_otus.fa file, which comes as a part of the QIIME formatted SILVA database?

We just started a discussion a few days back on the QIIME1 forum, pertaining to this
https://groups.google.com/forum/#!topic/qiime-forum/Cy5IXBvzXqE

Best,
Aditya

[joey711]

Ideally you use the reference sequences with their full resolution. Summarizing to 99% or less resolution can result in lost details relative to the information that is actually present in your data.

[joey711]

On the RDP site I just see:

RDP Release 11, Update 5 :: September 30, 2016

Is 11.5 what you meant by release 16?

[adityabandla]

@joey711 Thanks Joey for the pointers. Working on creating a 100% clustered file now

Perhaps @guidohooiveld was referring to the SILVA 128 release available here https://www.mothur.org/w/images/b/b4/Silva.nr_v128.tgz

Can this be formatted and made available for DADA2? Also if the steps required to create the formatted files for use with DADA2 can be made available, it would be great

[cdiener]

While converting HITDB a while ago I wrote a function that converted mothur dbs to DADA2. Dunno if it would work here but might be worth a shot...

[guidohooiveld]

@joey711 : I mixed things up: MOTHUR hosts a reformatted version of the "The Bacteria and Archaea hierarchy model used by RDP Classifier and RDP Hierarchy Browser [which] have been updated to training set No. 16. This new training set has over 300 new genera and 2000 new sequences added. There are some rearrangements in genera Gp1, Gp3 and Gp4 of the Acidobacteria due to addition of recently proposed new genera."
This training set is indeed based on RDP Release 11, Update 5 :: September 30, 2016. Source: Release 11 Update History.

@joey711 : I got confused now by your remark on "ideally you use the reference sequences with their full resolution." Since the MOTHUR team always reformat the "Ref NR 99" SILVA reference files, why are these then also used with DADA2? Or am I misisng something obvious...?

@adityabandla : yes, that is indeed the reformatted SILVA 128 file I was referring to; thanks.

@cdiener : thanks for pointing to your function.

[benjjneb]

DADA2-formatted fasta files for Silva version 128 and RDP trainset 16 are now available on Zenodo, and from the dada2 website.

In addition, the processing scripts to produce these re-formatted files from the original data-sources are now included as private functions w/in the dada2 R package. See the taxonomy.R file of the current (1.5.1) package version for the code.

[benjjneb]

Further, along similar lines, is it valid to assign taxonomies to DADA2 sequence variants, say using the 99_otus.fa file, which comes as a part of the QIIME formatted SILVA database?

@adityabandla The current way we recommend taking advantage of the high-resolution from DADA2 sequence variants for taxonomic assignment is to use the assignSpecies or addSpecies methods w/in the dada2 package.

These methods use exact matching to assign genus-species binomial names where possible.

The standard k-mer methods used for taxonomic assignment are necessarily fuzzy. While exact matching is limited by the reference database, we see 1/3-1/2 of abundant sequence variants in the human microbiome assigned this way using current databases. Furthermore, exact matching has a durable meaning -- you exactly matched a classified representative of that species -- that remains true as these databases are expanded in the future (not necessarily true for fuzzy matches).

The one downside, our current implementation is slow. This is a known issue #225 and we are working to address it.

[benjjneb]

Is 11.5 what you meant by release 16?

RDP has a separate "trainset" it uses for its Naive Bayesian Classifier (current version is 16) and full database release (current version is 11.5).