Minimap2: dbvar ground truth to compare minimap2 and ngmlr for SV

[ramcn]

Hello,

I am trying to compare the minimap2+sniffles combination with ngmlr+sniffles on nanopore rel5 dataset. [[link](https://s3.amazonaws.com/nanopore-human-wgs/rel5-guppy-0.3.0-chunk10k.fastq.gz)]

Below is the SV classification comparison obtained using SURVIVOR tool on the VCF file obtained using the 2 combinations.

It looks like both are quite comparable in finding different SVs. But I am trying to validate the results by comparing with suitable ground truth from dbvar. Could I get some guidance on how to go about validating the SV's detected.

[mdkeehan]

bedtools is handy for this sort of thing but in general it is not easy. Transform your SV's into a bed track. Compute a bedtools track intersection from NGMLR to dbvar and Minimap2 to dbvar.

[lh3]

Thanks, @ramcn. This is the first time I see such a comparison. Just curious: what minimap2 version were you using?

As @mdkeehan said, the simplest way to evaluate is to use bedtools to find the intersection between dbvar and minimap2/ngmlr calls. This usually gives you an ok impression overall. Careful/publication-quality evaluation is much harder, though.

[ramcn]

Thanks for the pointers. I will try out the suggested method and get back.

I am using the latest code on git and version shows up as 2.11-r819-dirty.

[lh3]

That is almost v2.12, which should give identical alignment to v2.11. Let me know what you find in the end. I have not evaluated minimap2+sniffles. I guess there should still be room for improvements. Thanks.

[ramcn]

Hi,

I did an intersection of minimap2+sniffles combination on dbvar dataset . Below is the methodology used and results. Kindly let me know if this is along the expected lines.

1. Convert the dbvar bed file to VCF.
   SURVIVOR bedtovcf GRCh38.nr_deletions.bed DEL GRCh38.nr_deletions.vcf
2. Find an intersection of the above vcf file and minimap2+sniffles output.
   SURVIVOR merge sample_files 1000 2 1 1 0 10 sample_merged.vcf
3. Count the number of SV of different length.
   SURVIVOR stats sample_merged.vcf 10 100 -1 res
4. The 2nd, 3rd and 4th column reports these counts for DEL, DUP and INS counts which were found in dbvar.

[lh3]

Thanks a lot for this analysis. Unfortunately, I am not familiar with SV evaluations. I am not sure what to expect when we compare an SV callset to dbVar. I will leave this thread open and let others chime in.

[ramcn]

Sure. Hopefully some one can give some insights and better ideas to evaluate wrt to ground truth.

Some of the numbers might be a bit off because my method is still work in progress. But what I understand is there are about 8K DELs and 12K INS which have been discovered by long read tech like pacific bio. Dbvar has a collection of these SV and can be used to evaluate the accuracy of a sequencing and variant calling pipeline.

Source: https://www.pacb.com/wp-content/uploads/Whitepaper-Human-Structural-Variation.pdf

[lh3]

I still don't know the best practice to use dbVar for benchmarking SV calls. You might be interested in a recent preprint on a related topic.

I am closing this for now. Please feel free to reopen or add more posts to this thread.