Scout: Case does not display BAM

[47KW]

Seems like some (all) cases delivered lately lack the BAM-files or do not connect them to the cases? Analysis date 2020-08-16
I have problems with:
F0035793
F0035649
F0035665

My colleague could not see the BAM´s in IGV on these cases either.

[northwestwitch]

Oh no, sounds bad, we'll check ASAP!

[dnil]

Hm, the only obvious thing I noticed on the housekeeper bundles was a higher hard link count than before. Something changed in how they are moved around?

[henrikstranneheim]

No I do not think so

[henrikstranneheim]

Here is a copy of the deepmarmot cram:

samtools idxstats test.cram
1   249250621   66285611    122090
2   243199373   70952832    134781
3   198022430   56122053    93406
4   191154276   54254613    87154
5   180915260   51418046    85369
6   171115067   48687123    81492
7   159138663   45139248    80060
8   146364022   42238126    73512
9   141213431   34526183    63710
10  135534747   37878262    70006
11  135006516   38072486    71619
12  133851895   37975134    65569
13  115169878   27494009    44245
14  107349540   25693021    45748
15  102531392   24353500    45709
16  90354753    25985181    52042
17  81195210    23156145    50124
18  78077248    21618988    37182
19  59128983    16726511    39867
20  63025520    17408470    36916
21  48129895    10938200    20599
22  51304566    10486982    25829
X   155270560   22793848    40087
Y   59373566    5237984 7631
MT  16569   518034  1417
GL000207.1  4262    0   0
GL000226.1  15008   0   0
GL000229.1  19913   0   0
GL000231.1  27386   0   0
GL000210.1  27682   0   0
GL000239.1  33824   0   0
GL000235.1  34474   0   0
GL000201.1  36148   0   0
GL000247.1  36422   0   0
GL000245.1  36651   0   0
GL000197.1  37175   0   0
GL000203.1  37498   0   0
GL000246.1  38154   0   0
GL000249.1  38502   0   0
GL000196.1  38914   0   0
GL000248.1  39786   0   0
GL000244.1  39929   0   0
GL000238.1  39939   0   0
GL000202.1  40103   0   0
GL000234.1  40531   0   0
GL000232.1  40652   0   0
GL000206.1  41001   0   0
GL000240.1  41933   0   0
GL000236.1  41934   0   0
GL000241.1  42152   0   0
GL000243.1  43341   0   0
GL000242.1  43523   0   0
GL000230.1  43691   0   0
GL000237.1  45867   0   0
GL000233.1  45941   0   0
GL000204.1  81310   0   0
GL000198.1  90085   0   0
GL000208.1  92689   0   0
GL000191.1  106433  0   0
GL000227.1  128374  0   0
GL000228.1  129120  0   0
GL000214.1  137718  0   0
GL000221.1  155397  0   0
GL000209.1  159169  0   0
GL000218.1  161147  0   0
GL000220.1  161802  0   0
GL000213.1  164239  0   0
GL000211.1  166566  0   0
GL000199.1  169874  0   0
GL000217.1  172149  0   0
GL000216.1  172294  0   0
GL000215.1  172545  0   0
GL000205.1  174588  0   0
GL000219.1  179198  0   0
GL000224.1  179693  0   0
GL000223.1  180455  0   0
GL000195.1  182896  0   0
GL000212.1  186858  0   0
GL000222.1  186861  0   0
GL000200.1  187035  0   0
GL000193.1  189789  0   0
GL000194.1  191469  0   0
GL000225.1  211173  0   0
GL000192.1  547496  0   0
NC_007605   171823  0   0
hs37d5  35477943    0   0
*   0   0   0

[northwestwitch]

If the files are not found then IGV should show a message. I'm inspecting the javascript code from the page and it doesn't return any error either.

[henrikstranneheim]

However, trying to read chromosome 1 from the file takes ages...

[dnil]

There is a weird entry in the scout load config giving both bam_path, alignment_path and mt_bam:

bam_path: /home/proj/production/housekeeper-bundles/deepmarmot/2020-08-15/ACC7051A8_lanes_1234_sorted_md_brecal_subsample_MT.bam
capture_kit:
alignment_path: /home/proj/production/housekeeper-bundles/deepmarmot/2020-08-15/ACC7051A8_lanes_1234_sorted_md_brecal.cram
chromograph: /home/proj/production/housekeeper-bundles/deepmarmot/2020-08-15/ACC7051A8_lanes_1234_sorted_md_brecal_tcov_chromograph.tar.gz
father: '0'
mother: '0'
mt_bam: /home/proj/production/housekeeper-bundles/deepmarmot/2020-08-15/ACC7051A8_lanes_1234_sorted_md_brecal_subsample_MT.bam

[dnil]

I guess this issue goes two ways. One for cg to revert whatever added the extra bam_path when there is both a cram and a subsample bam. And we add an extra check to the scout load to make sure this is reported in error - or handled with precedence for the cram,

[northwestwitch]

There is a weird entry in the scout load config giving both bam_path, alignment_path and mt_bam:

This is highly suspicious! We should fix the config files of the newly uploaded cases. And we could fix the ones that don't work at the database level

[henrikstranneheim]

Original cram looks alright to me. I cannot find anything wrong with it. Maybe, this is only a scout config thing. But then when did that change?

[dnil]

Original cram looks alright to me. I cannot find anything wrong with it. Maybe, this is only a scout config thing. But then when did that change?

There is a mixup at some point about the bam_file pointing to the reduced mt_bam, and alignment_file to the nuclear cram. CG (or is there some passthrough from MIP?) needs a little attention there. And scout should ofcourse handle the situation with conflicting input better.

[ielvers]

We have the same issue with our cases that were delivered yesterday and today:
20149
20157
16106
20144

The cases delivered before that are fine.
Thank you for working on it :)

[dnil]

To summarise: @henrikstranneheim found that it appears to be a side effect from this change to another system 11 days ago: https://github.com/Clinical-Genomics/cg/pull/674, and @northwestwitch notes that there are 36 cases affected. A fix for future cases is being reviewed #2014. Bear with us a few h and we should hopefully have this fixed. I will not say I hope you have other old cases lined up to work on, so lets leave it at hoping there is nothing super urgent on that list.

[northwestwitch]

It's 36 cases in total. I'll fix the path to the bam files manually.

• [x] _id:"acehen"
  display_name:"19274"

• [x] _id:"aptbison"
  display_name:"20035"

• [x] _id:"exactbear"
  display_name:"F0032316"

• [x] _id:"usablecondor"
  display_name:"F0012453-2"

• [x] _id:"readyfly"
  display_name:"F0033650-2"

• [x] _id:"famouscoyote"
  display_name:"F0016518-3"

• [x] _id:"comicspider"
  display_name:"F0014143-2"

• [x] _id:"helpedbream"
  display_name:"F0017095-2"

• [x] _id:"fleetbedbug"
  display_name:"20149"

• [x] _id:"soughtrat"
  display_name:"F0035794"

• [x] _id:"deepmarmot"
  display_name:"F0035793"

• [x] _id:"goodsatyr"
  display_name:"F0035667"

• [x] _id:"manypig"
  display_name:"F0035649"

• [x] _id:"briefibex"
  display_name:"F0035639"

• [x] _id:"grandgannet"
  display_name:"F0035638"

• [x] _id:"restedimp"
  display_name:"F0032219"

• [x] _id:"lovingcattle"
  display_name:"F0035668"

• [x] _id:"chieftoucan"
  display_name:"F0035666"

• [x] _id:"ablepiglet"
  display_name:"F0035665"

• [x] _id:"caringboar"
  display_name:"F0034761"

• [x] _id:"busypigeon"
  display_name:"F0034540"

• [x] _id:"wiredcalf"
  display_name:"F0034216"

• [x] _id:"chiefstork"
  display_name:"20157"

• [x] _id:"keenfox"
  display_name:"F0035810"

• [x] _id:"hardyasp"
  display_name:"F0035795"

• [x] _id:"toughmarten"
  display_name:"F0035602"

• [x] _id:"firstkrill"
  display_name:"F0035819"

• [x] _id:"likedlizard"
  display_name:"F0035815"

• [x] _id:"onesawfly"
  display_name:"F0035703"

• [x] _id:"alerttahr"
  display_name:"F0035455"

• [x] _id:"nativehound"
  display_name:"F0035669"

• [x] _id:"wiseserval"
  display_name:"16106"

• [x] _id:"comicboar"
  display_name:"CV011"

• [x] _id:"clearmoose"
  display_name:"F0035597"

• [x] _id:"cleandodo"
  display_name:"F0035063"

• [x] _id:"easybadger"
  display_name:"20144"

[dnil]

Reopening to track the remaining cases, but in principle there is a fix in place for future cases from next release.

[northwestwitch]

For reference: this is the query for finding cases with MT bam files in the bam_file field : {"individuals.bam_file" : /_MT.bam$/ }

[northwestwitch]

Found 2 new cases uploaded yesterday when the bug was still in place: F0000130-2(selectpoodle, cust002), and CV011 (comicboar, cust010). I've fixed them now.

Checked a case uploaded today and looks fine, so I'm closing this now.