Scout: Scout too slow to work in

[NicoleCMMS]

Scout has become so slow that it is no long possible to analyse our patients. Some parts of Scout are working without any delay but when we click on a variant it can take up to one minute for the variant page to open.

[moonso]

We're looking at it @NicoleCMMS , hopefully it will be fixed soon

[dnil]

We have found an issue with the total index size being larger than the memory allocated to us on the server. Fingers crossed we can get some more!

[henrikstranneheim]

does @ingkebil know?

[dnil]

He's in touch with IT!

[henrikstranneheim]

👍

[dnil]

What do you think now @NicoleCMMS @AnnHam? It is not super snappy, but tentatively we are at a sustainable performance? @ingkebil also has a conversation with IT about letting our LIMS live on another machine until we can migrate to a new server farm.

[NicoleCMMS]

Yes I agree it is much faster than it was earlier today. We can at least analyse some patients at this speed 😊 Thanks for working on this so fast!

Nicole

Från: Daniel Nilsson [e-mail]
Skickat: den 27 september 2019 14:56
Till: Clinical-Genomics/scout [e-mail]
Kopia: Nicole Lesko [e-mail]; Mention [e-mail]
Ämne: Re: [Clinical-Genomics/scout] Scout too slow to work in (#1337)

What do you think now @NicoleCMMShttps://github.com/NicoleCMMS @AnnHamhttps://github.com/AnnHam? It is not super snappy, but tentatively we are at a sustainable performance? @ingkebilhttps://github.com/ingkebil also has a conversation with IT about letting our LIMS live on another machine until we can migrate to a new server farm.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Clinical-Genomics/scout/issues/1337?email_source=notifications&email_token=AFK2GT4ZXRFQ3JYN45AX4F3QLX7EHA5CNFSM4I3DH4X2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7YZSVQ#issuecomment-535927126, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFK2GT63XKRH6PU3RHBQYYTQLX7EHANCNFSM4I3DH4XQ.

[dnil]

Thanks everyone! Closing this issue for now, but do get back if things are unbearable again. We hope this will be sorted more long term in a month or so. More hardware that will help has already been bought.

[northwestwitch]

I'm reopening this because it's still slow. In a rustic and horrible way I've been analyzing what is causing this behavior and I think I might have found the cause.
To do so I wrote some code that counts the microseconds it takes to execute each function in the variant view (see branch variant_find_bottleneck) that is on stage.

The specific code that takes most time is this:

if add_other:
        for other_variant in store.other_causatives(case_obj, variant_obj):
            # This should work with old and new ids
            case_id = other_variant['case_id']
            other_case = store.case(case_id)
            if not other_case:
                continue
            other_variant['case_display_name'] = other_case.get('display_name', case_id)
            other_causatives.append(other_variant)

[ingkebil]

Can you explain that code for people who are not familiar with scout?

Just throwing some ideas around:
Would it be possible to move whatever this piece of code does and load it after the page is displayed? Back in the day, that would've been an AJAX call. I'm not familiar with what fancy technology you kids are playing these days ;)

On other news, I have asked IT to give me an estimate on the following:

• time and complexity to move LIMS off of the server onto anything else we have
• add more RAM

Everyone is at the facility forum, so expect no response before Wednesday.

[ingkebil]

I can up the memory again to 50GB and disable all non-essential services on that server. This means e.g. shutting down the beacon.

Is that alright?

[dnil]

Nice @northwestwitch!

@ingkebil, it lists other cases which have the same variant as a causative variant. Postponing that querying is a general enough way to deal with it, but not quite satisfactory: a user may miss out on a matching causative if they look at the page too short a time. They will have to learn to wait until that process finishes. Possible, but not fun.

But, e.g. changing that store.other_causatives to do more of what check_causatives does should help. A sneak peak says it does close to the same thing, but lets a mongo query do the heavy lifting rather than pyton looping over a list of variants?

[northwestwitch]

@ingkebil I would try to optimize the code first. @dnil an even faster way to get all causatives display_names and cases (and even links to the other causative variants) would be to query the event collection using the 'mark_causative' verb. That collection contains exactly what we need I think. I'll try tomorrow!

[moonso]

Nice job @northwestwitch! Don’t forget to consider “unmark causative”