Scout: New index needed, with sort

[dnil]

2021-05-12 08:07:34,761 - scout.server.app - ERROR: Exception on /cust002/causatives [GET] [in /home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/flask/app.py:1892]
Traceback (most recent call last):
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
   response = self.full_dispatch_request()
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
   rv = self.handle_user_exception(e)
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
   reraise(exc_type, exc_value, tb)
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
   raise value
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
   rv = self.dispatch_request()
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
   return self.view_functions[rule.endpoint](**req.view_args)
 File "/home/hiseq.clinical/SERVER/apps/scout/git/scout/scout/server/utils.py", line 26, in decorated_function
   context = func(*args, **kwargs)
 File "/home/hiseq.clinical/SERVER/apps/scout/git/scout/scout/server/blueprints/institutes/views.py", line 143, in causatives
   for variant_obj in variants:
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/pymongo/cursor.py", line 1207, in next
   if len(self.__data) or self._refresh():
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/pymongo/cursor.py", line 1124, in _refresh
   self.__send_message(q)
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/pymongo/cursor.py", line 1001, in __send_message
   address=self.__address)
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
   exhaust=exhaust)
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
   return func(session, server, sock_info, slave_ok)
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
   unpack_res)
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/pymongo/server.py", line 135, in run_operation_with_response
   _check_command_response(first, sock_info.max_wire_version)
 File "/home/hiseq.clinical/miniconda/envs/prod/lib/python3.6/site-packages/pymongo/helpers.py", line 164, in _check_command_response
   raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: Executor error during find command: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit., full error: {'waitedMS': 0, 'ok': 0.0, 'errmsg': 'Executor error during find command: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.', 'code': 96}

[dnil]

Its not actually, really a bug, more like an oversight of an inefficiency leading to a very urgently needed enhancement. Potato-potato. If noone beats me to it I can perhaps make a PR with some of the less ginormous things from the other index PR.