React-select: [v2] react-select didn't work for long lists

[kahwee]

Hi there

I'm testing out react-select v2 and I noticed performance issues once when I have a long lists of 300,000 items. I wanted to know if there is any interest to build in something like react-virtualized into react-select so users can scroll through long lists without jank or is this out of scope of this project.

Thanks.

[roginfarrer]

You can either build this yourself, or use @bvaughn's react-virtualized-select. It's not yet build on v2, though.

[bvaughn]

Someone could also build an integration with react-window. That would be pretty nifty.

[tmacdonald]

react-window is working well. I just replaced MenuList with a FixedSizeList.

It definitely got sluggish at 300k items, and the first time I tried it I got a potential out-of-memory warning.

[bvaughn]

It definitely got sluggish at 300k items, and the first time I tried it I got a potential out-of-memory warning.

The react-window version got sluggish with 300k items?

I wouldn't expect this. Initially, I would expect the sluggishness and out-of-memory issues to be more related to e.g. parsing 300k items worth of JSON or something.

[bvaughn]

Btw, interested in sharing your integration? I'd love to get a Code Sandbox version I could link to from react-window docs.

[tmacdonald]

@bvaughn https://codesandbox.io/s/lxv7omv65l

[bvaughn]

Thank you! I think the sluggishness in this case is related to the filtering. This is the same reason I released react-select-fast-filter-options for the older version of react-select

[tmacdonald]

Cool. I'll take a look at that.

[paradoxxxzero]

I finally managed to integrate js-search from react-select-fast-filter-options in react select v2 for anyone interested, it's not optimal but it's a lot faster.
See: https://github.com/Kozea/formol/blob/master/src/async/SelectMenuField.jsx

[M1K3Yio]

@tmacdonald @paradoxxxzero I've looked over both of your solutions, both are great. I am currently using a similar implementation like paraoxxxzero version. I noticed that in both if you search and use the down arrow, once you get past the initial view-able values the list doesn't auto scroll. Do either of you any any solutions or have noticed this?

[Janpot]

Another small issue with @tmacdonald solution is that the menu doesn't shrink when there's small amount of options when typing in the search box.

PS: I slightly modified the solution with VariableSizeList to account for option groups

import React from 'react';
import Select from 'react-select';
import { VariableSizeList as List } from 'react-window';

const GROUP_HEADER_HEIGHT = 13;
const ITEM_HEIGHT = 34;

function MenuList (props) {
  const { options, getValue } = props;
  const [ value ] = getValue();

  const initialOffset = options.indexOf(value) * ITEM_HEIGHT;

  const children = React.Children.toArray(props.children);

  function getOptionSize (option) {
    if (option.options) {
      return option.options.length * ITEM_HEIGHT + GROUP_HEADER_HEIGHT;
    }
    return ITEM_HEIGHT;
  }

  function getItemSize (i) {
    return getOptionSize(options[i]);
  }

  const totalHeight = options.reduce((height, option) => {
    return height + getOptionSize(option);
  }, 0);

  const estimatedItemSize = totalHeight / options.length;

  return (
    <List
      height={Math.min(totalHeight, 300)}
      itemCount={children.length}
      itemSize={getItemSize}
      estimatedItemSize={estimatedItemSize}
      initialScrollOffset={initialOffset}
    >
      {({ index, style }) => <div style={style}>{children[index]}</div>}
    </List>
  );
}

[jnachtigall]

Also in the solution by @tmacdonald as in https://github.com/JedWatson/react-select/issues/2850#issuecomment-410318717 the scrollbar does not adapt to the position. In the given Codesandbox press ArrowDown key 15 times: You'll see that the scrollbar does not scroll with the focussed element.

I am not sure how this would be fixed tbh. Probably one would need to sync the react-window List position inside with the focussed element of MenuList. But it seems that the focussed element is not part of the props received by MenuList (https://react-select.com/props#menulist) So there's no way to do the syncing.

[paradoxxxzero]

I managed to fix this as you can see in this exemple by doing this. This scrolls to the selected item on props change.

[zachgibson]

@paradoxxxzero That example works really well, but what are listDefaultHeight and listApproximatedLengthBreak? I’m using a bigger font size for my select so I need to change these values to work.

[paradoxxxzero]

These are needed to approximate the height of a wrapping option text, since the height computation has to happen before the render. https://github.com/bvaughn/react-window/issues/6 should fix this mess.

[Kepro]

great work here! any working solution for optgroup? :)

[jacobworrel]

The proposed solutions do a good job of boosting performance but it still starts to get slow with more than 10K items. I identified two bottlenecks here and here.

It would be great if there was some way of opting out of these two expensive operations if the react-select consumer is using some sort of windowing solution like react-window.

[Guichaguri]

@paradoxxxzero Any chance that you release your solution as an independent package?

[Kepro]

@jacobworrel 10K in dropdown? waste of server time and browser time... change design of your application

[jacobworrel]

@Guichaguri I published a standalone package that draws heavily from @paradoxxxzero's react-window integration (without optimized filtering yet) here.

[melchin]

I tried to convert the old react-virtualized-select to something compatible with react-select v2.
I managed to get something rendering, but it is only non-laggy for arrow scrolling with lists with less than 1500 items.

It will scroll to the proper index while scrolling with keys.
https://codesandbox.io/s/l75lw5l5l7

[jacobworrel]

@Kepro by 10K, i mean 10,000

[jacobworrel]

@melchin I'm pretty sure this is an issue with the Select component from react-select and not your code. The Select component calls setState on option hover, causing a rerender. On every render the renderMenu method gets called, which iterates over every item in the list, creating a new React element for each item, which is the source of the bottleneck. I'm coming across the same limitations when trying to integrate react-select with react-window.

[carloslfu]

For anyone coming here, just found react-windowed-select by @jacobworrel, it works fine 👍.

[melchin]

@carloslfu That solution is extremely laggy at 30,000 options. Even 10,000 options isn't ideal.
https://codesandbox.io/s/0zl8444vw

[carloslfu]

Yeah, you are right @melchin. It speeded up things for my use case with 2000 options, but at 10000 it's so slow.

[akphi]

@M1K3Yio

For the scrolling with arrow key problem, here's my solution

[m-mattu]

react-virtualized-select from @jacobworrel worked for my project. Largest select in the project is 1,000 records

[liorbrauer]

I'm not completely positive on this, but it seems part of the reason react-select is slow for large lists of options even when rendering the options with react-window components is that react-select still gets ALL options and needs to process them all (see here: https://github.com/JedWatson/react-select/blob/98f469eeb34c689f2697ffabd43c920077130bbc/packages/react-select/src/Select.js#L1275). I think ideally we'd want to only pass the visible (+overscan) options to react-select in order for it to be performant. Just leaving my thoughts here if it's useful for anyone, I'll take a stab at this later.

[jacobworrel]

@liorbrauer you're absolutely correct - buildMenuOptions is the main reason for the component being slow/laggy with large lists and windowing/virtualization doesn't solve that (it only mitigates the general performance problem by limiting the number of DOM nodes added to the document). as noted, the implementations incorporating react-window including mine (react-windowed-select) max out somewhere between 5000-10000 items. as @Kepro noted, if you're dealing with lists above 1000 items, you should probably consider using the Async select which loads the options in asynchronously as the user types/filters.

[liorbrauer]

@jacobworrel Oh I just read your previous comment, you were describing the same issue. Thanks for replying.

Going async is also a viable option, but I think it should be possible to also render 10k+ in options in a performant way if react-select can incorporate some caching on the built options and only update when needed (or alternatively only update a subset when needed?).

[jacobworrel]

@liorbrauer I agree - it would be great to optimize the buildMenuOptions function so it isn't such a drag on performance. @gwyneplaine any thoughts on the direction we could take to accomplish this?

[liorbrauer]

@jacobworrel and @gwyneplaine

So looking at the code here:

https://github.com/JedWatson/react-select/blob/98f469eeb34c689f2697ffabd43c920077130bbc/packages/react-select/src/Select.js#L394-L400

I'm curious why line 398 (nextProps.menuIsOpen !== menuIsOpen) is needed?

The output of buildMenuOptions can be cached and only be rebuilt if one of the other conditions is true. As it stands today, it's basically building and destroying the Menu Options every time the Select is opened / closed, which probably isn't needed.

[guillaume86]

So I have the same issue, and it was working mostly fine in 1.x.
Is there a way to disable the CSS in JS system?
Would that work by just replacing components?
I think the main issue is the hover handling (it looks like it is re-rendering way too much stuff when hovering items).

[jacobworrel]

@guillaume86

there is a hack/workaround to disable the mouse events but this leads to other issues: https://github.com/JedWatson/react-select/issues/3128

[melissachang]

Has anyone gotten @tmacdonald solution to work when options are variable heights:

Sadly, I can't use react-windowed-select because I'm using AsyncSelect. react-windowed-select appears to only support sync react-select.

[ebonow]

Greetings, I will be closing this ticket as a duplicate for https://github.com/JedWatson/react-select/issues/3128 which is still receiving posting activity. We will be continuing to monitor performance for menu rendering.