Pry: TAB autocomplete is super-slow

[jjgh]

• ruby 2.3.0p0 (2015-12-25 revision 53290) [x64-mingw32]
• Win 7 64bit

Hi, I am seeing that the TAB auto-complete is dramatically slow.
Is this a know problem? Maybe it has already been addressed in a commit and only not just released in an updated gem yet?
Or should I start analyzing my system for problems?

Thanks

[jjgh]

sure:

pry -v
Pry version 0.10.3 on Ruby 2.3.0

Not a rails application, a simple pry console I start from console with the command "pry".

For testing purposes I decided to try the "dbscan" gem (implementing the DBSCAN machine learning algorithm) at https://github.com/matiasinsaurralde/dbscan.

So, in the pry console I require the "dbscan" gem, then use it like in the basic example at https://github.com/matiasinsaurralde/dbscan/blob/master/examples/simple.rb.

At that point I have a "dbscan.results" Hash object.

If I type "dbscan.results." and then I try to TAB-autocomplete that, then it will hang for about 5 seconds before it finds nothing and beeps. If I press TAB again then it hangs for another 5 seconds before showing "show all 3981 possibilities?"

I am not sure I can help with inspecting the code, I could try to take a look at it...

I hope this helps, thanks!

[mvz]

It seems the slow tab completion occurs when the receiver is not a regular variable. In the above case, it will be whatever dbscans.results returns.

As mentioned in #1446, Pry will use a different search for possible methods in this case, because it can't actually look at the receiver object itself (because dbscans.results hasn't been executed yet).

This search can take a long time if there are many modules already in the system, since Pry will iterate over all of them: https://github.com/pry/pry/blob/67c0b7fcd7ee81dede7d50d028afbb592ab83cbe/lib/pry/input_completer.rb#L171.

[dgutov]

will use a different search for possible methods in this case

IRB does something similar as well, but Pry completion is much slower in this case, while yielding the same results. Here's a benchmark that doesn't require installing third-party gems.

Launch pry, and inside it:

# Create a lot of modules, with one unique method per module.
# Whether the method names are unique or not, doesn't affect Pry completion speed much, BTW.
N = 20000

# Save the references to avoid GC.
modules = (1..N).map do |i|
  Module.new do
    module_eval <<-DEF
      def foo#{i}
      end
DEF
  end
end; nil

require 'benchmark'

Benchmark.bmbm do |x|
  x.report("Pry") { _pry_.complete("aaa.") }
end

If we try it with N = 40000 as well, we'll get results like:

| # of modules | Pry |
|--------------|------|
| 20000 | 0.93 |
| 40000 | 2.08 |

[dgutov]

I'm sorry everyone, please disregard the previous measurements, they were made against a modified version of IRB (I'll edit the messages to avoid further confusion). The problem here are the Module#name calls, this method is relatively slow for anonymous modules.

And they are not needed here, strictly speaking. The name is only used to blacklist a number of modules, and if we construct the list (or set!) of blacklisted modules once and cache it, that speeds up all subsequent completions, a lot. We also never need to refresh it, considering it only targets core modules.

EDIT: Constructing the blacklist in advance would require loading IRB first. However, scanning the namespaces this way is very fast, relatively speaking, so we don't really have to cache it.

[mvz]

Thanks for diving into this, @dgutov!

[dgutov]

With Pry 0.11 released, we can probably close this.