Netmiko: Improve performance/run time

[MichalTaratuta]

I have just realised that the ssh connection method ConnectHandler(**data) takes ~4s !!! any chance to improve it? When trying to run with Ansible this just creates 7s !!! tasks which in my opinion is not satisfactory.

[ktbyers]

SSH screen-scraping is going to be slow. There is a very strong trade-off between how fast you do it and reliability (i.e. if you try to be very aggressive on timing, things start breaking).

You can probably try setting global_delay_factor: .5 or to .25 and see if the performance improves.

I might also try to restructure the timeout code to allow you to be more aggressive on the timing (i.e. allow you to override the defaults and execute faster). There are somethings I currently do that cause global_delay_factor to get overridden when trying to go faster (i.e. you want to go faster, but the code doesn't allow you to).

[ddfrathb]

The send_command() by default expects the prompt - but it searches the entire output every line rather than each new line added to the output. Is this intentional or would it be worth changing?

Also, for the connection handler, would it not also be reasonable to expect the prompt rather than waiting for some time?

We're doing some additional testing but wondering f you intentionally do not do this for a reason.

[ktbyers]

@ddfrathb I don't think that is worth changing. That will be incredibly fast relative to the IO-wait times that we already have (i.e. the amount of time we spend interacting with the remote network device).

I think what is there is pretty reasonable. In general, it does this:

        self._test_channel_read()
        self.set_base_prompt()
        self.disable_paging()
        self.set_terminal_width()

So it tests it can read the channel and sets the router prompt (which inherently has an aspect of finding the current router prompt). Also at the beginning you have to dynamically figure out what the prompt is (before you can use it).

In general, Netmiko is optimized for reliability not performance which I think is the right choice when dealing with screen-scraping and automation.

[johnarnold]

@ddfrathb I noticed the search every line issue and logged it with suggested fix in https://github.com/ktbyers/netmiko/issues/427

If somebody has the time to code it up I can test it. I just haven't had time to code it myself.

[ktbyers]

@johnarnold I have been pretty swamped this summer...hoping to have more time to revamp a few things in the fall.

[ddfrathb]

Hey @johnarnold just read #427 and is essentially the exact reason we have an issue. We have some routers with hundreds of thousands of routes and millions of lines of output. Thanks for noticing me :D I will follow up and we have implemented a basic fix similar to the one you suggested of just reading the last n bytes for the prompt.

[ktbyers]

Implemented improved send_command() in PR #910 (feel free to test and comment there).

Also fast_cli=True can be passed to ConnectHandler which implements a different set of performance improvements.