Kibana: Allow http/https proxy for plugin installation

[seang-es]

Customers behind secure networks are unable to use 'kibana plugin --install' to install Kibana plugins. They need the ability to configure an http or https proxy to be used for the calls to the plugin repositories.

[stormpython]

@seang-es would we be able to implement a solution that uses HTTP Connect tunnelling?

[stormpython]

After discussion with @seang-es, it would seem that users behind secure networks have their own http proxy servers and would simply like the ability to direct Kibana through these proxies.

The users would like to modify an option in the kibana.yml to set the host and port to the proxy server.

[sumioka]

I need authenticate proxy setting, too.
I am looking forward to your implementation to be done.

[rashidkpc]

Yeah, you don't want CONNECT here, just plain old http proxies

[simianhacker]

@seang-es How does this work with other plugin installers like bin/plugin in Elasticsearch? I don't see CLI options for their installers to set up a authenticated proxy? Also after thinking about this shouldn't this be taken care of at the OS level?

[stormpython]

After a discussion with @spalger and @rashidkpc, the reason we don't want to use CONNECT here is because many companies don't support it. So here are the proposed options:

1. See if this is really needed by our users or if there are other work arounds that would allow users to accomplish the same thing. @seang-es, can you respond to @simianhacker's comments so that we have a better understanding of how the customer gets around this for Elasticsearch plugins.
2. Rewrite installedPlugins module to use node request module instead of wreck.js since wreck does not support proxying.
3. Write a http proxy plugin that extends wreck.js.

Of the 3, if this is a much needed feature, I am most comfortable with implementing option 2.

[stormpython]

After further discussion, we have decided not to add http/https proxying for plugin installs in Kibana. The main reason for this is because Elasticsearch does not support it either, and there is a work around for doing offline installs using the file installer.

For example:

bin/kibana plugin --install --url file:///home/username/plugin.tar.gz

_Note_: You must use absolute paths

So, I am removing the P1 label and closing this issue.

[stormpython]

I submitted issue #5998 to update the Kibana plugin documentation to include file installs from a local directory.

[avallen]

One of the reasons given above for not supporting this directly is invalid, because elasticsearch's plugin command _does_ support specifying proxies, albeit via the corresponding java system properties, as documented at:
https://www.elastic.co/guide/en/elasticsearch/reference/1.6/modules-plugins.html

bin/plugin -DproxyHost=host_name -DproxyPort=port_number --install mobz/elasticsearch-head

[simianhacker]

@avallen Does downloading the file and then installing it manually work for you?

[stachdude]

This so called workaround doesn't seem to work

$bin/kibana plugin --install --url file:///opt/kibana-4.4.1-linux-x64/marvel-latest.tar.gz
Invalid install option. Please use the format <org>/<plugin>/<version>.

Plus not allowing proxy, and/or offline plugin installs kills basically 90% of organisations from using this version...

[stachdude]

The correct way for offline install is:

bin/kibana plugin -i marvel -u file:///tmp/marvel-latest.tar.gz

[MrCoder]

@stormpython The PI label should be removed, but shall it not be kept open?

[mpe]

This breaks Dockerfiles that otherwise work correctly behind a HTTP proxy.

[peske]

@stormpython - ridiculous explanation for several reasons:

• Elasticsearch DOES support outgoing proxy (I know because it is only way for me to install plugins);
• No reasonable administrator will allow unrestricted outgoing traffic from its datacenter (servers), so you should expect that plugin installation will go through an outgoing proxy. Basically if Kibana cannot install plugins through outgoing proxy, it is next to not being able to install them at all.
• Workaround you've suggested significantly complicates administration and configuration management (i.e. Puppet, Chef). And introduces confusion: for example I'm not sure if my troubles installing sense plugin (see issue 7400) are caused by installation from file (your workaround) or not.

Guyz, discussed or not, decided or not, you should reconsider this again. Only one decision is the right one (the smart one), and you've missed it.

[epixa]

Consistency across the projects for installation is important, so we're going to look into the proxy support in Elasticsearch.

[yodog]

using puppet and behind a corp firewall, i can install plugins for elasticsearch and logstash (but not kibana) by

(1) setting an environment variable in the exec instance

exec {
    "$name":
        command     => $command,
        creates     => $creates,
        environment => [ "http_proxy=http://1.2.3.4:3128" ],
        logoutput   => $logoutput,
        onlyif      => $onlyif,
        path        => ["/bin", "/sbin", "/usr/bin", "/usr/sbin"],
        returns     => [0,74],
    ;
}

or by

(2) passing proxy paramaters to java.

"${bin_plugin} -DproxyHost=1.2.3.4 -DproxyPort=3128 install -b --verbose $name"

[barryib]

Here is a try #7967 to add proxy support for plugin installation

[timroes]

This has been fixed with #12753 and will be released with 6.1. The plugin command will understand the http_proxy, https_proxy and no_proxy environment variables then.

[fbaligand]

Great news @timroes !
Big Thanks for that !