Bloop: Bloop doesn't respect proxy env-vars

[japgolly]

I'm using OSX and have a local cntlm proxy running. These env vars are normally enough for everything:

ALL_PROXY=http://127.0.0.1:8080
FTP_PROXY=http://127.0.0.1:8080
HTTPS_PROXY=http://127.0.0.1:8080
HTTP_PROXY=http://127.0.0.1:8080
RSYNC_PROXY=http://127.0.0.1:8080
all_proxy=http://127.0.0.1:8080
ftp_proxy=http://127.0.0.1:8080
http_proxy=http://127.0.0.1:8080
https_proxy=http://127.0.0.1:8080
no_proxy=localhost,127.0.0.*,10.*,192.168.*,...
rsync_proxy=http://127.0.0.1:8080

but with 1.3.0-RC1 Bloop is ignoring those and only honouring the SOCKS proxy in the network adapter in Network Preferences.

If I go to Network Preferences, edit the adapter, go to the Proxies page, select SOCKS and set its url to http://127.0.0.1 and port to 1080 (yes, 1080 not 8080 is correct), Bloop seems to detect it and I get:

> bloop server -Xss=2m                                                            
Defaulting on nailgun port 8212
There is no server running at port 8212
Starting the bloop server... this may take a few seconds
Running /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server as a jar...
Shelling out with '['java', '-jar', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']' ...
Error: Invalid or corrupt jarfile /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server
Running /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server as a script...
Shelling out in Unix system with ['sh', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
Detected java options are ''
Error while downloading https://repo1.maven.org/maven2/io/get-coursier/coursier-cli_2.12/1.1.0-M14-4/coursier-cli_2.12-1.1.0-M14-4-standalone.jar: Can't connect to SOCKS proxy:http://127.0.0.1, ignoring it
Bloop server in /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server failed to run.
First invocation attempt: ['java', '-jar', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
-> Return code: 1
Second invocation attempt: ['sh', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
-> Return code: 255

If I change url to http://127.0.0.1:1080 and clear the port field I get:

> bloop server -Xss=2m
Defaulting on nailgun port 8212
There is no server running at port 8212
Starting the bloop server... this may take a few seconds
Running /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server as a jar...
Shelling out with '['java', '-jar', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']' ...
Error: Invalid or corrupt jarfile /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server
Running /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server as a script...
Shelling out in Unix system with ['sh', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
Detected java options are ''
Error while downloading https://repo1.maven.org/maven2/io/get-coursier/coursier-cli_2.12/1.1.0-M14-4/coursier-cli_2.12-1.1.0-M14-4-standalone.jar: Can't connect to SOCKS proxy:http://127.0.0.1:1080, ignoring it
Bloop server in /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server failed to run.
First invocation attempt: ['java', '-jar', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
-> Return code: 1
Second invocation attempt: ['sh', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
-> Return code: 255

If I disable the system SOCKS proxy I get:

> bloop server -Xss=2m
Defaulting on nailgun port 8212
There is no server running at port 8212
Starting the bloop server... this may take a few seconds
Running /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server as a jar...
Shelling out with '['java', '-jar', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']' ...
Error: Invalid or corrupt jarfile /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server
Running /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server as a script...
Shelling out in Unix system with ['sh', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
Detected java options are ''
Error while downloading https://repo1.maven.org/maven2/io/get-coursier/coursier-cli_2.12/1.1.0-M14-4/coursier-cli_2.12-1.1.0-M14-4-standalone.jar: repo1.maven.org, ignoring it
Bloop server in /usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server failed to run.
First invocation attempt: ['java', '-jar', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
-> Return code: 1
Second invocation attempt: ['sh', '/usr/local/Cellar/bloop/1.3.0-RC1/bin/blp-server', '-Xss=2m']
-> Return code: 255

This is with and without the proxy env vars set.

One thing I've had to do before was have my JVM-based apps check for env vars like http_proxy, and then when found set system (not env) vars to have the JVM respect them. Maybe a similar approach would be helpful here? Something like:

final case class Proxy(url: Option[URL], noProxy: List[String]) {

  def setAsJvmProxy(): Unit = {
    for (u <- url) {
      val host = u.getHost
      val port = u.getPort.toString
      for (scheme <- List("http", "https")) {
        System.setProperty(s"$scheme.proxyHost", host)
        System.setProperty(s"$scheme.proxyPort", port)
      }
    }
    noProxy match {
      case Nil => ()
      case _   => System.setProperty("http.nonProxyHosts", noProxy.mkString("|"))
    }
    ()
  }

[jvican]

@japgolly Thanks for such a detailed issue!

You're not passing the JVM options correctly to bloop. The Build Server guide in our docs explains that you need to append -J.

Bloop currently honors http_proxy and https_proxy et al. Check the file ProxySetup.scala.

The issue at hand seems to be caused by the python script not passing the environment variables to the process that shells out to blp-server. The way we run bloop is different in macOS than Linux/Windows (macOS uses coursier launch, which is the one causing problems), so this is most likely why it doesn't work. I'll try to fix in a bit.

[jvican]

I've pushed a commit fixing this issue.

[jvican]

Alright, false alarm, it doesn't. I don't understand why you're having these proxy issues. Can you investigate further based on the links I gave you above and the current support Bloop has for proxies @japgolly ?

[japgolly]

Hi @jvican, firstly congrats on 1.3.0 release btw! It's an awesome feat of work!! 👏

Ok I read through ProxySetup and played around with it a bit locally and I think I know the problem. On object initialisation, it populates the configuredProxySettings map but it doesn't apply them to the JVM. There's a updateProxySettings method which calls System.setProperty but there are no System.setProperty calls on init. I think when getEnvProxyProperties returns a result that will be used it should also call System.setProperty to instate it, else it will just go into the map and not be in use :)

[jvican]

Thanks for the kind words @japgolly!

Regarding your analysis, it's true we don't call that logic when we initialize the server, but we don't need to yet because on server initialization we don't download anything, we only force download of deps via coursier (for things like the compiler bridge) when a client asks for it, and in that case we do set the properties.

I think something else is at play here. If you look at your logs, you see you get a coursier error. I bet that error comes from the script we set up in macOS to launch coursier, which we install under /usr/local/Cellar/Bloop/1.3.0/bin/blp-server:

#!/usr/bin/env sh

java_opts=()
server_args=()
for arg in "$@"; do
  if [[ $arg == -J* ]]; then
    java_opts+=(${arg#"-J"})
  else
    server_args+=($arg)
  fi
done

echo "Detected java options are '${java_opts[@]}'"
BASE_BIN_DIR=$(dirname "$0")
COURSIER_BIN="$BASE_BIN_DIR/blp-coursier"
/usr/libexec/java_home -v 1.8 -F -R --exec java \
   "${java_opts[@]}" -jar "$COURSIER_BIN" launch ch.epfl.scala:bloop-frontend_2.12:1.3.0 \
  -r bintray:scalameta/maven \
  -r bintray:scalacenter/releases \
  -r https://oss.sonatype.org/content/repositories/staging \
  --main bloop.Server -- "${server_args[@]}"

This is the script called by the python script implementing bloop server. We system process it inheriting the environment of the caller of bloop server using check_call so, in theory, it should have access to your environment variables.

Can you try running coursier in your terminal and see if you can reproduce it?

[jvican]

@japgolly Is this issue no longer a problem given that you've managed to use build tracing in the latest 1.3.0? :smile:

[japgolly]

Alas no, it's cos I switched computers and was using bloop from home where no proxies are necessary :laughing: I'll be back in at EvilCorp on Wednesday where good men and women cower and proxies roam the lands unchallenged, I'll try your coursier suggestion then and report back.

[japgolly]

Hi @jvican, I've just tried running that coursier command directly and it fails. It's coursier that's lacking detection/propagation of env-var based proxy settings!

[japgolly]

Hmmm, this gets a bit worse. It seems that Coursier itself doesn't respect the JVM proxy.

> /usr/libexec/java_home -v 1.8 -F -R --exec java -Dhttp{,s}.proxyHost=http://127.0.0.1 -Dhttp{,s}.proxyPort=8080 -jar "$COURSIER_BIN" launch ch.epfl.scala:bloop-frontend_2.12:1.3.2 -r bintray:scalameta/maven -r bintray:scalacenter/releases -r https://oss.sonatype.org/content/repositories/staging --main bloop.Server --

Resolution error: Error downloading ch.epfl.scala:bloop-frontend_2.12:1.3.2
  not found: /Users/d883476/.ivy2/local/ch.epfl.scala/bloop-frontend_2.12/1.3.2/ivys/ivy.xml
  download error: Caught java.net.UnknownHostException: repo1.maven.org (repo1.maven.org) while downloading https://repo1.maven.org/maven2/ch/epfl/scala/bloop-frontend_2.12/1.3.2/bloop-frontend_2.12-1.3.2.pom
  download error: Caught java.net.UnknownHostException: dl.bintray.com (dl.bintray.com) while downloading https://dl.bintray.com/scalameta/maven/ch/epfl/scala/bloop-frontend_2.12/1.3.2/bloop-frontend_2.12-1.3.2.pom
  download error: Caught java.net.UnknownHostException: dl.bintray.com (dl.bintray.com) while downloading https://dl.bintray.com/scalacenter/releases/ch/epfl/scala/bloop-frontend_2.12/1.3.2/bloop-frontend_2.12-1.3.2.pom
  download error: Caught java.net.UnknownHostException: oss.sonatype.org (oss.sonatype.org) while downloading https://oss.sonatype.org/content/repositories/staging/ch/epfl/scala/bloop-frontend_2.12/1.3.2/bloop-frontend_2.12-1.3.2.pom

[jvican]

Looks like coursier is at fault here, @japgolly you might want to follow up with @alexarchambault to figure out what's happening.