isomorphisms@scheherezade:~/spark/spark-2.4.3-bin-hadoop2.7$ bin/spark-shell --master local[2]
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.spark.SparkConf$.<init>(SparkConf.scala:716)
at org.apache.spark.SparkConf$.<clinit>(SparkConf.scala)
at org.apache.spark.SparkConf$$anonfun$getOption$1.apply(SparkConf.scala:389)
at org.apache.spark.SparkConf$$anonfun$getOption$1.apply(SparkConf.scala:389)
at scala.Option.orElse(Option.scala:289)
at org.apache.spark.SparkConf.getOption(SparkConf.scala:389)
at org.apache.spark.SparkConf.get(SparkConf.scala:251)
at org.apache.spark.deploy.SparkHadoopUtil$.org$apache$spark$deploy$SparkHadoopUtil$$appendS3AndSparkHadoopConfigurations(SparkHadoopUtil.scala:463)
at org.apache.spark.deploy.SparkHadoopUtil$.newConfiguration(SparkHadoopUtil.scala:436)
at org.apache.spark.deploy.SparkSubmit$$anonfun$2.apply(SparkSubmit.scala:334)
at org.apache.spark.deploy.SparkSubmit$$anonfun$2.apply(SparkSubmit.scala:334)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:334)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:143)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.UnknownHostException: scheherezade: scheherezade: Name or service not known
at java.net.InetAddress.getLocalHost(InetAddress.java:1505)
at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:946)
at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress$lzycompute(Utils.scala:939)
at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress(Utils.scala:939)
at org.apache.spark.util.Utils$$anonfun$localCanonicalHostName$1.apply(Utils.scala:996)
at org.apache.spark.util.Utils$$anonfun$localCanonicalHostName$1.apply(Utils.scala:996)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.util.Utils$.localCanonicalHostName(Utils.scala:996)
at org.apache.spark.internal.config.package$.<init>(package.scala:302)
at org.apache.spark.internal.config.package$.<clinit>(package.scala)
... 18 more
Caused by: java.net.UnknownHostException: scheherezade: Name or service not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getLocalHost(InetAddress.java:1500)
... 27 more
https://stackoverflow.com/a/26686229/563329
echo $HOSTNAME
on your terminal.scheherezade
), try to ping it : ping scheherezade
. If it returns ping: cannot resolve scheherezade: Unknown host
then add an entry into your /etc/hosts
file.cat >> /etc/hosts
and add following:127.0.0.1 scheherezade
in step 3, i typed: cat >> /etc/hosts , then i got:
bash: /etc/hosts: Permission denied
However, i typed: sudo nano /etc/hosts, then typed my password, and replace : 127.0.1.1 OLD_HOSTNAME (mine is hero) with 127.0.1.1 CURRENT_HOSTNAME ( it's the name in your terminal, right after '@' sign, e.g: shaun@iamshaun then CURRENT_HOSTNAME is iamshaun )
https://stackoverflow.com/a/26686229/563329
- Do
echo $HOSTNAME
on your terminal.- Whatever hostname it shows (lets say
scheherezade
), try to ping it :ping scheherezade
. If it returnsping: cannot resolve scheherezade: Unknown host
then add an entry into your/etc/hosts
file.- For that
cat >> /etc/hosts
and add following:
127.0.0.1 scheherezade
Thanks for the snippets :)
Most helpful comment
https://stackoverflow.com/a/26686229/563329
echo $HOSTNAME
on your terminal.scheherezade
), try to ping it :ping scheherezade
. If it returnsping: cannot resolve scheherezade: Unknown host
then add an entry into your/etc/hosts
file.cat >> /etc/hosts
and add following:127.0.0.1 scheherezade