Procedure
- Open the <Spark_Job_Server_Path>/settings.sh file.
-
Edit the following line to specify the IP address that you want
the Spark Job Server
service to use. The service will listen on this IP address.
sjs_host=localhost
-
Edit the following line to specify the port that you want the
Spark Job Server service
to use. The service will listen on this port.
sjs_port=8098
-
Edit the following line to specify the path to the Hadoop cluster
settings directory. Note that this must be a local path: you can obtain the
settings files (such as hdfs-site.xml,
mapred-side.xml, core-site.xml, and
yarn-site.xml) from your Hadoop cluster and copy them to
the machine where you installed Spark Job Server.
hadoop_conf_dir=/path/to/hadoop/cluster/settings/directory
-
To use Spark Job Server server with a
secure Hadoop cluster (using Kerberos), add the following line to the file. Note
that this must be a local path: you can obtain the
krb5.conf file from your Hadoop cluster and copy it to the
machine where you installed Spark Job Server.
krb5.config=/path/to/Kerberos/configuration/file/krb5.conf
- Save your changes to the settings file.
- Restart Spark Job Server for your changes to be taken into account.