Configuring Spark Job Server - 7.3

Talend Installation Guide

Version
7.3
Language
English
Operating system
Linux
Subscription type
Subscription
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend DQ Portal
Talend ESB
Talend Identity and Access Management
Talend Installer
Talend JobServer
Talend LogServer
Talend MDM Server
Talend MDM Web UI
Talend Repository Manager
Talend Runtime
Talend SAP RFC Server
Talend Studio
Content
Installation and Upgrade
Last publication date
2022-10-30

Procedure

  1. Open the <Spark_Job_Server_Path>/settings.sh file.
  2. Edit the following line to specify the IP address that you want the Spark Job Server service to use. The service will listen on this IP address.
    sjs_host=localhost
  3. Edit the following line to specify the port that you want the Spark Job Server service to use. The service will listen on this port.
    sjs_port=8098
  4. Edit the following line to specify the path to the Hadoop cluster settings directory. Note that this must be a local path: you can obtain the settings files (such as hdfs-site.xml, mapred-side.xml, core-site.xml, and yarn-site.xml) from your Hadoop cluster and copy them to the machine where you installed Spark Job Server.
    hadoop_conf_dir=/path/to/hadoop/cluster/settings/directory
  5. To use Spark Job Server server with a secure Hadoop cluster (using Kerberos), add the following line to the file. Note that this must be a local path: you can obtain the krb5.conf file from your Hadoop cluster and copy it to the machine where you installed Spark Job Server.
    krb5.config=/path/to/Kerberos/configuration/file/krb5.conf
  6. Save your changes to the settings file.
  7. Restart Spark Job Server for your changes to be taken into account.