Double-click tHDFSConfiguration to open its
Component view. Note that tHDFSConfiguration is used because the Spark Yarn client mode is used to run Spark Jobs in this scenario.
Spark uses this component to connect to the HDFS system to which the jar files dependent on the Job are transferred.
- In the Version area, select the Hadoop distribution you need to connect to and its version.
- In the NameNode URI field, enter the location of the machine hosting the NameNode service of the cluster. If you are using WebHDFS, the location should be webhdfs://masternode:portnumber; if this WebHDFS is secured with SSL, the scheme should be swebhdfs and you need to use a tLibraryLoad in the Job to load the library required by the secured WebHDFS.
- In the Username field, enter the authentication information used to connect to the HDFS system to be used. Note that the user name must be the same as you have put in the Spark configuration tab.