Define the advanced settings - 6.3

How to Use Spark 2.0 with Studio 6.3

EnrichVersion
6.3
task
Design and Development > Designing Jobs > Hadoop distributions
Design and Development > Designing Jobs > Job Frameworks > Spark Batch
Design and Development > Designing Jobs > Job Frameworks > Spark Streaming
EnrichPlatform
Talend Studio
Define Spark advanced settings in the Studio to read Spark 2.0 jar files in your cluster.

Procedure

  1. In the Advanced properties table, to add a row, click the plus symbol (+).
  2. In the Property column, in double quotation marks, enter spark.yarn.jars. This parameter provides the names of jar files to be used by your Spark Job, as well as the paths to them in your cluster.
  3. In the Value column, in double quotation marks, enter the names of the jar files and their directories. In this example, since these jar files are centralized in the usr/lib/spark/jars directory, enter "usr/lib/spark/jars/*".