Skip to main content Skip to complementary content

Hiding sensitive information provided in the context of your Spark Job

When you execute your Talend Studio Jobs for Apache Spark on a Talend JobServer, if this Job uses a context and your user password is defined in this context, the Talend JobServer may fail to hide the password from its Talend CommandLine terminal.
Your Job runs on one of the following clusters:
  • Microsoft HDInsight
  • Google Cloud Dataproc
  • Cloudera Altus
  • Databricks
  • Qubole
  • All the other supported distributions when they run on Yarn Cluster.

Use tRunJob in a Standard Job to orchestrate your Spark Job.

Procedure

  1. Create a Standard Job and add tRunJob to it.
  2. Click the Context view to open its view and load the contexts to be used by your Spark Job.
    It is recommended to set up the contexts to be used under the Context node in the Repository of Talend Studio. This way, you can directly click the to import these contexts to your Job.
  3. Double-click tRunJob to open its Component view.

    Example

  4. Click the ... button next to the Job field.
  5. From the Context drop-down list, select the context to be used. The contexts that appear on this list are those you imported to this Job in the previous steps.
  6. Select the Use an independent process to run subjob check box to avoid memory limitation issue.
  7. Select the Transmit whole context check box to apply the all the context variables of this Standard Job on your Spark Job.
  8. If you need to add some supplementary context variables, use the Context Param table to add them.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!