Skip to main content

Spark mode

A method on which a Spark Job runs. A Talend Spark Job can be run in the following modes:
  • Local: Talend Studio builds the Spark environment in itself at runtime to run the Job locally in Talend Studio.
  • Standalone: Talend Studio connects to a Spark-enabled cluster to run the Job from this cluster.
  • YARN client: Talend Studio runs the Spark driver to orchestrate how the Job should be run and then send the orchestration to the YARN service of a given Hadoop cluster so that the Resource Manager of this YARN service requests execution resources accordingly.
  • YARN cluster: Talend Studio submits Jobs to and collects the execution information of your Job from YARN and ApplicationMaster. The Spark driver runs on the cluster and can run independently from Talend Studio.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!

Leave your feedback here