Default values in Big Data run profiles - Cloud

Talend Cloud Management Console for Pipelines User Guide

author
Talend Documentation Team
EnrichVersion
Cloud
EnrichProdName
Talend Cloud
task
Administration and Monitoring > Managing projects
Administration and Monitoring > Managing users
Deployment > Deploying > Executing Tasks
Deployment > Scheduling > Scheduling Tasks
EnrichPlatform
Talend Management Console

The following table lists the default values set in the Big Data run profiles.

You can overwrite certain parameter values in the ADVANCED CONFIGURATION section while creating a run profile. To change the values, add the parameter listed in the Overridable column of the table below and set the desired value in the form in Talend Cloud Management Console.

Property name Default value Category Overridable
Micro-batch interval 5000 ms Basic configuration N/A
Streaming timeout 10 ms Basic configuration N/A
Yarn queue - Executors spark.yarn.queue
Driver memory 512 MB Driver spark.driver.memory
Driver cores 1 Driver spark.driver.cores
Yarn strategy Dynamic Driver spark.dynamicAllocation.enabled
Initial number of executors 1 Executors spark.dynamicAllocation.initialExecutors
Min number of executors 0 Executors spark.dynamicAllocation.minExecutors
Max number of executors 0 Executors spark.dynamicAllocation.maxExecutors
Cores per executor 1 Executors spark.executor.cores
Executor memory 512 MB Executors spark.executor.memory
Executor memory overhead 384 MB Advanced configuration spark.executor.memoryOverhead
Checkpointing enabled Advanced configuration N/A
spark.submit.deployMode cluster N/A N/A
Temporary storage /tmp Basic configuration spark.local.dir