Default values in Databricks run profiles - Cloud

Talend Cloud Management Console for Pipelines User Guide

Talend Documentation Team
Talend Cloud
Administration and Monitoring > Managing projects
Administration and Monitoring > Managing users
Deployment > Deploying > Executing Tasks
Deployment > Scheduling > Scheduling Tasks
Talend Management Console

The following table lists the default values set in the Databricks run profiles.

You can overwrite certain parameter values in the ADVANCED CONFIGURATION section while creating a run profile. To change the values, add the parameter listed in the Overridable column of the table below and set the desired value in the form in Talend Cloud Management Console.

Property name Default value Category Overridable
spark_version 5.5.x-scala2.11 Basic configuration N/A
Micro-batch interval 5000 Basic configuration N/A
node_type_id Standard_DS3_v2 (Azure) or m4.large (AWS) Basic configuration N/A
destination dbfs:/tpd-cluster-logs Basic configuration N/A
num_workers 2 Basic configuration N/A
Log destination /tmp/beam-staging- Basic configuration N/A
StorageLevel MEMORY_AND_DISK Runtime set N/A