Recommended configuration - 7.3

Talend Data Management Platform Installation Guide for Linux

EnrichVersion
7.3
EnrichProdName
Talend Data Management Platform
EnrichPlatform
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend DQ Portal
Talend Identity and Access Management
Talend Installer
Talend JobServer
Talend Log Server
Talend Repository Manager
Talend Runtime
Talend SAP RFC Server
Talend Studio
task
Installation and Upgrade

Description

Location

Configuration property

Default/Minimal value

Recommended value

Maximum number of database connections in the Quartz connection pool

Talend Administration Center configuration file

"WEB-INF/classes/quartz.properties" :

org.quartz.dataSource.QRTZ_DS.maxConnections

30

MAX_CONCURRENT_TASK_EXECUTIONS + MAX_CONCURRENT_PLAN_EXECUTIONS

Maximum number of concurrent Jobs handled by the Scheduler

Talend Administration Center configuration file

"WEB-INF/classes/quartz.properties" :

org.quartz.threadPool.threadCount

30

MAX_CONCURRENT_TASK_EXECUTIONS + MAX_CONCURRENT_PLAN_EXECUTIONS

Maximum database connections for Talend Administration Center (apart from Quartz)

Talend Administration Center configuration file

"WEB-INF/classes/configuration.properties" :

hibernate.c3p0.max_size

32

MAX_CONCURRENT_TASK_EXECUTIONS + MAX_CONCURRENT_PLAN_EXECUTIONS + MAX_CONCURRENT_LOGGED_USERS

It defines the period between each remote Job check

Talend Administration Center database table configuration

scheduler.conf.taskStatusRefreshTime

1

MAX_CONCURRENT_TASK_EXECUTIONS / 500

Defines the size of thread pool which checks the latest executions at startup

Talend Administration Center database table configuration

dashboard.conf.taskExecutionsHistory.threadPoolSize

10

( MAX_CONCURRENT_TASK_EXECUTIONS + MAX_CONCURRENT_PLAN_EXECUTIONS ) / 25

Defines the size of thread pool which checks all the tasks at startup

Talend Administration Center database table configuration

scheduler.conf.simultaneousThreadsForStatusRefresh

5

MAX_CONCURRENT_TASK_EXECUTIONS / 50

Defines the number of maximum opened files for database process

Host of database server

Maximum opened files:

For example, under Linux set the Mysql configuration property "open_files_limit" and ensure that the system file limit is >= to the formula on the right

(depends on operating system)

( MAX_CONCURRENT_TASK_EXECUTIONS + MAX_CONCURRENT_PLAN_EXECUTIONS + MAX_CONCURRENT_LOGGED_USERS ) x 3

Defines the number of maximum connections allowed to the database

Database server

Max connections:

For example, set the Mysql configuration property "max_connections = 10000"

(depends on database vendor)

( MAX_CONCURRENT_TASK_EXECUTIONS + MAX_CONCURRENT_PLAN_EXECUTIONS + MAX_CONCURRENT_LOGGED_USERS ) x 3

Defines the maximum number of concurrent connections accepted by the JobServer

JobServer configuration file

"conf/TalendJobServer.properties" :

org.talend.remote.server.MultiSocketServer.

MAX_CONCURRENT_CONNECTIONS

1000

MAX_CONCURRENT_JOBS_EXECUTIONS x 2

Definition of variables used in the table above

Variable Description

MAX_CONCURRENT_JOBS_EXECUTIONS

Maximum expected number of concurrent executed Jobs on JobServer side

MAX_CONCURRENT_LOGGED_USERS

Maximum expected number of concurrent logged users (Talend Administration Center + Studio) on Talend Administration Center side

MAX_CONCURRENT_PLAN_EXECUTIONS

Maximum expected number of concurrent plan executions on Talend Administration Center side

MAX_CONCURRENT_TASK_EXECUTIONS

Maximum expected number of concurrent task executions on Talend Administration Center side

The following settings are recommended for a benchmark environment:
  • "WEB-INF/classes/quartz.properties" :

    org.quartz.dataSource.QRTZ_DS.maxConnections = 1000

    org.quartz.threadPool.threadCount = 1000"

  • WEB-INF/classes/configuration.properties" :

    hibernate.c3p0.max_size=1000

  • "conf/TalendJobServer.properties" :

    org.talend.remote.server.MultiSocketServer.MAX_CONCURRENT_CONNECTIONS=2000