Performance and tuning - 6.4

Talend Data Fabric Release Notes

Talend Data Fabric
Installation and Upgrade
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend DQ Portal
Talend ESB
Talend Identity Management
Talend Installer
Talend JobServer
Talend Log Server
Talend MDM Server
Talend MDM Web UI
Talend Project Audit
Talend Repository Manager
Talend Runtime
Talend SAP RFC Server
Talend Studio

The performance of reading and writing data records has been improved in general for MDM components which interact with an MDM server.

The improvement varies depending on the components you are using and the tuning configurations you have made:

  • If a Job uses the component tMDMBulkLoad to bulk load large volumes of data into MDM, you can tune the bulk load operation by providing the parameter bulkload.concurrent.http.requests to the JVM to limit the maximum number of concurrent requests sent to the MDM server.

  • If a Job interacts with the MDM server using any of the MDM components tMDMDelete, tMDMOutput, tMDMInput, tMDMRouteRecord, tMDMSP, tMDMViewSearch and tMDMBulkLoad, you are recommended to increase the timeout values before executing the Job to help process a large number of data records. You can customize two timeout values by providing the parameters ws_client_connection_timeout and ws_client_receive_timeout to the JVM.

  • When bulk loading large volumes of data into MDM, authorized users can configure two parameters bulkload.concurrent.database.requests and bulkload.concurrent.wait.milliseconds in the file <$INSTALLDIR>/conf/mdm.conf to avoid transaction and deadlock issues since too many threads on the server side are trying to obtain database connections.

  • For a cluster of MDM servers, an auto increment generator based on Hazelcast, which is a distributed in-memory data grid, is initialized automatically during the MDM server startup. You are allowed to change the basic Hazelcast configuration in the file <$INSTALLDIR>/conf/mdm.conf or make the advanced configuration in the file <TomcatPath>/webapps/talendmdm/WEB-INF/beans.xml. Note that the Hazelcast port is used only by MDM server nodes in cluster mode.

  • Authorized users can add a parameter -v to the CommandLine when running the dbmigration script for your system to perform only the validation of user-defined data models.