To install Talend Data Preparation
in cluster mode, you need to make some additions to the
<Data_Preparation_Path>/config/application.properties
configuration file.
To perform this installation, you need to install and configure as many instances of
Talend Data Preparation, and its
dependencies, as necessary.
Before you begin
- Each product, namely Talend Data Preparation and optionally
Talend Dictionary Service,
requires the use of a Load Balancer.
- You have configured MongoDB in cluster mode. For more information, see the MongoDB documentation.
- You have configured Kafka and Zookeeper in cluster mode. For more information,
see the Zookeeper documentation and the Kafka documentation.
Procedure
-
Install a first Talend Data Preparation instance.
-
In the
<Data_Preparation_Path>/config/application.properties
file, edit the
mongodb.host property to specify the hosts and
ports of the several MongoDB instances.
Use the following syntax:
mongodb.host=<host1>:<port1>,<host2>:<port2>,…,<hostN>.
The hosts and ports for the different URLs must be concatenated, except for
the last host, that will inherit the value of the
mongodb.port property. For example:
mongodb.host=mongorep-mongodb-replica-1.mongorep-mongodb-replica.default.svc.cluster.local:27017,
mongorep-mongodb-replica-0.mongorep-mongodb-replica.default.svc.cluster.local:27017,
mongorep-mongodb-replica-2.mongorep-mongodb-replica.default.svc.cluster.local:27017,
mongorep-mongodb-replica-3.mongorep-mongodb-replica.default.svc.cluster.local
mongodb.port=27017
-
Edit the
service.cache.file.location and
dataset.content.store.file.location properties to specify the
location of your Network File System, or shared folder that must be available to
all the Talend Data Preparation
instances.
service.cache.file.location=sharedContent/
dataset.content.store.file.location=sharedContent/store/datasets/content/
-
Edit the properties specifying the hosts and ports for the Kafka and Zookeeper
instances.
In the same way as the MongoDB URLs, the Kafka and Zookeeper hosts and ports
must be concatenated, except for the last port, that is inherited from the
dedicated properties.
spring.cloud.stream.kafka.binder.brokers=host1:9092,host2:9092,host3
spring.cloud.stream.kafka.binder.zkNodes=host1:2181,host2:2181,host3
spring.cloud.stream.kafka.binder.defaultBrokerPort=9092
spring.cloud.stream.kafka.binder.defaultZkPort=2181
-
To increase the session duration and reduce the risk of unexpected logouts, add
the following lines:
security.token.renew-after=600
security.token.invalid-after=3600
-
Repeat this installation and configuration procedure for each instance of
Talend Data Preparation that you
want to install.
Results
The several Talend Data Preparation
instances have been installed and configured to work in cluster mode.