To install Talend Data Preparation in cluster
mode, you need to make some modifications in the <Data_Preparation_Path>/config/application.properties
configuration file.
To perform this installation, you need to install and configure as many instances of
Talend Data Preparation and its dependencies as necessary.
Procedure
-
Install a first Talend Data Preparation instance.
-
In the <Data_Preparation_Path>/config/application.properties
file, edit the
mongo.host
property to specify the hosts and
ports of the several MongoDB instances.
Use the following
syntax:
spring.data.mongodb.host=<host1>:<port1>,<host2>:<port2>,...,<hostN>
The
hosts and ports for the different URLs must be concatenated, except for the
last host, that will inherit the value of the mongo.port
property. For
example:
mongodb.host=mongorep-mongodb-replica-1.mongorep-mongodb-replica.default.svc.cluster.local:27017,
mongorep-mongodb-replica-0.mongorep-mongodb-replica.default.svc.cluster.local:27017,
mongorep-mongodb-replica-2.mongorep-mongodb-replica.default.svc.cluster.local:27017,
mongorep-mongodb-replica-3.mongorep-mongodb-replica.default.svc.cluster.local
mongodb.port=27017
-
Edit the
service.cache.file.location
and
dataset.content.store.file.location
properties to specify
the location of your Network File System, or shared folder that must be
available to all the Talend Data Preparation instances. For
example:
service.cache.file.location=sharedContent/
dataset.content.store.file.location=sharedContent/store/datasets/content/
-
Edit the properties specifying the hosts and ports for the Kafka and ZooKeeper
instances.
In the same way as the MongoDB URLs, the Kafka and ZooKeeper hosts and ports must
be concatenated, except for the last port, that is inherited from the dedicated
properties.
spring.cloud.stream.kafka.binder.brokers=host1:9092,host2:9092,host3
spring.cloud.stream.kafka.binder.zkNodes=host1:2181,host2:2181,host3
spring.cloud.stream.kafka.binder.defaultBrokerPort=9092
spring.cloud.stream.kafka.binder.defaultZkPort=2181
-
To increase the session duration and reduce the risk of unexpected logouts, add
the following lines:
security.token.renew-after=600
security.token.invalid-after=3600
-
Repeat this installation and configuration procedure for each instance of
Talend Data Preparation that you want to install.
Results
The several
Talend Data Preparation instances have been installed
and configured to work in cluster mode.