Installing Talend Dictionary Service in cluster mode - Cloud

Talend Cloud Hybrid Installation Guide for Linux

Version
Cloud
Language
English (United States)
EnrichDitaval
linux
Product
Talend Cloud
Module
Talend Data Preparation
Talend Data Stewardship
Talend Management Console
Content
Installation and Upgrade

You can install Talend Dictionary Service in cluster mode and benefit from a high availability and a better scalability when working with semantic types in Talend Data Stewardship.

To install Talend Dictionary Service in cluster mode, you need to modify the <Tomcat>/conf/data-quality.properties configuration file.

To perform this installation, you need to install and configure as many instances of Talend Dictionary Service and its dependencies as necessary.

Before you begin

  • You have configured a Load Balancer for Talend Dictionary Service.
    Note: The same Load Balancer can be used for Talend Data Preparation, Talend Data Stewardship and Talend Dictionary Service. In addition, the Load Balancer can be either physical or logical.
  • You have configured MongoDB in cluster mode. For more information, see MongoDB documentation.
  • You have configured Kafka and Zookeeper in cluster mode. For more information, see Zookeeper documentation and Kafka documentation.

Procedure

  1. Install a first Talend Dictionary Service instance.
    For more information on the installation procedure, see Installing Talend Dictionary Service.
  2. In the <Tomcat>/conf/data-quality.properties file, edit the spring.data.mongodb.host property to specify the hosts and ports of the several MongoDB instances.
    Use the following syntax:
    spring.data.mongodb.host=<host1>:<port1>,<host2>:<port2>,...,<hostN>
    The hosts and ports for the different URLs must be concatenated, except for the last host, that will inherit the value of the spring.data.mongodb.port property. For example:
    spring.data.mongodb.host=mongorep-mongodb-replica-1.mongorep-mongodbreplica.
    default.svc.cluster.local:27017,
    mongorep-mongodb-replica-0.mongorep-mongodbreplica.
    default.svc.cluster.local:27017,
    mongorep-mongodb-replica-2.mongorep-mongodbreplica.
    default.svc.cluster.local:27017,
    mongorep-mongodb-replica-3.mongorep-mongodbreplica.
    default.svc.cluster.local
    mongodb.port=27017
  3. Edit the properties specifying the hosts and ports for the Kafka and Zookeeper instances.
    In the same way as the MongoDB URLs, the Kafka and Zookeeper hosts and ports must be concatenated, except for the last port, that is inherited from the dedicated properties.
    spring.cloud.stream.kafka.binder.brokers=host1:9092,host2:9092,host3
    spring.cloud.stream.kafka.binder.zkNodes=host1:2181,host2:2181,host3
    spring.cloud.stream.kafka.binder.defaultBrokerPort=9092
    spring.cloud.stream.kafka.binder.defaultZkPort=2181
  4. Repeat this installation and configuration procedure for each instance of Talend Dictionary Service that you want to install.

Results

You have installed several Talend Dictionary Service instances and configured them to work in cluster mode.