Installing Talend Dictionary Service manually

Talend Real-time Big Data Platform Installation Guide for Linux

EnrichVersion
6.4
EnrichProdName
Talend Real-Time Big Data Platform
task
Installation and Upgrade

This procedure contains the steps to manually install Talend Dictionary Service on your machine. For the automatic installation procedure using Talend Installer, see Using Talend Installer graphical installation mode.

Prerequisites:

Preparing your installation

  1. Download Apache Kafka from https://kafka.apache.org/downloads and install it.

    For more information on the supported Apache Kafka version, see Compatible Messaging Systems.

    For more information on how to install it, see Apache Kafka documentation.

  2. Download a MongoDB instance from https://www.mongodb.com/download-center and install it.

    For more information on the supported MongoDB databases, see Compatible Databases.

    For more information on how to install it, see MongoDB documentation.

    If you want to secure connections with MongoDB using SSL, MongoDB Enterprise Server has to be manually installed on your machine. For more information, see https://docs.mongodb.com/v3.2/security/.

  3. Download Apache Tomcat from http://tomcat.apache.org/download-80.cgi and install it.

    For more information on the supported Apache Tomcat versions, see Compatible web application servers and containers.

    For more information on how to install it, see Apache Tomcat documentation.

    For production environments, it is recommended to use a separate Tomcat instance for Talend Dictionary Service.

To manually install and configure Talend Dictionary Service, follow this procedure:

Installing and configuring Talend Dictionary Service

  1. Add mongo to the PATH environment variable.

  2. Create the dqdict database in MongoDB using the following command: use dqdict.

  3. Create the following user for the dqdict database in MongoDB:

    • Username: dqdict-user

    • Password: duser

    You can use the following command: db.createUser( { user: "dqdict-user", pwd: "duser", roles: [{ role: "readWrite", db: "dqdict"}]}).

  4. Stop your Tomcat instance if it was automatically started.

  5. Unzip the DQDictServer-A.B.C.zip to a dq_dict folder.

  6. Remove the contents of the <Tomcat>/webapps folder.

  7. Create a <Tomcat>/app folder and copy the dataquality-semantic-producer-A.B.C.war file from dq_dict.

  8. Copy the file contained in dq_dict/context to <Tomcat>/conf/Catalina/localhost.

  9. Copy the configuration file contained in dq_dict/config to <Tomcat>/conf.

  10. Open the <Tomcat>/conf/data-quality.properties file and edit the following Talend Dictionary Service properties:

    Note

    All the passwords entered in the properties file are encrypted when you start your Talend Dictionary Service instance.

    Field

    Action

    oidc.url

    Enter the URL to Talend Identity and Access Management, http://localhost:9080/oidc for example.

    scim.url

    Enter the URL to Talend Identity and Access Management SCIM, http://localhost:9080/scim for example.

    oidc.semanticservice.id

    Enter the Talend Identity and Access Management OIDC client identifier.

    oidc.semanticservice.secret

    Enter the Talend Identity and Access Management OIDC password.

    spring.cloud.stream.kafka.binder.brokers

    Enter the host corresponding to your Kafka broker.

    spring.cloud.stream.kafka.binder.defaultBrokerPort

    Enter the port corresponding to your Kafka broker.

    spring.cloud.stream.kafka.binder.zkNodes

    Enter the host corresponding to your Zookeeper node.

    spring.cloud.stream.kafka.binder.defaultZkPort

    Enter the port corresponding to your Zookeeper node.

  11. Update the following fields with your MongoDB settings:

    Field

    Description

    dq.mongo.host

    Host name of your MongoDB instance.

    dq.mongo.port

    Port number of your MongoDB instance.

    dq.mongo.username

    Username used to connect to the database.

    dq.mongo.password

    Password used to connect to the database.

    dq.mongo.database.name

    Name of the database on which Talend Dictionary Service is connected, dqdict by default.

  12. Launch MongoDB.

  13. Go to the dq_dict/database folder and edit the semantic-dictionary-import.sh script.

  14. Replace the variables with your MongoDB credentials.

  15. Execute the script.

  16. Start Talend Dictionary Service by launching, in order:

    1. Apache Zookeeper

    2. Apache Kafka

    3. Apache Tomcat