Talend Administration Center basic configuration

Talend Real-time Big Data Platform Installation Guide for Linux

EnrichVersion
6.3
EnrichProdName
Talend Real-Time Big Data Platform
task
Installation and Upgrade

The following pages detail basic configuration procedures you should follow in order to install Talend Administration Center:

Increasing the memory of Tomcat

To increase the memory heap size of the server in order to make all the applications work properly, proceed as follows:

  1. If you are using Ubuntu, change directory to <TomcatPath>/bin where <TomcatPath> is the Tomcat installation directory, and edit the file catalina.sh.

    If you are using another Linux distribution, edit the file /usr/share/tomcat/conf.

  2. Add the following line:

    set JAVA_OPTS=%JAVA_OPTS% -XX:MaxMetaspaceSize=512m -Xmx1024m -Xms256m

    The Tomcat memory size is now increased and the server can hold several web applications.

    For Oracle users only:

    Add the following line to the .sh file in order to specify the catalog and schema database parameters, and to avoid errors during Talend Administration Center startup:

    Xmx<1G> -Dtalend.catalog=<catalogName> -Dtalend.schema=<schemaName>

Installing database drivers in your Web application server

Installing the database drivers

If you are not using the embedded H2 database with Talend Administration Center or Talend Activity Monitoring Console, you must install the driver for the database to use in your Web application server.

For more information regarding the databases compatible with Talend Administration Center and Talend Activity Monitoring Console, see Compatible Databases.

To install database drivers in your Web application server, proceed as follows:

  1. Stop your Web application server.

  2. In case you use Tomcat, clean the <apache-tomcat>/work/Catalina/localhost folder, where <apache-tomcat> corresponds to Tomcat installation directory.

  3. Make sure that the driver for the database you want to use does not exist in any of these folders:

    Web application Server used

    Folders to check

    Tomcat

    • <apache-tomcat>/webapps/org.talend.administrator/WEB-INF/lib

    • <apache-tomcat>/webapps/amc/WEB-INF/lib

    • <apache-tomcat>/lib

    JBoss

    • <JBoss_installation_folder>/standalone/lib/ext

    • <JBoss_installation_folder>/standalone/lib/endorsed

    • <JBoss_installation_folder>/standalone/tmp/work/jboss.web/default-host/amc/eclipse/plugins/org.talend.amc.libraries_X.X.X.XXXXXXXX_XXXX/lib/ext

    If the driver already exists in one of these folders, skip the next step.

  4. Download the correct database driver(s) from the official provider website, according to the version of the JVM you use to run your Web application server and the version of the database you want to use.

    In case you use Oracle, use a copy of the ojdbcX.jar file from your Oracle installation.

    Note that those drivers are specific and that you should only download the one(s) that you need.

    Database used

    Driver to download

    MySQL

    http://dev.mysql.com/downloads/connector/j/

    Oracle

    http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

    Azure SQL

    Download the patched jTDS driver from https://sourceforge.net/p/jtds/bugs/_discuss/thread/16113049/7594/attachment/jtds-1.3.1.jar.

    For more information about the related bug, see https://sourceforge.net/p/jtds/bugs/725/?page=0. Note that there is no official release that includes the fix yet.

    MS SQL

    http://sourceforge.net/projects/jtds/files/jtds/

    PostgreSQL

    http://jdbc.postgresql.org/download.html

    Microsoft JDBC Drivers 6.0, 4.2, 4.1, and 4.0 for SQL Server

    http://www.microsoft.com/en-us/download/details.aspx?id=11774

    MariaDB

    https://downloads.mariadb.org/connector-java/

  5. For Tomcat, if it does not exist, create a new endorsed folder under <apache-tomcat>.

    For JBoss, if it does not exist, create a new ext folder under <JBoss_installation_folder>/standalone/tmp/work/jboss.web/default-host/amc/eclipse/plugins/org.talend.amc.libraries_X.X.X.XXXXXXXX_XXXX/lib/.

  6. Place the driver(s) you need in the folder you created.

  7. In case you install the patched jTDS driver to work with Microsoft Azure database, in the database settings of Talend Administration Center, fill in the information of the patched driver as follows:

    database.url=jdbc:jtds:sqlserver://talendtac.database.windows.net:1433/tac;user=tadmin@talendtac.database.windows.net;password=yourpassword;ssl=require;
    database.driver=net.sourceforge.jtds.jdbc.Driver
    database.username=tadmin
    database.password=yourpassword 

    Note that ssl=require is a mandatory parameter, as stated in the Microsoft documentation https://docs.microsoft.com/nl-nl/azure/sql-database/sql-database-develop-java-simple#Opmerking.

  8. Restart your Web application server.

(Best Practice) Using VACUUM with PostgreSQL for Talend Administration Center users

When using Talend Administration Center to retrieve, schedule and/or execute Jobs, many update/delete database operations are performed, which may result in performance slowdown if you are using PostgreSQL.

Indeed, it is recommended to execute the VACUUM command with PostgreSQL, as items that are deleted or obsoleted by an update are not physically removed from their table.

For more information on the VACUUM command, see the PostgreSQL documentation.

For more information on how to set up automatic vacuuming (which is a process launched at regular intervals by the PostgreSQL server to execute VACUUM only on the tables that have been updated), see the PostgreSQL documentation.

Configuring Tomcat to use a proxy server

To configure Tomcat to connect to a proxy server, proceed as follows:

  1. Stop your Tomcat server.

  2. If you are using Ubuntu, change directory to <TomcatPath>/bin where <TomcatPath> is the Tomcat installation directory, and edit the file setenv.sh.

    If the file does not exist, create it.

    If you are using another Linux distribution, edit the file /usr/share/tomcat/conf.

  3. Add the following lines, changing the parameters to match with your configuration:

    [Tomcat Proxy Property]
    -Dhttp.proxySet=true
    -Dhttp.proxyHost=http(s)://proxy.server.com # Specify the host name or IP address of the proxy. You can use this parameter for http and https host names.
    -Dhttp.proxyPort=3128 # Specify the port number of the proxy server.
    -Dhttp.nonProxyHost=localhost|host.mydomain.com|192.168.0 # Specify a list of hosts separated by "|" that do not require access through the proxy server.

  4. Restart your Tomcat server.

Synchronizing Web application and server time zones

To make sure that the DST change and the time zones are correctly taken into account, check that your OS includes an environment variable set as follows:

On Windows: TZ=Europe/Paris

On Linux: Export TZ="Europe/Paris"

Note

If you wish to implement high availability to the task execution scheduling using Talend clustering feature, then read Talend High Availability.

Launching Talend Administration Center

The recommended way to configure the connection to the database and to the shared repository (Git or SVN) is through the Web interface of Talend Administration Center.

  1. Start the application server on which Talend Administration Center is installed.

  2. Open a Web browser and type in the following URL:

    http://localhost:8080/<ApplicationPath>

    Replace <localhost> with the IP address or the hostname of the Web server if the Web browser IP is different from the machine you are on, and <ApplicationPath> with the Talend Administration Center Web application path. For example, http://localhost:8080/org.talend.administrator.

    Choose a port according to your environment. The default port 8080 may clash with another application.

    When connecting for the first time, a dialog box is displayed, asking you the password to access the Database Configuration page.

  3. Type in the default admin password. H2 database connection parameters are displayed and some automatic checks are performed on driver, url, connection, version information.

    The administration database (storing users, rights, etc.) being an H2 embedded database, its access information is automatically set. However:

    • for security reasons and to improve performances, the ;MV_STORE=FALSE;MVCC=TRUE additional parameters must be added to the H2 default URL. The H2 database URL should thus looks like the following:

      jdbc:h2:~/talend_administrator;MV_STORE=FALSE;MVCC=TRUE;AUTO_SERVER=TRUE;LOCK_TIMEOUT=15000
    • if you do not want to use the embedded H2 database, you can set up a different database server (MySQL, MSSQL or Oracle) and set the corresponding connection parameters. For more information, see Configuring Talend Administration Center to run on a different database than H2.

  4. Click Set new license, then browse your system to the License file you received from Talend and click Upload. A final License check is performed.

  5. Click Go to Login.

  6. On the Login page, type in the default connection login for your first access (login: admin@company.com, password: admin).

    Those credentials correspond to the default user of the Web application.

    After the first connection, it is strongly recommended not to use the default user account to access the application for security reasons. You can either change the default credentials of this account (admin@company.com/admin) or create another administrator user and remove the default account.

Note

If your Web access is restricted, you may need to click Validate your license manually to perform the validation of your license key. Follow the instructions on screen.

Once the license is validated, the navigation bar of Talend Administration Center opens with all the pages accessible for the default administrator user account.

For more information on which pages of Talend Administration Center an administrator user can access, see the Talend Administration Center User Guide.

Configuring Talend Administration Center to run on a different database than H2

By default, the Talend Administration Center Web application is configured to run with the default H2 embedded database.

For more information on Talend Administration Center database, see Database.

Prerequisite:

  • The external database must have been created with a utf8 collation.

To run Talend Administration Center on another database:

  1. If you want to use a MySQL, Oracle or MS SQL database for Talend Administration Center, install the right database driver in the application server as described in Installing database drivers in your Web application server.

    For MySQL users: to prevent further transaction issues on the Job Conductor page of Talend Administration Center, it is recommended to configure MySQL like described in the documentation on Talend Help Center.

  2. Start the application server, then open a Web browser and type the URL of the Talend Administration Center Web application.

  3. On the Login page, click Go to db config page, then enter the administrator password (by default, it is admin).

    Note that if you are starting Talend Administration Center for the first time, you already are on the database configuration page.

  4. In the Database type list, select your database. As a result, the Driver and Url fields are automatically updated with the template corresponding to this database.

  5. In the Url field, replace the parameters in brackets with your database details.

    Note that you can click the Reload from file button to reload your previous database as changes are not saved until you click Save.

  6. Click Save to take your changes into account.

Link Talend Administration Center to your version control system

  1. Click Configuration to access the setting page of Talend Administration Center.

  2. Change the following parameters for the Git or SVN module using the parameters you have set during the installation process of the Git or SVN server.

    - Server Location URL: your Git or SVN repository URL.

    - Username: your Git or SVN repository user.

    - Password: your Git or SVN repository password.

For examples of Git or SVN URLs, and more details, see Setting up your version control system.

Note

If you use several Git or SVN repositories to store your projects, refer to the User Guide of Talend Administration Center and check the Advanced settings procedure.

The link to Git or SVN is now established, you can thus create a new project in order for the Talend clients to have at least one project in their workspace. Next steps:

  • Create one or more users from the Users page.

  • Create a new, remote, collaborative project from the Projects page.

  • Associate the user(s) with the project from the Project authorizations page.

For more details, see the Talend Administration Center User Guide.

(Git only) Configuring Talend Administration Center to handle multiple repositories

You might need to connect Talend Administration Center to another Git repository than the one you entered on the Configuration page of the web application, for example when you are creating new projects. If so, you need to update a configuration file.

Prerequisite: You need to shut down Tomcat.

  1. Open the following file to edit it:

    <tomcat_path>WEB-INF\classes\configuration.properties

  2. Add the following:

    git.conf.enableHashRepositoryUrl=true

    Note that this configuration may increase disk space usage if you use different protocols (http / https / ssh, etc.) to access the same repository.

  3. Restart Tomcat.

    Now a separate local folder will be created for each Git repository URL entered in Talend Administration Center.

(Git only) Configuring Talend Administration Center to improve performance when creating execution tasks

When creating execution tasks on the Job Conductor page of Talend Administration Center, you might notice some slowness when retrieving the Jobs stored on the Git repository. If so, you have the possibility to update a configuration file in order to prevent Git from refreshing the repository and thus to improve speed of item selection.

Prerequisite: You need to shut down Tomcat.

  1. Open the following file to edit it:

    <tomcat_path>WEB-INF\classes\configuration.properties

  2. Add the following:

    git.conf.refreshDataProjectWhenSelectJob.enable=false

    Note that the recommended parameter value is true, and that you should only disable to prevent refresh of local Git repositories.

  3. Restart Tomcat.

    Now the selection of Git items from the Job Conductor page of Talend Administration Center will be quicker.

Configuring the log storage mode

The log outputs are stored by default in the server application standard log file (STDOUT) as defined in the Log4j.xml file located in the <ApplicationPath>/WEB-INF/classes folder. However you can store the log in a different file by setting the path to this file in the Log4j.xml file.

  • To do so, simply set the path in the Configuration page in Talend Administration Center. For more information, refer to your Talend Administration Center User Guide. If you leave the path field blank in the Configuration page, then you can also customize the Log4j.xml to address your custom needs.