Data storage location - 8.0

Talend Data Catalog Installation and Upgrade Guide

Version
8.0
Language
English
Operating system
Windows
Product
Talend Big Data Platform
Talend Data Fabric
Talend Data Management Platform
Talend Data Services Platform
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Data Catalog
Content
Installation and Upgrade
Last publication date
2024-01-26
Here are some recommendations on how to handle your application data and customizations during the migration process.

Database server

Most application data is located on your database server.
  • Make sure you backup your database before upgrading.

    Upgrading your application also upgrades the associated database content (database schema, stored procedures, indexes and data).

    A part of the Talend Data Catalog software is implemented as database stored procedures, such as tracing lineage. A given version of Talend Data Catalog corresponds to a version of the software in the Tomcat application server and a version of the software in the database.

  • Ensure that the database has at least 20% free space. The upgrade process may take several hours (on large repositories) and also need extra space for temporary data during the migration.

Software installation directory

The software installation directory contains some critical application data and setup customizations. This table describes where you can find them.

If you have customized some of these files, you will need to copy them from the old installation directory to the new one during the migration process.

Folder Description
\postgresql\data contains the actual PostgreSQL database data when configured with the Setup utility in the Database Server tab.
\data contains data from other applications:
  • download\
    • MIMB\ contains downloaded third-party software packages.
  • files\
    • mimb\ contains files for the upgrade of MIMBWebServices.
    • mm\
      • analytics\ contains the generated analytic data files.
      • backups\ contains model backups for model comparison purpose.
      • operations\ contains the operation execution generated files such as a troubleshooting package or a backup ready to download. As these generated files can be very large, it is critical to delete the operation logs and files. To do so, you can execute the Delete operation logs and files operation manually from the context menu of the repository root in the repository manager, or schedule the operation from the MANAGE Schedules page.
      • sessions\ contains any temporary files for user login session.
      • tmp\ contains other temporary files.
  • logs\
    • mimb\ contains the model import/export bridge log files.
    • search\ contains the search query log files.
    • tomcat\ contains the Apache Tomcat log files.
  • MIMB\
    • cache\ contains bridge execution files organized as BridgeId\ImportId\mir_Version,nativeVersion.
    • parameters\ contains bridge execution parameter files (only for some bridges like DI/ETL with runtime variables) organized as BridgeId\ImportId\Parameters.
  • search\ contains the Lucene search engine indexes. They can be automatically fully rebuilt from scratch if the folder is empty, which can take a lot of time.
  • temp\ for any temporary files, such as from the bridges.
  • webapps\ for the Apache Tomcat application server cache.

The actual location of this application server data directory can be configured with the Setup utility, in the Application Server tab.

\conf contains the configuration and customizations:
  • conf.properties contains the files including most of customizations defined with the Setup utility.
  • ModelBridgeList.xml contains the list of enabled bridges and their names.
  • resources\ contains any user interface customizations, such as MM.properties and MetadataExplorer.xml.
  • Template\ contains the default template files of all the above files and directories. For example, it can contain an updated ModelBridgeList.xml or MetadataExplorer.xml file after applying a cumulative patch.
\tomcat\conf The tomcat.properties file contains the Tomcat port and memory customizations.

The keystore file contains the Tomcat SSL certificates defined with the Setup utility.

\jre\lib\security contains some SSL customizations.

It is recommended to reinstall the SSL keys with the Setup utility and not reuse such directory.