Using the dbmigration tool for migration - 6.4

Talend MDM Platform Migration Guide

EnrichVersion
6.4
EnrichProdName
Talend MDM Platform
task
Installation and Upgrade
EnrichPlatform
Talend Activity Monitoring Console
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend Data Preparation
Talend Data Stewardship
Talend DQ Portal
Talend ESB
Talend Identity Management
Talend Installer
Talend JobServer
Talend Log Server
Talend MDM Server
Talend MDM Web UI
Talend Project Audit
Talend Repository Manager
Talend Runtime
Talend SAP RFC Server
Talend Studio

The dbmigration tool allows you to migrate system objects, data records and deployed objects (except Jobs and workflows) between relational databases.

Prerequisite(s):

  • Make sure that both MDM servers are up and running.

  • If you need to migrate between two databases on a single machine, you must run the two MDM servers side by side. You can do this by setting a different port binding on the target server.

    For further information about running multiple instances of Tomcat on the same machine, see the Talend Installation Guide.

  • The migration of UpdateReport records to the target MDM server may lead to unexpected results. For example, some eligible triggers are triggered, or processes or Jobs are executed. To avoid this issue, before the migration, you need to disable the Event Manager on the target MDM server by configuring subscription.engine.autostart=false in the file <$INSTALLDIR>/conf/mdm.conf, where <$INSTALLDIR> specifies the MDM server installation directory. You need to re-enable it and when the migration is completed. Make sure to restart the MDM server for the configuration to take effect.

  1. In the target MDM server, browse to:

    <$INSTALLDIR>/tools/dbmigration

  2. Save a copy of the dbmigration.properties.template file giving it the name dbmigration.properties, for example.

  3. Open the new dbmigration.properties file using a text editor.

  4. In the "from" list, set the connection information to the source server.

    In the "to" list, set the connection information to the target server.

    If you want to migrate from 5.X, one configuration example is given below:

    ####from 5.X using EJBs####
    from.host=localhost
    from.user=admin
    from.password=talend
    from.jndi.port=1199
    
    to.url=http://192.168.32.191:8280/talendmdm
    to.home=D:\\Talend\Talend-MDMServer
    to.user=administrator
    to.password=administrator
    to.cluster.batch.size=500

    If you want to migrate from 6.X, one configuration example is given below:

    ####from 6.X using REST####
    from.url=http://localhost:8180/talendmdm
    from.user=administrator
    from.password=administrator
    
    to.url=http://192.168.32.191:8280/talendmdm
    to.home=D:\\Talend\Talend-MDMServer
    to.user=administrator
    to.password=administrator
    to.cluster.batch.size=500

    If the data containers and data models in your source server do not have the same names (for example, if the ProductDemo data container uses the Product data model), you must also use the following syntax in the dbmigration.properties file to specify how they are connected: <data container>.datamodel=<datamodel>.

    For example:

    ProductDemo.datamodel=Product

    Note

    For entities that have Foreign Keys to themselves (for example, Person might have a FK relation 'is child of' to Person), the migration process does not ensure the correct insertion order. As a result, the validation of such records may fail even though data integrity is correct. To avoid this, it is recommended that you deploy the data model to the target MDM server first and then temporarily disable FK integrity checks for the FK field in the data model editor.

  5. If needed, modify the number of migration threads specified by the parameter db.migration.threads.

    The default value of this parameter is 8.

  6. Run the appropriate dbmigration file for your system (dbmigration.bat for Windows and dbmigration.sh for Linux).

    Give the CommandLine the file name you set as a parameter, as follows:

    Windows: dbmigration.bat dbmigration.properties

    Linux: dbmigration.sh dbmigration.properties

  7. If needed, you can add parameters to the CommandLine when running the dbmigration script for your system.

    Add the parameter...

    To...

    -i

    Do the migration in an interactive mode.

    For example, you will be prompted which data model needs to be migrated. Enter Y to select the data model you want to migrate, or enter N to skip the data model you do not want to migrate.

    -v

    Perform only the validation of user-defined data models.

    You will be prompted if a data model has any entity that refers to itself, or has circular dependencies between entities such as EntityA references EntityB, EntityB references EntityC, and EntityC references EntityA.

  8. Restart the target server in order to complete the migration process.

Note

If the source and target servers are running on the same machine and using different port bindings, stop the source server and then launch the target server on the desired port.

This will migrate the MDM instances stored in databases from the source server to the target server in line with the connection information defined in the file.

Checking the execution results of the dbmigration tool

The activity of the dbmigration tool can be logged using the log4j feature.

If needed, you can change the log4j configuration information in the file <$INSTALLDIR>/tools/dbmigration/log4j.properties. For more information on the log4j logging levels, see the Apache documentation at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html.

An example of the file log4j.properties is shown below:

log4j.rootCategory=INFO,CONSOLE,FILE

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%p [%c] - %m%n

log4j.appender.FILE=org.apache.log4j.RollingFileAppender
log4j.appender.FILE.File=dbmigration.log
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=%d %p [%c] - %m%n

By default, the log information is stored in the file <$INSTALLDIR>/tools/dbmigration/dbmigration.log.

When you migrate MDM instances using the dbmigration tool, you can check the log information in the file dbmigration.log to troubleshoot migration issues.

The following is an example of an error message in the file dbmigration.log:

2015-12-16 18:25:36,486 ERROR [org.talend.mdm.db.migration.rest.RESTXmlServer] - Unable to commit on 'Product' (total 2 records, some of them rollbacked)

Additionally, you can check the file <$INSTALLDIR>/logs/mdm.log on the source MDM server and target MDM server respectively to learn more information about the execution results of the dbmigration tool.