Managing data models - 6.5

Talend Data Fabric Studio User Guide

EnrichVersion
6.5
EnrichProdName
Talend Data Fabric
task
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

An authorized user can also import/export, copy/paste and delete created data models from Talend Studio.

Note

It is also possible to import and share MDM complete projects or only a data model or part of the data model from the community web page, Talend Exchange. For further information, see Projects/objects on Talend Exchange.

How to export data models

From Talend Studio you can export one or multiple data models in order to exchange them between two different MDM servers or repositories.

Prerequisite(s): You have already connected to the MDM server from Talend Studio. At least one data model exists.

To export one or multiple data models:

  1. In the MDM Repository tree view, right-click the Data Model node and select Export Items from the contextual menu.

    The [Export Repository items] dialog box appears and the Data Model check box is selected by default.

    Note

    If you want to export all items in the repository and not only data models, click the Select All button to select the check boxes of all items in the list.

  2. To export the selected data models in a folder, select the Select root directory option and click the three-dot button next to the option.

    A dialog box appears.

  3. Browse to where you want to save the data model and click OK to close the dialog box.

    The path to the target folder is set in the field next to the selected option.

  4. Click the Add Dependencies button to add dependencies for the selected item or items.

    You can click the Add Dependencies button for multiple times until all item dependencies are added or click the Add All Dependencies button to add all item dependencies in one go.

    For more information about item dependencies, see Managing item dependencies in the MDM Repository.

  5. In the [Export items] dialog box, click Finish to validate your changes and close the dialog box.

    A folder holding all data models in the current MDM Repository is created in the specified path. An XML file that lists all exported elements is also created.

  6. To export the selected data models in a zip file, select the Select archive file option in the [Export Items] dialog box and then click the three-dot button next to the option.

    A dialog box appears.

  7. Browse to where you want to save the data model archive file.

  8. In the File Name field, enter a name for the archive file and then click Open to close the dialog box.

    The path to the target archive file is set in the field next to the selected option.

  9. In the [Export items] dialog box, click Finish to validate your changes and close the dialog box.

    An archive file holding all data models in the current MDM Repository is created in the specified path. An XML file that lists all exported elements is also created.

Note

If you want to export only one data model of all those in the MDM Repository, expand Data Model in the [Export items] dialog box and then select the check box next to the name of the data model you want to export and proceed as usual.

How to import data models

From Talend Studio you can import data models into the current MDM Repository that have been created in other MDM Repositories.

Prerequisite(s): You have already connected to the MDM server from Talend Studio. You have exported one or multiple data models from other MDM repositories.

To import one or multiple data models into the current MDM Repository:

  1. In the MDM Repository tree view, right-click the Data Model node and select Import items from the contextual menu.

    The [Import items] dialog box displays.

  2. Select the root directory or archive file option according to whether the data models you want to import are saved in a folder/archive file and then click Browse next to the option.

    A dialog box appears.

  3. Browse to the folder/archive file holding the data models and click OK/Open to close the dialog box.

  4. In the [Import items] dialog box, click Finish to validate your changes and close the dialog box.

    All data models in the folder/archive file are exported and listed under the Data Model node in the current Talend Studio.

Note

If you want to import only one data model of all those saved in the folder/archive file, expand Data Model in the [Import items] dialog box and then select the check box next to the name of the data model you want to import and proceed as usual.

How to edit properties of a data model

You are allowed to edit properties of a data model.

Prerequisite(s): You have already connected to the MDM server from Talend Studio. At least one data model exists.

To edit properties of a data model from the data model editor, do the following:

  1. In the MDM Repository tree view, expand the Data Model node.

  2. Right-click the data model you want to edit and select Edit properties from the contextual menu.

    A dialog box pops up.

  3. Edit the properties of a data model, for example, name, purpose or description, according to your needs.

  4. Click Finish to save your changes.

You can also edit any element in the data model from the Properties view in the data model editor. For further information, see How to use the Properties view in the data model editor.

How to enable foreign key integrity checking

You can enable foreign key integrity checking to manage whether users are allowed to delete a record that is linked to another record through a foreign key. By default, this is not allowed.

To enable foreign key integrity checking:

  1. In the MDM Repository tree view, expand the Data Model node and double-click the data model for which you want to enable foreign key integrity checking.

    The corresponding editor opens with the selected data model displayed.

  2. Select the entity containing the foreign key for which you want to enable integrity checking.

  3. In the Properties view, click the Relationship tab.

  4. Under the Foreign Key section:

    • Select the Enforce FK integrity check box to enable foreign key integrity checking, that is, to raise an error whenever a user tries to delete a record if the record is linked to any other records through this foreign key.

    • Select the Allow FK integrity override check box to allow users to force the delete of a record even if it is linked to another record through this foreign key.

These two options can be combined in the following ways.

 Enforce FK integrity enabledEnforce FK integrity disabled
Allow FK integrity override enabledDelete allowed, with user prompted for overrideDelete allowed
Allow FK integrity override disabledDelete forbiddenDelete allowed

In the case of conflicts, the least tolerant rule is always applied.

If a user is prevented from deleting a record, details are provided in <$INSTALLDIR>/logs/mdm.log.

Handling circular dependencies

If you are using a SQL database to store your data records, you may encounter an error during initialization because a circular dependency is detected. This means that, for example, EntityA references EntityB, EntityB references EntityC, and EntityC references EntityA.

To fix the data model in such a case, simply turn off Foreign Key integrity checking for the relation, as described in How to enable foreign key integrity checking.

How to check the validity of a Data Model

Whenever you make changes to your Data Model, Talend Studio can perform a series of checks to ensure that the Data Model is valid. By default, these checks are performed when you save your Data Model and when you deploy your Data Model, and can also be launched manually.

After checking the validity of a data model, Talend Studio also provides the impact analysis result if the data model changes impact the database schema. For more information, see How to deal with the impact of data model changes.

Items in the Data Model that are validated include:

  • Entities (other than Reusable Types):

    • Lookup fields

    • Primary keys

    • Primary key information

    • Inheritance

  • Fields (may occur in Entities or Reusable Types):

    • Foreign keys

    • Foreign key information

The validation check reports both errors (issues which mean your Data Model is not valid) and warnings (issues which do not affect the validity of your Data Model, but which may not provide the result you expect). This report is shown in the MDM Problem View of the console.

How to launch a validation check manually

To launch a validation check manually, without saving any changes you may have made to your Data Model, do the following:

  1. In the MDM Repository tree view, expand Data Model, right-click the Data Model on which you want to perform a validation check, and then click Validate.

  2. In the [Validation Result Dialog] that opens, click the Details >> button for more information on any errors or warnings that may be found.

How to identify the source of validation issues

When the Data Model validation check reports issues with your Data Model, it also helps you identify where these issues occur so you can fix them.

To identify the source of validation issues, do the following.

  1. In the MDM Problem View of the console, double-click the error or warning.

    The Data Model opens on the Schema Source tab.

  2. Click the red error marker in the right-hand scrollbar to jump directly to the point where the error or warning has been identified, and perform any changes necessary to make the Data Model valid.

Note that you can also customize how the MDM Problem View displays errors and warnings in the console by clicking the View Menu button , for instance to group them by type (error or warning).

How to configure the validation check

By default, the validation check is performed when you launch it manually, when you save any changes to your Data Model, or when you deploy the Data Model to the server. You can configure this behavior in the preferences for the Data Model validation check.

To configure the preferences for the Data Model validation check, do the following.

  1. Click Window > Preferences.

  2. In the [Preferences] window that opens, expand Talend > MDM and then click Data Model Validation.

  3. In the [Data Model Validation] preferences, select the situations in which you want a validation check to be performed on your Data Model.

    • User initiated validation: Users can launch a check manually.

    • Validation on save: A check is performed when the user saves any changes to the Data Model.

    • Validation on deploy: A check is performed when the user deploys the Data Model to the server.

    If you choose to perform a validation when the user deploys the Data Model to the server, you can also specify what happens if issues are found.

How to deal with the impact of data model changes

When designing a data model, you can make as many changes as required to the data model before you deploy the modified data model to the MDM server to check if the changes make sense.

When you deploy the data model, Talend Studio will perform a series of checks to ensure the validity of the data model. For more information, see How to check the validity of a Data Model.

After the validity checking of a data model, if the changes you made to the data model impact the database schema, for example, adding new elements to one entity in the data model, Talend Studio provides the impact analysis result when you deploy the modified data model to the MDM server which helps you to decide whether to cancel the operation or continue the deployment.

The impact analysis result shows the relevant data model, the description of the changes made to the data model, the level of each change and the operation to be performed.

If a data model change has an impact level of Medium or High and leads to table recreation, the impact analysis result also lists the entities to be dropped and recreated.

In this dialog box, you can view the data model changes and select the appropriate operation according to your need.

Item

Description

DataModel/Change

It shows the relevant data model and the changes made to the data model.

Level

There are three levels of data model changes:

  • Low: The change will lead to the modification of the database schema, but it is perfectly OK.

  • Medium: The change will be reflected in the database schema, but the existing data may cause it to fail.

  • High: The change cannot be reflected without recreating part of the database schema that corresponds to the data model changes, and will result in a loss of data. Because of referential integrity, this change will cascade through the tables defined to have foreign key relationships and result in the loss of their data also. If you want to change an optional simple type element to a mandatory one, make sure the default value is defined for the element.

For more information, see Data model changes and their impact levels.

Operation

Three options are available:

  • Recreate table: recreate the table to reflect the changes made to the data model. This option is available only if the change level is Medium or High.

  • Apply changes: apply the changes to the data model.

  • Cancel deployment: cancel the deployment of the modified data model.

Once you have made your choice, click OK to validate your changes and close the dialog box.

Data model changes and their impact levels

The data model changes are classified into three levels according to their impacts on the data model: Low, Medium and High.

When you made changes to a data model, the database schema will be updated accordingly.

Below are some examples of the data model changes, their corresponding impact level, and the results in the database:

Action

Impact level

Result

Change an optional simple type element to a mandatory one without a default value (null values exist for the given element)

High

The entity table is dropped and recreated.

Change an optional complex type element to a mandatory one

High

The entity table is dropped and recreated.

Change the maximum occurrence of an element from 1 to n or vice versa

High

The entity table is dropped and recreated.

Make changes in inheritance

High

All tables involved in the inheritance tree are dropped and recreated.

Change the type of an element from one simple type to another

High

N/A

Modify the value of totalDigits or fractionDigits of a decimal element

High

The entity table is dropped and recreated.

Remove or add anonymous types

High

All entity types using anonymous types may be dropped and recreated.

Make high impact changes to an entity that has foreign key relationships

High

The entity table is dropped and recreated, together with the tables of all other entities that have foreign key relationships with this entity.

Add a new mandatory element without a default value

High

The entity table is dropped and recreated with the new column added.

Remove a mandatory simple type element

Medium

The column in the entity table will be removed.

Remove an optional element

Medium

The column in the entity table will be removed.

Rename an optional element

Medium

A column with the old name remains and a new column with a new name is created.

Change an optional simple type element to a mandatory one with a default value (null values exist for the given element)

Medium

The column in the entity table will be updated with the default value.

Change an optional simple type element to a mandatory one (no null values exist for the given element)

Low

The column definition in the entity table will be changed accordingly.

Change a mandatory element to an optional one

Low

The column definition in the entity table will be changed accordingly.

Change an optional multi-occurrence element to a mandatory multi-occurrence one or vice versa

Low

The column definition in the entity table will be changed accordingly.

Add a new optional element

Low

A new column is added to the entity table.

Increase the length of a string element

Low

The length of the column is increased accordingly.

Add a new element (of string, boolean or numeric type) with a default value

Low

A new column is added with the default value.