tDataStewardshipTaskOutput Standard properties - 6.5

Data Stewardship

author
Talend Documentation Team
EnrichVersion
6.5
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
task
Data Governance > Third-party systems > Data Stewardship components
Data Quality and Preparation > Third-party systems > Data Stewardship components
Design and Development > Third-party systems > Data Stewardship components
EnrichPlatform
Talend Data Stewardship
Talend Studio

These properties are used to configure the tDataStewardshipTaskOutput component which runs in the Standard Job framework.

The Standard tDataStewardshipTaskOutput component belongs to the Talend Data Stewardship family.

The component in this framework is available in all Talend subscription-based products.

Basic settings

Property Type

Either Built-in or Repository.

 

Built-In: You create and store the schema locally for this component only.

 

Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs.

Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to Repository. When you create a Spark Job, avoid the reserved word line when naming the fields.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

Once you select a campaign from the Find a campaign list, tDataStewardshipTaskOutput becomes aware of the campaign schema and it creates identical schema columns in its basic settings. The component also adds the TDS_DUE_DATE and TDS_EXTERNAL_ID columns to the schema:
  • TDS_DUE_DATE: holds the task due date using the format: YYYY-MM-DD HH:mm:ss. The due date is calculated automatically according to the task delay time you define when creating the campaign in Talend Data Stewardship. If you do not define a delay date for tasks when creating the campaign, this column stays empty.

  • TDS_EXTERNAL_ID: helps you set an identifier of your choice on tasks so that you can retrieve them easily from any other system such as Talend MDM Web UI.

    If you do not define an identifier for tasks when creating the Job, this column stays empty.

In case you select to write the tasks in a MERGING or a GROUPING campaign, the below columns are automatically added to the schema.

  • TDS_GID: holds the group identifier.

    This identifier is used by tDataStewardshipTaskOutput to group records in tasks. All source records that should be grouped in a single task must have the same TDS_GID.

  • TDS_MASTER (only for a MERGING campaign): indicates if the record is a master or a source record.

    Two cases to consider:
    • If no source record is set as master for a given task, Talend Data Stewardship determines which attributes of matched records to use to create the master record according to the survivorship rules you define when creating the campaign.

      For more information about survivorship rules, see Setting a data model in the Merging campaign.

    • If more than one source record is set as master for a given task, Talend Data Stewardship takes the first source set as master to be the master record. So the best practice is to have either 0 or one master record per task.
  • TDS_SOURCE: name of the source for the record, if any.

  • TDS_SCORE: lists the calculated distance between the input and the master records according to the matching algorithm.

  • TDS_CREATION: task creation date.
  • TDS_LAST_UPDATE: task last update date.

URL

Enter the address to access the Talend Data Stewardship server suffixed with /data-stewardship/, for example http://<server_address>:19999/data-stewardship/.

When working with the cloud version of Talend Data Stewardship, use https://tds.us.integrationcloud.talend.com or https://tds.eu.integrationcloud.talend.com to access the application, according to your data center.

Username and Password

Enter the authentication information to the Talend Data Stewardship server.

To enter the password, click the [...] button next to the password field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.

Campaign

Click Find a campaign to open a list of all the campaigns available on the server, and select the name of the campaign from which to delete the tasks.

Label

A read-only field which displays the technical name of the campaign once it is selected in the basic settings.

Type

A read-only list which shows the campaign type out of the predefined types once the campaign is selected.

Override enforcement of data model

Select this check box if you want to process data on the Talend Data Stewardship server even if the schema type is not valid, no input validation of the schema is performed. This check box is selected by default with the RESOLUTION campaigns. However, this check box should be selected for all campaign types to guarantee smooth processing of data.

State and Assignee

State: Select from the list the state of the tasks you want to create.

Assignee: Select the campaign participant whose tasks you want to create. Otherwise, select No Assignee to create the tasks without assigning them to anybody.

You can also select Custom and set custom expressions in the fields which are displayed.

Priority, Choice and Tags

Priority (optional): Select any of the task priorities. Otherwise, select Custom and set a custom expression in the field which is displayed.

If no level is selected, Medium is used by default.

Choice (optional, available only when an ARBITRATION campaign is selected): Select any of the choice options set on the records while defining the campaign in the web application. The default value is No Choice, this enables data stewards to do the work in the web application. However, setting an arbitration choice in the Job is a way to help the steward by pre-selecting the supposedly most relevant choice directly when creating the tasks.

Tags (optional): Enter the tag to associate with the tasks you want to create.

You can use the tag(s) to filter the tasks you want to load into the campaign.

Comments

(Optional): Select one or several schema columns and enter the comment you want to add to the tasks you want to create.

This table lists by default all the schema columns.

The campaign participant will be able to see the comment any time he/she places the pointer on the source record column in Talend Data Stewardship . This information can help him/her making a more informed decision when resolving the task.

Advanced settings

Max tasks per commit

Set the number of lines you want to have in each commit.

Do not change the default value unless you are facing performance issues. Increasing the commit size can improve the performance but setting a too high value could cause Job failures.

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at the Job level as well as at each component level.

Global variables

NB_LINE

The number of messages processed. This is an After variable and it returns an integer.

NB_REJECT

The number of rows rejected. This is an After variable and it returns an integer.

NB_SUCCESS

The number of rows successfully processed. This is an After variable and it returns an integer.

ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

Usage

Usage Rule

This component is usually used as an end component of a Job or Subjob and it always needs an input link.