tSetDynamicSchema Standard properties - Cloud - 8.0

Dynamic Schema

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Data Governance > Third-party systems > Custom code components (Integration) > Dynamic Schema components
Data Quality and Preparation > Third-party systems > Custom code components (Integration) > Dynamic Schema components
Design and Development > Third-party systems > Custom code components (Integration) > Dynamic Schema components
Last publication date
2024-02-20

These properties are used to configure tSetDynamicSchema running in the Standard Job framework.

The Standard tSetDynamicSchema component belongs to the Custom Code family.

The component in this framework is available in all subscription-based Talend products.

Basic settings

Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. When you create a Spark Job, avoid the reserved word line when naming the fields.

Built-In: You create and store the schema locally for this component only.

Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion.

    If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the Repository Content window.

This option is not available if you select the Guess schema from file option or the Read schema from file option.

Sync columns

Click to synchronize the output file schema with the input file schema. The Sync function is available only when the component is linked with an input component using a Row connection.

Guess schema from file

Select this option to generate the schema from a csv file. You need to enter the path to the csv file in the Filename field manually or by clicking the [...] button and navigating to the csv file.

This option is not available if the component is linked with an input component using a Row connection or if you select the Read schema from file option.

Read schema from file

Select this option to generate the schema from a schema definition file. You need to enter the path to the schema definition file in the Filename field manually or by clicking the [...] button and navigating to the file.

Note:
  • A schema definition file needs to be a csv file.
  • A schema definition file contains a header line and multiple column definitions, with each column definition occupies one line in the file. It can contain column definitions of multiple tables.
  • The header line contains multiple fields. The value of a field can only be an option of the Property drop-down list in the Parameters table (case sensitive).
  • When the schema of this component is used by a tSAPTableInput component, the value of a ExtendedConfiguration field needs to be in the format of {"TableName":"{your_table_name}"}.
  • This option is available only when you have installed the 8.0.1-R2022-04 Talend Studio Monthly update or a later one delivered by Talend. For more information, check with your administrator.

This option is not available if the component is linked with an input component using a Row connection or if you select the Guess schema from file option.

Parameters

This table contains two columns.

  • Property: Property of the schema entry: ColumnName, DBName, Type, DBType, Length, Precision, Nullable, Key, or ExtendedConfiguration.
  • Value: Value assigned to this property.

This option is not available if you select the Guess schema from file option or the Read schema from file option.

Note:
  • The ExtendedConfiguration property is available only when you have installed the 8.0.1-R2022-04 Talend Studio Monthly update or a later one delivered by Talend. For more information, check with your administrator.
  • The use of the ExtendedConfiguration property varies with the component that uses the schema of this component. When used with a tSAPTableInput component, this property holds table names and the value needs to be in the format of {"TableName":"{your_table_name}"}.

Advanced settings

Row limit

Specify the number of rows in the csv file to guess schema from.

This field is available if you select the Guess schema from file option in the Basic settings view.

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at a Job level as well as at each component level.

Global Variables

Global Variables

DYNAMIC: the last row of the data retrieved using the defined dynamic schema. This is an After variable and it returns a dynamic.

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl+Space to access the variable list and choose the variable to use from it.

For more information about variables, see Using contexts and variables.

Usage

Usage rule

This component can be used as the end component or be used independently in a subJob.

Limitation

To work with tSetDynamicSchema, components should have the option of Use existing dynamic, which allows you to select the tSetDynamicSchema component from the list.