Dynamic schema - Cloud - 8.0

Talend Studio User Guide

Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Talend Studio
Design and Development
Last publication date

Talend Studio allows you to add a dynamic column to the schema of certain components in a Job. The dynamic column may constitute the only column in the schema, or it may be added after known columns, as the last column in the schema.

The dynamic column retrieves the columns which are undefined in the schema. This means that source columns which are unknown when the Job is designed, become known at runtime and are added to the schema. This can make Job design much easier as it allows for simple one to one mapping of many columns. There are many uses for dynamic columns. For instance in data migration tasks, developers can copy columns of data to another location without having to map each column individually.

Note that any static object set in the schema editor, such as data pattern or default value, is not taken into account for a dynamic column.

Warning: The Dynamic Schema functionality is to help you configure a schema in a non-static way, so you won't have to redesign your Job for future schema alteration while ensuring it will work all the time. However, it is not a guarantee that your schema will stick 100% accurately to the schema of the actual data to handle.

While the dynamic schema feature significantly eases Job designs, it does not work in all components. For a list of components that support this feature, go to <install_dir>/plugins/, where <install_dir> is the Studio installation directory, extract the jar file org.talend.core.tis_<version>.jar to get the text file supportDynamic.txt in the resources folder.

Warning: In the Database Input components, the SELECT query must include the * wildcard, to retrieve all of the columns from the table selected.

For further information about defining dynamic schemas, see Defining dynamic schema columns.

For further information regarding the mapping of dynamic columns, see Mapping dynamic columns.

For an example of using the dynamic schema feature in a Job, see Dynamic Schema.