tMysqlOutputBulkExec - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

tMysqlOutputBulkExec properties

The tMysqlOutputBulk and tMysqlBulkExec components are used together in a two step process. In the first step, an output file is generated. In the second step, this file is used in the INSERT statement used to feed a database. These two steps are fused together in the tMysqlOutputBulkExec component.

Component family

Databases/MySQL

 

Function

Executes the Insert action on the data provided.

Purpose

As a dedicated component, it improves performance during Insert operations to a MySQL database.

Basic settings

Property type

Either Built-in or Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

 

 

Built-in: No property data stored centrally.

 

 

Repository: Select the repository file in which the properties are stored. The fields that follow are completed automatically using the data retrieved.

 

DB Version

Select the version of MySQL that you are using.

 

Host

Database server IP address

 

Port

Listening port number of DB server.

 

Database

Name of the database

 

Username and Password

DB user authentication data.

To enter the password, click the [...] button next to the password field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.

Warning

tCreateTable can be used as a substitute for this function.

Action on table

On the table defined, you can perform one of the following operations:

None: No operation is carried out.

Drop and create table: The table is removed and created again.

Create table: The table does not exist and gets created.

Create table if not exists: The table is created if it does not already exist.

Drop table if exists and create: The table is removed if it already exists and created again.

Clear table: The table content is deleted.

 

Table

Name of the table to be written.

Note

Note that only one table can be written at a time and that the table must already exist for the insert operation to succeed

 

Local FileName

Name of the file to be generated and loaded.

This file is generated on the same machine where the Studio is installed or where the Job using tMysqlOutputBulkExec is deployed. Then it is loaded to the database specified by Host field.

 

Append

Select the check box for this option to append new rows to the end of the file.

 

Schema and Edit Schema

A schema is a row description, it defines the number of fields to be processed and passed on to the next component. The schema is either Built-in or stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

 

 

Built-In: You create and store the schema locally for this component only. Related topic: see Talend Studio User Guide.

 

 

Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs. Related topic: see Talend Studio User Guide.

When the schema to be reused has default values that are integers or functions, ensure that these default values are not enclosed within quotation marks. If they are, you must remove the quotation marks manually.

For more details, see https://help.talend.com/display/KB/Verifying+default+values+in+a+retrieved+schema.

Advanced settings

Additional JDBC parameters

Specify additional connection properties for the DB connection you are creating. This option is not available if you have selected the Use an existing connection check box in the Basic settings.

Note

You can press Ctrl+Space to access a list of predefined global variables.

 

Row separator

String (ex: "\n"on Unix) to distinguish rows.

 

Field separator

Character, string or regular expression to separate fields.

 

Escape char

Character of the row to be escaped

 

Text enclosure

Character used to enclose the text.

 

Create directory if does not exist

This check box is selected by default. It creates a directory to hold the output table if required.

 

Custom the flush buffer size

Customize the amount of memory used to temporarily store output data. In the Row number field, enter the number of rows after which the memory is to be freed again.

 

Action on data

On the data of the table defined, you can carry out the following opertaions:

Insert records in table: Add new records to the table.

Update records in table: Make changes to existing records.

Replace records in table: Replace existing records with new one. Ignore records in table: Ignore existing records or insert the new ones.

 

Records contain NULL value

This check box is selected by default. It allows you to take account of NULL value fields. If you clear the check box, the NULL values will automatically be replaced with empty values.

 

Encoding

Select the encoding from the list or select Custom and define it manually. This field is compulsory for DB data handling.

 

tStatCatcher Statistics

Select this check box to collect the log data at the component level.

 

Enable parallel execution

Select this check box to perform high-speed data processing, by treating multiple data flows simultaneously. Note that this feature depends on the database or the application ability to handle multiple inserts in parallel as well as the number of CPU affected. In the Number of parallel executions field, either:

  • Enter the number of parallel executions desired.

  • Press Ctrl + Space and select the appropriate context variable from the list. For further information, see Talend Studio User Guide.

Warning

  • The Action on table field is not available with the parallelization function. Therefore, you must use a tCreateTable component if you want to create a table.

  • When parallel execution is enabled, it is not possible to use global variables to retrieve return values in a subjob.

Usage

This component is mainly used when no particular transformation is required on the data to be loaded onto the database.

Limitation

n/a

Scenario: Inserting data in MySQL database

This scenario describes a two-component Job which carries out the same operation as the one described for tMysqlOutputBulk properties and tMysqlBulkExec properties, although no data is transformed.

  • Drop a tRowGenerator and a tMysqlOutputBulkExec component from the Palette to the design workspace.

  • Connect the components using a link such as Row > Main.

  • Set the tRowGenerator parameters the same way as in Scenario: Inserting transformed data in MySQL database. The schema is made of four columns including: ID, First Name, Last Name, Address and City.

  • In the workspace, double-click the tMysqlOutputBulkExec to display the Component view and set the properties.

  • Define the database connection details in the corresponding fields, if necessary. Consult the recommendations detailed in Scenario: Inserting transformed data in MySQL database, concerning the conservation of connection details in the Repository, under the Metadata node. In the component view, select Repository in the Property Type field and then select the appropriate connection in the adjacent field. The following fields will be filled in automatically.

For further information, see Talend Studio User Guide.

  • In the Action on table field, select the None option as you want to insert the data into a table which already exists.

  • In the Table field, enter the name of the table you want to populate, the name being clients in this example.

  • In the Local filename field, indicate the access path and the name of the file which contains the data to be added to the table. In this example, the file is clients.txt.

  • Click on the Advanced settings tab to define the component's advanced parameters.

  • In the Action on data list, select the Insert records in table to insert the new data in the table.

  • Press F6 to run the Job.

The result should be pretty much the same as in Scenario: Inserting transformed data in MySQL database, but the data might differ as these are regenerated randomly everytime the Job is run.