tSalesforceOutputBulkExec - 6.3

Talend Open Studio for Big Data Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Open Studio for Big Data
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

The tSalesforceOutputBulk and tSalesforceBulkExec components are used together in a two step process. In the first step, an output file is generated. In the second step, this file is used to feed the Salesforce database. These two steps are fused together in the tSalesforceOutputBulkExec component. The advantage of using two separate steps is that the data can be transformed before it is loaded in the database.

Function

tSalesforceOutputBulkExec executes the intended actions on the .csv bulk data for Salesforce.com.

Purpose

As a dedicated component, tSalesforceOutputBulkExec gains performance while carrying out the intended data operations into your Salesforce.com.

tSalesforceOutputBulkExec Properties

Component family

Business/Cloud

Basic settings

Property Type

Either Built-In or Repository.

  • Built-In: No property data stored centrally.

  • Repository: Select the repository file where the properties are stored.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Connection

Select the component whose connection details will be used to set up the connection.

Connection type

Select the type of the connection from the drop-down list, either Basic or OAuth.

  • Basic: select this option to access Salesforce.com by entering your user ID and password. With this option selected, you need to specify the following properties:

    • User Id: the ID of the user in Salesforce.

    • Password: the password associated with the user ID.

    • Security Key: the security token.

  • OAuth: select this option to access Salesforce.com by entering your consumer key and consumer secret. This way, your user name and password will not be exposed, but extra work is required:

    With this option selected, you need to specify the following properties:

    • Client Id and Client Secret: the OAuth consumer key and consumer secret, which are available in the OAuth Settings area of the Connected App that you have created at Salesforce.com.

    • Callback Host and Callback Port: the OAuth authentication callback URL. This URL (both host and port) is defined during the creation of a Connected App and will be shown in the OAuth Settings area of the Connected App.

    • Token File: the path to the token file that stores the refresh token used to get the access token without authorization.

 

Module Name

Click the [...] button next to the field and in the dialog box displayed, select the module that will be used or select the Use custom object check box and specify the module name in the Object Name field.

 

Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Since version 5.6, both the Built-In mode and the Repository mode are available in any of the Talend solutions.

Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:

  • View schema: choose this option to view the schema only.

  • Change to built-in property: choose this option to change the schema to Built-in for local changes.

  • Update repository connection: choose this option to change the schema stored in the repository and decide whether to propagate the changes to all the Jobs upon completion. If you just want to propagate the changes to the current Job, you can select No upon completion and choose this schema metadata again in the [Repository Content] window.

Click Sync columns to retrieve the schema from the previous component connected in the Job.

 

Output Action

Select one of the following operations to be performed from the drop-down list.

  • INSERT: insert one or more new records into Salesforce.

  • UPDATE: update one or more existing records in Salesforce.

  • UPSERT: create new records and update existing records. In the Upsert Key Column field dispalyed, specify the key column for the upsert operation.

  • DELETE: delete one or more records in Salesforce.

 

Bulk File Path

Specify the path to the file that stores the data to be processed.

 

Append

Select this check box to append new data at the end of the file if it already exists, instead of overwriting the existing data.

 

Ignore Null

Select this check box to ignore NULL values.

Advanced settings

Salesforce URL

Enter the Webservice URL required to connect to the Salesforce database.

 

Need compression

Select this check box to activate SOAP message compression, which can result in increased performance levels.

 

Trace HTTP message

Select this check box to output the HTTP interactions on the console.

 

Client Id

Enter the ID of the real user to differentiate between those who use the same account and password to access the Salesforce website.

This field is available only when Query is selected from the Query Mode drop-down list.

 

Timeout (milliseconds)

Enter the intended number of query timeout in Salesforce.com.

 

Use Proxy

Select this check box to use a proxy server, and in the Host, Port, User Id, and Password fields displayed, specify the connection parameters of the proxy server.

 

Relationship mapping for upsert

Click the [+] button to add lines as needed and specify the external ID fields in the input flow, the lookup relationship fields in the upsert module, the lookup module as well as the external ID fields in the lookup module.

  • Column name of Talend Schema: the name of the external ID field in the input flow. It refers to the fields in the schema of the preceding component. Such columns are intended to match against the external ID fields specified in the External id name column, which are the fields of the lookup module specified in the Module name column.

  • Lookup relationship field name: the name of the lookup relationship field. It refers to the lookup relationship fields of the module specified in the Module Name field in the Basic settings view and is intended to establish relationship with the lookup module specified in the Module name column in this table. For how to define the lookup relationship fields and how to provide their correct names in the Lookup relationship field name field, go to the Salesforce website and launch the Salesforce Data Loader application for proper actions and information.

  • Module name: the name of the lookup module.

  • Polymorphic: select this check box when and only when polymorphic fields are used for relationship mapping. You will get an error if you keep this check box cleared for a polymorphic field or select it for a field that is not polymorphic. For more information about the polymorphic fields, search polymorphic at http://www.salesforce.com/us/developer/docs/api_asynch/.

  • External id name: the name of the external ID field in the lookup module specified in the Module name column.

This table is available only when UPSERT is selected from the Output Action drop-down list.

 

Concurrency Mode

Select the concurrency mode for the job.

  • Parallel: process batches in parallel mode.

  • Serial: process batches in serial mode.

 

Rows to Commit

Specify the number of lines per data batch to be processed.

 

Bytes to Commit

Specify the number of bytes per data batch to be processed.

 

Wait Time Check Batch State (milliseconds)

Specify the wait time for checking whether the batches in a Job have been processed until all batches are finally processed.

 

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at the Job level as well as at each component level.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component is mainly used when no particular transformation is required on the data to be loaded into Salesforce.com.

Limitation

The bulk data to be processed in Salesforce.com should be .csv format.

Scenario: Inserting bulk data into Salesforce

This scenario describes a four-component Job that submits bulk data in the file SalesforceAccount.txt used in Scenario 2: Gathering erroneous data while inserting data into a Salesforce object into Salesforce, executs your intended actions on the data, and ends up with displaying the Job execution results for your reference.

Setting up the Job

  1. Create a new Job and add a tFileInputDelimited component, a tSalesforceOutputBulkExec component, and two tLogRow components by typing their names on the design workspace or dropping them from the Palette.

  2. Link the tFileInputDelimited component to the tSalesforceOutputBulkExec component using a Row > Main connection.

  3. Link the tSalesforceOutputBulkExec component to the first tLogRow component using a Row > Main connection.

    Link the tSalesforceOutputBulkExec component to the second tLogRow component using a Row > Reject connection.

Configuring the components

  1. Double-click the tFileInputDelimited component to open its Basic settings view.

  2. In the File name/Stream field, browse to or enter the path to the input data file. In this example, it is D:/SalesforceAccount.txt.

  3. Click the [...] button next to Edit schema and in the pop-up schema dialog box, define the schema by adding four columns Name, ParentId, Phone and Fax of String type.

    Click OK to save the changes and close the dialog box.

Setting up the connection to Salesforce

  1. Double-click the tSalesforceOutputBulkExec component to open its Basic settings view.

  2. In the User Id, Password and Security Key fields, enter the user authentication information required to access Salesforce.

  3. Click the [...] button next to the Module Name field and in the pop-up dialog box, select the object you want to access. In this example, it is Account.

  4. In the Bulk File Path field, browse to or enter the path to the CSV file that stores the data for bulk processing.

    Note that the bulk file here to be processed must be in .csv format.

  5. Double-click the first tLogRow component to open its Basic settings view.

  6. In the Mode area, select Table (print values in cells of a table) for better readability of the results.

  7. Do the same to configure the second tLogRow component.

Executing the Job

  1. Press Ctrl + S to save the Job.

  2. Press F6 to execute the Job.

    On the console of the Run view, you can check the execution result.

    In the tLogRow_1 table, you can read the data inserted into Salesforce.

    In the tLogRow_2 table, you can read the rejected data due to the incompatibility with the Account objects you have accessed.

    Note that if you want to transform the input data before loading them into Salesforce, you need to use tSalesforceOutputBulk and tSalesforceBulkExec in cooperation to achieve this purpose. For further information about the usage of these two components, see Scenario: Inserting transformed bulk data into Salesforce.