tDropboxPut - 6.3

Talend Open Studio for Big Data Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Open Studio for Big Data
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

Function

This component uploads data to Dropbox from either a local file or a given data flow.

Purpose

The tDropboxPut component uploads files to a Dropbox account.

tDropboxPut properties

Component family

Cloud/Dropbox

 

Basic settings

Use existing connection

Select this check box and in the Component List click the relevant connection component to reuse the connection details you already defined.

 

Access token

Enter the access token required by the Dropbox account you need to connect to. This access token allows the Studio to make Dropbox API calls for that Dropbox account.

Note that a Dropbox App should have been created under that account before generating the access token. For further information about a Dropbox access token, see https://www.dropbox.com/developers/blog/94/generate-an-access-token-for-your-own-account.

 

Use HTTP proxy

If you are using a proxy, select this check box and enter the host and port information of that proxy in the corresponding fields that are displayed.

 

Path (File only)

Enter the path pointing to the file you need to write contents in. This file will be created on the fly if it does not exist.

Note that the path string should start with a slash (/). It is the root folder of the Dropbox App for which you are using the current access token.

 

Upload mode

Select upload mode to be used:

  • Rename if existing: the uploaded file is automatically renamed. For example, a file named test.txt might be renamed to test (1).txt.

  • Replace if existing: the uploaded file replaces the existing one.

  • Update specified revision: the file you are uploading is used to update a specific revision of that file. If the revision you specify is the latest revision, then the existing file on Dropbox is replaced; if it is an older revision, the file you are uploading is renamed to indicate that a conflict is encountered; if the revision does not exist, an error is returned.

 

Upload incoming content as file

Select this radio button to read data directly from the input flow of the preceding component and write the data into the file specified in the Path field.

 

Schema and Edit schema

A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. The schema is either Built-In or stored remotely in the Repository.

Note that the schema of this component is read-only with a single column named content and it receives data from the content column of its input schema only. This means that you must use a content column in the input data flow to carry the data to be uploaded. This type of column is typically provided by the tFileInputRaw component. For further information, see tFileInputRaw.

The Schema field is not available when you have selected the Expose as OutputStream or the Upload local file radio button.

 

Upload local file

Select this radio button to upload a locally stored file to Dropbox. In the File field that is displayed, you need to enter the path or browse to this file.

 

Expose as OutputStream

Select this check box to expose the output stream of this component as a variable named OUTPUTSTREAM so that the other components can reuse this variable to write the contents to be uploaded into the exposed output stream.

For example, you can use the Use output stream feature of the tFileOutputDelimited component to feed a given tDropboxPut's exposed output stream. For further information, see tFileOutputDelimited.

Advanced settings

tStatCatcher Statistics

Select this check box to collect log data at the component level.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

OUTPUTSTREAM: the contents of the uploaded file. This is a Flow variable and it returns an object.

Note that this OUTPUTSTREAM variable is available only when the Expose as OutStream check box has been selected.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

This component is used either standalone in a Subjob to directly upload a local file to Dropbox or as an end component of a Job flow to upload given data being handled in this flow.

Limitation

n/a

Scenario: Uploading files to Dropbox

In this scenario, a six-component Job consisting of three Subjobs is created to write data onto Dropbox using different upload modes.

Before replicating this scenario, you need to create a Dropbox App under the Dropbox account to be used. In this scenario, the Dropbox App to be used is named to talenddrop and thus the root folder in which files are uploaded is talenddrop, too. In addition, the access token to this folder has been generated from the App console provided by Dropbox.

For further information about a Dropbox App, see https://www.dropbox.com/developers/apps/.

Linking the components

  1. In the Integration perspective of the Studio, create an empty Job from the Job Designs node in the Repository tree view.

    For further information about how to create a Job, see Talend Studio User Guide.

  2. In the workspace, enter the name of the component to be used and select this component from the list that appears. In this scenario, the components are tDropboxConnection, tFixedFlowInput, tFileOutputDelimited, tFileInputRaw and two tDropboxPut components.

    The tFixedFlowInput component generates some data to be uploaded to Dropbox in this scenario. In the real-world case, you can use other components such as tMysqlInput or tMap in the place of tFixedFlowInut to design a sophisticated process to prepare your data to be handled.

  3. Connect tFixedFlowInput to tFileOutputDelimited using the Row > Main link.

  4. Do the same to connect tFileOutputDelimited to one of the two tDropboxPut components and connect tFileInputRaw to the other tDropboxPut component.

  5. Connect tDropboxConnection to tFixedFlowInput using the Trigger > On Subjob Ok link. Then connect tFixedFlowInput to tFileInputRaw using the same type of link.

Connecting to Dropbox

  1. Double-click tDropboxConnection to open its Component view.

  2. In the Access token field, paste the token that you have generated via the App console of Dropbox for accessing the Dropbox App folder to be used.

Generating the output stream

Defining the input data

  1. Double-click tFixedFlowInput to open its Component view.

    In this scenario, only three rows of sample data are created to indicate three countries and their calling codes.

    33;France
    86;China
    81;Japan
  2. Click the [...] button next to Edit schema to open the schema editor.

  3. Click the [+] button twice to add two rows and in the Column column, rename them to code and country, respectively.

  4. Click OK to validate these changes and accept the propagation prompted by the pop-up dialog box.

  5. In the Mode area, select the Use Inline Table radio button. The code and the country column have been automatically created in this table.

  6. Enter the sample data mentioned above in this table.

Defining the output stream

  1. Double-click tFileOutputDelimited to open its Component view.

  2. Select the Use output stream check box to write the data to be outputted into a given output stream.

  3. In the Output stream field, enter the code to define the output stream you need to write data in. In this scenario, it is the output stream of the tDropboxPut_1 component linked with the current component. Thus the code used to write the data reads as follows:

    ((java.io.OutputStream)globalMap.get("tDropboxPut_1_OUTPUTSTREAM"))

    Note that in this example code, the tDropboxPut component has the number 1 as its affix, which represents its component ID distributed automatically within this Job. If the tDropboxPut component you are using has a different ID, you need to adapt the code to that ID number.

  4. Click Edit schema to verify that the schema of this component is identical with that of the preceding tFixedFlowInput component. If not so, click the Sync columns button to make both of the schemas identical.

Exposing the tDropboxPut output stream

  1. Double-click the tDropboxPut component linked with tFileOutputDelimited to open its Component view.

  2. Select the Use existing connection check box to reuse the connection created by tDropboxConnection.

  3. In the Path field, enter the path pointing to the file you need to write data in, with a slash (/) at the beginning of the path. For example, enter /calling_code.csv.

  4. In the Upload mode area, select the Rename if Existing radio button.

  5. Select the Expose As OutputStream radio button to expose the output stream of this component so that the other component, tFileOutputDelimited in this scenario, can write data in the stream.

Defining the media data to be uploaded

  1. Double-click tFileInputRaw to open its Component view.

    This component is used to read a picture named esb_architecture.png into the data flow. In the real-world practice, this file can be of many other formats, such as pdf, xls, ppt or mp3.

  2. In the Filename field, enter the path or browse to the file you need to upload.

  3. In the Mode area, select the Read the file as a bytes array radio button.

Uploading the incoming contents

  1. Double-click the tDropboxPut component linked with tFileInputRaw to open its Component view.