tFileFetch Standard properties - 6.5

Internet (Integration)

author
Talend Documentation Team
EnrichVersion
6.5
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance > Third-party systems > Internet components (Integration)
Data Quality and Preparation > Third-party systems > Internet components (Integration)
Design and Development > Third-party systems > Internet components (Integration)
EnrichPlatform
Talend Studio

These properties are used to configure tFileFetch running in the Standard Job framework.

The Standard tFileFetch component belongs to the Internet family.

The component in this framework is available in all Talend products.

Basic settings

Protocol

Select the protocol you want to use from the list and fill in the corresponding fields: http, https, ftp, smb.

The properties differ slightly depending on the type of protocol selected. The additional fields are defined in this table, after the basic settings.

URI

Type in the URI of the site from which the file is to be fetched.

Use cache to save resource

Select this check box to save the data in the cache.

This option allows you to process the file data flow (in streaming mode) without saving it on your drive. This is faster and improves performance.

Domain

Enter the Microsoft server domain name.

Available for the smb protocol.

Username and Password

Enter the authentication information required to access the server.

To enter the password, click the [...] button next to the password field, and then in the pop-up dialog box enter the password between double quotes and click OK to save the settings.

Available for the smb protocol.

Destination Directory

Browse to the destination folder where the file fetched is to be placed.

Destination Filename

Enter a new name for the file fetched.

If the Upload file option on the Advanced settings view is selected, the upload response will be saved in this file.

Create full path according to URI

It allows you to reproduce the URI directory path. To save the file at the root of your destination directory, clear the check box.

Available for the http, https and ftp protocols.

Add header

Select this check box if you want to add one or more HTTP request headers as fetch conditions. In the Headers table, enter the name(s) of the HTTP header parameter(s) in the Name field and the corresponding value(s) in the Value field.

Available for the http and https protocols.

POST method

This check box is selected by default. It allows you to use the POST method. In the Parameters table, enter the name of the variable(s) in the Name field and the corresponding value in the Value field.

Clear the check box if you want to use the GET method.

Available for the http and https protocols.

Die on error

Clear this check box to skip the rows in error and to complete the process for the error free rows

Available for the http, https and ftp protocols.

Read Cookie

Select this check box for tFileFetch to load a web authentication cookie.

Available for the http, https, ftp and smb protocols.

Save Cookie

Select this check box to save the web page authentication cookie. This means you will not have to log on to the same web site in the future.

Available for the http, https, ftp and smb protocols.

Cookie file

Type in the full path to the file which you want to use to save the cookie or click [...] and browse to the desired file to save the cookie.

Available for the http, https, ftp and smb protocols.

Cookie policy

Choose a cookie policy from this drop-down list. Four options are available, BROWSER_COMPATIBILITY, DEFAULT, NETSCAPE and RFC_2109.

Available for the http, https, ftp and smb protocols.

Single cookie header

Check this box to put all cookies into one request header for maximum compatibility among different servers.

Available for the http, https, ftp and smb protocols.

Advanced settings

tStatCatcher Statistics

Select this check box to collect the log data at each component level.

Timeout

Enter the number of milliseconds after which the protocol connection should close.

Available for the http and https protocols.

Print response to console

Select this check box to print the server response in the console.

Available for the http and https protocols.

Upload file

Select this check box to upload one or more files to the server. Then in the Files table displayed, click the [+] button to add the file(s) to upload and define the following parameters for each file:

  • Name: the new name of the file after being uploaded, between double quotation marks.

  • File: the full path of the file to upload, e.g. "D:/filefetch.txt".

  • Content-Type: the content type of the file to upload. The default value is "application/octet-stream".

  • Charset: the character set of the file to upload. The default value is "ISO-8859-1".

Available for the http and https protocols, with the POST method option in the Basic settings view selected.

With this option selected, the upload response will be saved in the file specified in the Destination filename field in the Basic settings view.

Enable proxy server

Select this check box if you are connecting via a proxy and complete the fields which follow with the relevant information.

Available for the http, https and ftp protocols.

Enable NTLM Credentials

Select this check box if you are using an NTLM authentication protocol.

Domain: The client domain name.

Host: The client's IP address.

Available for the http and https protocols.

Need authentication

Select this check box and enter the username and password in the relevant fields, if they are required to access the protocol.

Available for the http and https protocols.

Support redirection

Select this check box to repeat the redirection request until redirection is successful and the file can be retrieved.

Available for the http, https and ftp protocols.

Global Variables

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

INPUT_STREAM: the content of the file being fetched. This is a Flow variable and it returns an InputStream.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Usage

Usage rule

This component is generally used as a start component to feed the input flow of a Job and is often connected to the Job using an OnSubjobOk or OnComponentOk link, depending on the context.

Limitation

Due to license incompatibility, one or more JARs required to use this component are not provided. You can install the missing JARs for this particular component by clicking the Install button on the Component tab view. You can also find out and add all missing JARs easily on the Modules tab in the Integration perspective of your studio. For details, see Installing external modules.