tAzureStorageGet properties - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

Component Family

Cloud / Azure Storage

 

Function

tAzureStorageGet connects to a given Azure storage account and retrieves blobs from a given container of that account.

Purpose

tAzureStorageGet allows you to specify filters you want to apply on the virtual hierarchy of the blobs and write selected blobs in a local folder.

Basic settings

Use an existing connection

Select this check box and in the Component List click the relevant connection component to reuse the connection details you already defined.

 

Account name

Enter the name of the storage account you need to access. A storage account name can be found in the Manage Access Keys dashboard of the Microsoft Azure Storage system to be used.

 

Account key

Enter the key associated with the storage account you need to access. Two keys are available for each account and by default, either of them can be used for this access.

 

Protocol

Select the protocol for this connection to be created.

 

Container

Enter the name of the container you need to retrieve blobs from.

 

Local folder

Enter the path, or browse to the folder in which you need to store the retrieved blobs.

 

Blobs

Complete this table to select the blobs to be retrieved. The parameters to be provided are:

  • Blob prefix: enter the common prefix of the names of the blobs you need to retrieve. This prefix allows you to filter the blobs which have the specified prefix in their names in the given container.

    A blob name contains the virtual hierarchy of the blob itself. This hierarchy is a virtual path to that blob and is relative to the container where that blob is stored. For example, in a container named photos, the name of a photo blob might be 2014/US/Oakland/Talend.jpg.

    For this reason, when you define a prefix, you are actually designating a directory level as the blob filter, for example, 2014/ or 2014/US/.

    If you want to select the blobs stored directly beneath the container level, that is to say, the blobs without virtual path in their names, remove quotation marks and enter null.

  • Include subdirectories: select this check box to retrieve all of the sub-folders and the blobs in those folders beneath the designated directory level in the Blob prefix column. If you leave this check box clear, tAzureStorageGet returns only the blobs directly beneath that directory level.

  • Create parent directories: select this check box to replicate the virtual directory of the retrieved blobs in the local folder.

    Note that if you leave this check box clear, there must be the same directory in the local folder as the retrieved blobs have in the container; otherwise, those blobs cannot be retrieved.

 

Die on error

Select this check box to stop the execution of the Job when an error occurs.

Clear the check box to skip any rows on error and complete the process for error-free rows. When errors are skipped, you can collect the rows on error using a Row > Reject link.

Advanced settings

tStatCatcher Statistics

Select this check box to gather the Job processing metadata at the Job level as well as at each component level.

Usage

This component is used as a standalone component.

Knowledge about Microsoft Azure Storage is required.

Global Variables

ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box.

ACCOUNT_NAME: the account name for accessing the storage. This is an After variable and it returns a string.

ACCOUNT_KEY: the key associated with the account for accessing the storage. This is an After variable and it returns a string.

CONTAINER: the container name used in this component. This is an After variable and it returns a string.

LOCAL_FOLDER: the local directory used in this component. This is an After variable and it returns a string.

A Flow variable functions during the execution of a component while an After variable functions after the execution of the component.

To fill up a field or expression with a variable, press Ctrl + Space to access the variable list and choose the variable to use from it.

For further information about variables, see Talend Studio User Guide.

Log4j

If you are using a subscription-based version of the Studio, the activity of this component can be logged using the log4j feature. For more information on this feature, see Talend Studio User Guide.

For more information on the log4j logging levels, see the Apache documentation at http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html.

Limitation

n/a