Scenario: Verifing the absence of a bucket, creating it and listing all the S3 buckets - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

In this scenario, tS3BucketExist is used to verify the absence of a bucket, tS3BucketCreate to create that bucket upon confirmation, and tS3BucketList to list all the buckets on Amazon S3.

Linking the components

  1. Drop tS3Connection, tS3BucketExist, tS3BucketCreate, tS3BucketList, tIterateToFlow and tLogRow onto the workspace.

  2. Link tS3Connection to tS3BucketExist using the OnSubjobOk trigger.

  3. Link tS3BucketExist to tS3BucketCreate using the Run if trigger.

  4. Link tS3BucketCreate to tS3BucketList using the OnSubjobOk trigger.

  5. Link tS3BucketList to tIterateToFlow using the Row > Iterate connection.

  6. Link tIterateToFlow to tLogRow using the Row > Main connection.

Configuring the components

  1. Double-click tS3Connection to open its Basic settings view.

  2. In the Access Key and Secret Key fields, enter the authentication credentials.

  3. Double-click tS3BucketExist to open its Basic settings view.

  4. Select the Use existing connection check box to reuse the connection.

  5. In the Bucket field, enter the bucket name to check if it exists.

  6. Double-click the If link to define the condition.

  7. In the Condition box, enter the expression:

    !((Boolean)globalMap.get("tS3BucketExist_1_BUCKET_EXIST"))

    This way, the rest of the Job will be executed if the specified bucket does not exist.

  8. Double-click tS3BucketCreate to open its Basic settings view.

    Select the Use existing connection check box to reuse the connection.

    In the Bucket field, enter the bucket name to create.

  9. Double-click tS3BucketList to open its Basic settings view.

    Select the Use existing connection check box to reuse the connection.

  10. Double-click tIterateToFlow to open its Basic settings view.

  11. Click Edit schema to open the schema editor.

    Click the [+] button to add one column, namely bucket_list of the String type.

    Click Ok to validate the setup and close the schema editor.

  12. In the Mapping area, press Ctrl + Space in the Value field to choose the variable tS3BucketList_1_CURRENT_BUCKET_NAME.

  13. Double-click tLogRow to open its Basic settings view.

    Select Table (print values in cells of a table) for a better display of the results.

Executing the Job

  1. Press Ctrl + S to save the Job.

  2. Press F6 to run the Job.

    As shown above, the bucket is created and all the buckets are listed.

  3. Go to the S3 web console:

    As shown above, the bucket has been created on the S3 server.

  4. Refresh the S3 Browser client:

    This shows that the S3 Create action was performed successfully.