Listing Amazon SQS queues in an AWS region - 6.3

Talend Open Studio for Big Data Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Open Studio for Big Data
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

Here's an example of using Talend components to iterate and list the URLs of all Amazon SQS (Simple Queue Service) queues in an AWS region.

Creating a Job for listing Amazon SQS queues

Create a Job to open a connection to access Amazon SQS, then iterate and list the URLs of all Amazon SQS queues in a specified region, finally display the total number of the queues in the region.

Prerequisites: You must create several Amazon SQS queues in an AWS region in order to replicate this scenario. In this example, four queues talend, talend-dev, talend-doc and talend-qa have been created in the Asia Pacific (Tokyo) region as shown in below figure.

  1. Create a new Job and add a tSQSConnection component, a tSQSQueueList component, a tIterateToFlow component, a tLogRow component, and a tJava component by typing their names in the design workspace or dropping them from the Palette.

  2. Link the tSQSQueueList component to the tIterateToFlow component using a Row > Iterate connection.

  3. Link the tIterateToFlow component to the tLogRow component using a Row > Main connection.

  4. Link the tSQSConnection component to the tSQSQueueList component using a Trigger > OnSubjobOk connection.

  5. Link the tSQSQueueList component to the tJava component using a Trigger > OnSubjobOk connection.

Opening a connection to access Amazon SQS

Configure the tSQSConnection component to open a connection to access Amazon SQS.

  1. Double-click the tSQSConnection component to open its Basic settings view.

  2. In the Access Key and Secret Key fields, specify the authentication information required to access Amazon SQS.

  3. Select an AWS region available for Amazon SQS from the Region drop-down list. In this example, it is Asia Pacific (Tokyo).

Listing all Amazon SQS queues in an AWS region

Configure the tSQSQueueList component, the tIterateToFlow component, the tLogRow component, and the tJava component to list the URLs of all Amazon SQS queues in an AWS region and display the total number of the queues in the region on the console of Talend Studio.

  1. Double-click the tSQSQueueList component to open its Basic settings view.

  2. Specify the connection details required to access Amazon SQS. In this example, select the Use an existing connection check box and from the Component List drop-down list displayed, select the connection component to reuse its connection details you have already defined.

  3. Double-click the tIterateToFlow component to open its Basic settings view.

  4. Click the button next to Edit schema to open the schema dialog box.

  5. Click the button to add one String type column CurrentQueueURL that will hold the URLs of the queues to be listed. When done, click OK to close the dialog box.

  6. In the Mapping table, set the value for the CurrentQueueURL column. In this example, the value is set to ((String)globalMap.get("tSQSQueueList_1_CURRENT_QUEUE_NAME")), which is the value of the global variable CURRENT_QUEUE_NAME for the tSQSQueueList component.

    Note that you can fill the value by pressing Ctrl + Space to access the global variables list and then selecting tSQSQueueList_1_CURRENT_QUEUE_NAME from the list.

  7. Double-click the tLogRow component to open its Basic settings view, and then select Table (print values in cells of a table) in the Mode area for better readability of the result.

  8. Double-click the tJava component to open its Basic settings view.

  9. In the Code field, enter the following code to display the total number of the queues in the region.

    System.out.println("The number of queues: " + ((Integer)globalMap.get("tSQSQueueList_1_NB_QUEUE")));

Executing the Job to list Amazon SQS queues

After setting up the Job and configuring the components used in the Job for listing Amazon SQS queues, you can then execute the Job and verify the Job execution result.

  1. Press Ctrl + S to save the Job.

  2. Press F6 to execute the Job.