Scenario: Handling data with Amazon Aurora - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

This scenario describes a Job that writes the user information into Amazon Aurora, and then reads the information in Amazon Aurora and displays it on the console.

The scenario requires the following seven components:

  • tAmazonAuroraConnection: opens a connection to Amazon Aurora.

  • tFixedFlowInput: defines the user information data structure, and sends the data to the next component.

  • tAmazonAuroraOutput: writes the data it receives from the preceding component into Amazon Aurora.

  • tAmazonAuroraCommit: commits in one go the data processed to Amazon Aurora.

  • tAmazonAuroraInput: reads the data from Amazon Aurora.

  • tLogRow: displays the data it receives from the preceding component on the console.

  • tAmazonAuroraClose: closes the connection to Amazon Aurora.

Adding and linking the components

  1. Create a new Job and add seven components listed previously by typing their names in the design workspace or dropping them from the Palette.

  2. Connect tFixedFlowInput to tAmazonAuroraOutput using a Row > Main connection.

  3. Do the same to connect tAmazonAuroraInput to tLogRow.

  4. Connect tAmazonAuroraConnection to tFixedFlowInput using a Trigger > OnSubjobOk connection.

  5. Do the same to connect tFixedFlowInput to tAmazonAuroraCommit, tAmazonAuroraCommit to tAmazonAuroraInput, and tAmazonAuroraInput to tAmazonAuroraClose.

Configuring the components

Opening a connection to Amazon Aurora

  1. Double-click tAmazonAuroraConnection to open its Basic settings view.

  2. In the Host, Port, Database, Username and Password fields, enter the information required for the connection to Amazon Aurora.

Writing the data into Amazon Aurora

  1. Double-click tFixedFlowInput to open its Basic settings view.

  2. Click the [...] button next to Edit schema and in the pop-up window define the schema by adding three columns: id of Integer type, and name and city of String type.

    Click OK to validate the changes and accept the propagation prompted by the pop-up dialog box.

  3. In the Mode area, select Use Inline Content (delimited file) and enter the following user information in the Content field.

    1;George;Bismarck
    2;Abraham;Boise
    3;Taylor;Nashville
    4;William;Jefferson City
    5;Alexander;Jackson
    6;James;Boise
    7;Gerald;Little Rock
    8;Tony;Richmond
    9;Thomas;Springfield
    10;Andre;Nashville
  4. Double-click tAmazonAuroraOutput to open its Basic settings view.

  5. Select the Use an existing connection check box and in the Component List that appears, select the connection component you have configured.

  6. In the Table field, enter or browse to the table into which you want to write the data. In this example, it is TalendUser.

  7. Select Drop table if exists and create from the Action on table drop-down list, and select Insert from the Action on data drop-down list.

  8. Double-click tAmazonAuroraCommit to open its Basic settings view.

  9. Clear the Close Connection check box if it is selected.

Retrieving the data from Amazon Aurora

  1. Double-click tAmazonAuroraInput to open its Basic settings view.

  2. Select the Use an existing connection check box and in the Component List that appears, select the connection component you have configured.

  3. Click the [...] button next to Edit schema and in the pop-up window define the schema by adding three columns: id of Integer type, and name and city of String type. The data structure is same as the structure you have defined for tFixedFlowInput.

  4. In the Table Name field, enter or browse to the table into which you write the data. In this example, it is TalendUser.

  5. Click the Guess Query button to generate the query. The Query field will be filled with the automatically generated query.

  6. Double-click tLogRow to open its Basic settings view.

  7. In the Mode area, select Table (print values in cells of a table) for better readability of the result.

Closing the connection to Amazon Aurora

  1. Double-click tAmazonAuroraClose to open its Basic settings view.

  2. In the Component List, select the connection component you have configured.

Saving and executing the Job

  1. Press Ctrl + S to save the Job.

  2. Press F6 or click Run on the Run tab to run the Job.

    As shown above, the user information is written into Amazon Aurora, and then the data is retrieved from Amazon Aurora and displayed on the console.