Scenario: Handling data with Amazon Aurora - 6.3

Talend Open Studio for Big Data Components Reference Guide

EnrichVersion
6.3
EnrichProdName
Talend Open Studio for Big Data
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

This scenario describes a Job that writes the user information into Amazon Aurora, and then reads the information in Amazon Aurora and displays it on the console.

The scenario requires the following seven components:

  • tAmazonAuroraConnection: opens a connection to Amazon Aurora.

  • tFixedFlowInput: defines the user information data structure, and sends the data to the next component.

  • tAmazonAuroraOutput: writes the data it receives from the preceding component into Amazon Aurora.

  • tAmazonAuroraCommit: commits in one go the data processed to Amazon Aurora.

  • tAmazonAuroraInput: reads the data from Amazon Aurora.

  • tLogRow: displays the data it receives from the preceding component on the console.

  • tAmazonAuroraClose: closes the connection to Amazon Aurora.

Adding and linking the components

  1. Create a new Job and add seven components listed previously by typing their names in the design workspace or dropping them from the Palette.

  2. Connect tFixedFlowInput to tAmazonAuroraOutput using a Row > Main connection.

  3. Do the same to connect tAmazonAuroraInput to tLogRow.

  4. Connect tAmazonAuroraConnection to tFixedFlowInput using a Trigger > OnSubjobOk connection.

  5. Do the same to connect tFixedFlowInput to tAmazonAuroraCommit, tAmazonAuroraCommit to tAmazonAuroraInput, and tAmazonAuroraInput to tAmazonAuroraClose.

Configuring the components

Opening a connection to Amazon Aurora

  1. Double-click tAmazonAuroraConnection to open its Basic settings view.

  2. In the Host, Port, Database, Username and Password fields, enter the information required for the connection to Amazon Aurora.

Writing the data into Amazon Aurora

  1. Double-click tFixedFlowInput to open its Basic settings view.

  2. Click the [...] button next to Edit schema and in the pop-up window define the schema by adding three columns: id of Integer type, and name and city of String type.

    Click OK to validate the changes and accept the propagation prompted by the pop-up dialog box.

  3. In the Mode area, select Use Inline Content (delimited file) and enter the following user information in the Content field.

    1;George;Bismarck
    2;Abraham;Boise
    3;Taylor;Nashville
    4;William;Jefferson City
    5;Alexander;Jackson
    6;James;Boise
    7;Gerald;Little Rock
    8;Tony;Richmond
    9;Thomas;Springfield
    10;Andre;Nashville
  4. Double-click tAmazonAuroraOutput to open its Basic settings view.

  5. Select the Use an existing connection check box and in the Component List that appears, select the connection component you have configured.

  6. In the Table field, enter or browse to the table into which you want to write the data. In this example, it is TalendUser.

  7. Select Drop table if exists and create from the Action on table drop-down list, and select Insert from the Action on data drop-down list.

  8. Double-click tAmazonAuroraCommit to open its Basic settings view.

  9. Clear the Close Connection check box if it is selected.

Retrieving the data from Amazon Aurora

  1. Double-click tAmazonAuroraInput to open its Basic settings view.

  2. Select the Use an existing connection check box and in the Component List that appears, select the connection component you have configured.

  3. Click the [...] button next to Edit schema and in the pop-up window define the schema by adding three columns: id of Integer type, and name and city of String type. The data structure is same as the structure you have defined for tFixedFlowInput.

  4. In the Table Name field, enter or browse to the table into which you write the data. In this example, it is TalendUser.

  5. Click the Guess Query button to generate the query. The Query field will be filled with the automatically generated query.

  6. Double-click tLogRow to open its Basic settings view.