Scenario: Sorting and aggregating the input data - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

This scenario describes a Job that sorts the entries of the input data based on two columns and displays the sorted data on the console, then aggregates the sorted data based on one column and displays the aggregated data on the console.

Adding and linking components

  1. Create a new Job and add the following components by typing their names in the design workspace or dropping them from the Palette: a tFixedFlowInput component, a tSortRow component, a tAggregateSortedRow component, and two tLogRow components.

  2. Link tFixedFlowInput to tSortRow using a Row > Main connection.

  3. Do the same to link tSortRow to the first tLogRow, link the first tLogRow to tAggregateSortedRow, and link tAggregateSortedRow to the second tLogRow.

Configuring the components

Sorting the input data

  1. Double-click tFixedFlowInput to open its Basic settings view.

  2. Click the [...] button next to Edit schema and in the pop-up window define the schema by adding four columns: Id and Age of Integer type, and Name and Team of String type.

    Click OK to close the schema editor and accept the propagation prompted by the pop-up dialog box.

  3. In the Mode area, select Use Inline Content(delimited file), and then in the Content field displayed, enter the input data to be sorted and aggregated. In this example, the input data is as follows:

    1;C01;28;Component Team
    2;D01;32;Doc Team
    3;C02;26;Component Team
    4;Q01;27;QA Team
    5;C03;24;Component Team
    6;D02;30;Doc Team
    7;Q02;23;QA Team
    8;C04;26;Component Team
  4. Double-click tSortRow to open its Basic settings view.

  5. Click the [+] button below the Criteria table to add as many rows as required and then specify the sorting criteria in the table. In this example, two rows are added, and the input entries will be sorted based on the column Team and then the column Age, both in ascending order.

  6. Double-click the first tLogRow to open its Basic settings view.

  7. In the Mode area, select Table (print values in cells of a table) for better readability of the sorting result.

Aggregating the sorted data

  1. Double-click tAggregateSortedRow to open its Basic settings view.

  2. Click the [...] button next to Edit schema and in the pop-up window define the schema by adding five columns: AggTeam of String type, AggCount, MinAge, MaxAge, and AvgAge of Integer type.

    Click OK to close the schema editor and accept the propagation prompted by the pop-up dialog box.

  3. In the Input rows count field, enter the exact number of rows of the input data. In this example, it is 8.

  4. Click the [+] button below the Group by table to add as many rows as required and specify the aggregation set in the table. In this example, the data will be aggregated based on the input column Team.

  5. Click the [+] button below the Operations table to add as many rows as required and specify the operation to be carried out and the corresponding input column from which the data will be taken for each output column. In this example, we want to calculate the number of the input entries, the minimum age, the maximum age, and the average age for each team.

  6. Double-click the second tLogRow to open its Basic settings view.

  7. In the Mode area, select Table (print values in cells of a table) for better readability of the sorting result.

Saving and executing the Job

  1. Press Ctrl + S to save the Job.

  2. Execute the Job by pressing F6 or clicking Run on the Run tab.

    As shown above, the input entries are sorted based on the column Team and then the column Age, both in ascending order, and the sorted entries are then aggregated based on the column Team.