Scenario: Writing and loading data through a named-pipe - 6.1

Talend Components Reference Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for Data Quality
Talend Open Studio for ESB
Talend Open Studio for MDM
Talend Real-Time Big Data Platform
task
Data Governance
Data Quality and Preparation
Design and Development
EnrichPlatform
Talend Studio

The following scenario creates a multi-component Job, which writes data into an open named-pipe and displays the data onto the console.

Dropping and linking the components

  1. Drop the following components from the Palette to the design workspace: tNamedPipeOpen, tParallelize, tNamedPipeClose, tFileInputDelimited, tSleep, tLogRow, tRowGenerator and tNamedPipeOutput.

  2. Connect tNamedPipeOpen to tParallelize using a Trigger > OnSubjobOk connection.

  3. Connect tParallelize to tFileInputDelimited using a Trigger > Parallelize connection.

  4. Connect tParallelize to tSleep using a Trigger > Parallelize connection.

  5. Connect tFileInputDelimited to tLogRow using a Row > Main connection.

  6. Connect tParallelize to tNamedPipeClose using a Trigger > Synchronize (Wait for all) connection.

  7. Connect tSleep to tRowGenerator using a Trigger > OnComponentOk connection.

  8. Connect tRowGenerator to tNamedPipeOutput using a Row > Main connection.

Configuring the components

Configuring the input component

  1. Double-click tNamedPipeOpen to define its propeties in its Basic settings view.

    Fill in the Name field with the name of a named-pipe and select Delete if already exist to avoid duplicate named-pipe.

  2. Double-click tParallelize to define its properties in its Basic settings view.

    Select end of all subjobs from the Wait for list.

    Fill in the Sleep Duration field with 100 to set the sleep duration.

  3. Double-click tFileInputDelimited to define its properties in its Basic settings view.

    Fill in the File name/Stream field with the following expression to use the name of the existing named-pipe defined in the Basic settings view of tNamedPipeOpen:

  4. ((String)globalMap.get("tNamedPipeOpen_1_PIPE_NATIVE_NAME"))

  5. Click the three-dot button next to Edit schema.

  6. Click the plus button to add three columns for tFileInputDelimited. Fill the three Column fields with id, first_name and last_name and set the Type of id to Integer. Keep the rest of the settings as default.

  7. Click OK to save the settings for the schema.

  8. Keep the rest of the settings in the Basic settings view of tFileInputDelimited as default.

  9. Double-click tSleep and fill the Pause (in seconds) field with 1.

  10. Double-click tRowGenerator to define its properties in its Basic settings view.

  11. Click RowGenerator Editor to define the schema.

  12. Click the plus button to add three columns for tRowGenerator. Fill the three Column fields with id, first_name and last_name and set the Type of id to Integer. Keep the rest of the settings of Type as default.

  13. Select sequence from the list in the Functions field for id.

  14. Select getFirstName from the list in the Functions field for Column first_name.

  15. Select TalendDataGenerator.getLastName from the list in the Functions field for Column last_name.

  16. Select id, fill the Value field under Function parameters tab with s1 for sequence identifier, 1001 for start value and 1 for step.

  17. Click OK to save the settings.

Configuring the output component

  1. Double-click tNamedPipeOutput to define its properties in its Basic settings view.

  2. Select the Use existing pipe connection check box and select tNamedPipeOpen_1 from the Pipe component list.

  3. Select Delete pipe if it exists to avoid duplicate named-pipe.

  4. Click Sync columns to retrieve the schema from the preceding component.

  5. Leave the rest of the settings as they are.

  6. Double-click tLogRow to define its properties in its Basic settings view.

  7. Click Sync columns to retrieve the schema from the preceding component.

  8. Select Table in the Mode area.

  9. Double-click tNamedPipeClose to define its properties in its Basic settings view.

  10. Select tNamedPipeOpen_1 from the Pipe list.

Saving and executing the Job

  • Press F6 to execute the Job.

The data written into the named-pipe is displayed onto the console.