Sending data to a Data Stewardship Resolution campaign to fix issues - Cloud

Talend Cloud Apps Connectors Guide
Version
Cloud
Language
English
Product
Talend Cloud
Module
Talend Data Inventory
Talend Data Preparation
Talend Pipeline Designer
Content
Administration and Monitoring > Managing connections
Design and Development > Designing Pipelines
Last publication date
2024-03-21

This scenario aims at helping you set up and use connectors in a pipeline. You are advised to adapt it to your environment and use case.

Before you begin

  • You have created a Test connection and dataset that contains data about customers, including ID, names, emails, countries, etc. See Test connection properties for more information.
  • You have already created a resolution campaign in Talend Cloud Data Stewardship in which you will send customer data with empty emails.
  • You have the Campaign owner role that allows you to display and retrieve the campaign in Talend Cloud Pipeline Designer. See this documentation for more information.

Procedure

  1. Click Add pipeline on the Pipelines page. Your new pipeline opens.
  2. Give the pipeline a meaningful name.

    Example

    Send customers with empty emails to Data Stewardship resolution campaign
  3. Click ADD SOURCE and select your source dataset, customers in the panel that opens. You can preview a sample of your data that contains information about customers.
  4. Click and add a Filter processor to the pipeline in order to filter the customers and keep only the ones that contain empty emails. The configuration panel opens.
  5. Give a meaningful name to the processor.

    Example

    filter on empty emails
  6. In the FILTERS area:
    1. Select .mail in the Input list, as it is the field that contains the email information.
    2. Select LENGTH in the Optionally select a function to apply list, as you want to count the length of the email field value.
    3. Select == in the Operator list and type in 0 in the Value field as you want to filter on email fields whose length equals to zero, that is to say empty email fields.
    4. Click SAVE to save your configuration.
      The data is filtered and two out of 14 records match the filtering condition.
  7. Click and add a Field Selector processor to the pipeline in order to select specific fields, change their case and simplify their names. The configuration panel opens.
  8. Give a meaningful name to the processor.

    Example

    select relevant fields
  9. In the SELECTORS area:
    1. Select .id in the Input list and enter ID in the Output field, as you want to select the fields corresponding to customer identifiers.
    2. Add a NEW ELEMENT and select .customer_first_name in the Input list and enter FIRST_NAME in the Output field, as you want to select the fields corresponding to customer first names.
    3. Add a NEW ELEMENT and select .customer_last_name in the Input list and enter LAST_NAME in the Output field, as you want to select the fields corresponding to customer last names.
    4. Add a NEW ELEMENT and select .company in the Input list and enter COMPANY in the Output field, as you want to select the fields corresponding to customer companies.
    5. Add a NEW ELEMENT and select .email in the Input list and enter EMAIL in the Output field, as you want to select the fields corresponding to customer emails.
    6. Click SAVE to save your configuration.
      The selected fields are kept and renamed according to your configuration.
  10. Click the ADD DESTINATION item and go the Campaigns tab to select the Resolution campaign in which the fields with empty emails will be inserted.
  11. Give a meaningful name to the destination; insert data into campaign for example.
  12. In the Configuration tab of the destination, click Main:
    1. Select Insert in the Action list to insert the data into the campaign when executing the pipeline.
    2. Enable the Override enforcement of the data model option in order to process data on the Talend Cloud Data Stewardship server even if the schema type is not valid (no input validation of the schema is performed).
    3. Select New in the State field and enter the user email of your choice (jane@talend.com for example) in the Assignee field as you want to assign this new resolution task to a data steward that will fix the email issues.
    4. Select High in the Priority list as it is a high priority task.
    5. Click SAVE to save your configuration.
  13. On the top toolbar of Talend Cloud Pipeline Designer, click the Run button to open the panel allowing you to select your run profile.
  14. Select your run profile in the list (for more information, see Run profiles), then click Run to run your pipeline.

Results

Your pipeline is being executed, the customer emails have been filtered and the output flow is inserted into the Data Stewardship Resolution campaign you have defined.

What to do next

  • The data steward that has been assigned to the Resolution task will review and fix the email issues in Talend Cloud Data Stewardship.
  • After that:
    • if the data steward is able to fix the issues, you will be able to retrieve the corrections in Talend Cloud Pipeline Designer.
    • if the data steward cannot find the missing emails, you will have the possibility to delete these fields from the campaign.