Python properties - Cloud

Talend Cloud Pipeline Designer Processors Guide

author
Talend Documentation Team
EnrichVersion
Cloud
EnrichProdName
Talend Cloud
task
Design and Development > Designing Pipelines
EnrichPlatform
Talend Pipeline Designer

Properties to configure in order to process your records using Python code.

The supported version of Python is 2.7.x.

Property

Configuration

Name

Enter a display name for the Python processor.

Description (optional)

Enter a description for the Python processor.

Map type

Select the type of transformation you want to to perform: either
  • MAP: only returns one element.
  • FLATMAP: returns a list of elements (0 or more) as an iterator.

For more information, see Differences between MAP and FLATMAP.

Python code

Enter the Python code in this block. You can click the button to open the editor in a separate dialog box.

Several features are available in the editor to help you write your code such as:
  • autocompletion when typing (for functions,record values, etc.),

  • element, line and syntax highlight (such as matching parentheses or brackets),

  • cut, copy and paste.

  • variables and Avpath syntax.

For useful tips on Python syntax, see Useful tips when using Python.

For more information on Python, see the Python online documentation.

Differences between MAP and FLATMAP

  • MAP:

    The Python processor automatically emits an output record for each input. It expects the result to be in a variable called output. The simplest Python code that simply passes through every record is therefore:

    output = input

  • FLATMAP:

    It is up to the python code to emit records explicitly. This is useful if the processing logic wants to emit less output records than input, typically for complex filtering (for simple/normal filtering, use the dedicated filter processor instead).

    To emit a record in output, you must explicitly append it in a variable called outputList. The simplest python code that simply passes through every record is therefore:

    outputList.append(input)

    But the following is also possible:
    if input["passengers"] > 4:
        outputList.append(input)

    In this case, the Python processor acts like a filter, as it will only pass input when passengers > 4