How to write a Job script - 6.1

Talend Big Data Studio User Guide

EnrichVersion
6.1
EnrichProdName
Talend Big Data
task
Design and Development
EnrichPlatform
Talend Studio

In the API Editor of the Talend Studio, you can use the auto-completion (Ctrl+Space) to help you write your Job script. Functions, parameters and some values will be suggested to you.

The code will display in color to easily differentiate the code, the values and the comment. And the syntax is checked for any mistake. If a parameter is wrongly set a red cross displays. Moreover, you can use template to write your Job script. For more information on the color definition of the text or on how to manage Job script templates, see Job script preferences (JobScript).

A Job script is mainly made as following:

  • First, the components of your process.

  • Second, the connections between those components.

  • Other information like context parameters, other parameters and notes.

Component properties

Component properties can be divided into three main parts:

  • component definition

  • component schema

  • specific settings

To add a component, type in the addComponent{} function and define its properties between the brackets:

Function

Parameter

Value

setComponentDefinition{}

TYPE

Type in the component you want to use, for example: tRowGenerator.

 

NAME

Type in the name you want to give to the component, for example: tRowGenerator_1.

 

POSITION (optional)

Type in the position of the component in the design workspace, for example: 2,3.

Note

The position of the component must be indicated in brackets.

setSettings{}

 

Define the component settings. This parameter is specific to each component.

addSchema{}

TYPE

Type in the type of schema used in the component, for example: FLOW, LOOKUP, etc.

 

LABEL

Give a label to the schema.

 

addColumn{}

Add as many columns as your schema needs and define the column parameters:

NAME: Type in the label of the column.

TYPE: Type in the type of data contained in this column.

KEY: Define the column as primary key.

NULLABLE: Define the column as nullable.

Note

All parameters have to be separated with a comma.

Connection properties

To add a connection, type in the addConnection{} function and define its properties between the brackets:

Parameter

Value

TYPE

Type in the type of schema used in the component, for example: FLOW, LOOKUP, etc.

NAME

Give a name to the connection.

LINESTYLE

Define the style of the row.

For example, 0 refers to a row main, 8 refers to a row lookup.

SOURCE

Type in the name of the source component.

TARGET

Type in the name of the target component.

Note

All parameters have to be separated with a comma.

Example of Job script

Below you can find a example of Job script with two tRowGenerator as input components, a tMap component, with the two input tables corresponding to the input components, a variable table and one output table, and a tLogRow component as output component.

/*
* This is a job with 2 tRowGenerator, 1 tMap and 1 tLogRow.
*/
 
//add component node 
addComponent{
//set basic infomation
setComponentDefinition {
		TYPE:tRowGenerator,
		NAME:"tRowGenerator_1",
		POSITION: 160, 64
	}
	//set element parameters
        	setSettings{
		NB_ROWS:100,
		VALUES{
			SCHEMA_COLUMN:tableOneColumn1,
			ARRAY:TalendString.getAsciiRandomString(6),
			SCHEMA_COLUMN:tableOneColumn2,
			ARRAY:TalendString.getAsciiRandomString(6),
			SCHEMA_COLUMN:tableOneColumn3,
			ARRAY:TalendString.getAsciiRandomString(6)
		}
	}
	//add the schema
    addSchema {
		NAME:"schema1",
		TYPE:"FLOW",
		LABEL:"tRowGenerator_1"
		addColumn {
			NAME:tableOneColumn1,
			TYPE:id_String,
			KEY:false,
			NULLABLE:false
		}
		addColumn {
			NAME:tableOneColumn2,
			TYPE:id_String,
			KEY:false,
			NULLABLE:true
		}
		addColumn {
			NAME:tableOneColumn3,
			TYPE:id_String,
			KEY:false,
			NULLABLE:true
		}
	}
}
addComponent{
	setComponentDefinition {
		TYPE:tRowGenerator,
		NAME:"tRowGenerator_2",
		POSITION: 160, 192
	}
	setSettings{
		NB_ROWS:100,
		VALUES{
			SCHEMA_COLUMN:tableTwoColumn1,
			ARRAY:TalendString.getAsciiRandomString(6),
			SCHEMA_COLUMN:tableTwoColumn2,
			ARRAY:TalendString.getAsciiRandomString(6)
		}
	}
	addSchema {
		NAME:"schema2",
		TYPE:"FLOW",
		LABEL:"tRowGenerator_2"
		addColumn {
			NAME:tableTwoColumn1,
			TYPE:id_String,
			KEY:false,
			NULLABLE:false
		}
		addColumn {
			NAME:tableTwoColumn2,
			TYPE:id_String,
			KEY:false,
			NULLABLE:true
		}
	}
}
addComponent{
	setComponentDefinition {
		TYPE:tLogRow,
		NAME:"tLogRow_1",
		POSITION: 544, 128
	}
	setSettings{
		LENGTHS{
			SCHEMA_COLUMN:var1,
			SCHEMA_COLUMN:var2,
			SCHEMA_COLUMN:var4
		}
	}
	addSchema {
		NAME:"schema3",
		TYPE:"FLOW",
		LABEL:"newOutPut"
		addColumn {
			NAME:var1,
			TYPE:id_String,
			KEY:false,
			NULLABLE:false
		}
		addColumn {
			NAME:var2,
			TYPE:id_String,
			KEY:false,
			NULLABLE:true
		}
		addColumn {
			NAME:var4,
			TYPE:id_String,
			KEY:false,
			NULLABLE:true
		}
	}
}
addComponent{
	setComponentDefinition{
		TYPE:tMap,
		NAME:"tMap_1",
		POSITION: 352, 128
	}
	setSettings{
		EXTERNAL:Map
	}
		addSchema {
			NAME: "newOutPut",
			TYPE: "FLOW",
			LABEL: "newOutPut"
			addColumn {
				NAME: "var1",
				TYPE: "id_String"
			}
			addColumn {
				NAME: "var2",
				TYPE: "id_String",
				NULLABLE: true
			}
			addColumn {
				NAME: "var4",
				TYPE: "id_String",
				NULLABLE: true
			}
		}
	//add special data, input tables, an output table, and a var table in tMap 	
    addMapperData{
		//the syntax is almost the same as the metatable
        		addInputTable{
			NAME:row1,
			SIZESTATE:INTERMEDIATE,
			EXPRESSIONFILTER:filter
			addColumn {
				NAME:tableOneColumn1,
				TYPE:id_String,
				NULLABLE:false
			}
			addColumn {
				NAME:tableOneColumn2,
				TYPE:id_String,
				NULLABLE:true
			}
			addColumn {
				NAME:tableOneColumn3,
				TYPE:id_String,
				NULLABLE:true
			}
		}
		//add another input table
        		addInputTable{
			NAME:row2,
			SIZESTATE:MINIMIZED
			addColumn {
				NAME:tableTwoColumn1,
				TYPE:id_String,
				NULLABLE:false
			}
			addColumn {
				NAME:tableTwoColumn2,
				TYPE:id_String,
				NULLABLE:true
			}
		}
		//add the var table
        		addVarTable{
			NAME:Var
			addColumn{
				NAME:var1,
				TYPE:id_String,
				EXPRESSION:row1.tableOneColumn1+row2.tableTwoColumn1
			}
			addColumn{
				NAME:var2,
				TYPE:id_String,
				EXPRESSION:row1.tableOneColumn2
			}
			addColumn{
				NAME:var3,
				TYPE:id_String,
				EXPRESSION:row1.tableOneColumn3+row2.tableTwoColumn2
			}
			addColumn{
				NAME:var4,
				TYPE:id_String
			}
		}
		//add the output table
        		addOutputTable{
			NAME:newOutPut
			addColumn{
				NAME:var1,
				TYPE:id_String,
				EXPRESSION:Var.var1+Var.var3
			}
			addColumn{
				NAME:var2,
				TYPE:id_String,
				EXPRESSION:Var.var2
			}
			addColumn{
				NAME:var4,
				TYPE:id_String,
				EXPRESSION:Var.var4
			}
		}
	}
}
//add connections 
addConnection{
	TYPE:"FLOW",
	NAME:"row1",
	LINESTYLE:0,
	SOURCE:"tRowGenerator_1",
	TARGET:"tMap_1"
}
addConnection{
	TYPE:"FLOW",
	NAME:"row2",
	LINESTYLE:8,
	SOURCE:"tRowGenerator_2",
	TARGET:"tMap_1"
}
addConnection{
	TYPE:"FLOW",
	NAME:"newOutPut",
	LINESTYLE:0,
	SOURCE:"tMap_1",
	TARGET:"tLogRow_1"
}