Working with Job execution tasks

Talend Administration Center User Guide

EnrichVersion
7.0
EnrichProdName
Talend Big Data
Talend Real-Time Big Data Platform
Talend Data Integration
Talend Data Fabric
Talend Data Services Platform
Talend Big Data Platform
Talend ESB
Talend Data Management Platform
task
Deployment
Administration and Monitoring
EnrichPlatform
Talend Administration Center

Warning

Only users that have the Operation Manager role and rights can have read-write access to the tasks list. Other types of users can have read-only access or no access to the list. For further information on access rights, see User roles/rights in the Administration Center. When a user of the Administration Center opens this page, he/she will have access only to the items for which the user has been granted the appropriate authorization by the Administrator.

Warning

Make sure your execution servers are configured (agent must be running), then you can create the various execution tasks that you want to launch.

A Job execution task represents a set of actions that you can configure in Talend Administration Center in order to execute the Jobs designed in Talend Studio directly from the Administration Center. You can also define different types of triggers to launch an execution task.

To access the execution task list, do the following:

In the Menu tree view, click Job Conductor to display the list of scheduled tasks that will deploy and execute Jobs on a remote server.

When you access this list for the first time, no task shows on the list.

The default Job Conductor page provides the following information:

Column label

Description

State

Priority among tasks that are not yet executed. Status can be: awaiting execution or frozen exec.

Note

Frozen exec will require a manual resume operation.

Warning

If the threshold for simultaneous executions is exceeded (by default more than 20 executions at the time), all simultaneous executions above the threshold will acquire the status "awaiting execution".

To reset the threshold according to your needs, look for quartz.properties in the installation folders of Talend Administration Center and change the threshold accordingly in the following parameter: org.quartz.threadPool.threadCount = 20

Status

Status of the current task. It can be: Ready to generate, Ready to deploy, Ready to install, Ready to run or Incomplete configuration.

Error status

Error message is displayed if the task does not complete properly.

Label

Name of the task to be executed.

Trigger status

Triggering state:

No trigger: no trigger has been set for this task.

Running: the trigger is activated and the task execution will start.

Completed: the task is complete.

All triggers paused: the trigger has been paused and needs to be resumed manually.

Actions

Actions you can do:

: opens a pop-up window with the last execution details of the selected task. From this window, you can visualize the logs, the context values and the advanced information of the task, as well as navigate between the different task executions.

: opens the execution history of the selected task.

Time left before next triggering

Time before the next triggering occurs.

Project

Name of the project containing the Job to be executed.

Branch

Name of the branch/tag containing the Job to be executed.

Job

Name of the Job to be executed.

Version

Version of the Job as defined in Talend Studio.

Context

Name of the context as defined for this Job in Talend Studio.

Execution server

Name of the server on which the task was last executed.

Virtual server

Name of the virtual server, if any.

Some extra columns are hidden by default but can be displayed in the table. For more information, see Customizing the display of the task list

These extra columns provide the following information:

Column

Description

Active

The task is activated when the corresponding Active check box is selected. If a task is deactivated, it can not be executed in the execution server.

CommandLine version

Revision number of the CommandLine.

Description

The description text typed at task creation.

Id

Unique identifier that can be used as parameter to launch the task via a Webservice, for example.

idQuartzJob

Quartz Job identifier associated to the corresponding task.

Last deployment

Date and time of the last time the Job was deployed.

Last ended run

Date and time of the last time the Job execution was complete.

Last run

Date and time of the last time the Job was executed.

Last script generation

Date and time of the last time the script was generated.

Next triggering on

Date and time of the next triggering.

SVN revision

SVN revision number of the Job.

Adding a Normal execution task

The Talend Administration Center allows you to add execution tasks on Jobs that are either based on the SVN or GIT repository, or pre-generated by the Studio as zip files. According to your needs, read one of the procedures below.

Note that once the task is added, you still can switch from one mode to the other.

Adding an execution task to a Job based on SVN/GIT repository

Note

SVN/GIT-based tasks are deprecated in Talend Administration Center. It is recommended to add execution tasks to a Job published to Nexus.

  1. From the toolbar on the Job Conductor page, click Add > Normal Task to clear the Execution task configuration panel.

  2. Enter/select the following information as necessary.

    Field

    Description

    Label

    Name of the task to be triggered.

    Description

    Provides any useful information regarding the task to be carried out.

    Active

    Select/clear this check box to activate/deactivate this task.

    Job

    By clicking the icon, you will open a dedicated Job filter to search for the Job of interest. This can avoid investing too much time in looking for the Job you need to execute. For further information, see the example below this table.

    You also have the possibility to associate a pre-generated zip file or a Job published to Nexus to your task. For more information, see Adding an execution task to a pre-generated Job and Adding an execution task to a Job published to Nexus.

    Project

    Select the project that holds the Job to be executed. This field will be automatically filled if you have selected the project of interest using the Job filter.

    Branch

    Select the SVN/GIT branch/tag, if any is created, of the project containing the Job to be executed. This field will be automatically filled if you have selected the branch/tag of interest using the Job filter. For more information about projects stored on SVN or GIT, see Managing SVN/Git branches and tags for a project.

    Name

    Select the Job to be executed. This field will be automatically filled if you have selected the Job of interest using the Job filter.

    From this field, you can select both Standard and Big Data Batch (Map/Reduce and Spark) Jobs depending on your license. For further information about Standard and Big Data Batch Jobs that you need to design in the Studio, see Talend Studio User Guide and the Talend Big Data Studio Getting Started Guide.

    Version

    If the Job exists in different versions, select the version you want to execute. Or select Latest version from the list in order to automatically execute the latest version of the selected Job.

    This field will be automatically filled if you have selected the Job version of interest using the Job filter.

    Note

    If you select a given version of the Job, except the latest, Job execution will be faster since there is no need to generate and deploy it before execution. However, when you select Latest version, Job execution will be longer since the Job will be regenerated and deployed before the execution.

    Context

    If several contexts are available for the selected Job, select the relevant context. This field will be automatically filled if you have selected the Job context of interest using the Job filter.

    Apply context to children

    Select this check box if you want to apply the selected context to all children Jobs.

    Regenerate Job on change

    Select this check box if you want to regenerate the selected Job before task deployment and execution every time a modification is made to the Job itself.

    This check box is only useful if you select the latest version of the Job for execution, since the modifications will only be made to the latest version.

    Warning

    Note that if you selected Latest version, in case a new version of the Job is created in Studio, the Job will be regenerated even if you did not select the Regenerate Job on change check box.

    Log4j Level

    Select the level of log you want to apply to the execution task. The levels are ranked in increasing order of severity, from TRACE (most detailed information) to OFF (logs turned off). For more information on how to enable and customize the log4j feature from the Studio, see the Talend Studio User Guide.

    Execution server

    Select the server on which the task should be deployed.

    If the relevant server does not display on the list, check the Servers page and make sure the server is correctly configured. For more information regarding the execution server configuration, see Configuring execution servers

    The list of execution servers offers virtual servers corresponding to physical servers grouped together. For details about Virtual Servers, see Configuring virtual servers

    Run Job as OS user (Unix)

    (Unix only)

    Enter the name of the Operating System user who is allowed to start the Job execution, root for example:

    • If the field is empty, any user is allowed to execute the Job. Note that these users must correspond to existing Operating System users.

    • To restrict this right to specific users, edit the list of allowed users in the following file:

      <JobServerInstallationDirectory>/conf/TalendJobServer.properties

    • You need to give the right umask to your user as well as specific permissions to the server directories to use this feature. For more information on how to do it, see the Talend Installation Guide.

    Tip

    By default, the user name must start with a lower-case letter from a to z, followed by a combination of lower-case letters (a to z) and numbers (from 0 to 9). To allow using characters other than those letters and numbers, you need to modify the regular expression ^[a-z][-a-z0-9]*\$ in the value of the org.talend.remote.jobserver.server.TalendJobServer.RUN_AS_USER_VALIDATION_REGEXP parameter in the file {Job_Server_Installation_Folder}\agent\conf\TalendJobServer.properties. For example:

    • To define a user name pattern that should include a dot, like firstname.lastname, modify the regular expression to ^[a-z][-a-z0-9]*.[a-z][-a-z0-9]*\$.

    • To allow using one or more underscores (_) in the user name, modify the regular expression to ^[a-z][-a-z_0-9]*\$.

    Statistic

    Decide whether you activate the statistics option during the execution of the selected task. The statistics option offered in Talend Administration Center is exactly the same as the one available in Talend Studio when you select the Statistics check box on the Run view. For more information about this option in the Studio, see Talend Studio User Guide.

    In the list, select:

    - enabled: It allows you to access the statistics of your task through the Real time statistics page by clicking the Show statistic view button of the toolbar or by opening the [Real time statistics] window every time you click Run to execute a task from Talend Administration Center.

    - disabled: this is the default mode. On this mode, the Show statistic view button is deactivated, so is the access to the Real time statistics page, and the [Real time statistics] window does not display when you execute a task.

    For more information on the Real time statistics facility, see Recovering the execution of a Job.

    Note

    The Statistic field is only available in certain editions of Talend Administration Center.

    Warning

    The task may need to be regenerated when you switch among the various modes. In this case, a short text in parenthesis is added behind the mode to notify the need to regenerate the task.

    On unavailable JobServer

    This option is part of the failover process provided by Talend. and allows you to decide what action you want to take on the selected task when the server is unavailable during a predetermined period of time (defined in Adding an execution server).

    Select one of the following from the list:

    - Wait: if the Job has an unknown status, usually caused by server problems, this option puts the task on hold throughout the period of time during which the server is inaccessible. When the server is accessible again, the task will restore the status it held prior to encountering the problem.

    - Reset task: if the Job has an unknown status, usually caused by server problems, this option puts the task status back to Ready to run when the predetermined period of time relating to the Job status elapses. However, the Job itself could continue running depending on the initial problem.

    - Restart task: if the Job has an unknown status, usually caused by server problems, this option will automatically start the task in its original state when the predetermined period of time relating to the Job status elapses.

    Note

    The Restart task option is only useful when you are using virtual servers.

    - Recover task: if the Job has an unknown status, usually caused by server problems, this option will automatically recover the Job execution from the last validated checkpoint.

    Note

    The context parameters used when the task is reset, restarted or recovered, are the same as those used during the last execution

    Timeout (s)

    Type in how long to wait, in seconds, before the task is killed.

    By default, if no value is entered in the field, the task runs indefinitely. This option might be useful if you need to limit the execution of a task to a certain period of time.

    Pause triggers on error

    Select the check box in order to pause the trigger(s) if an error occurs during the execution of the task.

    In this step, you can use the dedicated Job filter to help find the Job to be executed from numerous Job folders.

    To do this, click the Select Job button on the configuration panel and in the pop-up [Select Job from Repository] window, select the project, the branch or the tag, the Job, the Job version and the execution context, respectively, depending on your needs. If too many Jobs present in the list such that searching one of them becomes difficult, you can enter the name of the Job of interest in the filter field docked on the top of the Name area.

    The following figure presents an example of this filter with the Map/Reduce Job land_raw_data selected.

  3. Click Save to validate the configuration or Cancel to cancel the creation of the task.

    Once the task has been created, you can still edit its properties, manually or using the Metaservlet application. For more information, see Editing a task.

Adding an execution task to a pre-generated Job

Talend Administration Center allows you to associate a pre-generated .zip file to a Job Conductor task, and thus to skip the generation step, in order to make sure the task that is deployed is the same as the one you have previously deployed.

For example, if you have generated a task in a development environment and want to run the exact same task in the production environment, you may want to use this feature.

To perform this action via the MetaServlet application, use the associatePreGeneratedJob command. For more information about the MetaServlet parameters, see Parameters and actions in metaServlet.

Prerequisites:

  • A .zip file holding a specific task has been previously generated by the Studio, or the CommandLine using the buildJob command (see Building a Job using the CommandLine). The best practice is to put this file in the Job archive folder, which path is defined in the Job Conductor node of the Configuration page.

    IMPORTANT:

    - when exporting your Job from the Studio, you must select the items check box in the [Build Job] window. For more information, see the Talend Studio User Guide;

    - this .zip file must contain a jobInfo.properties file at the root level, which describes the project and the Job characteristics.

    These information are necessary for the .zip file to be imported without generating errors.

  • A project with the same name as the development project used to generate the Jobs has been created in the Projects page. For more information, see Adding a project.

  • Unix only: If you want to execute the Job as a specific Operating System user (Run Job as OS user option), this user must already exist in the system, he must have the right umask and some server directories must have been given specific permissions. For more information, see the Talend Installation Guide.

    Tip

    By default, the user name must start with a lower-case letter from a to z, followed by a combination of lower-case letters (a to z) and numbers (from 0 to 9). To allow using characters other than those letters and numbers, you need to modify the regular expression ^[a-z][-a-z0-9]*\$ in the value of the org.talend.remote.jobserver.server.TalendJobServer.RUN_AS_USER_VALIDATION_REGEXP parameter in the file {Job_Server_Installation_Folder}\agent\conf\TalendJobServer.properties. For example:

    • To define a user name pattern that should include a dot, like firstname.lastname, modify the regular expression to ^[a-z][-a-z0-9]*.[a-z][-a-z0-9]*\$.

    • To allow using one or more underscores (_) in the user name, modify the regular expression to ^[a-z][-a-z_0-9]*\$.

  1. From the toolbar on the Job Conductor page, click Add > Normal Task to display the Execution task configuration panel.

  2. Enter/select the following information as necessary.

    Field

    Description

    Label

    Name of the task to be triggered.

    Description

    Provides any useful information regarding the task to be carried out.

    Active

    Select/clear this check box to activate/deactivate this task.

    Job

    Click the icon to open the [Import generated code] window. This window allows you to retrieve the pre-generated file holding the desired Job and to associate this Job to the task.

    Note that, when exporting your Job from the Studio, you must select the items check box in the [Build Job] window to be able to import your Job successfully on the Job Conductor page. For more information, see the Talend Studio User Guide.

    You also have the possibility to add a task to a Job based on SVN/GIT repository or published to Nexus. For more information, see Adding an execution task to a Job based on SVN/GIT repository and Adding an execution task to a Job published to Nexus.

    Project

    This field is automatically completed and is read-only.

    Note that this project must exist in Talend Administration Center.

    Branch

    This field is automatically completed and is read-only.

    Name

    This field is automatically completed and is read-only.

    Version

    This field is automatically completed and is read-only.

    Context

    If you have defined several contexts in the Studio for the selected Job, select the relevant context in the list.

    Apply context to children

    This field is automatically completed and is read-only. It can be set when the Job is built from the Studio as a standalone Job.

    Regenerate Job on change

    Select this check box if you want to regenerate the selected Job before task deployment and execution every time a modification is made to the Job itself.

    This check box is only useful if you select the latest version of the Job for execution, since the modifications will only be made to the latest version.

    Warning

    Note that if you selected Latest version, in case a new version of the Job is created in Studio, the Job will be regenerated even if you did not select the Regenerate Job on change check box.

    Log4j Level

    Select the level of log you want to apply to the execution task. The levels are ranked in increasing order of severity, from TRACE (most detailed information) to OFF (logs turned off). For more information on how to enable and customize the log4j feature from the Studio, see the Talend Studio User Guide.

    Execution server

    Select the server on which the task should be deployed.

    If the relevant server does not display on the list, check the Servers page and make sure the server is correctly configured. For more information regarding the execution server configuration, see Configuring execution servers.

    The list of execution servers offers virtual servers corresponding to physical servers grouped together. For details about Virtual Servers, see Configuring virtual servers.

    Statistic

    This field is automatically completed. It is either:

    - enabled (read and write) if you enabled the statistics during the generation of the Job;

    - disabled (read and write) if you deactivated the statistics during the generation of the Job.

    On unavailable JobServer

    This option is part of the failover process provided by Talend. and allows you to decide what action you want to take on the selected task when the server is unavailable during a predetermined period of time (defined in Adding an execution server).

    Select one of the following from the list:

    - Wait: if the Job has an unknown status, usually caused by server problems, this option puts the task on hold throughout the period of time during which the server is inaccessible. When the server is accessible again, the task will restore the status it held prior to encountering the problem.

    - Reset task: if the Job has an unknown status, usually caused by server problems, this option puts the task status back to Ready to run when the predetermined period of time relating to the Job status elapses. However, the Job itself could continue running depending on the initial problem.

    - Restart task: if the Job has an unknown status, usually caused by server problems, this option will automatically start the task in its original state when the predetermined period of time relating to the Job status elapses.

    Note

    The Restart task option is only useful when you are using virtual servers.

    - Recover task: if the Job has an unknown status, usually caused by server problems, this option will automatically recover the Job execution from the last validated checkpoint.

    Note

    The context parameters used when the task is reset, restarted or recovered, are the same as those used during the last execution

    Timeout (s)

    Type in how long to wait, in seconds, before the task is killed.

    By default, if no value is entered in the field, the task runs indefinitely. This option might be useful if you need to limit the execution of a task to a certain period of time.

    Pause triggers on error

    Select the check box in order to pause the trigger(s) if an error occurs during the execution of the task.

  3. Click Save to validate the configuration or Cancel to cancel the creation of the task.

    Once the task has been created, you can still edit its properties, manually or using the Metaservlet application. For more information, see Editing a task.

Adding an execution task to a Job published to Nexus

Talend Administration Center allows you to add an execution task to a Job published in the Nexus artifact repository as a .zip file, and thus to skip the generation step, in order to make sure the task that is deployed is the same as the one you have previously deployed.

Prerequisites:

  • The parameters to connect to the Nexus artifact repository are filled in the Artifact Repository node of the Configuration page. For more information, see Setting up the artifact repository parameters.

  • The Job you want to add has been previously published to the Nexus artifact repository from the Studio.

  • Unix only: If you want to execute the Job as a specific Operating System user (Run Job as OS user option), this user must already exist in the system, he must have the right umask and some server directories must have been given specific permissions. For more information, see the Talend Installation Guide.

    Tip

    By default, the user name must start with a lower-case letter from a to z, followed by a combination of lower-case letters (a to z) and numbers (from 0 to 9). To allow using characters other than those letters and numbers, you need to modify the regular expression ^[a-z][-a-z0-9]*\$ in the value of the org.talend.remote.jobserver.server.TalendJobServer.RUN_AS_USER_VALIDATION_REGEXP parameter in the file {Job_Server_Installation_Folder}\agent\conf\TalendJobServer.properties. For example:

    • To define a user name pattern that should include a dot, like firstname.lastname, modify the regular expression to ^[a-z][-a-z0-9]*.[a-z][-a-z0-9]*\$.

    • To allow using one or more underscores (_) in the user name, modify the regular expression to ^[a-z][-a-z_0-9]*\$.

  1. From the toolbar on the Job Conductor page, click Add > Normal Task to display the Execution task configuration panel.

  2. Enter/select the following information as necessary.

    Field

    Description

    Label

    Name of the task to be triggered.

    Description

    Provides any useful information regarding the task to be carried out.

    Active

    Select/clear this check box to activate/deactivate this task.

    Job

    Click the icon to open the [Select artifact from Nexus repository] window. This window allows you to retrieve the .zip file holding the Job published and to associate it to the task. Note that if you misconfigured or did not configure the connection to Nexus on the Configuration page, this icon will be grayed out.

    In the Repository list, select the repository in which the Job has been published.

    In the Browse panel, browse to the Job you want to associate with the task then click OK to close the window.

    You also have the possibility to add a task to a Job based on SVN/GIT repository or on a pre-generated Job. For more information, see Adding an execution task to a Job based on SVN/GIT repository and Adding an execution task to a pre-generated Job.

    Project

    This field is automatically completed and is read-only.

    Note that the project must exist in Talend Administration Center and that you must have read/write access to this project.

    Branch

    This field is automatically completed and is read-only.

    Name

    This field is automatically completed and is read-only.

    Version

    This field is automatically completed and is read-only.

    Context

    If you have defined several contexts in the Studio for the selected Job, select the relevant context in the list.

    Apply context to children

    This field is automatically completed and is read-only. It can be set when the Job is published from the Studio as a standalone Job.

    Regenerate Job on change

    Select this check box if you want to regenerate the selected Job before task deployment and execution every time a modification is made to the Job itself.

    This check box is only useful if you select the latest version of the Job for execution, since the modifications will only be made to the latest version.

    Warning

    Note that if you selected Latest version, in case a new version of the Job is created in Studio, the Job will be regenerated even if you did not select the Regenerate Job on change check box.

    Log4j Level

    Select the level of log you want to apply to the execution task. The levels are ranked in increasing order of severity, from TRACE (most detailed information) to OFF (logs turned off). For more information on how to enable and customize the log4j feature from the Studio, see the Talend Studio User Guide.

    Execution server

    Select the server on which the task should be deployed.

    If the relevant server does not display on the list, check the Servers page and make sure the server is correctly configured. For more information regarding the execution server configuration, see Configuring execution servers.

    The list of execution servers offers virtual servers corresponding to physical servers grouped together. For details about Virtual Servers, see Configuring virtual servers.

    Run Job as OS user (Unix)

    (Unix only)

    Enter the name of the Operating System user who is allowed to start the Job execution, root for example:

    • If the field is empty, any user is allowed to execute the Job. Note that these users must correspond to existing Operating System users.

    • To restrict this right to specific users, edit the list of allowed users in the following file:

      <JobServerInstallationDirectory>/conf/TalendJobServer.properties

    • You need to give the right umask to your user as well as specific permissions to the server directories to use this feature. For more information on how to do it, see the Talend Installation Guide.

    Tip

    By default, the user name must start with a lower-case letter from a to z, followed by a combination of lower-case letters (a to z) and numbers (from 0 to 9). To allow using characters other than those letters and numbers, you need to modify the regular expression ^[a-z][-a-z0-9]*\$ in the value of the org.talend.remote.jobserver.server.TalendJobServer.RUN_AS_USER_VALIDATION_REGEXP parameter in the file {Job_Server_Installation_Folder}\agent\conf\TalendJobServer.properties. For example:

    • To define a user name pattern that should include a dot, like firstname.lastname, modify the regular expression to ^[a-z][-a-z0-9]*.[a-z][-a-z0-9]*\$.

    • To allow using one or more underscores (_) in the user name, modify the regular expression to ^[a-z][-a-z_0-9]*\$.

    Statistic

    This field is automatically completed. It is either:

    - enabled (read and write) if you enabled the statistics during the generation of the Job;

    - disabled (read and write) if you deactivated the statistics during the generation of the Job.

    On unavailable JobServer

    This option is part of the failover process provided by Talend. and allows you to decide what action you want to take on the selected task when the server is unavailable during a predetermined period of time (defined in Adding an execution server).

    Select one of the following from the list:

    - Wait: if the Job has an unknown status, usually caused by server problems, this option puts the task on hold throughout the period of time during which the server is inaccessible. When the server is accessible again, the task will restore the status it held prior to encountering the problem.

    - Reset task: if the Job has an unknown status, usually caused by server problems, this option puts the task status back to Ready to run when the predetermined period of time relating to the Job status elapses. However, the Job itself could continue running depending on the initial problem.

    - Restart task: if the Job has an unknown status, usually caused by server problems, this option will automatically start the task in its original state when the predetermined period of time relating to the Job status elapses.

    Note

    The Restart task option is only useful when you are using virtual servers.

    - Recover task: if the Job has an unknown status, usually caused by server problems, this option will automatically recover the Job execution from the last validated checkpoint.

    Note

    The context parameters used when the task is reset, restarted or recovered, are the same as those used during the last execution

    Timeout (s)

    Type in how long to wait, in seconds, before the task is killed.

    By default, if no value is entered in the field, the task runs indefinitely. This option might be useful if you need to limit the execution of a task to a certain period of time.

    Pause triggers on error

    Select the check box in order to pause the trigger(s) if an error occurs during the execution of the task.

  3. Click Save to validate the configuration or Cancel to cancel the creation of the task.

    Once the task has been created, you can still edit its properties, manually or using the Metaservlet application. For more information, see Editing a task.

Adding an execution task to an artifact

The Talend Administration Center also allows you to add execution tasks to Jobs that are retrieved on the Nexus artifact repository at execution time, or pre-published via the Publisher page.

Prerequisites:

  • The parameters to connect to the Nexus artifact repository are filled in the Artifact Repository node of the Configuration page. For more information, see Setting up the artifact repository parameters.

  • The Job you want to add has been previously published to the Nexus artifact repository from the Studio.

  • Unix only: If you want to execute the Job as a specific Operating System user (Run Job as OS user option), this user must already exist in the system, he must have the right umask and some server directories must have been given specific permissions. For more information, see the Talend Installation Guide.

    Tip

    By default, the user name must start with a lower-case letter from a to z, followed by a combination of lower-case letters (a to z) and numbers (from 0 to 9). To allow using characters other than those letters and numbers, you need to modify the regular expression ^[a-z][-a-z0-9]*\$ in the value of the org.talend.remote.jobserver.server.TalendJobServer.RUN_AS_USER_VALIDATION_REGEXP parameter in the file {Job_Server_Installation_Folder}\agent\conf\TalendJobServer.properties. For example:

    • To define a user name pattern that should include a dot, like firstname.lastname, modify the regular expression to ^[a-z][-a-z0-9]*.[a-z][-a-z0-9]*\$.

    • To allow using one or more underscores (_) in the user name, modify the regular expression to ^[a-z][-a-z_0-9]*\$.

Adding an execution task on an artifact

  1. From the toolbar on the Job Conductor page, click Add > Artifact Task to display the Execution task configuration panel.

  2. Enter/select the following information as necessary.

    Field

    Description

    Label

    Name of the task to be triggered.

    Description

    Provides any useful information regarding the task to be carried out.

    Active

    Select/clear this check box to activate/deactivate this task.

    Project

    Select the project containing the artifact to be executed

    Branch

    Select the SVN/GIT branch/tag, if any is created, of the project containing the artifact to be executed. For more information about projects stored on SVN or GIT, see Managing SVN/Git branches and tags for a project.

    Job

    Click the Nexus button icon to open the [Select artifact from Nexus repository] window. This window allows you to link dynamically the Job artifact and to associate it to the task, knowing that this artifact is only retrieved at execution time.

    In the Repository list, select the repository in which the Job is published. In the Browse panel, browse to the Job you want to associate with the task then click OK to close the window.

    OR:

    Click the Publisher button to retrieve the publishing task created from the Publisher page. Details of the artifact are then automatically completed in the form. For more information on publishing tasks, see Working with publishing tasks.

    Snapshot

    Select this check box to execute the artifact as a snapshot.

    Repository

    Repository in which the artifact is published. This field is automatically completed.

    Group ID

    Name of the group in which the artifact is published. This field is automatically completed.

    Artifact

    Name of the artifact item. This field is automatically completed.

    From this field, you can select both Standard and Big Data Batch Jobs (Map/Reduce and Spark), according to your license. For more information on the Standard and Big Data Batch Jobs created from the Studio, see the Talend Studio User Guide and the Talend Big Data Studio Getting Started Guide.

    Publish Version

    Version of the artifact to be executed. This field is automatically completed and is read-only.

    Use Latest Version

    If there are several versions of the same artifact, select this check box in order to automatically execute the latest version of the selected artifact.

    Context

    Context to be applied to the execution task. This field is automatically completed.

    Apply context to children

    This field is automatically completed and is read-only. It can be set when the Job is published from the Studio as a standalone Job.

    Log4j Level

    Select the level of log you want to apply to the execution task. The levels are ranked in increasing order of severity, from TRACE (most detailed information) to OFF (logs turned off). For more information on how to enable and customize the log4j feature from the Studio, see the Talend Studio User Guide.

    Execution server

    Select the server on which the task should be deployed.

    If the relevant server does not display on the list, check the Servers page and make sure the server is correctly configured. For more information regarding the execution server configuration, see Configuring execution servers.

    The list of execution servers offers virtual servers corresponding to physical servers grouped together. For details about Virtual Servers, see Configuring virtual servers.

    Run Job as OS user (Unix)

    (Unix only)

    Enter the name of the Operating System user who is allowed to start the Job execution, root for example:

    Tip

    By default, the user name must start with a lower-case letter from a to z, followed by a combination of lower-case letters (a to z) and numbers (from 0 to 9). To allow using characters other than those letters and numbers, you need to modify the regular expression ^[a-z][-a-z0-9]*\$ in the value of the org.talend.remote.jobserver.server.TalendJobServer.RUN_AS_USER_VALIDATION_REGEXP parameter in the file {Job_Server_Installation_Folder}\agent\conf\TalendJobServer.properties. For example:

    • To define a user name pattern that should include a dot, like firstname.lastname, modify the regular expression to ^[a-z][-a-z0-9]*.[a-z][-a-z0-9]*\$.

    • To allow using one or more underscores (_) in the user name, modify the regular expression to ^[a-z][-a-z_0-9]*\$.

    • If the field is empty, any user is allowed to execute the Job. Note that these users must correspond to existing Operating System users.

    • To restrict this right to specific users, edit the list of allowed users in the following file:

      <JobServerInstallationDirectory>/conf/TalendJobServer.properties

    • You need to give the right umask to your user as well as specific permissions to the server directories to use this feature. For more information on how to do it, see the Talend Installation Guide.

    Statistic

    This field is automatically completed and is read-only. It is either:

    - enabled (read and write) if you enabled the statistics during the generation of the Job;

    - disabled (read and write) if you deactivated the statistics during the generation of the Job.

    On unavailable JobServer

    This option is part of the failover process provided by Talend. and allows you to decide what action you want to take on the selected task when the server is unavailable during a predetermined period of time (defined in Adding an execution server).

    Select one of the following from the list:

    - Wait: if the Job has an unknown status, usually caused by server problems, this option puts the task on hold throughout the period of time during which the server is inaccessible. When the server is accessible again, the task will restore the status it held prior to encountering the problem.

    - Reset task: if the Job has an unknown status, usually caused by server problems, this option puts the task status back to Ready to run when the predetermined period of time relating to the Job status elapses. However, the Job itself could continue running depending on the initial problem.

    - Restart task: if the Job has an unknown status, usually caused by server problems, this option will automatically start the task in its original state when the predetermined period of time relating to the Job status elapses.

    Note

    The Restart task option is only useful when you are using virtual servers.

    - Recover task: if the Job has an unknown status, usually caused by server problems, this option will automatically recover the Job execution from the last validated checkpoint.

    Note

    The context parameters used when the task is reset, restarted or recovered, are the same as those used during the last execution

    Timeout (s)

    Type in how long to wait, in seconds, before the task is killed.

    By default, if no value is entered in the field, the task runs indefinitely. This option might be useful if you need to limit the execution of a task to a certain period of time.

    Pause triggers on error

    Select the check box in order to pause the trigger(s) if an error occurs during the execution of the task.

  3. Click Save to validate the configuration or Cancel to cancel the creation of the task.

    Once the task has been created, you can still edit its properties, manually or using the Metaservlet application. For more information, see Editing a task.

Organizing execution tasks

Editing a task

To edit a task:

  1. On the Job Conductor page, select the task you want to modify. Its details are displayed in the Execution task configuration panel.

  2. Make the relevant changes to task details where necessary.

  3. Click Save to validate the changes or Cancel to cancel the changes.

    Note that you can also edit a task (name, context values, Job .zip file, etc.) via the MetaServlet application using the updateTask command. For more information about the MetaServlet parameters, see Parameters and actions in metaServlet.

Duplicating a task

To avoid creating a new task from scratch, you can duplicate an existing one and work around its metadata to have a new task in the list.

To duplicate a task, do the following:

  1. On the Job Conductor page, select the task you want to duplicate.

  2. On the toolbar, click Duplicate. The panel to the right shows the metadata of the selected task.

  3. Modify the metadata as needed in order to create a new task.

  4. Click Save to validate the operation or Cancel to cancel it.

    The new task is listed in the Execution tasks list.

Deleting a task

To delete one or more tasks from the task list, follow the procedure below.

To perform this action via the MetaServlet application, use the deleteTask command. For more information about the MetaServlet parameters, see Parameters and actions in metaServlet.

  1. On the Job Conductor page, select the task you want to delete. Its details display in the Execution task configuration panel.

  2. On the toolbar, click Delete. A confirmation dialog box appears.

  3. Click OK to remove the task from the task list.

Searching a task

If you have a great number of tasks and want to filter them, you can do so by typing key words or by selecting some check boxes in the Filters fields of the columns listed below:

  • Active

  • Branch

  • CommandLine version

  • Context

  • Description

  • Error status

  • Label

  • Last deployment

  • Last ended run

  • Last run

  • Last script generation

  • Name

  • Next triggering on

  • Project

  • Server

  • Status

  • SVN revision

  • Version

  • Virtual server

Example of how to apply a filter on the label of several task(s)

  1. Click the arrow on the Label column to display its options, then select the Filters option.

  2. Type in the name by which you want to filter the tasks. Here, the text customer_data is used to display only the tasks whose names include these terms.

    The filtered column appears in blue bold. To remove the filter, clear the Filters check box in this column.

    To remove all filters and reset the page, click the cog icon on the right of the top toolbar.

Customizing the display of the task list

You can customize the execution task list view to restrict the number of displayed tasks according to different criteria. You can also show/hide one or more columns in the task list.

  1. On the Job Conductor page, put the pointer on a column name and click the drop-down arrow.

  2. In the drop-down list, select:

    Item

    To...

    Sort Ascending

    arrange the list in an ascending order

    Sort Descending

    arrange the list in an descending order

    Columns

    display a drop-down list where you can select/clear the check box next to the column(s) you want to show/hide

    Group by this field

    arrange the list by the name of the selected column

    Show in groups

    show the list as one group

The figure below shows the list view options in the drop-down list.

Once you have customized the list, your preferences are saved (column filters, order or width) and kept even after the page is refreshed. To reset the page, you have to click the cog icon on the right of the top toolbar.

Refreshing the task list

The task list is refreshed automatically at regular intervals of time, but you can refresh it any time by clicking Refresh on the toolbar.

Scheduling a task

You can schedule your tasks based on time or event, using time triggers or file triggers respectively.

Time-based scheduling can be linked to a simple trigger or to a CRONUI trigger depending on the way you want your time-based triggered task to be executed, either at a defined time and date or regularly over a period of time.

You can also schedule your task to be executed based on a file trigger. The presence of a file in a defined location will trigger the job execution task.

Or you can also schedule your task to be execute from a Webservice.

On the Job Conductor page, click the task you want to schedule then select the relevant trigger.

Note that if you define several types of triggers for the same task, the first trigger to be checked is the first to be executed.

For more information about these triggers, see:

Recovering the last execution of a task

From the Job conductor page, you can open the Error recovery management page which is designed to help the recovery of erroneous Jobs. It displays the graphical and analytical details of the Job execution. To access Error recovery management, do the following:

  1. On the Job Conductor page, select the task you want to recover.

  2. On the toolbar, click the Recover last execution button.

The page switches to Error recovery management. Note that at the moment, the page only displays the details of the last execution of your selected task.

To view the details of all previous executions, you should access the page via Task executions monitoring. For detailed information about this and the Error recovery management page, see Accessing the Error Recovery Management page.

To learn more about best practices to follow in case of Job generation issues in Talend Administration Center, see the documentation on Talend Help Center (https://help.talend.com)

Accessing the execution list of a task

To monitor all executions of a given task, do the following:

  1. On the Job Conductor page, select the task you want to monitor.

  2. In the Actions column, click the Show execution history icon.

    The page switches to Task execution monitoring, and a filter is applied on the task you have selected.

To monitor the executions of all tasks, click the Disable filters button on the toolbar.

For more information on task execution monitoring, see Monitoring task executions in the Monitoring node.