Designing Jobs, Routes, or Data Services - Cloud - 8.0

Talend Software Development Life Cycle Best Practices Guide

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Administration Center
Talend Artifact Repository
Talend CommandLine
Talend JobServer
Talend Management Console
Talend Remote Engine
Talend Studio
Content
Administration and Monitoring
Deployment
Design and Development
Last publication date
2024-02-29
At this stage, the conceptualization part is done and each team has been assigned some tasks. The development team designs Jobs, Routes, or Data Services in Talend Studio, which are the development unit in Talend.
Best practices: To ensure continuous integration during development and to help developers design and build consistent, efficient and optimised artifacts, here are some best practices you should follow:

Concept

Best practice example

Naming standards

In Talend Studio, define a naming convention for Jobs, Routes, or Data Services and folders and follow it.

In this document, the naming convention is the following, but feel free to adapt it to your requirements: job_|route_ |service_ prefixes for Job, Routes, and Data Services names respectively, test_ prefix for Test Case names, pub_ prefix for publishing task names and task_ prefix for execution task names.

For example, name your folder xxx. Folders should be used to group Jobs of a similar type. Then create a Job named job_xxx_description and its Test case named test_xxx_description.

At a more granular level, components should also have a meaningful name.

At the project level, name your project using upper case otherwise it might cause build failure.

Warning: If you are working on a Git-managed project, do not use any of the following reserved key words to name your Job or Job folder:
  • tests
  • target
  • src
If any of the above-mentioned key words is used in the name of a Job, a Job folder or any level of its parent folders, changes to your Job or your Jobs in the folder will not get pushed to Git.

Version control

Use GIT branches and tags as well as Talend Studio to handle artifact versions.

For more information on how to change the version of your artifacts centrally at once to publish them with the version of your choice, see Changing the deployment version of each artifact at once.

Project identifier

When first connecting to the project in Talend Studio, edit the parameters to set the project identifier (groupID) that will be used at deployment time.

For more information on how to set this project identifier, see Changing the deployment identifier of the project at once.

Metadata

Use schema metadata in your Jobs, Routes, or Data Services to share database connections between several artifacts and help designing source/target components.

Contexts

Use contexts in order to reuse variables (context parameters locally for artifacts, group contexts globally for projects) such as database connectivity, host names, ports, etc. If values need to be changed or are used in multiple places, then they should not be hard coded and it is recommended to use contexts.

These contexts are also useful to switch between environments (Development context then QA context then Production context).

Standard Job layout

Use a standard Job layout to ensure its readability, it is particularly useful for collaborative work.

Some examples include: putting data flows from left to right, top-to-bottom layout to show the process flow between subJobs, target components on the right, etc.

Complexity

Jobs should follow a logic and be split in steps, called subJobs, when necessary. It is also recommended to use parent Jobs to run one or several child Jobs in order to create a process flow and even though there is no limit, you should avoid using more than 20 components in a Job.

Once the artifact is designed in a remote project from Talend Studio, it can be published, deployed, and executed in Talend Cloud. Exporting as an artifact will also help to perform Quality assurance tests on the same exact Jobs than those created in the Development environment.

For more information, see Deploying to QA and Production environments.