Best Practice: Conventions for Return Codes

EnrichVersion
6.4
6.3
6.2
6.1
6.0
5.6
EnrichProdName
Talend Data Fabric
Talend ESB
Talend Data Integration
Talend Open Studio for Data Integration
Talend Big Data
Talend Open Studio for ESB
Talend Data Management Platform
Talend Real-Time Big Data Platform
Talend Data Services Platform
Talend Big Data Platform
Talend Open Studio for MDM
Talend Open Studio for Big Data
Talend MDM Platform
task
Design and Development
Data Governance > Third-party systems > Logs and errors components (Integration)
Data Quality and Preparation > Third-party systems > Logs and errors components (Integration)
Design and Development > Third-party systems > Logs and errors components (Integration)
EnrichPlatform
Talend Studio

Best Practice: Conventions for Return Codes

Overview

As discussed in the Best Practice on Conventions for Job Entry/Exit/Return Points, Talend jobs should have a clear methodology, structure, and defined standards on where and how a job starts and stops. Knowing the state upon entry to, exit from, or return from a Job can dramatically improve code structure, error handling, operational management, monitoring, notification, and debugging of these code modules.

Defining a common use mechanism for Job Return Codes not only deliver enterprise standards, they provide useful information about a jobs' execution and the hand-off between other connected jobs and/or joblets.

Talend components tDie and tWarn are discussed in more detail in the Best Practice on Design Elements for Job Components, however it is important to mention them here as they provide the means to specify the preferred Return Code from any job. These two components are also well defined in the Talend Component Reference Guide and can be found for use in the Logs & Errors palette of the Talend Studio.

These specialized components, placed into any job, explicitly control the exit condition from the job specifying a useful Return Code, programmable message, and in the case of the tDie component whether the current Java Virtual Machine (JVM) will close immediately on exit or not.

User defined messages for the tDie/tWarn components, a string value, are programmatically available in the DIE_MESSAGES and WARN_MESSAGES global variables, respectively.  

tLogCatcher is a closely related component which when used, catches tDie/tWarn invocations. A failed components' generated ERROR_MESSAGE is further detailed in the Best Practice on Conventions for Logging.

Priorities

The tDie/tWarn components organize a jobs exit using Priority Levels, an integer value, which are programmatically available in the DIE_PRIORITY and WARN_PRIORITY global variables, respectively.  At lease one Priority Level must be selected.

These Priorities, which are industry standard, include:
TRACE Fine-grained informational events that track job execution line by line
DEBUG Medium-grained informational events that are most useful when debugging
INFO Coarse-grained informational events to highlight execution process
WARNING Potentially harmful events where execution process continues
ERROR Problematic events where execution process may or may not continue
FATAL Critical events where execution process aborts

As TRACE and DEBUG Priority Levels can generate a huge amount of detail, the recommended best practice is to limit their use to actual debugging scenarios and rarely or never use them in a production environment.

For the remaining Priorities, the recommend best practice is to adopt programming conventions to be widely used across jobs and projects within the enterprise.

Return Codes

It is highly recommended that jobs utilize these tDie/tWarn components and that, in-conjunction with the Priority Levels, well defined Return Codes, are established and enforced.

These can be simple binary values that return 1=true/0=false, or 1=unsuccessful/0=successful. These can also be complex where returned values encode specialized meaning on the job/components condition.

The following example of a complex offers one possible convention.

Return Code Example

Have the Return Code defined as a 4 digit number where the first digit designates the Priority Level which allows calling jobs to determine the nature of the Return Code (as organized above).

The second digit designates a System Level which identifies where the code was generated.

The last two digits a specific Condition Type which when coupled with the first two digits clearly isolate what has occurred that warrants the Return Code.

Where Priority Codes are defined as:
PRIORITY LEVEL PRIORITY CODE
INFO 3
WARNING 4
ERROR 5
FATAL 6
Where System Codes are defined as:
SYSTEM LEVEL SYSTEM CODE
Operating System 1
Memory 2
Storage 3
Network 4
Internet 5
File System 6
Database 7
NoSQL 8
Other 9
Where Type Codes are defined as:
TYPE LEVEL TYPE CODE
Permission 01
Connection 02
Locate 03
Check 04
Open 05
Close 06
Read 07
Write 08
Create 09
Delete 10
Rename 11
Dump 20
Load 21
Get 30
Put 31
To further illustrate this example, here is how some Return Codes may be used:
RETURN CODE SAMPLE MESSAGE
3605 Open File Successful
4304 Disk Space is Low
5701 Invalid DB Login
6205 Insufficient Memory

Other numbering schemes will work just as well as long as the best practice is to have well defined Return Codes and wide adoption.

Summary

Jobs should have well defined entry/exit/return points, and along with them an established standard for Job Return Codes.

It is highly recommended to design them to provide flexibility when new codes are required so they can be easily added and to construct them to support a variety of conditions that can be programmatically controlled and monitored.