Best Practice: Conventions for Return Codes

Overview

As discussed in the Best Practice on Conventions for Job Entry/Exit/Return Points, Talend jobs should have a clear methodology, structure, and defined standards on where and how a job starts and stops. Knowing the state upon entry to, exit from, or return from a Job can dramatically improve code structure, error handling, operational management, monitoring, notification, and debugging of these code modules.

Defining a common use mechanism for Job Return Codes not only deliver enterprise standards, they provide useful information about a jobs' execution and the hand-off between other connected jobs and/or joblets.

Talend components tDie and tWarn are discussed in more detail in the Best Practice on Design Elements for Job Components, however it is important to mention them here as they provide the means to specify the preferred Return Code from any job. These two components are also well defined in the Talend Component Reference Guide and can be found for use in the Logs & Errors palette of the Talend Studio.

These specialized components, placed into any job, explicitly control the exit condition from the job specifying a useful Return Code, programmable message, and in the case of the tDie component whether the current Java Virtual Machine (JVM) will close immediately on exit or not.

User defined messages for the tDie/tWarn components, a string value, are programmatically available in the DIE_MESSAGES and WARN_MESSAGES global variables, respectively.

tLogCatcher is a closely related component which when used, catches tDie/tWarn invocations. A failed components' generated ERROR_MESSAGE is further detailed in the Best Practice on Conventions for Logging.

Priorities

The tDie/tWarn components organize a jobs exit using Priority Levels, an integer value, which are programmatically available in the DIE_PRIORITY and WARN_PRIORITY global variables, respectively. At lease one Priority Level must be selected.

These Priorities, which are industry standard, include:

TRACE	Fine-grained informational events that track job execution line by line
DEBUG	Medium-grained informational events that are most useful when debugging
INFO	Coarse-grained informational events to highlight execution process
WARNING	Potentially harmful events where execution process continues
ERROR	Problematic events where execution process may or may not continue
FATAL	Critical events where execution process aborts

As TRACE and DEBUG Priority Levels can generate a huge amount of detail, the recommended best practice is to limit their use to actual debugging scenarios and rarely or never use them in a production environment.

For the remaining Priorities, the recommend best practice is to adopt programming conventions to be widely used across jobs and projects within the enterprise.

Return Codes

It is highly recommended that jobs utilize these tDie/tWarn components and that, in-conjunction with the Priority Levels, well defined Return Codes, are established and enforced.

These can be simple binary values that return 1=true/0=false, or 1=unsuccessful/0=successful. These can also be complex where returned values encode specialized meaning on the job/components condition.

The following example of a complex offers one possible convention.

Return Code Example

Have the Return Code defined as a 4 digit number where the first digit designates the Priority Level which allows calling jobs to determine the nature of the Return Code (as organized above).

The second digit designates a System Level which identifies where the code was generated.

The last two digits a specific Condition Type which when coupled with the first two digits clearly isolate what has occurred that warrants the Return Code.

Where Priority Codes are defined as:

PRIORITY LEVEL	PRIORITY CODE
INFO	3
WARNING	4
ERROR	5
FATAL	6

Where System Codes are defined as:

SYSTEM LEVEL	SYSTEM CODE
Operating System	1
Memory	2
Storage	3
Network	4
Internet	5
File System	6
Database	7
NoSQL	8
Other	9

Where Type Codes are defined as:

TYPE LEVEL	TYPE CODE
Permission	01
Connection	02
Locate	03
Check	04
Open	05
Close	06
Read	07
Write	08
Create	09
Delete	10
Rename	11
Dump	20
Load	21
Get	30
Put	31

To further illustrate this example, here is how some Return Codes may be used:

RETURN CODE	SAMPLE MESSAGE
3605	Open File Successful
4304	Disk Space is Low
5701	Invalid DB Login
6205	Insufficient Memory

Other numbering schemes will work just as well as long as the best practice is to have well defined Return Codes and wide adoption.

Summary

Jobs should have well defined entry/exit/return points, and along with them an established standard for Job Return Codes.

It is highly recommended to design them to provide flexibility when new codes are required so they can be easily added and to construct them to support a variety of conditions that can be programmatically controlled and monitored.