When executing a Job, Talend Studio allows you to collect key runtime execution information,
statistics, logs and volumetrics, for tracking and analysis through the tStatCatcher Statistics option or through using a log
component.
Examples include monitoring how often a Job is executed, how long it
takes to run, how many records are processed, whether it completes successfully, what is
the reason it fails, etc.
The statistics, logs and volumetrics data can be written out to the Java
console, delimited files and database tables, or any combination of the three. If the
data is stored in delimited files or database tables, it is written to three separate
files or tables based on the configuration as described in the following procedure.
Procedure
-
Click
on the toolbar of the Studio main window or click
from the menu bar to open the Project
Settings dialog box.
-
In the tree view of the dialog box, expand the Job Settings node and click Stats & Logs to open the corresponding
view.
Note that the
Stats & Logs preferences can be set for
all Jobs in a project via
Project Settings if the
preferences for
Stats & Logs will not change
depending upon the context of execution. It can also be set separately for
individual Jobs via the
Stats & Logs view on the
Job tab. For more information, see
Automating the use of statistics & logs.
-
Specify the type of the information you want to capture by selecting the
Use statistics (tStatCatcher), Use logs
(tLogCatcher), and/or Use volumetrics
(tFlowMeterCatcher) check boxes where relevant.
- When the Use statistics (tStatCatcher) check box
is selected, the key information such as the Job start time, end time,
duration, and success or failure status is captured. If the
Catch components statistics (tStatCatcher
Statistics) check box is selected, or if the
tStatCatcher Statistics check box on the
Advanced settings view of a specific
component is selected, the key information for the component is also
captured.
- When the Use logs (tLogCatcher) check box is
selected, messages are harvested from Java Exception, the
tDie component, and/or the
tWarn component according to the state of the
Catch runtime errors, Catch user
errors, and Catch user warnings
check boxes.
- When the Use volumetrics (tFlowMeterCatcher)
check box is selected, the row counts from all
tFlowMeter components in a Job is
captured.
-
Select the On Console check box if you want to display
data on the console.
-
Select the On Files check box if you want to store data
in delimited files.
The relevant fields are enabled or disabled according to your settings.
Specify the name of the delimited files in the Stats File
Name, Log File Name, and/or
Meter file name fields according to the type of the
information you select.
-
Select the On Database check box if you want to store
data in database tables.
The relevant fields are enabled or disabled according to your settings. Specify the value
of the database connection parameters in the corresponding fields, and the name
of the database tables in the
Stats Table,
Logs Table, and/or
Meter Table
fields according to the type of the information you select.
Tip: The stats, logs, and/or meter tables are created
automatically by
Talend if they do not exist. You can also create those tables using
tCreateTable based on the schema of
tStatCatcher,
tLogCatcher, and
tFlowMeterCatcher, with customized column length to avoid
data truncation issues.
Warning: Stats or
logs is not supported with JDBC generic component for Teradata. To store
statistics and log data in a Teradata database, select
Teradata from the Db Type
drop-down list and provide related parameters for connecting the Teradata
database as prompted.
Note that the Stats & Logs preferences can be set for all Jobs in a project via Project Settings if the preferences for Stats & Logs will not change depending upon the context of execution. It can also be set separately for individual Jobs via the Stats & Logs view on the Job tab. For more information, see Automating the use of statistics & logs.