When executing a Job, Talend Studio allows you to collect key runtime execution information,
statistics, logs and volumetrics, for tracking and analysis through the tStatCatcher Statistics option or through using a log
component.
Examples include monitoring how often a Job is executed, how long it
takes to run, how many records are processed, whether it completes successfully, what is
the reason it fails, etc.
The statistics, logs and volumetrics data can be written out to the Java
console, delimited files and database tables, or any combination of the three. If the
data is stored in delimited files or database tables, it is written to three separate
files or tables based on the configuration as described in the following procedure.
Procedure
-
Click
on the toolbar of the Studio main window or click
from the menu bar to open the Project
Settings dialog box.
-
In the tree view of the dialog box, expand the Job Settings node and click Stats & Logs to open the corresponding
view.
Note: The
Stats & Logs preferences can
be set for all Jobs in a project via
Project
Settings if the preferences for
Stats & Logs will not change depending upon the context
of execution. It can also be set separately for individual Jobs via the
Stats & Logs view on the
Job tab. For more information,
see
Automating the use of statistics & logs.
-
Specify the type of the log information you want to capture by
selecting the Use statistics
(tStatCatcher), Use logs
(tLogCatcher) and/or Use volumetrics
(tFlowMeterCatcher) check boxes where relevant.
- When the Use statistics
(tStatCatcher) check box is selected, the key
information such as the Job start time, end time, duration, and success
or failure status is captured. If the Catch
components statistics (tStatCatcher Statistics) check
box is selected, or if the tStatCatcher
Statistics check box on the Advanced settings view of a specific component is
selected, the key information for the component is also captured.
- When the Use logs
(tLogCatcher) check box is selected, messages are
harvested from Java Exception and the tWarn and tDie
components.
- When the Use volumetrics
(tFlowMeterCatcher) check box is selected, the row
counts from all tFlowMeter
components in a Job is captured.
-
Select the On Console
check box to display the data in the console if needed.
-
Select the On Files
and/or On Database check boxes for the
storage of the log data if needed.
The relevant fields are enabled or disabled according to your
settings. Specify the name of the delimited files, the value of the database
connection parameters and the name of the database tables in the
corresponding fields according to the type of the log information and the
storage of the log data you select.
Warning: Stats or logs is not supported with JDBC generic component
for Teradata. To store statistics and log data in a Teradata database,
select Teradata from the Db Type drop-down list and provide related
parameters for connecting the Teradata database as prompted.
Note: The Stats & Logs preferences can be set for all Jobs in a project via Project Settings if the preferences for Stats & Logs will not change depending upon the context of execution. It can also be set separately for individual Jobs via the Stats & Logs view on the Job tab. For more information, see Automating the use of statistics & logs.