Some log messages are triggered by operations such as pipeline executions and thus are automatically displayed on screen.
However, if you would like to visualize all the logs related to your pipeline, for debugging purposes, you need to click the VIEW LOGS button in the Metrics tab of the Pipeline Details panel.
Spark on Yarn logs
If you have selected a Big Data/Spark on Yarn run profile before executing your pipeline, your logs are handled differently by Talend Pipeline Designer.
- the log events related to starting and preparing the pipeline execution are available directly from the Talend Cloud Pipeline Designer. window in
- the log events related to the pipeline execution itself are available
from your Hadoop Cluster. Note: You might need to replace the default host name in the URL that is provided in the LOGS with your Hadoop YARN ResourceManager host name.
The interface shown in the picture above is indicative only and might differ slightly according to your Hadoop YARN ResourceManager.