The operating principles of the Talend products could be summarized as briefly as the following topics:
building technical or business-related processes and data profiling analyses,
administrating users, projects, access rights and processes and their dependencies,
deploying and executing technical processes,
monitoring the execution of technical processes.
Depending on your license, some of the functional blocks may not be available to you.
Each of the above topics can be isolated in different functional blocks and the different types of blocks and their interoperability can be described as in the following architecture diagram :
Building and administrating
The CLIENTS block includes one or more Talend Studio APIs and Web browsers that could be on the same or on different machines.
From the Studio API, end-users can carry out technical processes: data integration or data service processes, mediation routes and services, and publish them on the Artifact Repository and data profiling analyses and reports regardless of data volume and process complexity. The Talend Studio allows the user to work on any project for which he has authorization. For more information, see the Talend Studio User Guide.
With MDM, the Talend Studio is also used by administrators to set up and operate a centralized master data repository. They can build data models that employ the necessary business and data rules to create a single "master" copy of the master data.
From a Web browser, end-users connect to the remotely based Administration Center through a secured HTTP protocol. The end-user category in this description may include developers, project managers, administrators and any other person involved in building data flows, Web, REST and data services, and mediation routes. Each of these end-users will use either the Studio or the Administration Center or both of them depending on the company policy.
Additionally, end-users also use a Web browser to connect to Talend Data Quality Portal that plugs to the data quality data mart and extends the dynamic reporting and monitoring capabilities of the Talend Studio, and to Talend Data Stewardship Console to resolve the data issues included in the listed tasks to reach the golden data record (if they subscribed to this option), and to the MDM Web User Interface, which is where they search, display or edit master data handled by the Talend Studio.
The TALEND SERVERS and DATABASES blocks and the Git/SVN grey circle include a web-based Administration Center (application server) connected to two shared repositories: one based on a Git or SVN server and one based on a database server (Admin).
The Administration Center enables the management and administration of all projects. Administration metadata (user accounts, access rights and project authorization for example) is stored in the database server and project metadata (Jobs, Business Models, Routines, Routes, Services for example) is stored in the Git or SVN server (to easily share them between the different end-users).
The Administration Center also enables to configure the tasks that handle job executions and triggers. It also looks after the job generation and deployment to the execution servers. For more information, see the Talend Administration Center User Guide.
Finally, the Administration Center enables you to access and manage the Routes or Services created from Talend Studio and published into the Artifact Repository, and set up and monitor their deployment and execution into the Talend Runtime. For more information, see the Talend Administration Center User Guide.
There is also one MDM server where the live version of the master data is stored. The MDM Repository contains a working copy of the data, and can be stored locally (that is, on the same machine as the Talend Studio) or remotely, based on a Git or SVN server. The data from the MDM Repository must be deployed to the MDM Server before it can be accessed by users of the Talend MDM Web User Interface.
Deploying and executing
The Nexus grey circle represents the artifact repository that stores all the:
Software Updates available for download.
Routes and Services that are published from the Studio and are ready to be deployed and executed in Talend Runtime.
The TALEND EXECUTION SERVERS block represents the execution servers that run technical processes according to the execution scheduling set up in the Talend Administration Center Web application. Those execution servers can be of:
One or more Talend Runtimes (execution container) deployed inside your information system. The Talend Runtime deploys and executes the technical processes according to the set up defined in the Talend Administration Center Web application. Those processes are Jobs built from the Studio and centralized on the Git or SVN server. Routes and Services retrieved from the artifact repository.
If you have several Talend Runtimes on which to deploy the Service and Route artifacts, you will be able to load balance their execution according to your needs. All instances of Talend Runtime will communicate between each other via the Service Locator to identify the one more likely to deploy and execute the artifact(s) set to deployment in Talend Administration Center. The Talend Runtime elected for the deployment will request for the artifact(s) to deploy and execute from the artifact repository and the artifact repository will thus send the artifact(s) requested along with all the dependencies needed for its/their execution to the Talend Runtime, that will deploy and execute it/them.
One or more JobServers deployed inside your information system that run technical processes (Jobs) according to scheduled time, date or event set in the Talend Administration Center Web application.
The end-user can transfer technical processes to a remote execution server directly from the Studio (distant run).
You must install the JobServer files ("Agent"), delivered by Talend, on each of the execution servers to become operational.
The Monitoring circle represents the monitoring: the Activity Monitoring Console and the Service Activity Monitoring.
The Activity Monitoring Console allows end-users to monitor the execution of technical processes. It provides detailed monitoring capabilities that can be used to consolidate log information collected, understand the interaction between underlying data flows, prevent faults that could be unexpectedly generated and support system management decisions. For more information on the Activity Monitoring Console, see the Talend Activity Monitoring Console User Guide.
The Service Activity Monitoring allows end-users to monitor service calls. It provides monitoring and consolidated event information that can be used to understand the underlying requests and replies that compose the event, monitor faults that may be unexpectedly generated and support the system management decisions. For more information on the Service Activity Monitoring, see the Talend Administration Center User Guide.