Talend Cloud functional architecture - Cloud - 8.0

Talend Studio User Guide

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Cloud
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Design and Development
Last publication date
2024-02-29
Available in...

Cloud API Services Platform

Cloud Big Data

Cloud Big Data Platform

Cloud Data Fabric

Cloud Data Integration

Cloud Data Management Platform

The following diagram illustrates the functional architecture of Talend Cloud Big Data and Talend Cloud Data Integration.

Functional architecture of Talend Cloud Big Data and Talend Cloud Data Integration

The following diagram illustrates the functional architecture of Talend Cloud Big Data Platform and Talend Cloud Data Management Platform.

Functional architecture of Talend Cloud Big Data Platform and Talend Cloud Data Management Platform.

The following diagram illustrates the functional architecture of Talend Cloud API Services Platform, Talend Cloud Data Fabric, and Talend Data Fabric.

Functional architecture of Talend Cloud API Services Platform, Talend Cloud Data Fabric, and Talend Data Fabric

The diagram of the Talend Cloud functional architecture is divided into two main parts: the local network and the cloud infrastructure.

Local network

The local network includes a web browser, Talend Studio, and a Remote Engine or a Remote Engine Gen2.

  • From your web browser, you can access the different cloud applications.

  • From Talend Studio, you can:
    • Publish data integration Jobs, Routes, or Services to Talend Management Console as Tasks, make them available to web users, and run them in the cloud.
    • Benefit from the Talend Cloud Data Preparation features through the use of the tDatasetInput, tDatasetOutput, and tDataprepRun components. You can create datasets from various databases and export them in Talend Cloud Data Preparation, or leverage a preparation directly in a data integration Job or Spark Job.
    • Use Jobs with tDataStewardshipTaskOutput, tDataStewardshipTaskInput and tDataStewardshipTaskDelete to load, retrieve, or delete tasks from the campaigns created in the Talend Cloud Data Stewardship.
    • Available in:

      Cloud API Services Platform

      Cloud Data Fabric

      Data Fabric

    • Import API definitions directly from Talend Cloud API Designer and use them to implement services. You can then test your Talend Cloud API Tester and publish them to Talend Cloud.
  • The Talend Remote Engine is used to run Jobs, Tasks, and preparations, on premises. The Remote Engine Gen2 is used to run pipelines everywhere (on premises or in your Virtual Private Cloud).

    Available in:

    Cloud API Services Platform

    Cloud Data Fabric

    Data Fabric

    You can also run Routes, Data Services, and Tasks on premises if Talend Runtime running on the same machine as the Talend Remote Engine.

  • Available in:

    Cloud API Services Platform

    Cloud Data Fabric

    Data Fabric

  • In Talend Cloud API Tester, you can create or import requests and scenarios to test your API. You can also automate your tests using the Maven plugin.

Cloud infrastructure

The cloud infrastructure includes the cloud applications and the Cloud Engines.
  • In Talend Management Console, you can administrate roles, users, projects, and access license information. You can create new users for the cloud applications and assign them to custom groups. You can then define roles and assign them to your users. Talend Management Console stores your Talend Studio license files and enables you to create projects to collaborate on in Talend Studio. In addition, you can enable data and file transfer, data integration, and access to shared data sources for web users. You can, for example, import and use preconfigured sample Tasks, or design Tasks that automate the exchange and synchronization of data between applications.
  • In Talend Cloud Pipeline Designer, you can design complex end-to-end pipelines to process, enrich and transform data at rest or in motion. You can run these pipelines either using the embedded Cloud Engine for Design or using a Remote Engine Gen2 installed on your Virtual Private Cloud or on premises.
  • In Talend Cloud Data Preparation, you can import your data from local files or other sources, and cleanse or enrich it by creating new preparations.
  • In Talend Cloud Data Stewardship, campaign owners manage data assets and organize the data interactions whenever human intervention is required to collaborate on data curation, arbitration, or validation.
  • Available in:

    Cloud API Services Platform

    Cloud Big Data Platform

    Cloud Data Fabric

    Cloud Data Management Platform

    Data Fabric

  • In Talend Dictionary Service, you can add, remove, or modify the semantic categories that are applied to each column in your data when opened in Talend Cloud Data Preparation or Talend Cloud Data Stewardship.
  • Available in:

    Cloud API Services Platform

    Cloud Data Fabric

    Data Fabric

  • In Talend Cloud API Designer, you can design APIs or import existing definitions, publish their documentation and create mocks. You can then export your definitions in OpenAPI Specification/Swagger or RAML. You can also test your APIs onTalend Cloud API Tester at any step in the design.

The Cloud Engines are used to run Artifacts, Tasks, and preparations, or pipelines, in the cloud.

Available in:

Cloud API Services Platform

Cloud Data Fabric

Data Fabric

Routes and Data Services can only be deployed on a Remote Engine/Runtime.