Available in...Big Data
Big Data Platform
Cloud Big Data
Cloud Big Data Platform
Cloud Data Fabric
Data Fabric
Real-Time Big Data Platform
In Talend Studio, if there
is no support for the Big Data Platform you want to use, follow the procedure explained
below to add this distribution yourself to make it available to Talend Studio.
In the current Talend Studio version,
you can use this procedure to add the Cloudera and the Hortonworks distributions only. This
procedure uses Cloudera to demonstrate how to add a dynamic distribution to Talend Studio.
With this dynamic support feature, you are empowered with more agility and flexibility to
use a Cloudera or Hortonworks version that was not released the moment your Talend Studio was
released, by simply adding this version yourself through several clicks.
The dynamic distributions added this way are
generally minor versions of a Talend-certified major release of your distribution. Talend
relies on the distribution vendors' compatibility statements to ensure the compatibility
of Talend Studio with these minor versions and, by this measure, provides official support for the use
cases that can be produced on these minor versions as well as on the Talend-certified versions.
For further information about the Talend-certified distribution versions and Talend
general support policy about the certified and the compatible versions, see Supported Big Data platform distribution versions for
Talend Jobs.
- On the version list of the distributions, some versions are labeled
Builtin. These versions were added by Talend
via the Dynamic distribution mechanism and delivered with Talend Studio
when it was released. They are certified by Talend,
thus officially supported and ready to use.
Note: For Cloudera distribution, Talend
recommends you to use CDP 7.x built-in distributions rather than CDP dynamic
distribution.
Procedure
-
In the Integration perspective, click to open the Project Settings
dialog box.
-
Click to open its view.
-
From the Distribution drop-down list, select
Cloudera.
-
Set up your local Nexus repository to store the dynamic distribution jar files
to be downloaded.
While not mandatory, this step allows other users or other Talend Studio
instances to download the same jar files much faster.
-
Set up a proxy on your local Nexus repository and link this proxy to
the dedicated Talend proxy:
https://talend-update.talend.com/nexus/content/groups/dynamicdistribution/
.
The credentials to be used to connect to this
Talend proxy are:
- Username: studio-dl-client
- Password: studio-dl-client
When you create your local proxy, you need to define the credentials
specific to this local proxy. For an example about how to create a Nexus
proxy, see Proxy settings from the Nexus documentation.
-
Click to open its view, select the Override default setup check box to activate the
Repository field.
-
In the Repository field, enter the URL of your
local proxy and the credentials defined for this proxy.
-
Click Check Connection to verify its connection
status.
-
Go back to the Dynamic distribution
settings view and click the Dynamic
distribution setup button to open the dynamic distribution
configuration wizard.
-
Select the Create new dynamic configuration radio button
and click Refresh to display, on the
Version drop-down list, the Cloudera versions that are
available in the connected Cloudera repository.
-
Select the Cloudera version for which you want to generate the configuration to be
used by Talend Studio.
-
Click Finish.
Talend Studio
starts to retrieve the configuration files for this distribution from the Cloudera
repository. This retrieval may take a while.
Once done, the [Dynamic distribution setup] wizard is
automatically closed to bring you back to the Dynamic
distribution settings view. The newly generated "dynamic"
distribution for the version you previously selected is displayed on the
Version list.
-
You can repeat the operations to add more versions if needs be.
Otherwise, click Apply and Close to close
the Project settings dialog box.
Results
You can then use this new version the same way as you use the built-in distributions
provided along with
Talend Studio. You can:
Although you can usually export a Job with its dependencies such as a connection
defined in the Repository, the connection to a dynamic
distribution cannot be exported the same way. If you need to export such a
connection, see Export or import the configuration of a dynamic Big Data platform distribution.