Creating an HDInsight cluster on Azure - 7.3

Spark Streaming
Version
7.3
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Design and Development > Designing Jobs > Job Frameworks > Spark Streaming
Last publication date
2024-02-21
Log in to your Azure account to create the HDInsight cluster to be used.

Before you begin

  • You have an Azure account with appropriate rights and permissions to the HDInsight service.

Procedure

  1. Navigate to the Azure portal: https://portal.azure.com/.
  2. Click All services on the menu bar on the left of the Azure welcome page.
  3. In the Analytics section, click HDInsight clusters to open the corresponding blade.
  4. Click Add to create an HDInsight cluster.
  5. Click Quick Create to display the Basics blade and enter the basic configuration information on this blade.
    Among the parameters, the Cluster type to be used must be the one officially supported by Talend. For example, select Spark 2.1.0 (HDI 3.6).

    For further information about the supported versions, see Supported Big Data platform distribution versions for Jobs.

    For further information about the supported versions, search for supported Big Data platforms on Talend Help Center (https://help.talend.com).

    For further information about the parameters on this blade, see Create clusters from Azure documentation.

  6. Click Next to open the Storage blade to set the storage settings for the cluster.
    1. From the Primary storage type list, select Azure storage.
    2. For the storage account to be used, select the My subscriptions radio button, then click Select a storage account and choose the account to be used from the blade that is displayed.
    3. For the other parameters, leave them as they are to use the default values or enter the values you want to use.
  7. Click Next to pass to the Summary step, in which you review and confirm the configuration made in the previous steps.
  8. Once you confirm the configuration, click Create.

Results

Azure starts to create your cluster. Once done, your cluster appears on the HDInsight clusters list.