Cloud Hadoop distributions
Talend supports the following cloud platforms for Big Data. Click your cloud platform to see the Big data support information.
- Google Dataproc
- Databricks on Azure
- Databricks on AWS
- Quoble on AWS
- Cloudera on Azure
- Cloudera on AWS
- Microsoft HD Insight
Google Dataproc version | Supported frameworks | Supported elements in Spark batch | Supported elements in Spark streaming | Supported elements in Standard |
---|---|---|---|---|
v1.1 | Standard Talend MapReduce Spark v2.0 |
Hive BigQuery Google Storage Avro Delimited Parquet Postional XML JSON |
Hive BigQuery Google Storage Avro Delimited Parquet Postional XML JSON Google PubSub |
Hive |
Databricks on Azure version | Supported frameworks | Supported elements in Spark batch | Supported elements in Spark streaming | Supported elements in Standard |
---|---|---|---|---|
3.5 LTS | Standard Spark v2.2 |
Hive Azure Blob Storage ADLS Gen1 |
Hive Azure Blob Storage ADLS Gen1 |
DBFS |
Databricks on AWS version | Supported frameworks | Supported elements in Spark batch | Supported elements in Spark streaming | Supported elements in Standard |
---|---|---|---|---|
3.5 LTS | Standard Spark v2.4 |
Hive S3 DynamoDB |
Hive S3 DynamoDB Kinesis |
DBFS |
Qubole version | Supported frameworks | Supported elements in Spark batch | Supported elements in Spark streaming | Supported elements in Standard |
---|---|---|---|---|
Qubole Spark 2 | Standard Spark v2.2 |
redshift S3 DynamoDB |
redshift S3 DynamoDB Kinesis |
S3 Hive |
Cloudera version | Supported frameworks | Supported elements in Spark batch | Supported elements in Spark streaming | Supported elements in Standard |
---|---|---|---|---|
CDH5.11 | Spark v2.1 | ADLS Gen1 Azure Blob Storage HDFS |
ADLS Gen1 Azure Blob Storage HDFS |
Cloudera version | Supported frameworks | Supported elements in Spark batch | Supported elements in Spark streaming | Supported elements in Standard |
---|---|---|---|---|
CDH5.11 | Spark v2.1 | S3 Redshift DynamoDB |
S3 Kinesis Redshift DynamoDB |
Microsoft HD Insight version | Supported frameworks | Supported elements in Spark batch | Supported elements in Spark streaming | Supported elements in Standard |
---|---|---|---|---|
3.4 | Talend MapReduce Spark v1.6 |
Hive | Hive | |
3.6 | Talend MapReduce Spark v2.1 |
Hive | Hive |