Amazon Kinesis properties - Cloud

Talend Cloud Apps Connectors Guide

Version
Cloud
Language
English
Product
Talend Cloud
Module
Talend Data Inventory
Talend Data Preparation
Talend Pipeline Designer
Content
Administration and Monitoring > Managing connections
Design and Development > Designing Pipelines
Last publication date
2024-03-21

Properties to configure to connect to an Amazon Kinesis stream.

Amazon Kinesis is unidirectional and can only be used as a source dataset in your pipeline.

Amazon Kinesis connection

Property

Configuration

Selection Select or enter Amazon Kinesis.
Configuration
Engine Select your engine in the list.
Connection Specify credentials Toggle this option OFF if your authentication type does not require credentials. By default, the authentication with credentials is enabled.
AWS access key Enter the Amazon Access Key ID that uniquely identifies your AWS Account. To know how to get your Access Key and Access Secret, read Getting Your AWS Access Keys.
AWS secret key Enter the Secret Key that uniquely identifies your AWS Account. To know how to get your Access Key and Access Secret, read Getting Your AWS Access Keys.
Specify Kinesis endpoint Toggle this option ON to enter the endpoint of the Kinesis service to be used.

For example, https://kinesis.us-east-1.amazonaws.com. More valid Kinesis endpoint URLs can be found in the Amazon documentation.

Use STS Toggle this option ON to enable the AWS Security Token Service and create a new assumed role session:
  • In the ARN Role field, enter the Amazon Resource Name (ARN) of the role to assume.
  • In the Role session name field, enter an identifier for the assumed role session.
  • (Optional) Toggle the Specify STS endpoint ON to specify the AWS Security Token Service endpoint where the session credentials are retrieved from.
  • (Optional) Toggle the Specify external role ID ON to enter the external ID of a third-party that will access your AWS resources. For more information, see the Amazon documentation.
Description Enter a display name (mandatory) and a description (optional) for the connection.

Amazon Kinesis dataset

Property Configuration
Dataset name Enter a display name for the dataset. This name will be used as a unique identifier of the dataset in all Talend Cloud apps.
Connection Select your connection in the list. If you are creating a dataset based on an existing connection, this field is read-only.
Dataset AWS region Select in the list an available AWS region.
Stream Select or enter the name of the Kinesis stream you want to pull data from.
Value format Select in the list the format of the data to be retrieved:
  • Avro: If you select this format, you need to enter the schema of the Avro records to be retrieved.
  • CSV: If you select this format, you need to indicate the type of field delimiter used in the file.