tKafkaCreateTopic Standard properties - Cloud - 8.0

Kafka

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Open Studio for Big Data
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Data Governance > Third-party systems > Messaging components (Integration) > Kafka components
Data Quality and Preparation > Third-party systems > Messaging components (Integration) > Kafka components
Design and Development > Third-party systems > Messaging components (Integration) > Kafka components

These properties are used to configure tKafkaCreateTopic running in the Standard Job framework.

The Standard tKafkaCreateTopic component belongs to the Internet family.

The component in this framework is available in all Talend products with Big Data and in Talend Data Fabric.

Basic settings

Use an existing connection

Select this check box and in the Component List drop-down list, select the desired connection component to reuse the connection details you already defined.

Version

Select the version of the Kafka cluster to be used.

Action on topic

Select how a topic is created.

  • Create topic: it creates a topic. If this topic exists already, the entire Job is stopped.

  • Create topic if not exists: it creates a topic when this topic does not exist. If this topic exists, the Job skip the creation to move to the next step.

Topic name

Enter the name of the topic to be created.

Broker list

Enter the addresses of the broker nodes of the Kafka cluster to be used.

The form of this address should be hostname:port. This information is the name and the port of the hosting node in this Kafka cluster.

If you need to specify several addresses, separate them using a comma (,).

Replication factor

Enter the number of the replicas to be created for the logs (the message sets) of the topic partitions.

Number of partitions

Enter the number of partitions to be created for the topic.

Set topic retention time (ms)

Select this check box to define the maximum time in milliseconds during which the contents of the topic are retained.

At the end of the time, the contents are removed or compacted depending on the policy you define in the Topic properties table in the Advanced settings view.

Use SSL/TLS

Select this check box to enable the SSL or TLS encrypted connection.

Note: This option is available when you have installed the 8.0.1-R2022-10 Studio Monthly update or a later one delivered by Talend. For more information, check with your administrator.

Set keystore

Select this check box to enable the SSL or TLS encrypted connection via a tSetKeystore component.

Then you need to use the tSetKeystore component in the same Job to specify the encryption information.

This check box is available when you select the Use SSL/TLS check box.

Note: This option is available when you have installed the 8.0.1-R2022-10 Studio Monthly update or a later one delivered by Talend. For more information, check with your administrator.

Advanced settings

Zookeeper security is enabled

Select this check box to connect to a security-enabled ZooKeeper to increase the security in your Kafka cluster.

For more information about the security features introduced in Kafka 0.9.0.0 version, see the official Kafka documentation.

This property is not available with Kafka 2.4.x onwards versions.

Topic properties

Add the Kafka topic properties you need to customize to this table.

For more information about the topic-level properties you can define in this table, see the section describing the topic-level configuration from the official Kafka documentation.

Kafka properties

Add the Kafka Admin properties you need to customize to this table.
For example, you can use this table to specify SSL configuration by providing the following properties and associated values:
  • "security.protocol" property with "SSL" value
  • "ssl.truststore.location" property with "context.ssl_trustore_location" value
  • "ssl.truststore.password" property with "context.ssl_password" value

For more information about the Kafka Admin properties you can define in this table, see Admin Configs from the official Kafka documentation.

This property is only available when you use Kafka 3.2.x version.

Timeout (ms)

Specify the amount of time in milliseconds you want Talend Studio to wait for:
  • the connection to Kafka, for Kafka 2.4.x onwards versions
  • the session of and connection to ZooKeeper, for Kafka 2.2.1 backwards versions

The default value is 10000 milliseconds.

Note: This option is available when you have installed the 8.0.1-R2022-11 Studio Monthly update or a later one delivered by Talend. For more information, check with your administrator.

tStatCatcher Statistics

Select this check box to gather the processing metadata at the Job level as well as at each component level.

Global Variables

ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After variable and it returns a string.

Global Variables

ERROR_MESSAGE

The error message generated by the component when an error occurs. This is an After variable and it returns a string.

Usage

Usage rule

This component is used standalone to create a topic that the other Kafka components can use in the other Jobs or subJobs.

Limitation

Due to license incompatibility, one or more JARs required to use this component are not provided. You can install the missing JARs for this particular component by clicking the Install button on the Component tab view. You can also find out and add all missing JARs easily on the Modules tab in the Integration perspective of your studio. For details, see Installing external modules. You can find more details about how to install external modules in Talend Help Center (https://help.talend.com).