Reading Avro messages using schema registry - Cloud - 8.0

Kafka

Version
Cloud
8.0
Language
English
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Data Governance > Third-party systems > Messaging components (Integration) > Kafka components
Data Quality and Preparation > Third-party systems > Messaging components (Integration) > Kafka components
Design and Development > Third-party systems > Messaging components (Integration) > Kafka components
Last publication date
2024-02-29

Configure tKafkaInputAvro to read Avro-formatted messages using schema registry.

Procedure

  1. Double-click tKafkaInputAvro to open its Component view.
  2. Click the button next to Edit schema to open the schema dialog box, and define the schema. In this scenario, the schema must be the same as the Avro schema.
    When done, click OK to save the changes and close the schema dialog box.
  3. In the Broker list field, enter the locations of the brokers of the Kafka cluster to be used, separating these locations using comma (,). In this scenario, only one broker exists and its location is localhost:9093.
  4. From the Starting offset drop-down list, select the starting point from which the messages of a topic are consumed. In this scenario, select From beginning, meaning to start from the oldest message of the entire topic.
  5. In the Topic name field, enter the name of the topic from which this Job consumes flights information. In this scenario, the topic is flightsTopic.
  6. In the Group id field, enter the ID of the group from which this Job consumes flights information. In this scenario, the topic is myFlightsGroup.
  7. Select the Use SSL/TLS check box and select the tSetKeystore component you previously configured in your Job from the tSetKeystore drop-down list. In this scenario, the component is tSetKeystore_1.
  8. Select the Use Schema Registry check box and set the following parameters:
    1. Url: enter the Schema Registry instance URL. In this scenario, the URL is https://localhost:8085.
    2. Basic authentication: select this check box and enter your credentials in the Username and Password fields.
    3. Set schema registry keystore: select this check to enable the SSL or TLS encrypted connection and select the tSetKeystore component you previously configured in your Job from the tSetKeystore drop-down list. In this scenario, the component is tSetKeystore_1 but it is possible to use a second tSetKeystore component if you use a different keystore file with schema registry.

Results

tKafkaInputAvro is configured to read messages using schema registry.