Retrieving information from schema registry - Cloud - 8.0

Kafka

Version
Cloud
8.0
Language
日本語
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Open Studio for Big Data
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
ジョブデザインと開発 > サードパーティーシステム > メッセージングコンポーネント > Kafka
データガバナンス > サードパーティーシステム > メッセージングコンポーネント > Kafka
データクオリティとプレパレーション > サードパーティーシステム > メッセージングコンポーネント > Kafka
Last publication date
2023-09-14

This scenario explains how to retrieve flight information from schema registry using tKafkaInputAvro in your Spark Streaming Jobs.

For more technologies supported by , see Talendコンポーネント.

In this scenario, you create the following Spark Streaming Job:

Before replicating this scenario, you need to ensure that your Kafka system is up and running and you have correct rights and permissions to access the Kafka topic to be used.

This scenario retrieves data from the following Avro schema:
{
    "type": "record",
    "name": "flightRecord",
    "namespace": "flightInformation,
    "fields": [
        {
            "name": "flightNumber",
            "type": "string",
        }
        {
            "name": "departure",
            "type": "string",
        }
        {
            "name": "destination",
            "type": "string",
        }
        {
            "name": "nbPassengers",
            "type": "int",
        }
        {
            "name": "aircraftSize",
            "type": "string",
        }
    ]
}
This scenario retrieves data from the following Avro messages:
{"flightNumber":"OMP45","departure":"Paris","destination":"Athens","nbPassengers":120,"aircraftSize":"Medium"}
{"flightNumber":"FGH34","departure":"Paris","destination":"Oslo","nbPassengers":122,"aircraftSize":"Medium"}
{"flightNumber":"XHK20","departure":"Madrid","destination":"Buenos Aires","nbPassengers":247,"aircraftSize":"Large"}
{"flightNumber":"TUI09","departure":"Zurich","destination":"Johannesburg","nbPassengers":322,"aircraftSize":"Large"}
{"flightNumber":"CDI03","departure":"Frankfurt","destination":"New-York","nbPassengers":366,"aircraftSize":"Large"}
{"flightNumber":"JKF77","departure":"Paris","destination":"Los-Angeles","nbPassengers":380,"aircraftSize":"Large"}
{"flightNumber":"LBZ23","departure":"London","destination":"Shanghai","nbPassengers":416,"aircraftSize":"Large"}
{"flightNumber":"NSV50","departure":"London","destination":"Vienna","nbPassengers":95,"aircraftSize":"Small"}
{"flightNumber":"LRS12","departure":"Roma","destination":"Rio de Janeiro","nbPassengers":395,"aircraftSize":"Large"}
{"flightNumber":"ALJ67","departure":"Roma","destination":"Warsaw","nbPassengers":102,"aircraftSize":"Small"}

Note that the sample data is created for demonstration purposes only.