このシナリオでは、Spark StreamingジョブでtKafkaInputAvroを使ってスキーマレジストリーからフライト情報を取得する方法を説明します。
でサポートされているテクノロジーの詳細は、Talendコンポーネントをご覧ください。
このシナリオでは次のSpark Streamingジョブを作成します。
このシナリオを複製する前に、Kafkaシステムが稼動していること、また、使用するKafkaトピックにアクセスするための適切な権限とアクセス権があることを確認してください。
このシナリオでは、次ののAvroスキーマからデータを取得します。
{
"type": "record",
"name": "flightRecord",
"namespace": "flightInformation",
"fields": [
{
"name": "flightNumber",
"type": "string",
},
{
"name": "departure",
"type": "string",
},
{
"name": "destination",
"type": "string",
},
{
"name": "nbPassengers",
"type": "int",
},
{
"name": "aircraftSize",
"type": "string",
}
]
}このシナリオでは、次ののAvroメッセージからデータを取得します。
{"flightNumber":"OMP45","departure":"Paris","destination":"Athens","nbPassengers":120,"aircraftSize":"Medium"}
{"flightNumber":"FGH34","departure":"Paris","destination":"Oslo","nbPassengers":122,"aircraftSize":"Medium"}
{"flightNumber":"XHK20","departure":"Madrid","destination":"Buenos Aires","nbPassengers":247,"aircraftSize":"Large"}
{"flightNumber":"TUI09","departure":"Zurich","destination":"Johannesburg","nbPassengers":322,"aircraftSize":"Large"}
{"flightNumber":"CDI03","departure":"Frankfurt","destination":"New-York","nbPassengers":366,"aircraftSize":"Large"}
{"flightNumber":"JKF77","departure":"Paris","destination":"Los-Angeles","nbPassengers":380,"aircraftSize":"Large"}
{"flightNumber":"LBZ23","departure":"London","destination":"Shanghai","nbPassengers":416,"aircraftSize":"Large"}
{"flightNumber":"NSV50","departure":"London","destination":"Vienna","nbPassengers":95,"aircraftSize":"Small"}
{"flightNumber":"LRS12","departure":"Roma","destination":"Rio de Janeiro","nbPassengers":395,"aircraftSize":"Large"}
{"flightNumber":"ALJ67","departure":"Roma","destination":"Warsaw","nbPassengers":102,"aircraftSize":"Small"}
サンプルデータはあくまでも例示用です。