tIcebergTable properties for Apache Spark Batch

These properties are used to configure tIcebergTable running in the Spark Batch Job framework.

The Spark Batch tIcebergTable component belongs to the Big Data family.

The component in this framework is available in all Talend products.

This component is available when you have installed the 8.0.1-R2024-07 Studio Monthly or a later one provided by Talend. For more information, check with your administrator.

Basic settings

Schema and Edit schema	A schema is a row description that defines the number of fields (columns) to be processed and passed on to the next component. Select the type of schema from the drop-down list: Built-In: You create and store the schema locally for this component only. Repository: You have already created the schema and stored it in the Repository. You can reuse it in various projects and Job designs. Click Edit schema to make changes to the schema. If you make changes, the schema automatically becomes built-in. This parameter is only available when you select Create or Create if not exists from the Action on table drop-down list.
Table name	Enter the name of the Iceberg table on which you want to work.
Action on table	Select the action you want to perform on the Iceberg table from the drop-down list: Alter: makes changes to a table. Create: creates a table. Create if it does not exist: creates a table only if no table already exists. Drop: removes table. Drop if exists: removes a table only if it already exists. Replace as select: replaces a table using a SQL query.
Create as select	Select this check box to create a table at the beginning of a query execution. The query needs to be specified in the As select query parameter. For more information, see Create table as select from Apache Iceberg documentation. This parameter is only available when you select Create or Create if it does not exist from the Action on table drop-down list.
Set table comment	Select this check box to add a comment to the table. This parameter is only available when you select Create or Create if it does not exist from the Action on table drop-down list.
Set table location	Select this check box to specify a location path in which you want the table to be created. This parameter is only available when you select Create or Create if it does not exist from the Action on table drop-down list.
Set partitions	Select this check box and click the plus button to create partitions inside the table. This parameter is only available when you select Create or Create if it does not exist from the Action on table drop-down list.
Set table properties	Select this check box to enter any extra configuration properties following the key and values format by clicking the plus button. For example, you can add properties such as format-version. For more information on available configuration properties, see Configuration from Apache Iceberg documentation. This parameter is only available when you select Create or Create if it does not exist from the Action on table drop-down list.
Purge	Select this check box to delete the table content. For more information, see DROP TABLE PURGE from the Iceberg documentation. This parameter is only available when you select Drop or Drop if exists from the Action on table drop-down list.
As select query	Enter the SQL SELECT query to be performed to replace the table. For more information, see Create table as select from Apache Iceberg documentation. This parameter is only available when you select Replace as select from the Action on table drop-down list.
Alter table action	Select from the drop-down list the action you want to perform on the table: Create branch: creates a branch. Create tag: creates a tag. Create branch if not exists: creates a branch if no branch already exists. Create tag if not exists: creates a tag if no tag already exists. Create or replace branch: creates or replaces a branch. Create or replace tag: creates or replaces a tag. Drop branch: removes a branch. Drop tag: removes a tag. Drop branch if exists: removes a branch without failing if the branch does not exist. Drop tag if exists: removes a tag without failing if the branch does not exist. Replace branch: replaces a branch. Replace tag: replaces a tag. For more information about branches and tags, see Branches and tags from the Iceberg documentation. This parameter is only available when you select Alter from the Action on table drop-down list.
Set snapshot	Select this check box to specify the snapshot from which you want to create either the branch or the tag. This parameter is only available when you select Alter from the Action on table drop-down list, and when you perform a creation of branches or tags.
Set retain	Select this check box to specify a retain duration for the branch or the tag. This parameter is only available when you select Alter from the Action on table drop-down list, and when you perform a creation of branches or tags.
Set min snapshots to keep	Select this check box to specify a minimum snapshots to be kept for the branch. This parameter is only available when you select Alter from the Action on table drop-down list, and when you perform a creation of branches.
Set max snapshot age	Select this check box to specify the snapshots maximum age for the branch. This parameter is only available when you select Alter from the Action on table drop-down list, and when you perform a creation of branches.

Global variables

ERROR_MESSAGE	The error message generated by the component when an error occurs. This is an After variable and it returns a string.

Usage

Usage rule	This component is generally used with other Iceberg components.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here