Google BigQuery properties

Google BigQuery properties - Cloud

Talend Cloud Apps Connectors Guide

Version

Cloud

Language

English

Product

Talend Cloud

Module

Talend Data Inventory

Talend Data Preparation

Talend Pipeline Designer

Content

Administration and Monitoring > Managing connections

Design and Development > Designing Pipelines

Last publication date

2024-03-21

Properties to configure to establish a connection to Google BigQuery.

Google BigQuery connection

Property		Configuration
Selection		Select or enter Google BigQuery.
Configuration
Engine		Select your engine in the list.
Connection	Project name	Enter the unique identifier of your Google Cloud Platform project. This information is available on the Dashboard page of your Google Cloud console. To know more about Google Cloud projects, read the Google Cloud documentation.
Connection	Google credentials	Paste the content of the JSON file that contains your service account key. To know more about the Google Cloud authentication process, read the Google Cloud documentation.
Description		Enter a display name (mandatory) and a description (optional) for the connection.

Google BigQuery dataset

Property		Configuration
Dataset name		Enter a display name for the dataset. This name will be used as a unique identifier of the dataset in all Talend Cloud apps.
Connection		Select your connection in the list. If you are creating a dataset based on an existing connection, this field is read-only.
Type		Select the type of dataset you want to create: Table: to access the dataset and dataset table located in your BigQuery bucket using their unique names. Query: to query the data in your existing tables.
Table	Dataset	Select or enter the unique name of your BigQuery dataset that contains your table. For more information, see the Google BigQuery documentation about datasets.
	Table name	Select or enter the unique name of your BigQuery table.
	Storage bucket	Enter the name of the bucket, the Google BigQuery container, which holds the data.
Query	Query	Enter the query to access the data of your choice located in your BigQuery table.
Query	Use legacy SQL	If the query to be used is the legacy SQL of BigQuery, enable this option. For further information about this legacy SQL, see the Legacy SQL query reference from the Google BigQuery documentation.

Google BigQuery destination configuration

When Google BigQuery is used as a destination dataset in your pipeline, you can select the type of operation you want to do while inserting data in your BigQuery table:

Property	Configuration
Table operation	Select how the processed data will be inserted in your BigQuery table: Append: select this option to append the processed data to the existing BigQuery table at execution time. Create if not exists: select this option to create a BigQuery table and insert the processed data in this table at execution time. Truncate: select this option to remove all the existing rows and thus overwrite the content of your BigQuery table at execution time.
Max batch size	Set the maximum number of lines allowed in each batch. Changing the default value may impact the pipeline performances.

Property

Configuration

Table operation

Select how the processed data will be inserted in your BigQuery table:

Append: select this option to append the processed data to the existing BigQuery table at execution time.
Create if not exists: select this option to create a BigQuery table and insert the processed data in this table at execution time.
Truncate: select this option to remove all the existing rows and thus overwrite the content of your BigQuery table at execution time.

Max batch size

Set the maximum number of lines allowed in each batch. Changing the default value may impact the pipeline performances.

Additional JDBC parameters might be displayed depending on whether the connector is used as a source or destination dataset, read this section to know more about these parameters.