Talend Snowflake Components - 7.3

Snowflake

EnrichVersion
Cloud
7.3
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
EnrichPlatform
Talend Studio
task
Data Governance > Third-party systems > Cloud storages > Snowflake components
Data Quality and Preparation > Third-party systems > Cloud storages > Snowflake components
Design and Development > Third-party systems > Cloud storages > Snowflake components
Two methods are available for connecting to, and interacting with, Snowflake from within a Talend Job: Snowflake components and JDBC components.

The preferred method is to use the new Snowflake components, which offer native connectivity and direct data manipulation (DML) of data within the Snowflake service.

Alternatively, you can use generic JDBC components, which offer a wider range of features when data definition (DDL) is required. Together these Snowflake integration capabilities provide Talend developers with Snowflake data processing options within Talend Jobs.

tSnowflakeConnection and tJDBCConnection

Take a closer look at how Snowflake works within Talend Jobs.

In this first example, the tSnowflakeConnection component establishes the connection. Use it either by specifying the connection definition and all the required parameters (refer to the details described under Talend Snowflake Connectivity), or preferably, change the Property Type connection setting from Built-In to Repository and select one of the defined connections you previously created.

Alternatively, the second example shows use of the tJDBCConnection component. Fill in the required fields, or change from the Built-In to the Repository Property Type and select one of the defined connections you previously created.

In either case, use the suggested best practice to check for a valid connection in the Job design, and exit if a connection cannot be made at runtime.

tWarn and tDie

A best practice is to incorporate the tWarn and tDie components. Additionally, once a Job design incorporates a Snowflake connection, it can be used by the other components needed for data manipulation.

tSnowflakeClose and tJDBCClose

A best practice when using Snowflake connections is to ensure that you close the connection before exiting the Talend Job. Use the tSnowflakeClose component or the tJDBCClose component to accomplish this.
They are typically triggered by the tPostJob component.

tSnowflakeInput and tSnowflakeOutput

The final steps involve using the tSnowflakeInput and tSnowflakeOutput components to manipulate data on a Snowflake connection. Both of these components allow the selection of the Property Type connection setting for Built-In or Repository. Alternatively, you may choose from a connection object created in the Talend Job, as shown previously. These two components work very much like many of the other database components found on the Studio Palette supporting Oracle, MS SQL Server, MySQL, and others. If you are familiar with using those, these new Snowflake components behave in a very similar manner.

This example shows the tSnowflakeInput component using a connection declared within the Job. The Schema selected determines the table name, and the Condition allows a filter (similar to a WHERE clause) to be added to the retreival of data. The tSnowflakeInput component is also shown being used for a lookup into a tMap component, which will make the appropriate SQL join based upon its design.
Note: The tSnowflakeInput component supports an implied SELECT functionality based upon the specified schema of the selected table.
The next example shows the tSnowflakeOutput component where the connection is selected from the Project Repository, essentially making the actual connection for the specific component at runtime. Once the operation of the component is complete (the dataset being passed in has finished being inserted) the connection will close. This technique provides the job designer a choice for connection management.