Querying data in a cloud file through a materialized view and a Snowflake external table - 7.3

Snowflake

Version
7.3
Language
English (United States)
Product
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for Big Data
Talend Open Studio for Data Integration
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
Module
Talend Studio
Content
Data Governance > Third-party systems > Cloud storages > Snowflake components
Data Quality and Preparation > Third-party systems > Cloud storages > Snowflake components
Design and Development > Third-party systems > Cloud storages > Snowflake components

For more technologies supported by Talend, see Talend components.

Data in Snowflake is maintained in databases. You can query this data by using:
  • External tables, which reference data files located in a cloud storage. These tables stores file-level metadata (such as the filename, a version identifiers, and other properties) about a data file stored in an external stage, thus providing users a database table interface for querying the data in the file. For information about the Snowflake external table feature, see https://docs.snowflake.net/manuals/user-guide/tables-external-intro.html

  • Materialized views, which store pre-computed data derived by a query. Since the data is pre-computed, querying a materialized view is faster than executing the original query. For information about the Snowflake materialized view feature, see https://docs.snowflake.net/manuals/user-guide/views-materialized.html.
This scenario describes the way to query data in a file stored in AWS S3 bucket through a Snowflake external table and a materialized view. It assumes that:
  • You have a valid Amazon S3 user account.

  • The data file (log1.json in this example) is in the logs folder under your S3 bucket named S3://my-bucket.
  • You have a valid Snowflake user account.