Querying data in a cloud file through a materialized view and a Snowflake external table - 7.3


Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Data Integration
Talend Data Management Platform
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Real-Time Big Data Platform
Talend Studio
Data Governance > Third-party systems > Cloud storages > Snowflake components
Data Quality and Preparation > Third-party systems > Cloud storages > Snowflake components
Design and Development > Third-party systems > Cloud storages > Snowflake components
Last publication date

For more technologies supported by Talend, see Talend components.

Data in Snowflake is maintained in databases. You can query this data by using:
  • External tables, which reference data files located in a cloud storage. These tables stores file-level metadata (such as the filename, a version identifiers, and other properties) about a data file stored in an external stage, thus providing users a database table interface for querying the data in the file. For information about the Snowflake external table feature, see https://docs.snowflake.net/manuals/user-guide/tables-external-intro.html

  • Materialized views, which store pre-computed data derived by a query. Since the data is pre-computed, querying a materialized view is faster than executing the original query. For information about the Snowflake materialized view feature, see https://docs.snowflake.net/manuals/user-guide/views-materialized.html.
This scenario describes the way to query data in a file stored in AWS S3 bucket through a Snowflake external table and a materialized view. It assumes that:
  • You have a valid Amazon S3 user account.

  • The data file (log1.json in this example) is in the logs folder under your S3 bucket named S3://my-bucket.
  • You have a valid Snowflake user account.