Apache Sqoop - Import - 7.1

Talend Data Catalog Bridges

author
Talend Documentation Team
EnrichVersion
7.1
EnrichProdName
Talend Big Data Platform
Talend Data Fabric
Talend Data Management Platform
Talend Data Services Platform
Talend MDM Platform
Talend Real-Time Big Data Platform
EnrichPlatform
Talend Data Catalog

Bridge Specifications

Vendor Apache
Tool Name Sqoop
Tool Version 1.0
Tool Web Site http://sqoop.apache.org/
Supported Methodology [Data Integration] Multi-Model, Data Store (Physical Data Model), (Source and Target Data Stores, Transformation Lineage, Expression Parsing) via SQL TXT File
Data Profiling
Multi-Model Harvesting
Incremental Harvesting
Remote Repository Browsing for Model Selection

SPECIFICATIONS
Tool: Apache Sqoop version 1.0 via SQL TXT File
See http://sqoop.apache.org/
Metadata: [Data Integration] Multi-Model, Data Store (Physical Data Model), (Source and Target Data Stores, Transformation Lineage, Expression Parsing)
Bridge: ApacheSqoop version 11.0.0

OVERVIEW
This bridge imports metadata from Apache SQOOP scripts.


Bridge Parameters

Parameter Name Description Type Values Default Scope
Directory Select a directory that contains Sqoop scripts to import DIRECTORY     Mandatory
File Filter Specify a search filter for files. Use regular expressions in java format if needed (e.g. '.*\.sql'). Multiple conditions can be defined by using a space as a separator (e.g. 'file1 file2'). The condition must be escaped with double quotes if it contains any spaces inside (e.g. "my file.sql"). Negation can be defined with the preceeding dash character (e.g. '-\.tar\.gz'). STRING      
Miscellaneous Specify miscellaneous options identified with a -option followed by a value if required:

GENERAL OPTIONS
-m <Java Memory's maximum size>
1G by default on 64bits JRE or as set in conf/conf.properties, e.g.
-m 8G
-m 2500M

-j <Java Runtime Environment command line options>
This option must be the last one in the Miscellaneous parameter as all the text after -j is passed "as is" to the JRE, e.g.
-j -Dname=value -Xms1G

-jre <Java Runtime Environment full path name>
It can be an absolute path to javaw.exe on Windows or a link/script path on Linux, e.g.
-jre "c:\Program Files\Java\jre1.8.0_211\bin\javaw.exe"

-v <Environment variable value>
None by default, e.g.
-v var1=value1 -v var2="value2 with spaces"

-model.name <model name>
Override the model name, e.g.
-model.name "My Model Name"

-prescript <script name>
The script must be located in the bin directory, and have .bat or .sh extension.
The script path must not include any parent directory symbol (..).
The script should return exit code 0 to indicate success, or another value to indicate failure.
For example:
-prescript \"script.bat\"

-pppd
Create the connections and connection data sets in DI/ETL design models. This feature should only be used when intending to export to another DI/ETL tool.

APACHE SQOOP OPTIONS
-s <pathname>
Path to the key/value pair file that will be used to preprocess the script files and replace shell parameters with the actual values. The path can be escaped with double quotes if it contains spaces or any special characters. The colon character (:) must be escaped with '\' if it is part of a parameter name. The file (referred to by the pathname) must have the following format:
key1:value1
"key 2":value2
\:key3withcolon:value3
E.g.
-s J:\MIMB\map_of_shell_parameters.txt

-e <encoding>
This value will be used to load text from the specified script files. By default, UTF-8 will be used. E.g.
-e UTF-16
-e UTF-16BE
-e US-ASCII
-e ISO-8859-1

-p <path>
HDFS destination path for tables when --target-dir and --warehouse-dir are not specified. If an imported table is 'country' then the target directory will be '/user/cloudera/country', E.g.
-p /user/cloudera
STRING      

 

Bridge Mapping

Mapping information is not available