Amazon Web Services (AWS) Aurora Database (via JDBC) - Import - 7.1

Talend Data Catalog Bridges

author
Talend Documentation Team
EnrichVersion
7.1
EnrichProdName
Talend Big Data Platform
Talend Data Fabric
Talend Data Management Platform
Talend Data Services Platform
Talend MDM Platform
Talend Real-Time Big Data Platform
EnrichPlatform
Talend Data Catalog

Bridge Specifications

Vendor Amazon
Tool Name AWS Aurora Database
Tool Version 2.x
Tool Web Site https://aws.amazon.com/rds/aurora/
Supported Methodology [Relational Database] Data Store (Physical Data Model, Stored Procedure Expression Parsing) via JDBC API
Remote Repository Browsing for Model Selection
Data Profiling
Multi-Model Harvesting
Incremental Harvesting

BRIDGE INFORMATION
Import tool: Amazon AWS Aurora Database 2.x (https://aws.amazon.com/rds/aurora/)
Import interface: [Relational Database] Data Store (Physical Data Model, Stored Procedure Expression Parsing) via JDBC API from Amazon Web Services (AWS) Aurora Database (via JDBC)
Import bridge: 'JdbcMySQLImport.AmazonAurora' 10.1.0

BRIDGE DOCUMENTATION
IMPORTING FROM A MYSQL SERVER DATABASE USING JDBC.

This bridge establishes a JDBC connection with a physical database in order to extract the physical metadata. It is critical that the parameters are filled correctly for the local connection requirements on the client workstation that runs the bridge.

Important note about permissions:
The username specified must have the SELECT, SHOW VIEW and EXECUTE permissions for each target MySQL schema that contains metadata.


Bridge Parameters

Parameter Name Description Type Values Default Scope
Driver location
This parameter is optional.

By default, all necessary JARs will be loaded from:
'${MODEL_BRIDGE_HOME}/java/Jdbc/mysql'

Please use the official JDBC driver for MySQL, MySQL Connector/J. You can downloaded it from:
https://dev.mysql.com/downloads/connector/j/

However, you may use this parameter to specify a different location. All JARs found in the specified folder will be added to the CLASSPATH.

The 'com.mysql.jdbc.Driver' JDBC driver with the 'MySQL-AB JDBC Driver' name is required.
DIRECTORY      
Host Host name, IP address where the database server is running or full qualified
JDBC connection string for MySQL JDBC driver.
STRING   localhost Mandatory
Port Database engine service port number. Default value is 3306. NUMERIC   3306 Mandatory
User The database user name on whose behalf the connection is being made.

Important note about permissions:
This bridge reads the metadata for a database, rather than the data. Hence, permission assignments need to be quite different than for a standard database user. The bridge requires access to the system tables, not the actual data tables, thus it is highly recommended that one engages with their DB Admin staff to ensure proper permissions for the username entered here.

We do not have any specific instructions as to what are the minimal set of permissions required as it varies from different types of databases, as well as different versions. In order to ensure that the bridge is working properly, one must first import using the bridge with a FULL Sys Admin type username (all permissions), i.e., the same type of username that is used to create and maintain the DB. Once the bridge is working properly against a given DB, then the DB Admin staff may create a user with lesser privileges that still produce a successful result.

Note, this parameter is not required.
STRING      
Password The database user's password on whose behalf the connection is being made PASSWORD      
Schema Description: you can specify a list of database schemas to import.
When the list is empty all available schemas are imported.
The list can have one or more schema names separated by semicolons (e.g. schema1; schema2).
You can specify schema name patterns using SQL LIKE expressions syntax.
Patterns support inclusions and exclusions.
Here is an example of inclusion syntax, "A%; %B; %C%; D" that tries to get schema names that:
- start with A or
- end with B or
- contain C or
- equal D

Note: when a pattern has special characters, like spaces enclose it in single quote marks (e.g. "'two wo%'; onewo%;").

To exclude a pattern prefix it with NOT. Here is an example of exclusion syntax, "A%; %B; NOT %SYS; NOT 'SYS%'"
that contributes to the following SQL filter: "where (name like A% or name like %B) and (name not like %SYS) and (name not like 'SYS%')"
REPOSITORY_SUBSET      
Stored procedure details Specify what stored procedure details you would like to include

- 'Signature' - name and parameters of stored procedures (default)
- 'Code, signature' - above plus code
- 'None' - stored procedures are not included
ENUMERATED
None
Signature
Code, signature
Signature  
Import indexes Specifies if the Indexes should be imported or not.

- 'False' : Indexes are not imported(default)
- 'True' : Indexes are imported
BOOLEAN
False
True
False  
Miscellaneous Specify miscellaneous options identified with a -letter and value.

For example, -m 4G -f 100 -j -Dname=value -Xms1G

-f the database driver fetch size in number of rows (e.g. -f 100)

-zip exportFile to store DDL create tables statements (e.g -zip c:/temp/ddl/export.zip)

-m the maximum Java memory size whole number (e.g. -m 4G or -m 2500M ).
-s import system objects that are skipped by default (e.g. -s)
-v set environment variable(s) (e.g. -v var1=value -v var2="value with spaces").
-j the last option that is followed by Java command line options (e.g. -j -Dname=value -Xms1G).

-diff difference detection level (e.g. -diff ikm for disabling checking indexes, keys, classifier and feature maps)

-synonyms disable import of all synonyms
STRING      

 

Bridge Mapping

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Amazon Web Services (AWS) Aurora Database (via JDBC)"
Metamodel
JdbcMySQL
Mapping Comments
     
Name Name  
Argument Procedure Column The stored procedure parameters are stored in the arguments
Description Comments on the column  
Kind Type The kind of parameter
Name Name  
Association Exported Keys  
Aggregation   True if all the attributes in the Exported Key a+C105re in the Primary Key
AssociationRole Exported Keys  
ExtraConstraint Update Rule, Delete Rule  
Multiplicity   Nullable property of the columns of the Exported Key
Source   Based on the multiplicity of each role
AssociationRoleNameMap Exported Keys The rolename map is created if the columns in the primary and foreign keys are different
Attribute Table Column  
Comment Comments on the column  
InitialValue Default value  
Name Name  
Optional Based on the nullable property  
PhysicalName Name  
Position Position If position is not provided, the order in which the attributes are retrieved is used.
BaseType Types  
DataType Data Type See datatype conversion array
Length Size  
Name   The name is computed from the datatype
PhysicalName Name  
Scale Maximum scale  
CandidateKey Index, Primary Key  
Name Name  
PhysicalName Name  
UniqueKey Non-Unique property  
Class Table of type "TABLE"
Comment Comments on the table  
CppClassType   Set to ENTITY
CppPersistent   Set to True
Name Name  
PhysicalName Name  
ClassDiagram Schema A class diagram is created for each package and contains all the elements of the package
DerivedType Column Table column, stored procedure column SQL View column or type
DataType Data Type See datatype conversion array
Length Size  
Name   The name is computed from the datatype
PhysicalName Name  
Scale Decimal digits  
UserDefined   True for Type
DesignPackage Schema A Package is created for each retrieved schema. If there is no schema a default package is created.
Name Name Set to "Schema" if there is no schema or the schema has no name.
ForeignKey Exported Keys  
Name Name  
PhysicalName Name  
Index Index, Primary Key, Exported Keys  
Clustered Index type true if index type is tableIndexClustered
Name Name  
PhysicalName Name  
IndexMember Index Member, Key Member  
Position Position in the Index or key  
SortOrder Ascending/descending order  
Model Catalog The model is built using the elements contained in the catalog (The default catalog used for Teradata)
Name Name Set to "Catalog".
SQLViewAttribute View Column  
Comment Comments on the column  
Name Name  
PhysicalName Name  
Position Ordinal position  
SQLViewEntity Table of type "VIEW"
Comment Comments on the table  
Name Name  
PhysicalName Name  
StoredProcedure Stored Procedure  
Description Comments on the stored procedure  
Name Name