Microsoft Azure SQL Database (via JDBC) - Import - 7.1

Talend Data Catalog Bridges

author
Talend Documentation Team
EnrichVersion
7.1
EnrichProdName
Talend Big Data Platform
Talend Data Fabric
Talend Data Management Platform
Talend Data Services Platform
Talend MDM Platform
Talend Real-Time Big Data Platform
EnrichPlatform
Talend Data Catalog

Bridge Specifications

Vendor Microsoft
Tool Name Azure SQL Database
Tool Version 12.x or newer
Tool Web Site http://azure.microsoft.com/en-us/services/sql-database/
Supported Methodology [Relational Database] Multi-Model, Data Store (Physical Data Model, Stored Procedure Expression Parsing) via JDBC API
Remote Repository Browsing for Model Selection
Data Profiling
Multi-Model Harvesting
Incremental Harvesting

BRIDGE INFORMATION
Import tool: Microsoft Azure SQL Database 12.x or newer (http://azure.microsoft.com/en-us/services/sql-database/)
Import interface: [Relational Database] Multi-Model, Data Store (Physical Data Model, Stored Procedure Expression Parsing) via JDBC API from Microsoft Azure SQL Database (via JDBC)
Import bridge: 'JdbcMicrosoftAzureSqlDatabase' 10.1.0

BRIDGE DOCUMENTATION
IMPORTING FROM A MICROSOFT AZURE SQL DATABASE USING JDBC.

This bridge establishes a JDBC connection with a physical database in order to extract the physical metadata. It is critical that the parameters are filled correctly to satisfy the local connection requirements on the client workstation that runs the bridge. Please refer to the individual parameter's tool tips for more detailed examples.

Important note about permissions:
The username specified must have the VIEW DEFINITION and CONNECT permissions.

System objects are ignored by default. To import them specify '-s' option in Miscellaneous parameter.

Driver details:
- driver name: Microsoft JDBC Driver 4.2 for SQL Server for JVM 64-bits (com.microsoft.sqlserver.jdbc.SQLServerDriver)
- driver version: 4.2
- file: sqljdbc42.jar (653,574 bytes)
- file location: ${MODEL_BRIDGE_HOME}/java/Jdbc/mssql

Connecting with Integrated Authentication On Windows
The JDBC URL should include 'integratedSecurity=true' or other settings according to your configuration
Also please use 'Miscellaneous' option to specify path for required files as described on docs.microsoft.com.
-Djava.library.path=C:\Microsoft JDBC Driver 6.4 for SQL Server\sqljdbc_<version>\enu\auth\x86
https://docs.microsoft.com/en-us/sql/connect/jdbc/building-the-connection-url?view=sql-server-2017#Connectingintegrated


Bridge Parameters

Parameter Name Description Type Values Default Scope
Driver location
This parameter is optional.

By default, all necessary JARs will be loaded from:
'${MODEL_BRIDGE_HOME}/java/Jdbc/mssql'

However, you may use this parameter to specify a different location. All JARs found in the specified folder will be added to the CLASSPATH.

The 'com.microsoft.sqlserver.jdbc.SQLServerDriver' JDBC driver with the 'Microsoft JDBC Driver 4.2 for SQL Server' name is required.
DIRECTORY      
Host Enter the host name or IP address where MS SQL Server database server is running (examples 1 and 2). Optionally, the parameter may contain a fully qualified JDBC connection string as defined for MS SQL Server (example 3).

For Azure SQL Database, bridge requires hostNameInCertificate when specified encrypt=true (example 4).

Example #1:
someservername.com

Example #2:
192.169.10.20

Example #3:
jdbc:sqlserver://192.168.56.61:1433;databaseName=AdventureWorksDW;integratedSecurity=false;

Example #4:
jdbc:sqlserver://abc.database.windows.net:1433;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;
STRING   localhost Mandatory
Port MS SQL Server database engine service port number. Default value is 1433.

For optimal connection performance, you should set the 'Port' option when you connect to a named instance.
This will avoid a round trip to the server to determine the port number.
If both a 'Port' option and 'Instance' option are used, the 'Port' option will take precedence and the 'Instance' option will be ignored.
NUMERIC      
Instance MS SQL Server instance name.

SQL Server allow for the installation of multiple database instances per server.
Each instance is identified by a specific name.
To connect to a named instance of SQL Server, you can either specify the port number of the named instance (preferred),
or you can specify the instance name.
STRING      
User The database username on whose behalf the connection is being made.

The user should have at least the VIEW DEFINITION permission and a permission to CONNECT to the database.
Also, for the Microsoft SQL Server specific bridge, if this parameter is empty, then it is assumed that integrated security is being used (see http://msdn.microsoft.com/en-us/library/ms378428.aspx#Connectingintegrated). In this case, the bridge will attempt to connect with this type of signature:

jdbc:sqlserver://; integratedSecurity=true
instead of

jdbc:sqlserver://; user=userid;password=userpassword

However, in order for this to work, the user must have the sqljdbc_auth.dll available on the PATH environment variable. Also the version of the library must match the version of the sqljdbc4.jar that they are using.
STRING      
Password The database user's password on whose behalf the connection is being made PASSWORD      
Database This option can be used as a constraint to import a SQL Server database by specifying one database name.

The possible values are one database name: only the schemas and tables that belong to this database will be imported.

Example:
AdventureWorksDW - import tables from the 'AdventureWorksDW' database.
REPOSITORY_SUBSET     Mandatory
Schema Description: you can specify a list of database schemas to import.
When the list is empty all available schemas are imported.
The list can have one or more schema names separated by semicolons (e.g. schema1; schema2).
You can specify schema name patterns using Microsoft SQL LIKE expressions syntax.
Patterns support inclusions and exclusions.
Here is an example of inclusion syntax, "A%; %B; %C%; D" that tries to get schema names that:
- start with A or
- end with B or
- contain C or
- equal D

Note: when a pattern has special characters, like spaces enclose it in single quote marks (e.g. "'two wo%'; onewo%;").

To exclude a pattern prefix it with 'NOT'. Here is an example of exclusion syntax, "A%; %B; NOT %SYS; NOT 'SYS%'"
that contributes to the following SQL filter: "where (name like A% or name like %B) and (name not like %SYS) and (name not like 'SYS%')"
STRING      
Tables, Views Description: you can specify a list of database tables to import.
When the list is empty all available tables are imported.
The list can have one or more table names separated by semicolons (e.g. table1; table2).
You can specify table name patterns using SQL LIKE expressions syntax.
Patterns support inclusions and exclusions.
Here is an example of inclusion syntax, "A%; %B; %C%; D" that tries to get table names that: - start with A or
- end with B or
- contain C or
- equal D

Note: when a pattern has special characters, like spaces enclose it in single quote marks (e.g. "'two wo%'; onewo%;").

To exclude a pattern prefix it with NOT. Here is an example of exclusion syntax, "A%; %B; NOT %SYS; NOT 'SYS%'" that
contributes to the following SQL filter: "where (name like A% or name like %B) and (name not like %SYS) and (name not like 'SYS%')"
STRING      
Stored Procedures, Functions You can specify a list of operations (stored procedures and functions) to import from the selected databese.
When the list is empty all available operations will be imported.
The list can have one or more operation names separated by semicolons (e.g. operation1; operation2).
You can specify operation name patterns using Microsoft SQL LIKE expressions syntax.
Patterns support inclusions and exclusions.
Here is an example of inclusion syntax, "A%; %B; %C%; D" that tries to get operation names that:
- start with A or
- end with B or
- contain C or
- equal D

Note: when a pattern has special characters, like spaces enclose it in single quote marks (e.g. "'two wo%'; onewo%;").

To exclude a pattern prefix it with NOT. Here is an example of exclusion syntax, "A%; %B; NOT %SYS; NOT 'SYS%'"
that contributes to the following SQL filter: "where (name like A% or name like %B) and (name not like %SYS) and (name not like 'SYS%')"

If there is a need to prevent all of the operations from being imported, specify a value that doesn't match any operation's name in the database.
REPOSITORY_SUBSET      
Stored procedure details Specify what stored procedure details you would like to include

- 'Signature' - name and parameters of stored procedures (default)
- 'Code, signature' - above plus code
- 'Lineage, code, signature' - above plus data lineage derived from the code
- 'None' - stored procedures are not included
ENUMERATED
None
Signature
Code, signature
Lineage, code, signature
Signature  
Import indexes Specifies if the Indexes should be imported or not.

- 'False' : Indexes are not imported(default)
- 'True' : Indexes are imported
BOOLEAN
False
True
False  
Miscellaneous Specify miscellaneous options identified with a -letter and value.

For example, -m 4G -f 100 -j -Dname=value -Xms1G

-f the database driver fetch size in number of rows (e.g. -f 100)

-zip exportFile to store DDL create tables statements (e.g -zip c:/temp/ddl/export.zip)

-m the maximum Java memory size whole number (e.g. -m 4G or -m 2500M ).
-s import system objects that are skipped by default (e.g. -s)
-v set environment variable(s) (e.g. -v var1=value -v var2="value with spaces").
-j the last option that is followed by Java command line options (e.g. -j -Dname=value -Xms1G).

-diff difference detection level (e.g. -diff ikm for disabling checking indexes, keys, classifier and feature maps)

-synonyms disable import of all synonyms
STRING      

 

Bridge Mapping

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Microsoft Azure SQL Database (via JDBC)"
Metamodel
Jdbc
Mapping Comments
     
Name Name  
Attribute Table Column  
Comment Comments on the column If JDBC driver supports
InitialValue Default value  
Name Name  
Optional Based on the nullable property For Access databases, set to false only if the attribute is in a Unique Index
PhysicalName Name  
Position Position If position is not provided, the order in which the attributes are retrieved is used.
BaseType Types  
DataType Data Type See datatype conversion array
Length Size  
Name   The name is computed from the datatype
PhysicalName Name  
Scale Maximum scale  
Class Table of type "TABLE"
Comment Comments on the table If JDBC driver supports
CppClassType   Set to ENTITY
CppPersistent   Set to True
Name Name  
PhysicalName Name  
ClassDiagram Schema A class diagram is created for each package and contains all the elements of the package
DerivedType Column Table column, stored procedure column SQL View column or type
DataType Data Type See datatype conversion array
Length Size  
Name   The name is computed from the datatype
PhysicalName Name  
Scale Decimal digits  
UserDefined   True for Type
DesignPackage Schema A Package is created for each retrieved schema. If there is no schema a default package is created.
Name Name Set to "Schema" if there is no schema or the schema has no name.
SQLViewAttribute View Column  
Comment Comments on the column If JDBC driver supports
Name Name  
PhysicalName Name  
Position Ordinal position  
SQLViewEntity Table of type "VIEW"
Comment Comments on the table If JDBC driver supports
Name Name  
PhysicalName Name  
StoreModel Catalog The model is built using the elements contained in the catalog (e.g. the database for MS SQL server)
Name Name Set to "Catalog" if the catalog has no name.
Synonym Table Synonym If JDBC driver supports
Name Name