erwin 9.x and 2019 R1 Mart - Import - 7.1

Talend Data Catalog Bridges

author
Talend Documentation Team
EnrichVersion
7.1
EnrichProdName
Talend Big Data Platform
Talend Data Fabric
Talend Data Management Platform
Talend Data Services Platform
Talend MDM Platform
Talend Real-Time Big Data Platform
EnrichPlatform
Talend Data Catalog

Bridge Requirements

This bridge:
  • requires the tool to be installed to access its SDK.

  • is only supported on Microsoft Windows.

Bridge Specifications

Vendor erwin, Inc.
Tool Name erwin Mart
Tool Version 9.x to 2020
Tool Web Site https://erwin.com/products/erwin-data-modeler/
Supported Methodology [Data Modeling] Multi-Model, Metadata Repository, Data Store (Physical Data Model, Logical Data Model, Stored Procedure Expression Parsing), Graphical Layout via COM API
Incremental Harvesting
Remote Repository Browsing for Model Selection
Data Profiling
Multi-Model Harvesting

SPECIFICATIONS
Tool: erwin, Inc. / erwin Mart version 9.x to 2020 via COM API
See https://erwin.com/products/erwin-data-modeler/
Metadata: [Data Modeling] Multi-Model, Metadata Repository, Data Store (Physical Data Model, Logical Data Model, Stored Procedure Expression Parsing), Graphical Layout
Component: CaMart9 version 11.0.0

OVERVIEW
This bridge imports data models from Erwin Mart, including logical only models with Entities, Attributes, Domains, physical only models with Tables, Columns, Data Types, or Logical/Physical models with both. This bridge also imports all relationships and their implementation (e.g. PK/FK) as well as the graphical layout of the data models (diagrams).

REQUIREMENTS
- Erwin Installation:
This bridge connects to the erwin Data Modeler Workgroup Edition mart using its COM API.
Thus, erwin Data Modeler Workgroup Edition must be installed with a valid license on the machine where this bridge is running. This erwin installation must be able to connect to and open models in the same mart instance with the same connection information as specified in this bridge parameters. Successful import depends upon the erwin API connecting to the mart server.
The erwin libraries used are EAL.dll and ERXML.dll
Starting with Erwin Data Modeler 9.8 / 2020 R1, Erwin requires the user to re-confirm the license in the UI if the tool has not been used for more than 72 hours.
In order to avoid this inconvenience, Erwin Data Modeler should be installed through the 'Offline Activation' email process which is available only on a node locked license.

- Erwin User:
This bridge must run on behalf of the Windows user that can run the erwin software.
If this bridge is running as part of a Windows desktop application, then this bridge must log on windows with that user.
If this bridge is running as part of a Windows service, then go to the Windows services and click on the 'Properties' menu of such service, go to the 'Log On' tab, select 'This account' and specify the same user name and password used as Windows session when testing connectivity from erwin to the mart, and restart the Windows service.

FREQUENTLY ASKED QUESTIONS
Q: What are the required configuration steps to support large models?
A: First, configure the Mart server with larger java memory settings for the Tomcat application server.
Read this Knowledge Base article for details: http://erwin-knowledgebase.com/index.php?View=entry&EntryID=3724
Then, configure MIMB with larger java memory settings to use in this bridge.
Update the file $MetaIntegration/conf/conf.properties with appropriate -Xmx parameter for a 64 bit JVM, for example:
M_JAVA_HOME=..\jre
M_JAVA_OPTIONS=-Xmx7G
This bridge Miscellaneous parameter can also be used to specify the amount of java memory to allocate.

LIMITATIONS
Refer to the current general known limitations at http://metaintegration.com/Products/MIMB/MIMBKnownLimitations.html or bundled in Documentation/ReadMe/MIMBKnownLimitations.html

Erwin Marts on Oracle databases are only supported with erwin and Mart version 9.1 or later.

SUPPORT
Provide a trouble shooting package with:
- the debug log (can be set in the UI or in conf/conf.properties with MIR_LOG_LEVEL=6)
- the metadata backup if available (can be set in the Miscellaneous parameter with option -backup)

-Erwin Windows Com Api debugging:
This bridge uses the Erwin Windows COM API via simple scripts (ErwinApi.js and ErwinXml.js).
If these scripts are not working properly, contact Erwin support to debug the issue (which may require re-installing erwin).
The scripts can be tested as follows:
cscript.exe //E:jscript //Nologo .\conf\MIRModelBridgeTemplate\ERwin\ERwinApi.js -version 9 -GetVersion
cscript.exe //E:jscript //Nologo .\conf\MIRModelBridgeTemplate\ERwin\ERwinXml.js -version 9 -from eMovies.erwin -to eMovies.xml -log eMoviesLog.txt

- Warning On Potential Popup Messages:
When importing .erwin models, there is a possibility that erwin may present a popup window that is unavailable for this bridge to respond to.
The result is this bridge will hang during import. A possible workaround recommended by the vendor is to modify the registry as follows, to avoid popup messages:
* Edit the registry on the machine where this bridge is running, using Regedit.exe
* Look for the key:
for version 9.0 to 9.6: HKEY_CURRENT_USER\Software\CA\ERwin Data Modeler\9.0\Messages
for version 9.7: HKEY_CURRENT_USER\Software\erwin\Data Modeler\9.7\Messages
* Add data 'GDM-46=1;' to the value 'Message Status'


Bridge Parameters

Parameter Name Description Type Values Default Scope
Server name The network host name used when connecting to the erwin Mart server, e.g.
localhost
STRING   localhost Mandatory
Server port The network port used when connecting to the erwin Mart server. NUMERIC   18170  
Use IIS The use of Microsoft Internet Information Services (IIS) when connecting to the erwin Mart server:

'True'
Connects to MartServer using Microsoft IIS, the port number may be dynamically assigned.

'False'
Use the specified port number when connecting to the erwin Mart Server, in which case the port number must be specified.
BOOLEAN
False
True
False  
Use SSL The use of Secure Sockets Layer (SSL) when connecting to the erwin Mart server (this parameter is only applicable for erwin Mart version 9.1 or later):

'True'
Connects to MartServer using SSL (encrypted HTTPS connection).

'False'
Connect to MartServer using a standard HTTP connection.
BOOLEAN
False
True
False  
Application name The name of the application when connecting to the erwin Mart Server. STRING   MartServer Mandatory
Authentication The type of authentication used when connecting to the erwin Mart server:

'Server Authentication'
Mart Server user name and password.

'Windows Authentication'
Windows username and password for secure database access.
In Windows Authentication mode, this bridge parameters 'User name' and 'password' are not used. Instead, the login credentials are inferred from the current windows session running this bridge. If this bridge is executed from a parent application running as a windows service, this bridge uses the credentials under which the windows service was started.
ENUMERATED
Server Authentication
Windows Authentication
Server Authentication  
User name The user name for Server Authentication. This parameter is not used in Windows Authentication mode. STRING      
Password The user password for Server Authentication. This parameter is not used in Windows Authentication mode. PASSWORD      
Browse mode The mode of browsing the erwin Mart:

'Libraries and Models'
The tree of Libraries and Models is retrieved.

'Libraries only'
Only libraries may be selected in this mode, which is more efficient on large Marts.
ENUMERATED
Libraries and Models
Libraries only
Libraries and Models  
Models The semicolon separated list of erwin model locator strings, e.g.
mart://Mart/Samples/eMovies
REPOSITORY_SUBSET      
Incremental import Incremental import only extracts what has changed since the last import. The initial full metadata harvesting (model import) of a very large source system can take a long time. However the extracted metadata are organized as a multi-model, where each model is a unit of change (e.g. Schema of a RDBMS server, or report of BI server). Subsequent model imports are dramatically faster than the initial import as this bridge will automatically try to detect changes in the source system, in order to only process the modified, added or deleted models and reuse all unchanged metadata from the model cache. Note however that the detection of change is more or less efficient depending on the sources system: e.g. BI servers can quickly provide the list of new, modified or deleted reports, but not all data stores offer a schema level change detection.

'True'
Import only the changes made since the last import

'False'
import all metadata. This option is required after upgrading the bridge in particular to take full advantage of any additional metadata coverage.

For debugging purpose, the option -cache.clear of the Miscellaneous parameter can be used to clear one model from the cache which is located (by default) in: $HOME/data/MIMB/cache/<BridgeId>/<ModelId>
BOOLEAN
False
True
True  
Import UDPs In erwin data modeler, a User Defined Property (UDP) is defined with a name (e.g. Threat Level), a definition (e.g. an enumeration with values: green, yellow, orange, red), and a possible default value (e.g. green). Applying a UDP to an object (e.g. table or column) can be done with an explicit value (e.g. orange) or without value therefore the default value of the UDP (e.g. green).

UDP can be imported:

'As metadata'
Import an explicit value as Property Value, implicit values are not imported, the default value is kept on the Property Type only.

'As metadata, migrate default values'
Import explicit and implicit values as Property Value objects.

'In description, migrate default values'
Append the property name and value, even if implicit, to the object's description and/or comment (depending upon the design level). This allows you to export the UDP values to another tool which does not support the notion of UDP.

'Both, migrate default values'
Import the UDP value, even if implicit, both as metadata and in the object's description and/or comment (depending upon the design level).
ENUMERATED
As metadata
As metadata, migrate default values
In description, migrate default values
Both, migrate default values
As metadata  
Import relationship name Controls how the erwin relationship names are imported:

'From relationship name'
from the name property.

'From derived name'
from the verb phrase derived name.
ENUMERATED
From relationship name
From derived name
From relationship name  
Import column order from In Erwin data modeler, the columns physical order can viewed through edit the current subject area 'Stored Display Settings', then click on tab 'Physical', group 'Display Level', then on the 'Physical Order' radio button.

Controls how the erwin table's column order is imported:

'Column order'
Order of the columns displayed in the erwin physical view.

'Physical order'
Order of the columns in the database, as generated in SQL DDL.
ENUMERATED
Column order
Physical order
Physical order  
Import owner schemas Controls how the erwin owner schemas are imported:

'True'
The owner schema for Tables, Views and other database objects is imported.

'False'
The owner schema for Tables, Views and other database objects is ignored. This option may be useful for models where physical names are not unique, to indicate the model should be treated as a logical model (not ready for SQL DDL generation) while physical information is still maintained the model.
BOOLEAN
False
True
True  
Import diagrams Controls how the erwin data model diagrams are imported:

'None'
Diagrams are not imported.

'Current view'
Only the current view of the diagrams are imported.

'All'
Diagrams are imported twice: once for the logical view and once for the physical view.
ENUMERATED
None
Current view
All
Current view  
Move entities to subject areas In erwin data modeler, entities belong under the model and may be assigned to one or more subject areas.

Move entities to subject areas:

'True'
In order to use subject areas as conventional (UML) design packages with their own namespace where any entity may belong to ONLY one subject area. Specifically, if entities belong to only one subject area, they will be placed in the corresponding package. Other entities are placed under the model. Additionally, this option assumes that all entities are defined within a user defined subject area.

'False'
BOOLEAN
False
True
False  
Miscellaneous Specify miscellaneous options identified with a -option followed by a value if required:

GENERAL OPTIONS
-m <Java Memory's maximum size>
1G by default on 64bits JRE or as set in conf/conf.properties, e.g.
-m 8G
-m 8000M

-j <Java Runtime Environment command line options>
This option must be the last one in the Miscellaneous parameter as all the text after -j is passed "as is" to the JRE, e.g.
-j -Dname=value -Xms1G
The following option must be set when a proxy is used to access internet (this is critical to access https://repo.maven.apache.org/maven2/ (and exceptionally a few other tool sites) in order to download the necessary third party software libraries.
-j -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=3128 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=3128 -Dhttp.proxyUser=user -Dhttp.proxyPassword=pass -Dhttps.proxyUser=user -Dhttps.proxyPassword=pass

-jre <Java Runtime Environment full path name>
It can be an absolute path to javaw.exe on Windows or a link/script path on Linux, e.g.
-jre "c:\Program Files\Java\jre1.8.0_211\bin\javaw.exe"

-v <Environment variable value>
None by default, e.g.
-v var1=value1 -v var2="value2 with spaces"

-model.name <model name>
Override the model name, e.g.
-model.name "My Model Name"

-prescript <script name>
The script must be located in the bin directory, and have .bat or .sh extension.
The script path must not include any parent directory symbol (..).
The script should return exit code 0 to indicate success, or another value to indicate failure.
For example:
-prescript "script.bat arg1 arg2"

-cache.clear
Clears the cache before the import, and therefore will run a full import without incremental harvesting.
Warning: this is a system option managed by the application calling the bridge and should not be set by users.
STRING      

 

Bridge Mapping

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"erwin 9.x to 2020 Mart"
Metamodel
CA ERwin Data Modeler (Mart)
Mapping Comments
     
DirectoryStructureModel Mart  
CreationTime Creation Time  
Description Description  
ImportDate Import Date  
Name Name  
NativeId Native Id  
NativeType Native Type  
Folder Library  
Author Author  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Modifier Modifier  
Name Name  
NativeId Native Id  
NativeType Native Type  
StoreContent Model, Template  
Author Author  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Modifier Modifier  
Name Name  
NativeId Native Id  
NativeType Native Type  

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"erwin 9.x to 2020 Mart"
Metamodel
CA ERwin Data Modeler (Model)
Mapping Comments
     
Argument Argument  
Comment Comment  
DefaultValue Default Value  
Description Definition  
DesignLevel Design Level  
Kind Kind  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
Position Position  
Association Relationship  
Aggregation Identifying True if identifying
Comment Comment  
Description Definition  
DesignLevel Design Level  
ExtraConstraint Constraint  
Name Name  
NativeId Native Id  
AssociationRole Unnamed RelationshipEnd, Named RelationshipEnd  
Multiplicity Cardinality  
Name Name  
OnDelete On Delete  
OnInsert On Insert  
OnUpdate On Update  
Source Source Based on the multiplicity of each role and the Identifying/Non Identifying property
Attribute Attribute, Column, Primary Key Column - Attribute, Primary Key Attribute, Column - Attribute, Primary Key Column  
Comment Comment  
Description Definition  
DesignLevel Design Level  
ExtraConstraint Constraint if the validation rule cannot be set on the type
Generate Generate  
InitialValue Default Value imported from the default value tag associated to this type
Name Physical Name, Name  
NativeId Native Id  
Optional Nullable  
PhysicalName Physical Name  
Position Position  
BusinessRule Min - Max, Valid Values List, Default Value, User-Defined  
Comment Comment  
Description Definition  
DesignLevel Design Level  
LowerBound Min  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
UpperBound Max  
Value Default Value, Text  
CandidateKey Candidate Key Group  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
PhysicalName Physical Name if set by the user
UniqueKey Unique Key True for Primary Key/Alternate Key, False for Inversion Entries
Class Entity, Table, Table - Entity, Data Source Table  
Comment Comment  
Description Definition  
DesignLevel Design Level  
DimensionalRole Dimensional Role  
DimensionalType Dimensional Type  
Generate Generate  
Name Physical Name, Name  
NativeId Native Id  
PhysicalName Physical Name  
ClassDiagram ER Diagram  
Comment Comment  
Description Definition  
DesignLevel Design Level  
MaxDefinitionLength Max Definition Length  
Name Name  
NativeId Native Id  
Notation Notation  
PhysicalName Physical Name  
Condition Condition  
Name Name  
ConnectionNamespace Folder  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
ConnectionPackage Data Source Schema, Directory  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
DatabaseSchema Database Schema  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
DerivedType Domain The MIR UserDefined property is set to True for User Defined Domains
Comment Comment  
DataType Data Type See datatype conversion arrays
Description Definition  
DesignLevel Design Level  
InitialValue Initial Value imported from the default value tag associated to this type
Length Length extracted from the datatype
LowerBound Lower Bound imported from the validation rule associated to this type
Name Name  
NativeDataType Native Data Type  
NativeId Native Id  
NativeLogicalDataType Native Logical Data Type  
PhysicalName Physical Name computed from the datatype, length and precision
Scale Scale extracted from the datatype
UpperBound Upper Bound imported from the validation rule associated to this type
UserDataType User Data Type  
DesignPackage Subject Area  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
FlatTextFile Data Source File, File  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Name Physical Name, Name  
NativeId Native Id  
PhysicalName Physical Name  
ForeignKey Foreign Key Group  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Generate Generate  
Name Name  
NativeId Native Id  
PhysicalName Physical Name if set by the user
Generalization Subtype Relationship  
Complete Complete  
Description Definition  
DesignLevel Design Level set to Logical Only if involved in a transformation
Exclusive Exclusive  
Name Name  
NativeId Native Id  
GeneralizationRole Subtype Relationship Role  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
OnDelete On Delete  
OnInsert On Insert  
OnUpdate On Update  
Identity Identity  
Cache Cache  
Cycle Cycle  
IncrementValue Increment Value  
MaximumValue Maximum Value  
MinimumValue Minimum Value  
Name Name  
StartValue Start Value  
Index Index  
Clustered Clustered  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Generate Generate  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
Unique Unique  
IndexMember Index Member  
Comment Comment  
Description Definition  
DesignLevel Design Level  
NativeId Native Id  
Position Position  
SortOrder Sort Order  
Note Note  
Author Author  
Importance Importance  
ModificationTime Modification Time  
Name Name  
NativeId Native Id  
Status Status  
Value Note  
PropertyElementTypeScope UDPs used for UDP values and column physical information specific to Teradata: COMPRESS, CASE and FORMAT
Name Name  
Scope Scope  
PropertyType UDP UDP design level: Logical and/or Physical
DataType Data Type  
Description Definition  
DesignLevel Design Level  
Group Group  
InitialValue Initial Value  
Name Name  
NativeId Native Id  
Position Position  
Usage Usage set to User
SQLViewAssociation View Relationship  
AliasName Alias Name  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
SQLTableSequence Sequence  
SQLViewAttribute View Column  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Expression Expression  
InitialValue Default Value  
Name Name  
NativeId Native Id  
Optional Nullable  
Position Position  
SQLViewEntity View  
Comment Comment  
Description Definition  
DesignLevel Design Level  
DimensionalRole Dimensional Role  
Generate Generate  
GroupByClause Group By Clause  
HavingClause Having Clause  
Name Name  
NativeId Native Id  
OrderByClause Order By Clause  
SelectType Select Type  
ViewStatement View Statement  
ViewStatementType View Statement Type  
ViewType View Type  
WhereClause Where Clause The View SQL if user defined or else the Where clause
Sequence Sequence  
Comment Comment  
DataType Data Type  
Description Definition  
DesignLevel Design Level  
Generate Generate  
Length Length  
Name Name  
NativeDataType Native Data Type  
NativeId Native Id  
Scale Scale  
StoreConnection Model Source, Data Source  
Comment Comment  
Connection Connection  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
StoreType Store Type  
SystemMajorVersion System Major Version  
SystemMinorVersion System Minor Version  
SystemReleaseVersion System Release Version  
SystemType System Type  
SystemTypeOld System Type Old  
StoreModel Model  
Author Author  
Comment Comment  
CreationTime Creation Time  
Description Definition  
DesignLevel Design Level  
ModificationTime Modification Time  
Modifier Modifier  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
StoreType Store Type  
SystemMajorVersion System Major Version  
SystemMinorVersion System Minor Version  
SystemReleaseVersion System Release Version  
SystemType System Type  
SystemTypeOld System Type Old  
StoredProcedure Stored Procedure  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Generate Generate  
Name Name  
NativeId Native Id  
Postcondition Postcondition  
Precondition Precondition  
Semantics Code  
Synonym Synonym  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Generate Generate  
Name Name  
NativeId Native Id  
Trigger Trigger  
Comment Comment  
Description Definition  
DesignLevel Design Level  
FireOnDelete Fire On Delete  
FireOnInsert Fire On Insert  
FireOnUpdate Fire On Update  
Generate Generate  
Name Name  
NativeId Native Id  
Semantics Code  
TriggerType Trigger Type  
TypeValue Type Value  
Comment Comment  
Description Definition  
DesignLevel Design Level  
Name Name  
NativeId Native Id  
PhysicalName Physical Name  
Position Position  
Value Value