Oracle Data Integrator (ODI) - Import - 7.1

Talend Data Catalog Bridges

author
Talend Documentation Team
EnrichVersion
7.1
EnrichProdName
Talend Big Data Platform
Talend Data Fabric
Talend Data Management Platform
Talend Data Services Platform
Talend MDM Platform
Talend Real-Time Big Data Platform
EnrichPlatform
Talend Data Catalog

Bridge Requirements

This bridge:
  • requires the tool to be installed to access its SDK.

Bridge Specifications

Vendor Oracle
Tool Name Oracle Data Integrator (ODI)
Tool Version 11.1.1.6 to 12c
Tool Web Site http://www.oracle.com/technetwork/middleware/data-integrator/overview/index.html
Supported Methodology [Data Integration] Multi-Model, Data Store (Physical Data Model, Logical Data Model), ETL (Source and Target Data Stores, Transformation Lineage, Expression Parsing) via Java API
Incremental Harvesting
Multi-Model Harvesting
Remote Repository Browsing for Model Selection
Data Profiling

BRIDGE INFORMATION
Import tool: Oracle Oracle Data Integrator (ODI) 11.1.1.6 to 12c (http://www.oracle.com/technetwork/middleware/data-integrator/overview/index.html)
Import interface: [Data Integration] Multi-Model, Data Store (Physical Data Model, Logical Data Model), ETL (Source and Target Data Stores, Transformation Lineage, Expression Parsing) via Java API from Oracle Data Integrator (ODI)
Import bridge: 'OracleDataIntegrator' 10.1.0

BRIDGE DOCUMENTATION
This bridge imports models and mappings from an Oracle Data Integrator work repository.
The bridge relies on the Oracle Data Integrator java API to import metadata. Therefore, the Oracle Data Integrator tool must be properly installed and accessible from the machine executing this bridge.
This bridge does not supports odiref for ODI v12c.

FREQUENTLY ASKED QUESTIONS

Q: What are the configuration steps required for External Authentication (against an LDAP server) ?
A: Make sure the External Authentication mode is correctly configured for the ODI tool, then copy the files jps-config.xml and cwallet.sso to folder /MetaIntegration/bin/config/. Templates of the jps-config.xml and cwallet.sso files are located in the ODI installation folder (/Oracle_ODI1/modules/oracle.jps_11.1.1/client_config).
Please refer to the Oracle documentation for details:
http://www.oracle.com/webfolder/technetwork/tutorials/obe/fmw/odi/odi_11g/odi_external_authentication/odi_external_authentication.htm

SUPPORT

Q: How do I provide metadata to the support team in order to reproduce an issue?
A: In order to reproduce an issue, a backup of both the master and work repositories must be created. Instructions are available as part of the ODI documentation:
- Backup Master Repository: http://docs.oracle.com/cd/E21764_01/integrate.1111/e12643/export_import.htm#autoId10
- Backup Work Repository: http://docs.oracle.com/cd/E21764_01/integrate.1111/e12643/export_import.htm#autoId12

Q: How do I validate the ODI Home directory (Oracle Home path) ?
A: The directory structure depends from ODI version and contains many subfolders and many files, but we can check the directory structure using few files from list below:

For ODI 11 the root folder contains at least that 23 folders and 170 files.
[Oracle Home path]\modules\oracle.jps_11.1.1\jps-api.jar
[Oracle Home path]\modules\oracle.jdbc_11.1.1\ojdbc6dms.jar
[Oracle Home path]\modules\oracle.idm_11.1.1\identitystore.jar
[Oracle Home path]\oracledi.sdk\lib\odi-core.jar
[Oracle Home path]\setup\manual\oracledi-sdk\oracle.odi-sdk-jse_11.1.1.jar

For ODI 12.1.3 the root folder contains at least that 213 folders and 2461 files.
[Oracle Home path]\odi\sdk\lib\oracle.odi-sdk-jse_12.1.3.jar
[Oracle Home path]\oracle_common\modules\oracle.security-api_12.1.3.jar
[Oracle Home path]\oracle_common\modules\oracle.jps_12.1.3\jps-api.jar
[Oracle Home path]\oracle_common\modules\oracle.jdbc_12.1.0\ojdbc6.jar
[Oracle Home path]\oracle_common\modules\oracle.ldap_12.1.3\ldapjclnt11.jar


Bridge Parameters

Parameter Name Description Type Values Default Scope
Driver Name Driver Name (e.g. oracle.jdbc.OracleDriver) to connect to master repository's database. The full name (including the package name) of the class should be specified.

Here are examples for some databases:

Oracle: oracle.jdbc.OracleDriver
SQLServer: sun.jdbc.odbc.JdbcOdbcDriver
IBM DB2 8.1 Net driver: COM.ibm.db2.jdbc.net.DB2Driver
STRING   oracle.jdbc.OracleDriver Mandatory
Oracle Home path The Oracle home directory where ODI Studio and SDK are installed. The SDK is used to read the ODI metadata. The directory should have 'odi' and 'oracle_common' sub-directories. The \odi\sdk\lib\oracle.odi-sdk-jse_[VERSION].jar file must be present. It is used to identify the ODI version and load dependent libraries. DIRECTORY     Mandatory
URL Driver URL to connect to the master repository's database.
URL is driver and database dependent.

Examples for some databases:
SQLServer: jdbc:odbc:YOUR_DSN
IBM DB2 8.1 Net: jdbc:db2://COMPUTER_NAME_OR_IP:6789/DATABASE_NAME
IBM DB2 8.1 App: jdbc:db2:DATABASE_NAME
Oracle Thin: jdbc:oracle:thin:@COMPUTER_NAME_OR_IP:1521:ORACLE_SID
Oracle Thin: jdbc:oracle:thin:@(description=(address=(host=DATABASE_HOST)(protocol=tcp)(port=1521))(connect_data=(sid=SID)))
NCR Teradata: jdbc:odbc:YOUR_DSN, or when using the TeraDriver (native driver): jdbc:odbc:<DB_Server_Name>
MS Access: jdbc:odbc:YOUR_DSN
STRING   jdbc:oracle:thin:@hostname:port:sid Mandatory
Master Repository User User name used to connect to Master Repository's Database STRING     Mandatory
Master Repository Password Password for specified Master Repository Database User PASSWORD      
ODI User Oracle Data Integrator Username STRING     Mandatory
ODI User Password Password for specified Oracle Data Integrator Username PASSWORD     Mandatory
Work repository Work repository name REPOSITORY_MODEL     Mandatory
Content Browsing Scope Specify the method used to browse ODI repository content:

'Projects' - collection of Packages and Interfaces that can be selected individually or in bulk using their Folders and Projects.
Selecting a Package selects all its dependent objects, like Interfaces.
All selected objects are considered to be executable using the specified Context.

'Load Plans and Scenarios' - collection of Load Plans and Scenarios that can be selected individually or in bulk using their Folders.
Selecting a Load Plans or Scenario selects all its dependent objects, like Packages and Interfaces.
ENUMERATED
Load Plans and Scenarios
Projects
Load Plans and Scenarios  
Content Specify the list of identifiers of objects you would like to import. Identifiers must be separated by the semicolon (;).

Identifiers use TYPE_ID format (e.g. LP_1234) where ID is the Internal ID of the ODI object.

ODI Studio UI shows Internal ID in the Version tab. Different types of ODI objects can have the same Internal ID.
A combination of Type and ID makes the object identifier unique.

The TYPE prefix can be:
- 'LP' for Load Plans
- 'SC' for Scenarios
- 'SF' for Scenario Folders
- 'PR' for Projects
- 'PF' for Project Folders
- 'IN' for Interfaces
- 'MP' for ODI 12 Mappings
- 'PG' for Packages

When you need to import all available projects but you cannot specify them explicitly (e.g. because projects come and go from the repository) you can specify the PR_ALL identifier that instructs the application to import all projects.

You can specify the list of identifiers using the browsing UI. The UI allows you to browse objects by either 'Projects' or 'Load Plans and Scenarios' type. You should use the Content Browsing Scope parameter to pick the browsing type.

The browsing UI needs to connect to the repository. You must specify all repository connection parameters before invoking the browsing UI.
REPOSITORY_SUBSET     Mandatory
Context Type the Context name (e.g. 'Global') or select a Context using a browser user interface. It is used to resolve Logical Schemas to Physical schemas.
The selected Context should define Physical Schemas for all Logical Schemas involved.
REPOSITORY_MODEL     Mandatory
JavaScript API Specify the ANSI file with ODI Substitution API JavaScript implementation.

This file must contains definition of OdiRef class with required methods. You may copy/paste JavaScript example below and start coding in JavaScript to implement the OdiRef class.

function OdiRef () {
this.getSysDate = function() {
var d = new Date();
var m = d.getMonth() + 1;
var D = d.getDate();
var y = d.getFullYear();
return m + '/' + D + '/' + y;
}

// any others odiRef methods
}

var odiRef = new OdiRef();

You may use in your JavaScript code some predefined variables that represents the ODI expression context:
- isCatalogSupported
- isSchemaSupported
- technologyName
- contextName
- logicalSchemaName
- dataServerInstanceName
- catalogName
- workCatalogName
- databaseSchemaName
- workSchemaName
- errorTablePrefix
- integrationTablePrefix
- journalTablePrefix
- journalTriggerPrefix
- journalViewPrefix
- loadingTablePrefix
- temporaryIndexPrefix
- localObjectMask
- localSequenceMask
- partitionMask
- remoteObjectMask
- remoteSequenceMask
- subPartitionMask
- partitionName
FILE *.*    
Miscellaneous Specify miscellaneous options identified with a -letter and optional value.

For example, -m 4G -j -Dname=value -Xms1G

-m the maximum Java memory size whole number (e.g. -m 4G or -m 2500M ).
-v set environment variable(s) (e.g. -v var1=value -v var2="value with spaces").
-j the last option that is followed by Java command line options (e.g. -j -Dname=value -Xms1G).
-pppd enables the DI/ETL post-processor processing of DI/ETL designs in order to create the design connections and connection data sets.
-jre path to the java executable. It could be an absolute path to javaw.exe on Windows or link/script path on Linux.
for example:
-jre "c:\Program Files\Java\jre1.8.0_211\bin\javaw.exe"
STRING      

 

Bridge Mapping

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Oracle Data Integrator (ODI)"
Metamodel
Oracle Data Integrator 12 (Mapping)
Mapping Comments
     
QueryColumn Column  
Description Description  
Name Name  
Optional Mandatory  
Position Order  
QueryFilter Filter  
Description Description  
Name Name  
QueryJoin Join  
Description Description  
JoinType Type  
Name Name  
QueryLookup Lookup  
Description Description  
Name Name  
QueryMapping Mapping, Reusable Mapping  
CreationTime Creation Time  
Description Description  
ModificationTime Modification Time  
Name Name  
NativeId Internal Id  
QueryTable Set Input, Input Signature, Distinct, Set, Datastore, Expression, Output Signature, Aggregate  
Name Name  
SqlQuery Reusable Mapping Component  
Name Name  

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Oracle Data Integrator (ODI)"
Metamodel
Oracle Data Integrator 12 (Package)
Mapping Comments
     
ActivityModel Package  
Description Description  
Name Name  
NativeId Internal Id  
EmbeddedCallStep Start Load Plan, Mapping, Start Scenario  
Description Description  
Name Name  
NativeId Internal Id  
GenericStep Unknown  
Name Name  

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Oracle Data Integrator (ODI)"
Metamodel
Oracle Data Integrator 12 (Repository Browse)
Mapping Comments
     
DiDesignActivityContent Load Plan, Scenario  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  
DirectoryStructureModel ODI Repository  
CreationTime Creation Time  
Description Description  
ImportDate Import Date  
ModificationTime Modification Time  
Name Name  
NativeType Native Type  
Type Type  
Folder Scenario Folder  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Oracle Data Integrator (ODI)"
Metamodel
Oracle Data Integrator 12 (Model)
Mapping Comments
     
Attribute Column  
Description Description  
InitialValue Initial Value  
Name Name  
Optional Is Null  
Position Order  
CandidateKey Alternate Key, Primary Key  
Description Description  
Name Name  
NativeId Internal Id  
Class Datastore  
Description Description  
Name Name  
NativeId Internal Id  
PhysicalName Alias  
DatabaseSchema Schema  
Description Description  
Name Name  
NativeId Internal Id  
DesignPackage Sub-Model  
Description Description  
Name Name  
NativeId Internal Id  
FlatTextFile FileDatastore  
Description Description  
Name Name  
NativeId Internal Id  
PhysicalName Alias  
Index Not Unique Index  
Description Description  
Name Name  
NativeId Internal Id  
StoreModel Model  
CreationTime Creation Time  
Description Description  
ModificationTime Modification Time  
Name Name  
StoreType Store Type  
SystemMajorVersion System Major Version  
SystemMinorVersion System Minor Version  
SystemReleaseVersion System Release Version  
SystemType System Type  

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Oracle Data Integrator (ODI)"
Metamodel
Oracle Data Integrator 12 (Repository Organization)
Mapping Comments
     
Connection Runtime Connection  
Name Name  
DiDesignActivityContent Executable Activity, Package, Load Plan, Scenario  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  
DiParameterContent Variables, Sequences, Runtime Context  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  
DiQueryMappingContent Executable Mapping  
Description Description  
Name Name  
NativeId Internal Id  
DiRuntimeActivityContent Scenario Package Summary Lineage, Executable Summary Lineage, Runtime Package, Scenario Interface Summary Lineage, Load Plan Summary Lineage  
Description Description  
Name Name  
NativeId Internal Id  
DiRuntimeTaskContent Runtime Mapping  
Name Name  
DirectoryStructureModel ODI Repository  
CreationTime Creation Time  
Description Description  
ImportDate Import Date  
ModificationTime Modification Time  
Name Name  
NativeType Native Type  
Type Type  
Folder Context, Physical Architecture, Reusable Mappings, Database Schema, Projects, Topology, Contexts, Project, Mappings, Logical Architecture, Model Folder, Technology, Packages, Scenario Folder, Context Item, Logical Schema, Project Folder, Models, Load Plans and Scenarios, Physical Schema  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  
LineageSummaryContent LineageSummary  
Description Description  
Name Name  
NativeId Internal Id  
ParameterDefinition Variable, Sequence  
Description Description  
Name Name  
NativeId Internal Id  
ParameterModel Repository Variables, Repository Sequences  
Description Description  
Name Name  
NativeId Internal Id  
StoreContent Connection, Model, Physical Connection  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Oracle Data Integrator (ODI)"
Metamodel
Oracle Data Integrator 12 (Repository Browse Projects)
Mapping Comments
     
Connection Runtime Connection  
Name Name  
DiDesignActivityContent Package, Load Plan, Scenario  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  
DiParameterContent Runtime Context  
Name Name  
DiQueryMappingContent Executable Mapping  
Description Description  
Name Name  
NativeId Internal Id  
DiRuntimeActivityContent Scenario Package Summary Lineage, Runtime Package, Scenario Interface Summary Lineage, Load Plan Summary Lineage  
Name Name  
DiRuntimeTaskContent Runtime Mapping  
Name Name  
DirectoryStructureModel ODI Repository  
CreationTime Creation Time  
Description Description  
ImportDate Import Date  
ModificationTime Modification Time  
Name Name  
NativeType Native Type  
Type Type  
Folder Context, Project, Reusable Mappings, Mappings, Model Folder, Packages, Scenario Folder, Project Folder, Projects  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  
StoreContent Model, Physical Connection  
CreationTime Creation Time  
Description Description  
LastModificationTime Last Modification Time  
Name Name  
NativeId Internal Id  

Meta Integration Repository (MIR)
Metamodel
(based on the OMG CWM standard)
"Oracle Data Integrator (ODI)"
Metamodel
Oracle Data Integrator 12 (Mapping)
Mapping Comments
     
QueryColumn Column  
Description Description  
Name Name  
Optional Mandatory  
Position Order  
QueryFilter Filter  
Description Description  
Name Name  
QueryJoin Join  
Description Description  
JoinType Type  
Name Name  
QueryLookup Lookup  
Description Description  
Name Name  
QueryMapping Mapping, Reusable Mapping  
CreationTime Creation Time  
Description Description  
ModificationTime Modification Time  
Name Name  
NativeId Internal Id  
QueryTable Set Input, Input Signature, Distinct, Set, Datastore, Expression, Output Signature, Aggregate  
Name Name  
SqlQuery Reusable Mapping Component  
Name Name