Getting Started with the Talend Metadata Bridge
In IT systems, each tool, including Talend, stores a proprietary definition of its own metadata. A modeling tool, such CA ErWin or SAP PowerDesigner, uses metadata to represent data and process models; Business Intelligence tools heavily use metadata to expose information to business users as a self-service and to model navigation paths such as hierarchies, dimensions and metrics; databases such as Oracle has a catalog that defines tables, columns, stored procedures, and so does Hadoop through environments such as Hive or HCatalog; an ETL tool has mappings, rules, input and output schemas, transformation routines...
This translates into reduced time to deliver new applications, reduced costs for changes related to applications updates, conversions and re-platforming, and better agility for continuous improvement.
The Talend Metadata Bridge brings into our unified platform import and export capabilities for metadata. For example, using the Talend Metadata Bridge you can import into and export from our Studio:
- Modeling metadata : This allows reusing and staying in sync with objects designed with a data and/or process modeling tools such as CA ErWin, Embarcadero ER/Studio, IBM Infosphere Data Architect or SAP Power Designer.
- Business Intelligence metadata : This allows reusing and staying in sync with semantic layers needed in Business Intelligence databases or front-ends such as Oracle Essbase and OBI, SAP BW or Business Objects, Microstrategy, QlikView, Tableau, and so on.
- ETL metadata : This allows importing/exporting metadata from other ETL tools such as IBM Infosphere DataStage, Informatica PowerCenter, and so on.
- Productivity Tools: This allows exposing or editing Talend metadata through Excel, Access, or Visio. For example a business user can use it to review, validate or even update Talend metadata definitions and mappings without having to learn our Studio.
The Talend Metadata Bridge is included with licensed Studio distributions (on subscription) for all versions.Which platform and OS does the Metadata Bridge work on?
Windows is the only operating system supported at this time. We are in the process of porting most bridges to Java in the next versions of Talend. However some bridges will always depend on Windows, simply because they exchange metadata with tools that are Windows only.
The Talend Metadata Bridge is supported on Microsoft Windows 2008 Server, Windows 2012 Server, Windows 7, Windows 8, or Windows 8.1.How do I get the Metadata Bridge? How do I install it?
The Talend Metadata Bridge is a Studio plug-in that extends the Studio with a new import/export capability through the File > Import or File > Export menus. This plug-in is provided as an Eclipse RCP “update site” which you can install right from within the Talend Studio.
Once again, this plug-in only works when the Talend Studio runs on a Windows machine or appliance.
To install the Talend Metadata Bridge plug-in:
- Start your Talend Studio and go to
Help > Install Additional Packages.
- In the Additional Talend Packages dialob box, select
Talend Metadata Bridge and click Finish to
- When the plug-in installation completes, Eclipse offers you to restart or continue. You
must restart the Studio.
Sorry, you just can't at this moment. You need to find a Windows machine or VM to perform the metadata import/export on. Hopefully this is not an every day activity so you will be able to get back to your Linux or Mac machine after the import/export is done.
Note however that there are two active projects in the roadmap:
- Port the bridges to pure Java
- Remote Bridge “agent” that you will be able to install on a separate Windows machine or VM
The Excel bridge indeed assumes the spreadsheet is formatted in a certain way. This is why the Metadata Bridge also comes with a Microsoft-compliant Excel add-in to help you follow the expected format.
Once again Windows is a prerequisite (the add-in will cause a pile of many VBA error dialogs on Excel for Mac).
The Bridge add-in extends Excel with a new “Metadata” ribbon. The new buttons help you insert pre-formatted worksheets.
The Excel add-in is located within the plug-in (at the time of this writing; actual path may vary slightly depending on the version of Talend):
Excel "add-ins" is a Microsoft Office standard so you may want to follow Microsoft’s directions to install it. Here is a typical sequence of steps to install the Metadata Bridge add-in:
In your Excel, go to File > Options , then in the Excel Options dialog box go to the Add-Ins pane, and then click on the Go... button.
In the Add-Ins dialog box browse to (actual path may vary slightly depending on the version of Talend):
then point to the file: Standard-Addin.xlam
Click OK . You should see the new Metadata ribbon in Excel from now on. You can go back to the Add-Ins dialog box and uncheck the add-in to uninstall.Does the Metadata Bridge software require a license to work?
No it does not when used on a licensed Talend Studio (on subscription). The Bridge plug-in has a built-in license to make it transparent to Talend users. Note however that this license only allows the Bridge to work from within the Talend Studio.How do I use the Metadata Bridge in the Studio?
The Metadata Bridge plug-in extends the standard Eclipse import/export wizards through the File > Import or File > Export menu, not the Import Items or Export Items tools, which are specific to the Talend repository.
There are other Eclipse import/export extensions so you may use the text filter box with the “metadata” keyword to find the bridge the first time; next time it should be preselected:
Yes, we have more than 100 bridges for you. Fortunately, we added a search button. If you do not know the formal name of the software you want to import from or export to, begin typing a keyword such as ‘ERWin’.
You are more than welcome to. The Studio plug-in automatically filers the list with the bridges that make sense and/or we are allowed to use, so you are free to pick any bridge that appears.
For information, the limitations are as follows:
- Import limitations:
- No BI tools allowed
- No ETLs except DataStage and Powercenter
- Export limitations:
- No ETLs, without exception
- No MDMs without exception
The bridges to modeling tools and Excel are bidirectional. The bridge to BI tools are export only, from Talend to BI. The ETL bridges are import-only bridges.
The Metadata Bridge automatically enforces these limitations in the import and export drop-down lists anyway.On my first import attempt I get the following error message: “[FATAL 3] [JBDG_F0003] Cannot start the Java Virtual Machine. Check your CLASSPATH”
The Metadata Bridge invokes the default Java Runtime on the Windows machine. You get this error on machines where the default JRE is not configured. To fix the problem, you may either add the full path to javaw.exe in the %PATH% environment variable, or add it in the Bridge plug-in config file located in (actual path may vary depending on your version of Talend):
Locate the <Executable> tag and add the full path to javaw.exe ; for instance:
Contact the Talend Support .
Importing and exporting metadata using Talend Metadata Bridge
Powered by Meta Integration® Model Bridge (MIMB), the Talend Metadata Bridge brings into the Talend Studio import and export capabilities for metadata. Using the Talend Metadata Bridge you can import metadata into your Talend Studio and export metadata from your Talend Studio to external IT systems including:
- process modeling tools such as CA ErWin, Embarcadero ER/Studio, IBM Infosphere Data Architect or SAP Power Designer
- Business Intelligence databases or front-ends such as Oracle Essbase and OBI, SAP BW or Business Objects, Microstrategy, QlikView, Tableau, and so on
- ETL tools such as IBM Infosphere DataStage and Informatica PowerCenter
- Applications such as Excel, Access, or Visio
Importing metadata from external systems
From your Talend Studio, select
the File > Import... menu.
In the [Import] wizard, select Import
metadata under the Metadata Bridge node,
and click Next.
Select a bridge from the Bridges list.
In this example, we will import metadata from a data file exported from CA ErWin 9.x Data Modeler.Tip: The parameters to set vary depending on the bridge you choose. Select any parameter to view a detailed description about it, and click in the Value field to set the value of the parameter.
- Click in the Value field for the File parameter, and click the [...] button to browse to the source file to import metadata from.
- Set the optional parameters if needed, and click Next.
Select a date format and/or a directory to hold the data files as needed. Then
click Next to start importing metadata from the selected