Accessing the semantic concepts stored in the Ontology repository

EnrichVersion
6.4
6.3
EnrichProdName
Talend Real-Time Big Data Platform
Talend Data Services Platform
Talend MDM Platform
Talend Data Fabric
Talend Big Data Platform
Talend Data Management Platform
task
Administration and Monitoring > Managing repositories
Administration and Monitoring > Monitoring logs
Data Quality and Preparation > Profiling data
EnrichPlatform
Talend Administration Center
Talend Studio
Talend Log Server

The ontology repository

The ontology repository built on Talend Log Server stores hundreds of semantic concepts and attributes which you can increase every time you define and run a Semantic-aware analysis in the Profiling perspective of Talend Studio.

Accessing the semantic concepts stored in the ontology repository

You can access the list of the semantic concepts and their attributes stored in the ontology repository and applied on several domains including customer, company, geography, product, finance, etc.

Since there is not a universal way to access all ontology repositories, we propose this procedure to make the ontology repository, used with Talend Studio, accessible and searchable while maintaining its unique functionalities and strengths:
  1. Run a Semantic-aware analysis in Talend Studio to initialize the data stored in the ontology repository.

  2. Configure Kibana in Talend Administration Center.

  3. Create the index pattern in Kibana.

  4. Define your dashboard to visualize the concepts and attributes stored in the ontology repository.

Before you begin

  • Retrieve the semantic_repository_content-20170128.json file from the Downloads tab in the left panel of this page.

  • Talend Administration Center and Talend Log Server must be installed.

Initializing the data stored in the ontology repository

Before you begin

  • Launch Apache Tomcat which embeds your Talend Administration Center.

  • Launch Talend Log Server.

Procedure

  1. Start Talend Studio.
  2. From the top menu bar, select Window > Preferences to open the Preferences window.
  3. From the Preferences window, select Talend > Profiling > Semantic-aware analysis to open the Semantic-aware analysis view.

    The connection information to the semantic repository on Talend Log Server is set by default depending on your installation.

  4. If you modified the port or the cluster name, edit the parameters in the corresponding fields.
  5. Click Check Connection to test if you can connect to the repository stored on Talend Log Server, and click OK.
  6. Create and run a Semantic-aware analysis from the Profiling perspective of Talend Studio to index and enrich the ontology repository on Talend Log Server.

    For more information about the Semantic-aware Analysis, see the documentation on Talend Help Center (https://help.talend.com).

Configuring Kibana in Talend Administration Center

From Talend Administration Center, you can set up the parameters of the monitoring modules that allow you to display the content of the ontology repository.

Procedure

  1. Open your Talend Administration Center web application.
  2. In the Menu tree view, click Configuration.
  3. Click the Monitoring node to display the parameters.
  4. In the Kibana URL field, type in the URL address of the Kibana application:

    http://localhost:8080/kibana

    http://localhost:8080/kibana is only given as example. Depending on your configuration, you may have to replace <localhost> with the IP address of the Web server application and <8080> with its actual port.

Creating the index pattern in Kibana

To be able to visualize the concepts and attributes stored in the ontology repository in Kibana, you must create an index pattern.

Procedure

  1. In the Talend Administration Center Menu tree view, click Logging to display the Kibana dashboard, or open a Web browser and go to http://localhost:8080/kibana/#/dashboard/.
  2. From the top menu bar, click Settings > Indices to open the Configure an index pattern page.
  3. Clear the Index contains time-based events check box.
  4. In the Index name or pattern field, enter an index name or pattern that matches the index pattern of the concepts and attributes loaded to Elasticsearch.

    If you want to import the semantic_repository_content-20170128.json file, enter tdq*.

  5. Click Create to add the index pattern.

Opening the Kibana dashboard

You can open the Kibana dashboard to visualize the content of the ontology repository.

Procedure

  1. In the Talend Administration Center Menu tree view, click Logging to display the Kibana dashboard, or open a Web browser and go to http://localhost:8080/kibana/#/dashboard/.
  2. From the top menu bar, click Settings > Objects to open the Edit Saved Objects page.
  3. Click Import and browse to the file you retrieved from the Downloads tab in the left panel of this page (semantic_repository_content-20170128.json).

    The semantic_repository_content-20170128.json file is provided as an example of dashboard that displays visualizations related to the ontology repository.

    You can define your own dashboards and visualizations.

  4. From the top menu bar, click Dashboard.
  5. In the top right corner, click the Load Saved Dashboard icon and select Semantic Repository Dashboard from the list.

    The dashboard displays the most frequent concepts and the attributes stored in the ontology repository.

  6. Use the search field on top of the dashboard to search for a concept or an attribute and filter the results listed in the dashboard.