MDM query language and REST data access - 7.2

EnrichVersion
7.2
EnrichProdName
Talend Data Fabric
Talend MDM Platform
task
Data Governance > Consolidating data
Data Governance > Retrieving data
EnrichPlatform
Talend MDM Server
Talend MDM Web UI

MDM query language and REST data access

Talend MDM proposes a REST API that enables users to perform CRUD (Create/Request/Update/Delete) operations on MDM records in a more convenient manner.

This article focuses mainly on the read and write operations of the MDM REST API, and then gives examples of basic and more complex queries that you can include in REST API calls.

This article applies to all Talend Platform products with MDM 6.0 and later.

MDM REST API resources

Talend MDM REST API is a modern, developer-friendly API that simplifies the interfaces to MDM, allowing developers to interact with MDM data more easily and directly.

Talend MDM Web UI provides access to the MDM REST resources documentation, which allows you to refer to the query language syntax and related parameters and test the execution of operations.

You can also integrate data API and transactions API. For more information, see Transactions API integration.

You can enable the support of cross-origin resource sharing (CORS) in MDM when developing a web application consuming MDM REST resources. For more information, see Cross-origin resource sharing support in MDM.

User data management

Create a record

Creates a new record in the specified data container. The record will be provided in the request content as XML or JSON. The ID of the record will be returned inside the location response header field.

Note: Your submitted ID will be used to create the record even if the ID is set to be auto-generated in the data model. An update operation will be performed if a record with the same ID already exists in MDM.
Request
POST /services/rest/data/{containerName}

The body is an XML or JSON representation of the record to be created.

Parameters
  • containerName: This is a String value which specifies the name of the data container in which you want to create a new data record.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • updateReport: This is a Boolean value which controls whether to generate a journal entry (update report). By default, it is true.
  • beforeSaving: This is a Boolean value which controls whether to invoke a Before Saving process. By default, it is false. If the beforeSaving parameter is set to true, the updateReport parameter must also be set to true, since a journal entry (update report) needs to be generated to invoke the Before Saving process.
Headers
  • Content-Type: application/xml or text/xml or application/json
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name, invalid storage type, or wrong XML document.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required WRITE permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors, for example, the Before Saving process returns error.
Limitation This REST API does not support creating a new record for the Role entity in the PROVISIONING system data container.

Sample request in XML format

<Product>
  <Picture>/imageserver/upload/TalendShop/tshirt.jpg</Picture>
  <Id>231035936</Id>
  <Name>Talend Fitted T-Shirt</Name>
  <Description>Fitted T. ultra-fine combed ring spun cotton</Description>
  <Features>
    <Sizes/>
    <Colors/>
  </Features>
  <Availability>false</Availability>
  <Price>15.99</Price>
  <Family>[1]</Family>
  <OnlineStore>Talend Shop@@http://www.cafepress.com/Talend.231035936</OnlineStore>
  <Stores>
    <Store></Store>
  </Stores>
</Product>

Sample request in JSON format

{
    "Product": {
        "Picture": "/imageserver/upload/TalendShop/tshirt.jpg",
        "Id": "231035936",
        "Name": "Talend Fitted T-Shirt",
        "Description": "Fitted T. ultra-fine combed ring spun cotton",
        "Features": {
            "Sizes": {},
            "Colors": {}
        },
        "Availability": "false",
        "Price": "15.99",
        "Family": "[1]",
        "OnlineStore": "Talend Shop@@http://www.cafepress.com/Talend.231035936",
        "Stores": {}
    }
}

Sample response headers

{
  "pragma": "No-cache",
  "date": "Tue, 18 Dec 2018 02:35:36 GMT",
  "location": "231035936"
}

Update a record

Updates a data record completely in the specified data container. The record will be provided in the request content as XML or JSON.

If you want to preserve some existing values, use the partial update operation instead. For more information, see Update a record partially.

Request
PUT /services/rest/data/{containerName}

The body is an XML or JSON representation of the record to be updated. The element Id in the request body determines which record will be updated.

Parameters
  • containerName: This is a String value which specifies the name of the data container in which you want to update a data record.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • updateReport: This is a Boolean value which controls whether to generate a journal entry (update report). By default, it is true.
  • beforeSaving: This is a Boolean value which controls whether to invoke a Before Saving process. By default, it is false. If the beforeSaving parameter is set to true, the updateReport parameter must also be set to true, since a journal entry (update report) needs to be generated to invoke the Before Saving process.
Headers
  • Content-Type: application/xml or text/xml or application/json
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully, and the whole data record is replaced by the provided input.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, nonexistent storage name, wrong XML document, invalid storage type, or nonexistent record Id.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required WRITE permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors, for example, the Before Saving process returns error.
Limitation This REST API does not support updating a record for the Role entity in the PROVISIONING system data container.

Sample request in XML format

<Product>
  <Picture>/imageserver/upload/TalendShop/tshirt.jpg</Picture>
  <Id>231035936</Id>
  <Name>Talend Fitted T-Shirt</Name>
  <Description>Fitted T. ultra-fine combed ring spun cotton</Description>
  <Features>
    <Sizes/>
    <Colors/>
  </Features>
  <Availability>false</Availability>
  <Price>18</Price>
  <Family>[1]</Family>
  <OnlineStore>Talend Shop@@http://www.cafepress.com/Talend.231035936</OnlineStore>
  <Stores>
    <Store></Store>
  </Stores>
</Product>

Sample request in JSON format

{
    "Product": {
        "Picture": "/imageserver/upload/TalendShop/tshirt.jpg",
        "Id": "231035936",
        "Name": "Talend Fitted T-Shirt",
        "Description": "Fitted T. ultra-fine combed ring spun cotton",
        "Features": {
            "Sizes": {},
            "Colors": {}
        },
        "Availability": "false",
        "Price": "18",
        "Family": "[1]",
        "OnlineStore": "Talend Shop@@http://www.cafepress.com/Talend.231035936",
        "Stores": {}
    }
}

Delete a container

Deletes a container, either a Master one or a Staging one.
Request
DELETE /services/rest/data/{containerName}
Parameters
  • containerName: This is a String value which specifies the name of the data container you want to delete.
  • drop: This is a Boolean value which determines if all data such as full text index will be deleted.
Headers
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 204 NO CONTENT: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors, for example, nonexistent storage name.

Update a record partially

Updates a data record partially in the specified data container. The record will be provided in the request content as XML.
Request
PATCH /services/rest/data/{containerName}

The body is an XML representation of part of the record to be updated. The element Id in the request body determines which record is to be updated.

Parameters
  • containerName: This is a String value which specifies the name of the data container in which you want to update a data record partially.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • updateReport: This is a Boolean value which controls whether to generate a journal entry (update report). By default, it is true.
  • pivot: This is a String value which represents the XPath to a multi-occurrence element where data needs to be added, replaced, or deleted in the item of interest.
  • overwrite: This is a Boolean value which controls whether to overwrite the existing list of values for a multi-occurrence element with the new one or just append new values to the existing list without deduplication. By default, it is true, which means the existing list of values will be replaced by the new one. Its value becomes invalid when the delete parameter value is set to true.
  • delete: This is a Boolean value which controls whether to remove the values that match the input values for a multi-occurrence element. By default, it is false.
  • key: This is a String value which represents the XPath relative to the pivot and helps identifying a complex type sub-element of the multi-occurence element defined by the pivot parameter. If this parameter is not specified, all sub-elements of the multi-occurence element with an XPath matching that of the sub-element of the source XML will be replaced. If more than one sub-element matches the key, the first one will be updated. If no sub-element matches the key, it will be added at the end.
  • position: This is an integer value which represents the position in which new values will be appended. If not specified, new values will be appended at the end.
Headers
  • Content-Type: application/xml or text/xml
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully, and the data record identified by the element Id is updated partially according to the provided input.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, nonexistent storage name, wrong XML document, invalid storage type, or nonexistent record Id.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required WRITE permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support updating a record partially for the Role entity in the PROVISIONING system data container.

Sample requests

Suppose there is a record like below in an entity Family:

<Family>
    <Id>1</Id>
    <Name>Lee</Name>
    <Kids>
        <Kid>
            <Name>David</Name>
            <Age>10</Age>
            <Habits>
                <Habit>Basketball</Habit>
                <Habit>Tennis</Habit>
            </Habits>
        </Kid>
        <Kid>
            <Name>James</Name>
            <Age>8</Age>
        </Kid>
        <Kid>
            <Name>Kate</Name>
            <Age>6</Age>
        </Kid>
    </Kids>
</Family>

With the delete value set to true, the pivot value set to Family/Kids/Kid, and the key value set to /Name, the following sample request will remove the multi-occurence element Kid identified by James:

<Person>
    <Id>1</Id>
    <Kids>
        <Kid>
            <Name>James</Name>
        </Kid>
    </Kids>
</Person>

With the overwrite value set to false, the delete value set to false, the pivot value set to Family/Kids/Kid[1]/Habits/Habit (Kid[1] indicates the first kid) , the key value not specified or set to ., and the position value set to 2, the following sample request will append two values to the multi-occurence element Habit after <Habit>Basketball</Habit> for the first kid:

<Person>
    <Id>1</Id>
    <Kids>
        <Kid>
            <Habits>
                <Habit>Football</Habit>
                <Habit>Table tennis</Habit>
            </Habits>
        </Kid>
    </Kids>
</Person>

Create or update records in batch mode

Creates or updates multiple data records pertaining to one or several entities in a data container in batch mode. The records will be provided in the request content as XML or JSON.

Note: Your submitted ID will be used to create the record even if the ID is set to be auto-generated in the data model. An update operation will be performed if a record with the same ID already exists in MDM.

You can also perform partial update of many records that pertain to one entity in one single bulk operation through the REST API. For more information, see How to perform mass partial update on MDM data records through the REST API.

Request
POST /services/rest/data/{containerName}/batch

The body is an XML or JSON representation of one or more records to be created or updated. The records in XML representation must be enclosed by a root tag with an arbitrary name, such as records. Each record contains:

  • one or more primary key fields with values, which are mandatory because they will be used to specify the record you want to create or update. If the record does not exist in the data container, it will be created. If the record already exists in the data container, it will be updated accordingly.
  • other fields defined for the specified entity, in which the mandatory fields must have values.
Parameters
  • containerName: This is a String value which specifies the name of the data container in which you want to create or update multiple data records.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • updateReport: This is a Boolean value which controls whether to generate a journal entry (update report). By default, it is false.
  • beforeSaving: This is a Boolean value which controls whether to invoke a Before Saving process. By default, it is false. If the beforeSaving parameter is set to true, the updateReport parameter must also be set to true, since a journal entry (update report) needs to be generated to invoke the Before Saving process.
Headers
  • Content-Type: application/xml or text/xml or application/json
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name, invalid storage type, or wrong XML document.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required WRITE permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors, for example, the Before Saving process returns error.
Limitation This REST API does not support creating or updating multiple records in batch mode for the Role entity in the PROVISIONING system data container.

Sample request in XML format

<records>
    <Store>
        <Id>100001</Id>
        <Address>Beijing APM</Address>
    </Store>
    <Store>
        <Id>100002</Id>
        <Address>Beijing Department Store</Address>
    </Store>
    <Product>
        <Picture>/imageserver/upload/TalendShop/dog.jpg</Picture>
        <Id>231035933</Id>
        <Name>Talend Dog T-Shirt</Name>
        <Description>Doggie t-shirt from American Apparel</Description>
        <Features>
            <Sizes>
                <Size>Small</Size>
                <Size>Medium</Size>
                <Size>Large</Size>
                <Size>X-Large</Size>
            </Sizes>
            <Colors>
                <Color>White</Color>
            </Colors>
        </Features>
        <Availability>false</Availability>
        <Price>16.99</Price>
        <Family>[3]</Family>
        <OnlineStore>Talend Shop@@http://www.cafepress.com/Talend.231035933</OnlineStore>
        <Stores>
            <Store>100001</Store>
            <Store>100002</Store>
        </Stores>
    </Product>
</records>

Sample request in JSON format

[
    {
        "Store": {
            "Id": "100001",
            "Address": "Beijing APM"
        }
    },
    {
        "Store": {
            "Id": "100002",
            "Address": "Beijing Department Store"
        }
    },
    {
        "Product": {
            "Picture": "/imageserver/upload/TalendShop/dog.jpg",
            "Id": "231035933",
            "Name": "Talend Dog T-Shirt",
            "Description": "Doggie t-shirt from American Apparel",
            "Features": {
                "Sizes": {
                    "Size": [
                        "Small",
                        "Medium",
                        "Large",
                        "X-Large"
                    ]
                },
                "Colors": {"Color": "White"}
            },
            "Availability": "false",
            "Price": "16.99",
            "Family": "[3]",
            "OnlineStore": "Talend Shop@@http://www.cafepress.com/Talend.231035933",
            "Stores": {
                "Store": [
                    "100001",
                    "100002"
                ]
            }
        }
    }
]

Delete records by query

Deletes one or more records that match the provided query. Query is provided in the request content as JSON.
Request
PUT /services/rest/data/{containerName}/delete

The body is a query language in JSON format. The parameter content type is application/json.

Parameters
  • containerName: This is a String value which specifies the name of the data container from which the data record(s) will be deleted.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • updateReport: This is a Boolean value which controls whether to generate a journal entry (update report). By default, it is false.
  • logical: This is a Boolean value which controls whether to delete records that match the provided query logically. By default, it is false, which implies that the records will be physically removed from the database and cannot be recovered from the Recycle Bin.
    Note: Logical delete is only supported on master data records.
Headers
  • Content-Type: application/json
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully, and all records that match the provided query are deleted.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name, invalid storage type, or wrong XML document.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required WRITE permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support deleting records by query for the Role entity in the PROVISIONING system data container.

Sample request

To delete the Product instances where Product/Price is greater than 18, use the following query:

{
   "select": {	
      "from": ["Product"],
      "where": {
         "gt":[
           {"field": "Product/Price"},
           {"value": "18"}
         ]
      }
   }
}

Get XML documents

Gets many XML documents from the database.
Request
POST /services/rest/data/{containerName}/documents

The body is a unique ids array. The parameter content type is application/json.

Parameters
  • containerName: This is a String value which specifies the name of the data container from which you want to query the record.
  • encoding: It represents the char set of all request parameters and defaults to UTF-8.
Headers
  • Content-Type: application/json
  • Authorization: Basic Authentication scheme
  • Accept: application/json
Response A JSON array that contains the documents of the records.
Status
  • 200 OK: The validation operation is executed successfully, no matter what the validation result is.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name.
  • 403 FORBIDDEN: Missing required permission, for example, no required READ permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support getting XML documents for the Role entity in the PROVISIONING system data container.

Sample request

[
  "Product.Product.231035933",
  "Product.Product.231035934",
  "Product.Product.231035935"
]

Sample response

[
  	"<ii>
	  <c>Product</c>
	  <dmn>Product</dmn>
	  <dmr/>
	  <sp/>
	  <t>1531809768682</t>
	  <taskId>null</taskId>
	  <i>231035933</i>
	  <p>
		<Product>
		  <Id>231035933</Id>
		  <Name>Talend Dog T-Shirt</Name>
		  <Description>Doggie t-shirt from American Apparel</Description>
		  <Features>
			<Sizes>
			  <Size>Small</Size>
			  <Size>Medium</Size>
			  <Size>Large</Size>
			  <Size>X-Large</Size>
			</Sizes>
			<Colors>
			  <Color>White</Color>
			</Colors>
		  </Features>
		  <Availability>true</Availability>
		  <Price>16.99</Price>
		  <Family>[14]</Family>
		  <OnlineStore>Talend Shop@@http://www.cafepress.com/Talend.231035933</OnlineStore>
		  <Stores/>
		</Product>
	  </p>
	</ii>",
	"<ii>
	  <c>Product</c>
	  <dmn>Product</dmn>
	  <dmr/>
	  <sp/>
	  <t>1531809768659</t>
	  <taskId>null</taskId>
	  <i>231035934</i>
	  <p>
		<Product>
		  <Id>231035934</Id>
		  <Name>Talend Jr. Spaghetti Tank</Name>
		  <Description>Spaghetti tank from American Apparel</Description>
		  <Features>
			<Sizes>
			  <Size>Small</Size>
			  <Size>Medium</Size>
			  <Size>Large</Size>
			  <Size>X-Large</Size>
			</Sizes>
			<Colors>
			  <Color>White</Color>
			  <Color>Light Blue</Color>
			  <Color>Light Pink</Color>
			  <Color>Lemon</Color>
			</Colors>
		  </Features>
		  <Availability>true</Availability>
		  <Price>16.99</Price>
		  <Family>[8]</Family>
		  <OnlineStore>Talend Shop@@http://www.cafepress.com/Talend.231035934</OnlineStore>
		  <Stores/>
		</Product>
	  </p>
	</ii>",
	"<ii>
	  <c>Product</c>
	  <dmn>Product</dmn>
	  <dmr/>
	  <sp/>
	  <t>1531809767902</t>
	  <taskId>null</taskId>
	  <i>231035935</i>
	  <p>
		<Product>
		  <Id>231035935</Id>
		  <Name>Talend Golf Shirt</Name>
		  <Description>Golf-style, collared t-shirt</Description>
		  <Features>
			<Sizes>
			  <Size>Medium</Size>
			</Sizes>
			<Colors>
			  <Color>White</Color>
			</Colors>
		  </Features>
		  <Availability>false</Availability>
		  <Price>16.99</Price>
		  <Family>[8]</Family>
		  <OnlineStore>Talend Shop@@http://www.cafepress.com/Talend.231035935</OnlineStore>
		  <Stores/>
		</Product>
	  </p>
	</ii>"
]

Get all document unique IDs

Gets the unique IDs of all documents as array.
Request
GET /services/rest/data/{containerName}/documents/uniqueIds
Parameters
  • containerName: This is a String value which specifies the name of the data container from which you want to get the unique ids of all documents.
  • type: This is a String value which specifies the name of the entity.
Headers
  • Authorization: Basic Authentication scheme
  • Accept: application/json
Response A JSON array that contains all primary keys of the entity in the specified container.
Status
  • 200 OK: The query operation is executed successfully, and the documents unique ids of the specified entity are returned in an array.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name or invalid storage type.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required READ permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support getting the unique IDs of all documents for the Role entity in the PROVISIONING system data container.

Sample response

[
  "Product.Product.231035935",
  "Product.Product.231035941",
  "Product.Product.231035937",
  "Product.Product.231035938",
  "Product.Product.231035939",
  "Product.Product.231035940",
  "Product.Product.231035934",
  "Product.Product.231035936",
  "Product.Product.231035933"
]

Get an XML string of a document

Gets an XML string of a document.
Request
GET /services/rest/data/{containerName}/documents/{uniqueId}
Parameters
  • containerName: This is a String value which specifies the name of the data container from which you want to get an XML string of a document.
  • uniqueId: This is a String value which represents the unique id of the document.
  • encoding: It represents the char set of the result and defaults to UTF-8.
Headers
  • Authorization: Basic Authentication scheme
  • Accept: application/xml
Response The document of the record in the specified container.
Status
  • 200 OK: The query operation is executed successfully, and the record that matches the provided ID will be returned as a string.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name or invalid storage type.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required READ permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support getting an XML string of a document for the Role entity in the PROVISIONING system data container.

Sample response

<ii>
    <c>Product</c>
    <dmn>Product</dmn>
    <dmr/>
    <sp/>
    <t>1531728848296</t>
    <taskId>null</taskId>
    <i>1</i>
    <p>
        <Product>
            <Id>1</Id>
            <Name>New Name</Name>
            <Description>2</Description>
            <Features>
                <Sizes>
                    <Size>Small</Size>
                    <Size>Medium</Size>
                    <Size>Large</Size>
                </Sizes>
                <Colors>
                    <Color>White</Color>
                    <Color>Light Blue</Color>
                    <Color>Light Pink</Color>
                </Colors>
            </Features>
            <Availability>true</Availability>
            <Price>3.00</Price>
            <Stores>
            </Stores>
        </Product>
    </p>
</ii>

Get records by query

Gets data records from the specified data container. Query is provided in the request content as JSON.

You can also retrieve a snapshot of a specific MDM record and navigate through the record history. For more information, see MDM Time Machine.

Request
PUT /services/rest/data/{containerName}/query

The body is a query language in JSON format. The parameter content type is application/json.

Parameters
  • containerName: This is a String value which specifies the name of the data container from which you want to read data records.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • ignoreCase: This is a Boolean value which controls whether to ignore the case of the entity and attribute names in JSON response. By default, it is true, which means all entity and attributes names in JSON response will be in lowercase.
Headers
  • Content-Type: application/json
  • Authorization: Basic Authentication scheme
  • Accept: application/xml or application/json
Response The documents of the records in the specified container.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name or invalid storage type.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support getting records by query for the Role entity in the PROVISIONING system data container.

Sample request

{
    "select": { 
        "from": ["Product"],
        "where": {
            "gt":[
                {"field": "Product/Price"},
                {"value": "16"}
            ]   
        }
    }
}

Sample response in XML format

<results>
   <Product>
      <Picture>/imageserver/upload/TalendShop/dog.jpg</Picture>
      <Id>231035933</Id>
      <Name>partialNameUpdate1</Name>
      <Description>partialDescriptionUpdate1</Description>
      <Features>
         <Sizes>
            <Size>Small</Size>
            <Size>Medium</Size>
            <Size>Large</Size>
            <Size>X-Large</Size>
         </Sizes>
         <Colors>
            <Color>White</Color>
         </Colors>
      </Features>
      <Availability>true</Availability>
      <Price>111.00</Price>
      <Family>[14]</Family>
      <OnlineStore>Talend Shop@@http://www.cafepress.com/Talend.231035933</OnlineStore>
      <Stores/>
   </Product>
   <Product>
      <Picture>/imageserver/upload/TalendShop/tshirt.jpg?width=150&amp;amp;height=90&amp;amp;preserveAspectRatio=true</Picture>
      <Id>231035935</Id>
      <Name>partialNameUpdate3</Name>
      <Description>partialDescriptionUpdate3</Description>
      <Features>
         <Sizes>
            <Size>Medium</Size>
         </Sizes>
         <Colors>
            <Color>White</Color>
         </Colors>
      </Features>
      <Availability>false</Availability>
      <Price>333.00</Price>
      <Family>[1]</Family>
      <OnlineStore>Talend Shop@@http://www.cafepress.com/Talend.231035936</OnlineStore>
      <Stores/>
   </Product>
</results>

Sample response in JSON format

[
  {
    "product": {
      "picture": "/imageserver/upload/TalendShop/dog.jpg",
      "id": "1",
      "name": "New Name",
      "description": "Doggie t-shirt from American Apparel",
      "features": {
        "sizes": {
          "size": [
            "Small",
            "Medium",
            "Large",
            "X-Large"
          ]
        },
        "colors": {
          "color": [
            "White"
          ]
        }
      },
      "availability": "true",
      "price": "16.99",
      "family": "[14]",
      "onlinestore": "Talend Shop@@http://www.cafepress.com/Talend.231035933",
      "stores": {}
    }
  },
  {
    "product": {
      "picture": "/imageserver/upload/TalendShop/dog.jpg",
      "id": "231035933",
      "name": "Talend Dog T-Shirt",
      "description": "Doggie t-shirt from American Apparel",
      "features": {
        "sizes": {
          "size": [
            "Small",
            "Medium",
            "Large",
            "X-Large"
          ]
        },
        "colors": {
          "color": [
            "White"
          ]
        }
      },
      "availability": "true",
      "price": "16.99",
      "family": "[14]",
      "onlinestore": "Talend Shop@@http://www.cafepress.com/Talend.231035933",
      "stores": {}
    }
  },
  {
    "product": {
      "picture": "/imageserver/upload/TalendShop/spaghetti.jpg",
      "id": "231035934",
      "name": "Talend Jr. Spaghetti Tank",
      "description": "Spaghetti tank from American Apparel",
      "features": {
        "sizes": {
          "size": [
            "Small",
            "Medium",
            "Large",
            "X-Large"
          ]
        },
        "colors": {
          "color": [
            "White",
            "Light Blue",
            "Light Pink",
            "Lemon"
          ]
        }
      },
      "availability": "true",
      "price": "16.99",
      "family": "[8]",
      "onlinestore": "Talend Shop@@http://www.cafepress.com/Talend.231035934",
      "stores": {}
    }
  }
]

Validate records in a container

Validates records in the specified container. The records will be provided in the request content as XML.
Request
POST /services/rest/data/{containerName}/validate

The body is an XML representation of one or more records to be validated, which are enclosed by a root tag with an arbitrary name, records for an example.

Parameters
  • containerName: This is a String value that represents the name of the container in which the records will be validated.

  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • beforeSaving: This is a Boolean value which controls whether to invoke a Before Saving process associated to the record(s) during validation. By default, it is true. The validation operation should be an immutable action, thus it allows to disable the Before Saving process which in some cases can modify the data.

  • returnSource: This is a Boolean value which indicates whether to include the input records in the response. By default, it is false.

Headers
  • Content-Type: application/xml or text/xml
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
  • Accept: application/xml or application/json
Response The list of the validation result for each record in JSON format, including the validity status, a message from the validation rule or the Before Saving process or empty, and optionally the input record as an echo.
Status
  • 200 OK: The validation operation is executed successfully, no matter what the validation result is.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name or wrong XML document.
  • 403 FORBIDDEN: Missing required permission, for example, no required WRITE permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid storage type.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support validating records for the following entity:
  • the User entity in the PROVISIONING system data container,
  • the Role entity in the PROVISIONING system data container,
  • the Conf entity in the CONF system data container,
  • the AutoIncrement entity in the CONF system data container,
  • the BrowseItem entity in the SearchTemplate system data container, and
  • the HierarchySearchItem entity in the SearchTemplate system data container.

Sample request

<records> 
    <ProductFamily> 
        <Name>Literature</Name> 
        <Status>Approved</Status> 
    </ProductFamily> 
    <ProductFamily> 
        <Name>Comics</Name> 
        <Status>Unknown</Status> 
    </ProductFamily> 
</records>

Sample response in XML format

<results>
   <result>
      <isValid>false</isValid>
      <message>&lt;msg>[EN:Validation faild]&lt;/msg></message>
      <sourceXml>
         <ProductFamily>
            <Name>Literature</Name>
            <Status>Approved</Status>
         </ProductFamily>
      </sourceXml>
   </result>
   <result>
      <isValid>false</isValid>
      <message></message>
      <sourceXml>
         <ProductFamily>
            <Name>Comics</Name>
            <Status>Unknown</Status>
         </ProductFamily>
      </sourceXml>
   </result>
</results>

Sample response in JSON format

[
  {
    "isValid": false,
    "message": "<msg>[EN:Validation faild]</msg>\n",
    "sourceXml": "<ProductFamily><Name>Literature</Name><Status>Approved</Status></ProductFamily>"
  },
  {
    "isValid": true,
    "message": "",
    "sourceXml": "<ProductFamily><Name>Comics</Name><Status>Unknown</Status></ProductFamily>"
  }
]

List primary keys for container and type

Lists all primary keys for container and type.
Request
GET /services/rest/data/{containerName}/{type}
Parameters
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • containerName: This is a String value which represents the name of the container.
  • type: This is a String value which specifies the name of the entity.
Headers
  • Authorization: Basic Authentication scheme
  • Accept: application/json
Response Returns the json array that contains all primary keys of entity in container.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name or invalid storage type.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required READ permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support listing primary keys for the Role entity in the PROVISIONING system data container.

Sample response

[
  {
    "$explicitprojection$": {
      "id": "231035933"
    }
  },
  {
    "$explicitprojection$": {
      "id": "231035934"
    }
  },
  {
    "$explicitprojection$": {
      "id": "231035935"
    }
  },
  {
    "$explicitprojection$": {
      "id": "231035936"
    }
  },
  {
    "$explicitprojection$": {
      "id": "231035937"
    }
  }
]

Update records partially in one bulk operation

Performs partial update of records in one bulk operation. The records will be provided in the request content as XML or JSON.
Request
PATCH /services/rest/data/{containerName}/{type}/bulk
Parameters
  • containerName: This is a String value which specifies the name of the data container in which you want to update records partially in one bulk operation.
  • type: This is a String value which specifies the name of the entity.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • updateReport: This is a Boolean value which controls whether to generate a journal entry (update report). By default, it is true.
Headers
  • Content-Type: application/xml or text/xml or application/json
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name, invalid storage type, or wrong XML document.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission, for example, no required WRITE permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support updating records partially in one bulk operation for the Role entity in the PROVISIONING system data container.

Sample request in XML

<records>
    <Product>
        <Id>231035933</Id>
        <Name>PartialNameUpdate1</Name>
        <Price>111</Price>
        <Stores>
            <Store>[100001]</Store>
        </Stores>
    </Product>
    <Product>
        <Id>231035934</Id>
        <Name>PartialNameUpdate2</Name>
        <Price>222</Price>
        <Family>[2]</Family>
    </Product>
</records>

Sample request in JSON (1)

[
    {
        "Product": {
            "Id": "231035933",
            "Name": "PartialNameUpdate1",
            "Price": "111",            
            "Stores": {
                "Store": ["100001"]
            }
        }
    },
    {
        "Product": {
            "Id": "231035934",
            "Name": "PartialNameUpdate2",
            "Price": "222",
            "Family": "[2]"
        }
    }
]

Sample request in JSON (2)

{
    "records": {
        "Product": [
            {
                "Id": "231035933",
                "Name": "PartialNameUpdate1",
                "Price": "111",
                "Stores": {
                    "Store": ["100001"]
                }
            },
            {
                "Id": "231035934",
                "Name": "PartialNameUpdate2",
                "Price": "222",
                "Family": "[2]"
            }
        ]
    }
}

Get a record by ID

Gets a data record by ID.
Request
GET /services/rest/data/{containerName}/{type}/{id}
Parameters
  • containerName: This is a String value which represents the name of the container where you want to query the record.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • type: This is a String value which specifies the name of the entity.
  • id: This is a String value which specifies the ID of the record.
  • datetime: This is a String value that specifies the date at which the record you want to retrieve was updated. The date can be provided as number of milliseconds since EPOCH or an XML formatted date.
  • swing: It controls how datetime is interpreted, CLOSEST, AFTER, or BEFORE.
  • ignoreCase: This is a Boolean value which controls whether to ignore the case of the entity and attribute names in JSON response. By default, it is true, which means all entity and attributes names in JSON response will be in lowercase.
Headers
  • Authorization: Basic Authentication scheme
  • Accept: application/json
Response The content of the record in JSON array format.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name.
  • 403 FORBIDDEN: Missing required permission, for example, no required READ permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support getting a record by ID for the Role entity in the PROVISIONING system data container.

Sample response

[
  {
    "product": {
      "picture": "/imageserver/upload/TalendShop/dog.jpg",
      "id": "231035933",
      "name": "Talend Dog T-Shirt",
      "description": "Doggie t-shirt from American Apparel",
      "features": {
        "sizes": {
          "size": [
            "Small",
            "Medium",
            "Large",
            "X-Large"
          ]
        },
        "colors": {
          "color": [
            "White"
          ]
        }
      },
      "availability": "false",
      "price": "16.99",
      "family": "[10]",
      "onlinestore": "Talend Shop@@http://www.cafepress.com/Talend.231035933",
      "stores": {}
    }
  }
]

Delete a record by ID

Deletes a data record by the specified ID.
Request
DELETE /services/rest/data/{containerName}/{type}/{id}
Parameters
  • containerName: This is a String value which specifies the name of the data container from which you want to delete a data record.
  • container: This is a String value which represents the type of the data container. It is either MASTER (default) or STAGING.
  • type: This is a String value which specifies the name of the entity from which you want to delete a data record.
  • id: The ID of the data record to be deleted. For composite IDs, you can use a dot-separated value. For example, to delete an instance Address with the ID 1 in the data container Customer, use http://localhost:8180/talendmdm/services/rest/data/Customer/Address/1, and to delete an instance Address with the ID 1 and 2, use http://localhost:8180/talendmdm/services/rest/data/Customer/Address/1.2.
  • updateReport: This is a Boolean value which controls whether to generate a journal entry (update report). By default, it is true.
  • logical: This is a Boolean value which controls whether to delete a record logically. By default, it is false, which implies that the record will be physically removed from the database and cannot be recovered from the Recycle Bin.
    Note: Logical delete is only supported on master data records.
Headers
  • Content-Type: application/json
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully, and the data record specified by the element Id will be deleted.
  • 400 BAD REQUEST: The request contains invalid parameter, for example, invalid storage name or invalid entity name.
  • 403 FORBIDDEN: Missing required permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid service URL.
  • 500 INTERNAL SERVER ERROR: Other errors.
Limitation This REST API does not support deleting a record by ID for the Role entity in the PROVISIONING system data container.

Administration

Get names of all containers

Gets the names of all containers, including system containers and deployed data containers.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
GET /services/rest/system/containers
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON array that contains the names of all containers.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

[
  "amaltoOBJECTSRole",
  "amaltoOBJECTSView",
  "amaltoOBJECTSDataModel",
  "UpdateReport",
  "Product"  
]

Create a data container

Creates a new data container with the specified data container name.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
POST /services/rest/system/containers
Parameters
  • containerName: This is a String value which specifies the name of the data container to be created.
Headers
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 204 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 500 INTERNAL SERVER ERROR: Other errors.

Check if a container has the staging area

Checks if a specified system or data container is configured with the staging storage.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
GET /services/rest/system/containers/{containerName}/hasStaging
Parameters
  • containerName: This is a String value which represents the name of the container to be checked.
Headers
  • Authorization: Basic Authentication scheme
Response Returns true if the specified container is configured with the staging storage, false otherwise.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 500 INTERNAL SERVER ERROR: Other errors.

Check if a container exists

Checks if a specified system or data container exists.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
GET /services/rest/system/containers/{containerName}/isExisting
Parameters
  • containerName: This is a String value which represents the name of the container.
Headers
  • Authorization: Basic Authentication scheme
Response Returns true if the specified container exists, false otherwise.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 500 INTERNAL SERVER ERROR: Other errors.

Create a data model

Creates a new data model with a given name and its content provided in request body as an XML Schema.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
POST /services/rest/system/models
Parameters
  • name: This is a String value which specifies the name of the data model to be created.
Headers
  • Content-Type: application/xml or text/xml
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample request

The following XML schema sample will create a data model with two entities Employee and Team.

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <xsd:import namespace="http://www.w3.org/2001/XMLSchema"/>
    <xsd:element name="Employee">
        <xsd:annotation>
            <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
            <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
        </xsd:annotation>
        <xsd:complexType>
            <xsd:all>
                <xsd:element maxOccurs="1" minOccurs="1" name="Id" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Name" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Team" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_ForeignKey">Team/Id</xsd:appinfo>
                        <xsd:appinfo source="X_ForeignKey_NotSep">true</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
            </xsd:all>
        </xsd:complexType>
        <xsd:unique name="Employee">
            <xsd:selector xpath="."/>
            <xsd:field xpath="Id"/>
        </xsd:unique>
    </xsd:element>
    <xsd:element name="Team">
        <xsd:annotation>
            <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
        </xsd:annotation>
        <xsd:complexType>
            <xsd:all>
                <xsd:element maxOccurs="1" minOccurs="1" name="Id" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Name" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
            </xsd:all>
        </xsd:complexType>
        <xsd:unique name="Team">
            <xsd:selector xpath="."/>
            <xsd:field xpath="Id"/>
        </xsd:unique>
    </xsd:element>
</xsd:schema>

Get details of an entity by its name

Gets details of an entity by its name, including the ID (name) of the data model in which the entity is created.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
GET /services/rest/system/models/entities/{entityName}
Parameters
  • entityName: This is a String value which specifies the name of the entity.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response The details of the entity in JSON format.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid entity name.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{
  "dataModelId": "Product",
  "entity": "Product"
}

Get the XML schema of a data model

Gets the XML schema of a data model by its name.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
GET /services/rest/system/models/{model}
Parameters
  • model: This is a String value which represents the name of the data model whose XML schema you want to get.
Headers
  • Accept: application/xml
  • Authorization: Basic Authentication scheme
Response The XML schema of the specified data model.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid data model name.
  • 500 INTERNAL SERVER ERROR: Other errors, for example, incorrect schema.

Sample response

The following sample shows the XML schema of a data model Company with two entities Employee and Team.

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <xsd:import namespace="http://www.w3.org/2001/XMLSchema"/>
    <xsd:element name="Employee">
        <xsd:annotation>
            <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
            <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
        </xsd:annotation>
        <xsd:complexType>
            <xsd:all>
                <xsd:element maxOccurs="1" minOccurs="1" name="Id" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Name" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Team" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_ForeignKey">Team/Id</xsd:appinfo>
                        <xsd:appinfo source="X_ForeignKey_NotSep">true</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
            </xsd:all>
        </xsd:complexType>
        <xsd:unique name="Employee">
            <xsd:selector xpath="."/>
            <xsd:field xpath="Id"/>
        </xsd:unique>
    </xsd:element>
    <xsd:element name="Team">
        <xsd:annotation>
            <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
        </xsd:annotation>
        <xsd:complexType>
            <xsd:all>
                <xsd:element maxOccurs="1" minOccurs="1" name="Id" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Name" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
            </xsd:all>
        </xsd:complexType>
        <xsd:unique name="Team">
            <xsd:selector xpath="."/>
            <xsd:field xpath="Id"/>
        </xsd:unique>
    </xsd:element>
</xsd:schema>

List the potential change impacts to a data model

Lists the potential change impacts to an existing data model by providing a different version of the XML schema of this data model into the request. By performing this call, no change will be applied to the existing data model though.

For more information about the data model change impact level, see Data model changes and their impact levels.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
POST /services/rest/system/models/{model}
Parameters
  • model: This is a String value which represents the name of the data model whose potential change impacts you want to list.
  • lang: This is a String value of the language code which specifies the language in which the change impacts description will be returned, for example, en for English. If this parameter is not specified, the change impacts description in English will be returned.
Headers
  • Content-Type: application/xml or text/xml
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
Response The potential data model change impacts description in XML format.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid data model name.
  • 500 INTERNAL SERVER ERROR: Other errors, for example, incorrect schema.

Sample response

The following sample shows the potential change impacts to the data model Company in Get the XML schema of a data model by providing a different version of its XML schema described in Update a data model with XSD.

<result>
    <high>
        <change>
            <message>Element 'Gender' was added.</message>
        </change>
    </high>
    <medium/>
    <low/>
    <entitiesToDrop>
        <entity>Employee</entity>
    </entitiesToDrop>
</result>

Update a data model with XSD

Updates a data model with the XSD (XML Schema Definition) provided in the request content as XML.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
PUT /services/rest/system/models/{model}
Parameters
  • model: This is a String value which represents the name of the data model whose XML schema you want to update.
  • force: This is a Boolean value which controls whether to update the data model with the new XSD when the change has an impact level of Medium or High. By default, it is false, which means the data model will not be updated when the change has an impact level of Medium or High.
Headers
  • Content-Type: application/xml or text/xml
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid data model name.
  • 500 INTERNAL SERVER ERROR: Other errors, for example, incorrect schema.

Sample request

The following sample updates the data model Company in Get the XML schema of a data model by adding a new mandatory element Gender in the entity Employee.

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <xsd:import namespace="http://www.w3.org/2001/XMLSchema"/>
    <xsd:element name="Employee">
        <xsd:annotation>
            <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
            <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
        </xsd:annotation>
        <xsd:complexType>
            <xsd:all>
                <xsd:element maxOccurs="1" minOccurs="1" name="Id" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Name" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Gender" type="xsd:string"> 
                    <xsd:annotation> 
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>  
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo> 
                    </xsd:annotation> 
                </xsd:element> 
                <xsd:element maxOccurs="1" minOccurs="1" name="Team" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_ForeignKey">Team/Id</xsd:appinfo>
                        <xsd:appinfo source="X_ForeignKey_NotSep">true</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                        <xsd:appinfo source="X_Write">Company_User</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
            </xsd:all>
        </xsd:complexType>
        <xsd:unique name="Employee">
            <xsd:selector xpath="."/>
            <xsd:field xpath="Id"/>
        </xsd:unique>
    </xsd:element>
    <xsd:element name="Team">
        <xsd:annotation>
            <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
        </xsd:annotation>
        <xsd:complexType>
            <xsd:all>
                <xsd:element maxOccurs="1" minOccurs="1" name="Id" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
                <xsd:element maxOccurs="1" minOccurs="1" name="Name" type="xsd:string">
                    <xsd:annotation>
                        <xsd:appinfo source="X_Write">Company_Manager</xsd:appinfo>
                    </xsd:annotation>
                </xsd:element>
            </xsd:all>
        </xsd:complexType>
        <xsd:unique name="Team">
            <xsd:selector xpath="."/>
            <xsd:field xpath="Id"/>
        </xsd:unique>
    </xsd:element>
</xsd:schema>

Clean caching information of basic authentication

Cleans the caching information of basic authentication.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
DELETE /services/rest/system/security/authentication/cache
Headers
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 500 INTERNAL SERVER ERROR: Other errors.

Get a list of all views

Gets a list of all Web Filter views that correspond to the deployed data models, including the name, description, and data model ID (name) for each view, and also the roles authorized to access each view.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
GET /services/rest/system/views
Parameters
  • lang (optional): This is a String value of the language code which specifies the language in which the description of the view will be returned, for example, en for English. If this parameter is not specified or the description in the specified language does not exist, all defined description will be returned.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON array that contains a list of all Web Filter views that correspond to the deployed data models.
Status
  • 200 OK: The operation is executed successfully.
  • 403 FORBIDDEN: Missing required permission.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

[
  {
    "name": "Browse_items_Product",
    "description": "[EN:Product][FR:Produit][ZH:产品]",
    "dataModelId": "Product",
    "roles": [
      "Demo_Manager",
      "Demo_User"
    ]
  },
  {
    "name": "Browse_items_Product#Stores",
    "description": "[FR:Produit avec Magasins][EN:Product with Stores]",
    "dataModelId": "Product",
    "roles": [
      "Demo_Manager",
      "Demo_User"
    ]
  },
  {
    "name": "Browse_items_ProductFamily",
    "description": "[FR:Famille Produit][EN:Product Family]",
    "dataModelId": "Product",
    "roles": [
      "Demo_Manager",
      "Demo_User"
    ]
  },
  {
    "name": "Browse_items_Product#Unavailable",
    "description": "[EN:Unavailable Products][FR:Produits non disponible]",
    "dataModelId": "Product",
    "roles": [
      "Demo_Manager",
      "Demo_User"
    ]
  },
  {
    "name": "Browse_items_Store",
    "description": "[EN:Store][FR:Magasin]",
    "dataModelId": "Product",
    "roles": [
      "Demo_Manager",
      "Demo_User"
    ]
  },
  {
    "name": "Browse_items_Product#AndFamily",
    "description": "[EN:Product with Family][FR:Produit avec famille]",
    "dataModelId": "Product",
    "roles": [
      "Demo_Manager",
      "Demo_User"
    ]
  }
]

Get details of a view by its name

Gets details of a view by the specified name, including the view name, description, data model ID (name), viewable business elements, searchable business elements, etc.

Note: Only the Super Admin type users with the administration role have access to this API. For more information about MDM user types, see MDM user types and system roles in Talend Administration Center User Guide.
Request
GET /services/rest/system/views/{viewName}
Parameters
  • viewName: This is a String value which specifies the name of the view.
  • lang (optional): This is a String value of the language code which specifies the language in which the description of the view will be returned, for example, en for English. If this parameter is not specified or the description in the specified language does not exist, all defined description will be returned.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response The details of the view in JSON format.
Status
  • 200 OK: The operation is executed successfully.
  • 400 BAD REQUEST: The request contains invalid parameter.
  • 403 FORBIDDEN: Missing required permission.
  • 404 NOT FOUND: The resource does not exist, for example, invalid view ID.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{
  "name": "Browse_items_Product#Stores",
  "description": "[FR:Produit avec Magasins][EN:Product with Stores]",
  "dataModelId": "Product",
  "sortField": "",
  "isAsc": false,
  "isTransformerActive": false,
  "customForm": "",
  "viewableBusinessElements": [
    "Product/Id",
    "Product/Name",
    "Product/Description",
    "Product/Availability",
    "Product/Price",
    "Product/Family",
    "Store/Address"
  ],
  "searchableBusinessElements": [
    "Product",
    "Product/Id",
    "Product/Name",
    "Product/Description",
    "Product/Price",
    "Product/Family",
    "Store/Address"
  ],
  "whereConditions": [
    {
      "leftPath": "Product/Stores/Store",
      "operator": "JOINS",
      "rightValueOrPath": "Store/Id",
      "spellCheck": "false",
      "stringPredicate": "NONE"
    }
  ]
}

Statistics

Get statistics about data

Gets statistics about master data records in a container, including the number of the records and the percentage of the records per each entity.
Request
GET /services/rest/system/stats/data/{container}
Parameters
  • container: This is a String value which represents the name of the container.
  • lang (optional): This is a String value of the language code which specifies the language in which the label of each entity will be returned, for example, en for English. If this parameter is not specified or the label in the specified language does not exist, the name of the entity will be returned.
  • top (optional): This is an integer value which limits the statistics data to the top N entities.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON response that contains statistics about master data records in the specified container.
Status
  • 200 OK: The operation is executed successfully.
  • 204 QUERY FAILS: Missing configuration information.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 404 NOT FOUND: The resource does not exist.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{
    "data": [
        {
            "Product": [
                {"count": 9},
                {"percentage": "60"}
            ]
        },
        {
            "Product Family": [
                {"count": 4},
                {"percentage": "26.67"}
            ]
        },
        {
            "Store": [
                {"count": 2},
                {"percentage": "13.33"}
            ]
        }
    ]
}

Get statistics about events

Gets the number of times each trigger has been called with and without success, grouped by status.
Request
GET /services/rest/system/stats/events
Parameters
  • timeframe (optional): This is an integer value which limits the statistic data to the calls executed in the last N seconds.
  • top (optional): This is an integer value which limits the statistics data to the top N triggers.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON response that contains statistics about events, i.e., the number of times each trigger has been called with and without success.
Status
  • 200 OK: The operation is executed successfully.
  • 204 QUERY FAILS: Missing configuration information.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 404 NO CONTENT: The request contains invalid parameter.

Sample response

{
    "events": [
        {
            "failed": [
                {"GoogleMap": 0},
                {"CompleteStoreURLOnCreate": 0},
                {"SynchronizeOnUpdate": 0},
                {"CheckAvailabilityOnCreate": 0}
            ]
        },
        {
            "completed": [
                {"SynchronizeOnUpdate": 5},
                {"GoogleMap": 2},
                {"CompleteStoreURLOnCreate": 2},
                {"CheckAvailabilityOnCreate": 2}
            ]
        }
    ]
}

Get statistics about journal

Gets the number of creation and update journal events during a period of time for each entity in a container, grouped first by the entity and then by the operation.
Request
GET /services/rest/system/stats/journal/{container}
Parameters
  • container: This is a String value which represents the name of the container.
  • lang (optional): This is a String value of the language code which specifies the language in which the label of each entity will be returned, for example, en for English. If this parameter is not specified or the label in the specified language does not exist, the name of the entity will be returned.
  • timeframe (optional): This is an integer value which limits the statistic data to the calls executed in the last N seconds.
  • top (optional): This is an integer value which limits the statistics data to the top N entities with most frequent updates. If the value is greater than 0, the statistics for the entities without any journal event will not be returned.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response

A JSON response that contains statistics about creation and update journal events for each entity in the specified container.

Note that the object from and to represented as epoch time in the JSON response give a period of time and the object create and update give the number of creation and update journal events that happen during the time period.

Status
  • 200 OK: The operation is executed successfully.
  • 204 QUERY FAILS: Missing configuration information.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 404 NO CONTENT: The request contains invalid parameter.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{
    "journal": [
        {
            "Product": [
                {
                    "creations": [
                        {
                            "create": 1,
                            "from": 1548143636461,
                            "to": 1548143636461
                        }
                    ]
                },
                {
                    "updates": [
                        {
                            "update": 2,
                            "from": 1548143639124,
                            "to": 1548143685771
                        },
                        {
                            "update": 1,
                            "from": 1548143825716,
                            "to": 1548143872363
                        }
                    ]
                }
            ]
        },
        {
            "ProductFamily": [
                {
                    "creations": [
                        {
                            "create": 1,
                            "from": 1548142670616,
                            "to": 1548142670616
                        }
                    ]
                },
                {
                    "updates": []
                }
            ]
        },
        {
            "Store": [
                {
                    "creations": [
                        {
                            "create": 1,
                            "from": 1548142762672,
                            "to": 1548142772475
                        }
                    ]
                },
                {
                    "updates": []
                }
            ]
        }
    ]
}

Get statistics about matching

Gets statistics about integrated matching for each entity with a match rule attached in a data container.
Request
GET /services/rest/system/stats/matching/{container}
Parameters
  • container: This is a String value which represents the name of the container.
  • lang (optional): This is a String value of the language code which specifies the language in which the label of each entity will be returned, for example, en for English. If this parameter is not specified or the label in the specified language does not exist, the name of the entity will be returned.
  • top (optional): This is an integer value which limits the statistics data to the top N entities.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response

A JSON response that contains statistics about integrated matching for each entity with a match rule attached in the specified data container.

Status
  • 200 OK: The operation is executed successfully.
  • 204 NO CONTENT: No match configuration information for the container.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 404 NOT FOUND: The resource does not exist.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{
    "matching": [
        {"Store": "100"},
        {"Product Family": "0"},
        {"Product": "100"}
    ]
}

Data matching

Return a matching explanation about input records

Returns a detailed explanation about how input data records for an entity with a match rule attached can be matched and grouped. The data records are provided in the request content as XML fragments.
Request
POST /services/rest/tasks/matching/explain
Parameters
  • model: This is a String value which represents the name of the MDM data model.
  • type: This is a String value which specifies the name of the entity.
Headers
  • Content-Type: application/xml
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON response that gives a detailed explanation about how the input data records for the specified entity with a match rule attached can be matched and grouped.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample request

<Store>
	<Id>200001</Id>
	<Address>Beijing Book Building (17 Xichang'an Jie)</Address>
	<Lat>116.383681</Lat>
	<Long>39.913855</Long>
</Store>
<Store>
	<Id>200002</Id>
	<Address>Beijing book building</Address>
	<Lat>116.383680</Lat>
	<Long>39.913856</Long>
</Store>

Sample response

{
  "groups": [
    {
      "group": [
        {
          "result": [
            {
              "id": "4244168a-acd7-42a0-91b7-873422c4982b"
            },
            {
              "confidence": 0.9157568693161011
            },
            {
              "minimum_confidences": [
                {
                  "minimum_confidence": 0.85
                }
              ]
            },
            {
              "match_confidences": [
                {
                  "match_confidence": 0.9
                }
              ]
            },
            {
              "related_ids": [
                "200002",
                "200001"
              ]
            },
            {
              "values": [
                {
                  "value": [
                    {
                      "field": "Store/Id"
                    },
                    {
                      "value": "200001"
                    }
                  ]
                },
                {
                  "value": [
                    {
                      "field": "Store/Address"
                    },
                    {
                      "value": "Beijing Book Building (17 Xichang'an Jie)"
                    }
                  ]
                },
                {
                  "value": [
                    {
                      "field": "Store/Lat"
                    },
                    {
                      "value": "116.383681"
                    }
                  ]
                },
                {
                  "value": [
                    {
                      "field": "Store/Long"
                    },
                    {
                      "value": "39.913856"
                    }
                  ]
                },
                {
                  "value": [
                    {
                      "field": "Store/Map"
                    },
                    {
                      "value": ""
                    }
                  ]
                }
              ]
            }
          ]
        },
        {
          "details": [
            {
              "detail": [
                {
                  "id": "200002"
                },
                {
                  "match": [
                    {
                      "is_match": true
                    },
                    {
                      "score": 0.9157568693161011
                    },
                    {
                      "field_scores": [
                        {
                          "field_score": [
                            {
                              "pair_id": "200001"
                            },
                            {
                              "field": "Store/Id"
                            },
                            {
                              "fieldValue": "200002"
                            },
                            {
                              "value": 1
                            },
                            {
                              "algorithm": "Dummy"
                            },
                            {
                              "threshold": 0
                            }
                          ]
                        },
                        {
                          "field_score": [
                            {
                              "pair_id": "200001"
                            },
                            {
                              "field": "Store/Address"
                            },
                            {
                              "fieldValue": "Beijing book building"
                            },
                            {
                              "value": 0.9157568693161011
                            },
                            {
                              "algorithm": "Jaro-Winkler"
                            },
                            {
                              "threshold": 0.9
                            }
                          ]
                        },
                        {
                          "field_score": [
                            {
                              "pair_id": "200001"
                            },
                            {
                              "field": "Store/Lat"
                            },
                            {
                              "fieldValue": "116.38368"
                            },
                            {
                              "value": 1
                            },
                            {
                              "algorithm": "Dummy"
                            },
                            {
                              "threshold": 0
                            }
                          ]
                        },
                        {
                          "field_score": [
                            {
                              "pair_id": "200001"
                            },
                            {
                              "field": "Store/Long"
                            },
                            {
                              "fieldValue": "39.913856"
                            },
                            {
                              "value": 1
                            },
                            {
                              "algorithm": "Dummy"
                            },
                            {
                              "threshold": 0
                            }
                          ]
                        },
                        {
                          "field_score": [
                            {
                              "pair_id": "200001"
                            },
                            {
                              "field": "Store/Map"
                            },
                            {
                              "fieldValue": ""
                            },
                            {
                              "value": 1
                            },
                            {
                              "algorithm": "Dummy"
                            },
                            {
                              "threshold": 0
                            }
                          ]
                        }
                      ]
                    }
                  ]
                },
                {
                  "values": [
                    {
                      "value": [
                        {
                          "field": "Store/Long"
                        },
                        {
                          "value": "39.913856"
                        }
                      ]
                    },
                    {
                      "value": [
                        {
                          "field": "Store/Map"
                        },
                        {
                          "value": ""
                        }
                      ]
                    },
                    {
                      "value": [
                        {
                          "field": "Store/Address"
                        },
                        {
                          "value": "Beijing book building"
                        }
                      ]
                    },
                    {
                      "value": [
                        {
                          "field": "Store/Id"
                        },
                        {
                          "value": "200002"
                        }
                      ]
                    },
                    {
                      "value": [
                        {
                          "field": "Store/Lat"
                        },
                        {
                          "value": "116.38368"
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "detail": [
                {
                  "id": "200001"
                },
                {
                  "match": [
                    {
                      "is_match": true
                    },
                    {
                      "score": 0.9157568693161011
                    },
                    {
                      "field_scores": [
                        {
                          "field_score": [
                            {
                              "pair_id": "200002"
                            },
                            {
                              "field": "Store/Id"
                            },
                            {
                              "fieldValue": "200002"
                            },
                            {
                              "value": 1
                            },
                            {
                              "algorithm": "Dummy"
                            },
                            {
                              "threshold": 0
                            }
                          ]
                        },
                        {
                          "field_score": [
                            {
                              "pair_id": "200002"
                            },
                            {
                              "field": "Store/Address"
                            },
                            {
                              "fieldValue": "Beijing book building"
                            },
                            {
                              "value": 0.9157568693161011
                            },
                            {
                              "algorithm": "Jaro-Winkler"
                            },
                            {
                              "threshold": 0.9
                            }
                          ]
                        },
                        {
                          "field_score": [
                            {
                              "pair_id": "200002"
                            },
                            {
                              "field": "Store/Lat"
                            },
                            {
                              "fieldValue": "116.38368"
                            },
                            {
                              "value": 1
                            },
                            {
                              "algorithm": "Dummy"
                            },
                            {
                              "threshold": 0
                            }
                          ]
                        },
                        {
                          "field_score": [
                            {
                              "pair_id": "200002"
                            },
                            {
                              "field": "Store/Long"
                            },
                            {
                              "fieldValue": "39.913856"
                            },
                            {
                              "value": 1
                            },
                            {
                              "algorithm": "Dummy"
                            },
                            {
                              "threshold": 0
                            }
                          ]
                        },
                        {
                          "field_score": [
                            {
                              "pair_id": "200002"
                            },
                            {
                              "field": "Store/Map"
                            },
                            {
                              "fieldValue": ""
                            },
                            {
                              "value": 1
                            },
                            {
                              "algorithm": "Dummy"
                            },
                            {
                              "threshold": 0
                            }
                          ]
                        }
                      ]
                    }
                  ]
                },
                {
                  "values": [
                    {
                      "value": [
                        {
                          "field": "Store/Long"
                        },
                        {
                          "value": "39.913855"
                        }
                      ]
                    },
                    {
                      "value": [
                        {
                          "field": "Store/Map"
                        },
                        {
                          "value": ""
                        }
                      ]
                    },
                    {
                      "value": [
                        {
                          "field": "Store/Address"
                        },
                        {
                          "value": "Beijing Book Building (17 Xichang'an Jie)"
                        }
                      ]
                    },
                    {
                      "value": [
                        {
                          "field": "Store/Id"
                        },
                        {
                          "value": "200001"
                        }
                      ]
                    },
                    {
                      "value": [
                        {
                          "field": "Store/Lat"
                        },
                        {
                          "value": "116.383681"
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Return an explanation about a matching group

Returns a detailed explanation about why data records in an existing matching group are matched and grouped together.
Request
GET /services/rest/tasks/matching/explain/{container}/groups
Parameters
  • container: This is a String value which represents the name of the container.
  • type: This is a String value which specifies the name of the entity.
  • group: This is a String value which represents the ID of the matching group.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON response that gives a detailed explanation about why data records in an existing matching group are matched and grouped together.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

The format of the response for this REST API is same as the format of the response for Return a matching explanation about input records.

Return a matching explanation about data records in the staging area

Returns a detailed explanation about how a list of data records in the staging area is matched and survived. You need to submit the data records by providing their IDs in the request content, with each ID separated by a line feed character.
Request
POST /services/rest/tasks/matching/explain/{container}/records
Parameters
  • container: This is a String value which represents the name of the container.
  • type: This is a String value which specifies the name of the entity.
Headers
  • Content-Type: text/plain
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON response that gives a detailed explanation about how a list of data records in the staging area is matched and survived.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample request

200001
200002

Sample response

The format of the response for this REST API is same as the format of the response for Return a matching explanation about input records.

Return a list of similar records

Returns a list of data records in the staging area similar to the one provided in the request content as XML, including the similar golden record with status 205 and its source records.
Request
POST /services/rest/tasks/matching/similar/{container}
Parameters
  • container: This is a String value which represents the name of the container.
  • model: This is a String value which represents the name of the MDM data model.
  • type: This is a String value which specifies the name of the entity.
Headers
  • Content-Type: application/xml
    Note: If the XML declaration is included in the XML request content and the encoding attribute is used, you must set its value to UTF-8.
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON response that contains a list of data records in the staging area similar to the input data record.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample request

<Product>
  <Id>300000001</Id>
  <Name>the old man and the sea</Name>
  <Description>Author - Ernest Miller Hemingway</Description>
  <Availability>false</Availability>
  <Price>21.99</Price>
</Product>

Sample response

{
  "items": [
    {
      "id": "200000005",
      "confidence": 0.9279589414596557,
      "golden": false
    },
    {
      "id": "200000006",
      "confidence": 0.960698914527893,
      "golden": false
    },
    {
      "id": "dc5bfccb-010a-4f3e-8b32-7b00ef0e7cd7",
      "confidence": 0.960698914527893,
      "golden": true
    }
  ]
}

Staging area management

Get staging area statistics for the current container

Gets staging area statistics for the current container, including the container name, the model name, the number of total records, valid records, invalid records, and records waiting for validation.
Request
GET /services/rest/tasks/staging
Headers
  • Accept: application/json, application/xml or text/xml
  • Authorization: Basic Authentication scheme
Response The staging area statistics for the current container.
Status
  • 200 OK: The operation is executed successfully.
  • 204 NO CONTENT: The container or model has not been selected.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response in JSON

{
  "total_records": 100,
  "waiting_validation_records": 15,
  "valid_records": 80,
  "invalid_records": 5,
  "data_container": "Product",
  "data_model": "Product"
}

Sample response in XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<staging>
  <data_container>Product</data_container>
  <data_model>Product</data_model>
  <invalid_records>5</invalid_records>
  <total_records>100</total_records>
  <valid_records>80</valid_records>
  <waiting_validation_records>15</waiting_validation_records>
</staging>

Start a new validation task for the current container

Starts a new validation task for the current container and returns its ID.
Request
POST /services/rest/tasks/staging
Headers
  • Accept: text/plain
  • Authorization: Basic Authentication scheme
Response The ID of the new validation task.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

This REST API returns the ID of the new validation task, like the following:
2ff91416-b25b-4cbb-8fb1-45bc12f34806

Get staging area statistics for a container

Gets staging area statistics for a given container, including the container name, the model name, the number of total records, valid records, invalid records, and records waiting for validation.
Request
GET /services/rest/tasks/staging/{container}
Parameters
  • container: This is a String value which represents the name of the container.
  • model: This is a String value which represents the name of the MDM data model.
Headers
  • Accept: application/json, application/xml or text/xml
  • Authorization: Basic Authentication scheme
Response The staging area statistics for the specified container.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response in JSON

{
  "total_records": 100,
  "waiting_validation_records": 15,
  "valid_records": 80,
  "invalid_records": 5,
  "data_container": "Product",
  "data_model": "Product"
}

Sample response in XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<staging>
  <data_container>Product</data_container>
  <data_model>Product</data_model>
  <invalid_records>5</invalid_records>
  <total_records>100</total_records>
  <valid_records>80</valid_records>
  <waiting_validation_records>15</waiting_validation_records>
</staging>

Start a new validation task for a container

Starts a new validation task for a given container and returns its ID.
Request
POST /services/rest/tasks/staging/{container}
Parameters
  • container: This is a String value which represents the name of the container.
  • model: This is a String value which represents the name of the MDM data model.
Headers
  • Accept: text/plain
  • Authorization: Basic Authentication scheme
Response The ID of the new validation task.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

This REST API returns the ID of the new validation task, like the following:
2ff91416-b25b-4cbb-8fb1-45bc12f34806

List all completed validation tasks IDs for a container

Lists the IDs of all completed validation tasks for a given container.
Request
GET /services/rest/tasks/staging/{container}/execs
Parameters
  • container: This is a String value which represents the name of the container.
  • before: This is a String value which represents the date in the format of yyyy-MM-ddTHH:mm:ss that limits the search to the validation tasks started before it.
  • start: This is an integer value which represents the pagination start offset.
  • size: This is an integer value which represents the pagination size.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response The IDs of all completed validation tasks for the specified container.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{executions:["2ff91416-b25b-4cbb-8fb1-45bc12f34806","a3ca3305-d80c-4594-adf9-bb52ffd50ec5"]}

Get the number of all completed validation tasks for a container

Gets the number of all completed validation tasks for a given container.
Request
GET /services/rest/tasks/staging/{container}/execs/count
Parameters
  • container: This is a String value which represents the name of the container.
  • before: This is a String value which represents the date in the format of yyyy-MM-ddTHH:mm:ss that limits the search to the validation tasks started before it.
Headers
  • Authorization: Basic Authentication scheme
Response An integer that represents the number of all completed validation tasks for the specified container.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Get statistics of the current validation execution for a container

Gets statistics of the current validation execution for a given container, including the execution ID, the execution start time, end time and running time, and also the number of total records, processed records, and invalid records.
Request
GET /services/rest/tasks/staging/{container}/execs/current
Parameters
  • container: This is a String value which represents the name of the container.
  • model: This is a String value which represents the name of the MDM data model.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response The statistics of the current validation execution for the specified container.
Status
  • 200 OK: The operation is executed successfully.
  • 204 NO CONTENT: No validation task is currently running for the container.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{
    "id": "e5aa900c-ddc4-47e4-a81a-9dc1c863b6a2",
    "start_date": "2019-03-12T03:31:25.755",
    "end_date": "2019-03-12T03:32:00.594",
    "running_time": "9:40:39",
    "total_record": 6,
    "processed_records": 6,
    "invalid_records": 0
}

Cancel the current validation execution for a container

Cancels the current validation execution for a given container.
Request
DELETE /services/rest/tasks/staging/{container}/execs/current
Parameters
  • container: This is a String value which represents the name of the container.
  • model: This is a String value which represents the name of the MDM data model.
Headers
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 200 OK: The operation is executed successfully.
  • 204 NO CONTENT: No validation task is currently running for the container.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Get execution statistics of a validation task for a container

Gets execution statistics of a validation task for a container by its execution ID, including the execution ID, the execution start time, end time and running time, and also the number of total records, processed records, and invalid records.
Request
GET /services/rest/tasks/staging/{container}/execs/{executionId}
Parameters
  • container: This is a String value which represents the name of the container.
  • model: This is a String value which represents the name of the MDM data model.
  • executionId: This is a String value which represents the execution ID of the validation task.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response The execution statistics of the specified validation task for a container.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{
    "id": "e5aa900c-ddc4-47e4-a81a-9dc1c863b6a2",
    "start_date": "2019-03-12T03:31:25.755",
    "end_date": "2019-03-12T03:32:00.594",
    "running_time": "9:40:39",
    "total_record": 6,
    "processed_records": 6,
    "invalid_records": 0
}

Check if a container has a staging area

Checks if a container has a staging area.
Request
GET /services/rest/tasks/staging/{container}/hasStaging
Parameters
  • container: This is a String value which represents the name of the container.
Headers
  • Authorization: Basic Authentication scheme
Response Returns true if the specified container has a staging area, false otherwise.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Data Stewardship

Get the number of Data Stewardship tasks

Gets the number of Data Stewardship tasks for the current user.
Request
GET /services/rest/tds/amount
Headers
  • Authorization: Basic Authentication scheme
Response An integer that represents the number of Data Stewardship tasks for the current user.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 500 INTERNAL SERVER ERROR: Other errors.
  • 503 SERVICE UNAVAILABLE: Talend Data Stewardship server is unavailable.

Set up the Data Stewardship metadata for a data model

Sets up the metadata in Talend Data Stewardship for an MDM data model with one or more match rules attached to its entities, including a data stewardship data model and a merging campaign for each entity that has a match rule attached to it.

The new data stewardship data model is created with attributes that correspond to each simple type element in the MDM entity, and the new merging campaign is created using the new data stewardship data model.

Request
PUT /services/rest/tds/setup
Parameters
  • model: This is a String value which represents the name of the MDM data model.
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON response that contains the name and the label of the data model and the merging campaign created for each entity that has a match rule attached to it. The name is in the format of tmdm_<mdm_data_model_name>_<mdm_entity_name> and the label is in the format of <mdm_data_model_name> - <mdm_entity_name> - tmdm, all in lower case.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 403 FORBIDDEN: Missing required permission.
  • 420 METHOD FAILURE: The setup of the data stewardship metadata fails.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{
  "success": {
    "datamodel": [
      {
        "name": "tmdm_product_product",
        "label": "product - product - tmdm"
      },
      {
        "name": "tmdm_product_store",
        "label": "product - store - tmdm"
      }
    ],
    "campaign": [
      {
        "name": "tmdm-product-product",
        "label": "product - product - tmdm"
      },
      {
        "name": "tmdm-product-store",
        "label": "product - store - tmdm"
      }
    ]
  }
}

Transactions

Get all active transaction IDs

Gets the IDs of all active transactions.

Request
GET /services/rest/transactions
Headers
  • Accept: application/json
  • Authorization: Basic Authentication scheme
Response A JSON array that contains the IDs of all active transactions.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

{transactions:["db4655dd-dbae-4dc2-a6f9-5b1b08ca1410","9197782a-3375-427f-9eb5-a77c76e92305"]}

Start a new transaction

Starts a new transaction and returns its ID.

Request
PUT /services/rest/transactions
Headers
  • Accept: text/plain
  • Authorization: Basic Authentication scheme
Response The ID of the new transaction.
Status
  • 200 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Sample response

This REST API returns the ID of the new transaction, like the following:
2ff91416-b25b-4cbb-8fb1-45bc12f34806

Commit a transaction

Commits a transaction by its ID.

Request
POST /services/rest/transactions/{id}
Parameters
  • id: This is a String value which specifies the ID of the transaction.
Headers
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 204 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Rollback a transaction

Rollbacks a transaction by its ID.

Request
DELETE /services/rest/transactions/{id}
Parameters
  • id: This is a String value which specifies the ID of the transaction.
Headers
  • Authorization: Basic Authentication scheme
Response No content.
Status
  • 204 OK: The operation is executed successfully.
  • 401 UNAUTHORIZED: Login fails, invalid username or password.
  • 500 INTERNAL SERVER ERROR: Other errors.

Query language

In the REST API calls, you can use the query language to narrow down the data records of interest, order the query results or implement other functions according to your needs.

SELECT clauses

You can use basic SELECT clauses in REST API calls.

Select all instances of an entity

You can query all instances of an entity. For example, to return all instances of "Type1" records, use the query:

{
  "select": {
    "from": ["Type1"]
  }
}

Note that the "from" element is an array type, allowing you to select multiple types. This does not mean the query can return all Type1 and Type2 instances. The array is used for joins.

Paging: start and limit

Paging may be needed in some use cases such as building a table. This can be done with "start" and "limit" elements.

For example, to return results which start with the first instance of "Type1" (index is 0 based), together with the next 10 instances, use the query:

{
  "select": {
    "from": ["Type1"],
    "start": 0,
    "limit": 10
  }
}
Pay attention to the following:
  • Standard execution of such queries implies a creation of a list of size limit on the server side. If you specify a large value, it may lead to performance issues or an OutOfMemoryError. Therefore, make sure to use paging with relatively small values of limit.

  • A limit of Integer.MAX_VALUE (2^31 -1) is considered by MDM code as "no limit". Make sure to use a streaming strategy instead of a list of results. However, this means a limit of (2^31 - 2) will try to create a list of that size (usually causing OutOfMemoryError).

Select field values

Instead of querying the whole record, you can query only one field in the record.

For example, to return values of "id" field of "Type1" type, use the query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"field": "Type1/id"}
    ]
  }
}

To select more than one field, repeat the "field" element. For example, to return values of "id", "value1" and "value2" for each record of "Type1", use this query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"field": "Type1/id"},
      {"field": "Type1/value1"},
      {"field": "Type1/value2"}
    ]
  }
}

Aliasing

For naming or client reasons, you may want to alias the returned element. Aliasing can help differentiate values in case of name collision. This can be achieved with the "alias" element.

For example, to return all "id" values in all Type1 instances, but instead of wrapping values inside an element "id", the wrapping element will be named "a", use the query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"field": "Type1/id"},
      {
        "alias": [
          {"name": "a"},
          {"field": "Type1/id"}
        ]
      }
    ]
  }
}

For example, use the query to return results with "Type1/containedField1/value1" wrapped in "v1" and "Type1/containedField2/value1" in "v2", thus avoiding the possible naming collision that both "Type1/containedField1/value1" and "Type1/containedField2/value1" would be returned wrapped in a "value1" element:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {
        "alias": [
          {"name": "v1"},
          {"field": "Type1/containedField1/value1"}
        ]
      },
      {
        "alias": [
          {"name": "v2"},
          {"field": "Type1/containedField2/value1"}
        ]
      }
    ]
  }
}

Distinct field values

You can use the "distinct" keyword to get all distinct values for an entity field.

For example:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"distinct": {"field": "Type1/value1"}}
    ]
  }
}

This query will return all distinct values for "value1" in type "Type1".

You can also alias a result using the "alias" element. For example:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {
        "alias": [
          {"name": "a0"},
          {"distinct": {"field": "Type1/value1"}}
        ]
      }
    ]
  }
}

This returns all distinct values for "value1" but inside an element named "a0".

Count

You can count how many results are returned by a query.

For example, to count how many Type1 instances are available, use the query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"count": {}}
    ]
  }
}

You can also combine the count operation with conditions to count how many instances match the specified criteria.

For example, to return how many instances of Type1 have a value1 greater than 1, use the query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"count": {}}
    ],
    "where": {
      "gt": [
        {"field": "Type1/value1"},
        {"value": "1"}
      ]
    }
  }
}

Count may also be used in order_by clauses.

Maximum and minimum

You can select the maximum and/or minimum value for a field in an entity.

For example, to retrieve the smallest value of "id" in the entity Type1, use the query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"min": {"field": "Type1/id"}}
    ]
  }
}

For example, to retrieve the largest value of "id" in Type1, use the query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"max": {"field": "Type1/id"}}
    ]
  }
}

For example, to retrieve both the largest and the smallest values of "id" in Type1, use the query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"max": {"field": "Type1/id"}},
      {"min": {"field": "Type1/id"}}
    ]
  }
}

Order by field's value

You can use as many "order by" clauses as needed in a query either in descending or ascending order to set the order in the returned result.

For example, to select all instances of Type1 and order results by id (from the largest to the smallest) and value1 (from the smallest to the largest), use the query:

{
  "select": {
    "from": ["Type1"],
    "order_bys": [
      {
        "order_by": [
          {"field": "Type1/id"},
          {"direction": "DESC"}
        ]
      },
      {
        "order_by": [
          {"field": "Type1/value1"},
          {"direction": "ASC"}
        ]
      }
    ]
  }
}

Order by field value occurrence

You can also order by field value occurrence so that the most common value is returned first down to the least common value (in case of DESC direction).

Here is one query example to return the most common value for "value1" down to the least common:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"field": "Type1/value1"},
    ],
    "order_bys": [
      {
        "order_by": [
          {"count": {"field": "Type1/value1"}},
          {"direction": "DESC"}
        ]
      }
    ]
  }
}

You can also include "limit" in the query to obtain the top N common values.

Here is one query example to obtain the top 10 values:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"field": "Type1/value1"},
    ],
    "order_bys": [
      {
        "order_by": [
          {"count": {"field": "Type1/value1"}},
          {"direction": "DESC"}
        ]
      }
    ],
    "limit": 10
  }
}
Note: This kind of query does not work if you are using the DB2 or MS SQL Server database, because they don't support the order by clause with the count function.

Conditions

The following operands are available for use:

Query language operand

Meaning

eq

equals

gt / gte

greater than/greater than or equals

lt / lte

less than / less than or equals

startsWith

starts with

contains

contains

isEmpty

is empty ('')

isNull

is null

Bear in mind the following:

  • The structure is always "operand": { field, value }.
  • If the field is of numeric type (int, short, integer, long, float, double, decimal) and you provide a numeric value for it, the value can be interpreted with or without being enclosed in double quotes. Otherwise, the value must be enclosed in double quotes. Make sure that the value matches the element type. For example, "value": "abc" is incorrect if the field "Type1/id" is of numeric type.
  • If you have no access to any of the fields used in the condition, the query will return nothing.

Example 1: The query below using the eq operand returns all instances of Type1 where the id element equals 1:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "eq": [
        {"field": "Type1/id"},
        {"value": "1"}
      ]
    }
  }
}

Example 2: The query below using the isEmpty operand returns all instances of Store where the value of the Address element equals ''.

{
  "select": {
    "from": ["Store"],
    "where": {
      "isEmpty": {
        "field": "Store/Address"
      }
    }
  }
}

Example 3: The query below using the not operator and the isNull operand returns all instances of Store where the value of the Address element is not null.

{
    "select": {
        "from": ["Store"],
        "where": {
            "not": {
                "isNull": {
                    "field": "Store/Address"
                }
            }
        }
    }
}

Index based condition

You may specify an index for your condition in case the field is a repeatable element. The index is 0-based.

For example, to return all instances of Type1 where the second value of "list" equals "1", use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "eq": [
        {"index": [
          {"field": "Type1/list"},
          1
        ]},
        {"value": "1"}
      ]
    }
  }
}

Conditions between fields

You may use conditions between different fields. Only "eq" (EQUALS) is currently supported for such comparisons.

For example, to return the instances of Type1 where "id" equals "value1", use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "eq": [
        {"field": "Type1/id"},
        {"value": {"field": "Type1/value1"}}
      ]
    }
  }
}

in operator

You can use the operator in within a query.

For example, to return all instances of Type1 where "id" equals 1 or 2 or 5, use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "in": [
        {"field": "Type1/id"},
        {"value": ["1","2","5"]}
      ]
    }
  }
}

Bear in mind the following:

  • If the field is of numeric type (int, short, integer, long, float, double, decimal) and you provide a numeric value for it, the value can be interpreted with or without being enclosed in double quotes. Otherwise, the value must be enclosed in double quotes. Make sure that the value matches the element type. For example, "value": ["a", "b", "c"] is incorrect if the field "Type1/id" is of numeric type.
  • The "value" is formatted as a JSON array, and the values listed must be static. For example, ["value1", "value2", "value3"...].
  • Only simple type fields at the root level are supported. However, the mandatory multi-occurrence foreign key fields are not supported.
  • The operator in cannot be used together with the full text search.
  • The operator in can work with the operator not. For more information, see the section not operator.

Logic operators

To allow more complex queries, boolean operators (and/or/not) may be included. The structure is similar to what is previously presented "operator": { left, right } where left and/or right can be logic operators too.

For example, to return all Type1 instances where "id = 1 OR value1 = 0", use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "or": [
        {
          "eq": [
            {"field": "Type1/id"},
            {"value": "1"}
          ]
        },
        {
          "eq": [
            {"field": "Type1/value1"},
            {"value": "0"}
          ]
        }
      ]
    }
  }
}

A binary logic operator (and/or) must have 2 and only 2 children. For example, you can write a condition with "a OR b OR c":

{
  "select": {
    "from": ["Type1"],
    "where": {
      "or": [
        {
          "eq": [
            {"field": "Type1/id"},
            {"value": "1"}
          ]
        },
        {
          "or": [
            {
              "eq": [
                {"field": "Type1/id"},
                {"value": "2"}
              ]
            },
            {
              "eq": [
                {"field": "Type1/value1"},
                {"value": "4"}
              ]
            }
          ]
        }
      ]
    }
  }
}

You can also use mixed logic operators. There is no ambiguity in the evaluation order of conditions.

For example, to return all Type1 instances where "(id = 1 OR (id = 2 and value1 = 4))", use the query which mix both and and or:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "or": [
        {
          "eq": [
            {"field": "Type1/id"},
            {"value": "1"}
          ]
        },
        {
          "and": [
            {
              "eq": [
                {"field": "Type1/id"},
                {"value": "2"}
              ]
            },
            {
              "eq": [
                {"field": "Type1/value1"},
                {"value": "4"}
              ]
            }
          ]
        }
      ]
    }
  }
}  

not operator

The operator not is supported and follows the structure "not": { condition }.

For example, to return all Type1 instances where "id" is not equal to 1, use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "not": {
        "eq": [
          {"field": "Type1/id"},
          {"value": "1"}
        ]
      }
    }
  }
}

You can also use the operator not in a more complex condition. For example:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "not": {
        "or": [
          {
            "eq": [
              {"field": "Type1/id"},
              {"value": "2"}
            ]
          },
          {
            "eq": [
              {"field": "Type1/id"},
              {"value": "4"}
            ]
          }
        ]
      }
    }
  }
}

You can use the operator not together with the operator in. The queried result does not include instances where the specified field has a null value.

For example:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "not": {
        "in": [
            {"field": "Type1/id"},
            {"value": ["1","3","4","5"]}        
            ]
      }
    }
  }
}

This will return all instances of Type1 where "id" does not equal any of the values listed, that is, 1, 3, 4 and 5.

Full text (field scope)

The query language includes support for full text search.

For example, to perform a full text search on the field "Type1/id" with the value "1", use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "full_text": [
        {"field": "Type1/id"},
        {"value": "1"}
      ]
    }
  }
}

The full text query also works on subelements. For example, if you have the following type structure:

Type1
  * id
  * element
    * value1
    * value2

Then the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "full_text": [
        {"field": "Type1/element"},
        {"value": "1"}
      ]
    }
  }
}

Is equivalent to:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "or": [
        {
          "full_text": [
            {"field": "Type1/element/value1"},
            {"value": "1"}
          ]
        },
        {
          "full_text": [
            {"field": "Type1/element/value2"},
            {"value": "1"}
          ]
        }
      ]
    }
  }
}

Full text (entity scope)

If you do not want to specify any field but want to look for a value in the entity, the following query gives an example of how to achieve this:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "full_text": [
        {"value": "1"}
      ]
    }
  }
}

If you omit the field, MDM performs full text search on all Type1 fields.

Metadata fields

MDM adds "metadata" fields to the records it manages. These fields are not present in the user data model but can be used in the query.

Supported metadata fields are:

  • taskId (also known as groupId)
  • timestamp (last modification time)
  • groupSize

For the staging storage, there are additional metadata fields:

  • Staging error
  • Staging source
  • Staging status
  • Staging block key
  • Staging has task

    Note that the field "Staging has task" indicates whether a Talend Data Stewardship task is linked with the record.

If the query contains staging-only fields, you may expect an error that indicates the incompatible expressions. The incompatible expressions are checked in selected fields, conditions, order by and join expressions.

Get metadata field values

To get a metadata field, use "metadata" instead of "field" in the selected fields as shown in the following example:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"metadata": "timestamp"},
      {"metadata": "task_id"},
      {"metadata": "group_size"},
      {"metadata": "staging_error"},
      {"metadata": "staging_source"},
      {"metadata": "staging_status"},
      {"metadata": "staging_blockkey"},
      {"metadata": "staging_hastask"}
    ]
  }
}

You can also mix fields from the entity with fields from the record:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"metadata": "timestamp"},
      {"metadata": "task_id"},
      {"field": "Type1/id"}
    ]
  }
}

You can also use the "distinct" keyword for metadata fields. For example, to return all distinct values of "taskId" wrapped in an element named "distinctTaskId", use the query:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {
        "alias": [
          {"name": "distinctTaskId"},
          {"distinct": {"metadata": "task_id"}}
        ]
      }
    ]
  }
}

Conditions involving metadata fields

You can involve metadata fields in conditions. As for the selected fields, just use "metadata" instead of "field". For example:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "eq": [
        {"metadata": "task_id"},
        {"value": "1"}
      ]
    }
  }
}

Cache

MDM can cache query results. In this case, it keeps the query in cache and serves the cached result every time this query is executed.

For example, to cache a query result, use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "eq": [
        {"field": "Type1/id"},
        {"value": "1"}
      ]
    },
    "cache": true
  }
}

By default, "cache" is false, but this query is also valid:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "eq": [
        {"field": "Type1/id"},
        {"value": "1"}
      ]
    },
    "cache": false
  }
}

A cached result can have at most 50 records. When the query yields too many results, the following log will appear in the MDM log at DEBUG level: Query is yielding more results than cache is allowed to keep, bypassing cache for query.

Join to other types

Joins are a way to retrieve data from another type following the foreign keys.

For example, you can retrieve the Address's street element by following the FK between Person and Address types.

A simple example of join would be:

{
  "select": {
    "from": [
      "Type1",
      "Type2",
    ],
    "fields": [
      {"field": "Type1/id"},
      {"field": "Type2/value1"}
    ],
    "joins": [
      {
        "from": "Type1/fk2",
        "on": "Type2/id"
      }
    ]
  }
}

This query returns all Type1's "id" element as well as the "value1" element located in Type2. The join is performed using the "fk2" in Type1.

You may chain joins in the query too. For example:

{
  "select": {
    "from": [
      "Type1",
      "Type2",
      "Type3"
    ],
    "fields": [
      {"field": "Type1/id"},
      {"field": "Type2/value1"},
      {"field": "Type3/value2"}
    ],
    "joins": [
      {
        "from": "Type1/fk2",
        "on": "Type2/id"
      },
      {
        "from": "Type2/fk3",
        "on": "Type3/id"
      }
    ]
  }
}

This query displays "id" from Type1, "value1" from Type2 and "value1" from Type3. The value for Type3's "value1" is obtained via a join between Type1, Type2 and Type3.

Note: Having more than one join to the same entity in a single query is not supported.

History related operators - “as of" operator

The "as_of" operator allows you to retrieve a record as it was at any date.

For example:

{
  "select": {
    "from": ["Type1"],
    "as_of": {
      "date": "1000"
    }
  }
}

The "as of" operator will not fail in case no history is available. Rather, it will return the record at its current state.

History browsing relies on the journal. If you inserted data straight into the SQL database or disabled event creation for tMDMOutput/tMDMDelete components, you should expect only partial history.

MDM can only build a history of the records based on information it stored about updates: if you disabled journal for some updates, it cannot be a complete and accurate history.

For example, to return all Type1 instances with the values they had one second after January 1st 1970 (1000 ms after EPOCH), use the query:

{
  "select": {
    "from": ["Type1"],
    "as_of": {
      "date": "1970-01-01T00:00:01"
    }
  }
}

History navigation shortcuts

The element "date" for the "as_of" element supports some handy shortcuts:

  • yesterday: date 24 hours ago.
  • creation: date of record creation. When you use "creation", you are not allowed to use the swing parameter. Otherwise, an expected error will be thrown.
  • now: current date.

For example, to return all instances of Type1 as they were yesterday (24 hours ago), use the query:

{
  "select": {
    "from": ["Type1"],
    "as_of": {
      "date": "yesterday"
    }
  }
}

Filter history (get by id)

You may want to filter the history records instead of getting all instances.

For example, to return the instance of Type1 with "id = 1" as it was on January 1st 1970, use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "eq": [
        {"field": "Type1/id"},
        {"value": "1"}
      ]
    },
    "as_of": {
      "date": "1000"
    }
  }
}

An improvement on this query would be to use all conditions with "as_of", which allows you to query on previous values. Note that conditions do not need to be only on id elements.

For example, to return the Type1 instance that contained "text" on January 1st, 1970, use the query:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "contains": [
        {"field": "Type1/value"},
        {"value": "text"}
      ]
    },
    "as_of": {
      "date": "1000"
    }
  }
}

However, this has scalability issues since it might force MDM to build a complete set of Type1 instances, thus leading to memory issues since states are computed in memory.

Using Joins together with as_of

You can use "joins" in a query together with the "as_of" operator.

For example, using the query below returns three fields ("id", "value1", "value2") coming from two types ("Type1", "Type2"):

{
  "select": {
    "from": [
      "Type1",
      "Type2"
    ],
    "fields": [
      {"field": "Type1/id"},
      {"field": "Type1/value1"},
      {"field": "Type2/value2"}
    ],
    "joins": [
      {
        "from": "Type1/fk2",
        "on": "Type2/id"
      }
    ],
    "as_of": {
      "date": "1000"
    }
  }
}

The returned values have the features below:

  • The "as_of" returns the values of "Type1" as they were on 01/01/1970.
  • The FK value used for Type2 is the FK value as of 01/01/1970.
  • The values for Type2 are those on 01/01/1970.

Security and user roles

Bear in mind that the query passed to the service is not always the query MDM will execute. Before executing the query, MDM will prune all elements the logged user is not allowed to access. This includes selected fields, conditions, and order by. All elements to hide from users are removed from the query without reporting any errors.

For example,

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"field": "Type1/id"},
	  {"field": "Type1/secret_field"}
    ]
  }
}

In this example, MDM will automatically remove "secret_field" from the query if the current user (that is, the user used to log on the REST API) is not allowed to access "secret_field". Therefore, the actual query executed will be:

{
  "select": {
    "from": ["Type1"],
    "fields": [
      {"field": "Type1/id"}
    ]
  }
}

Another example of such queries is:

{
  "select": {
    "from": ["Type1"],
    "where": {
      "gt": [
        {"field": "Type1/secret_field"},
        {"value": "1"}
      ]
    }
  }
}

This query will be read as "return all instances of Type1 where Type1's secret_field is greater than 1", and the returned result is expected to be Type1 instances where secret_field > 1.

However, if the current user's roles do not have access right to "secret_field", the actual query becomes:

{
  "select": {
    "from": ["Type1"]
  }
}

This query will be read as "return all instances of Type1", and the returned result now is expected to be all Type1 instances since the current user is not allowed to access "secret_field".

This prevents unprivileged users to indirectly guess what are the values in "secret_field" using value conditions on this field.

An example of how to execute an HTTP GET operation with MDM REST API from Talend MDM Web UI

From Talend MDM Web UI, authorized users can use the Tools menu to access the MDM REST API documentation page based on Swagger, and test REST API operations on this page if needed.

The following introduces an example of executing an HTTP GET operation with MDM REST API to list all primary keys of the master data records in a container on the MDM server.

Before you begin

Make sure that you have already imported the MDM Demo project and created several master data records for the Product entity of the Product container.

Procedure

  1. Open the REST API documentation page from Talend MDM Web UI.
  2. From the list of operations under the User data management category, click the GET link which is followed by /data/{containerName}/{type} to displays its details. You can also click Lists all primary keys for container and type to display the operation details.
  3. Select Master from the Container list.
  4. Enter Product in the containerName field.
  5. Enter Product in the type field.
  6. Click Execute to execute the GET operation.

    The HTTP request URL, response body, response code and response headers are displayed accordingly. In this example, all primary keys of the Product container are listed.

  7. To verify the execution result of this GET operation, go to the Master Data Browser page, then search the master data records of the Product container, and validate that the execution result matches the search result.

Transactions API integration

Transactions API can be integrated with data API.

For more information about the MDM transactions, see MDM Transactions.

Attaching a REST request to an existing transaction

Modifications made through the REST data API can be attached to a transaction created through the transactions API.

To attach a REST request to an existing transaction, you just need to set the custom HTTP header "transaction-id" to the id of the transaction. If the provided transaction id does not exist, it will be created.

For example:
PUT /talendmdm/services/rest/transactions, creates a new transaction and returns its id
POST /talendmdm/services/rest/data/Product (with transaction-id set to the id returned at step1), creates a new product but does not commit changes to the database
POST /talendmdm/services/rest/transactions/<id> commits the transactions and thus changes made at step 2

This feature also works with batch insert operation (/talendmdm/services/rest/data/Product/batch).